Survey Methods Design in Psychology.ppt

1、Survey Methods & Design in Psychology,Lecture 10 ANOVA (2007)Lecturer: James Neill,Overview of Lecture,Testing mean differences ANOVA models Interactions Follow-up tests Effect sizes,Parametric Tests of Mean Differences,One-sample t-test Independent samples t-test Paired samples t-test One-way ANOVA

2、 One-way repeated measures ANOVA Factorial ANOVA Mixed design ANOVA ANCOVA MANOVA Repeated measures MANOVA,Correlational statistics vs tests of differences between groups,Correlation/regression techniques reflect the strength of association between continuous variables Tests of group differences (t-

3、tests, anova) indicate whether significant differences exist between group means,Are The Differences We See Real?,Major Assumptions,Normally distributed variables Homogeneity of variance Robust to violation of assumptions,A t-test or ANOVA is used to determine whether a sample of scores are from the

4、 same population as another sample of scores.(in other words these are inferential tools for examining differences in means),Why a t-test or ANOVA?,t-tests,An inferential statistical test used to determine whether two sets of scores come from the same populationIs the difference between two sample m

5、eans real or due to chance?,Use of t in t-tests,Question: Is the t large enough that it is unlikely that the two samples have come from the same population? Decision: Is t larger than the critical value for t (see t tables depends on critical and N),Ye Good Ol Normal Distribution,Use of t in t-tests

6、,t reflects the ratio of differences between groups to within groups variability Is the t large enough that it is unlikely that the two samples have come from the same population? Decision: Is t larger than the critical value for t (see t tables depends on critical and N),One-tail vs. Two-tail Tests

7、,Two-tailed test rejects null hypothesis if obtained t-value is extreme is either direction One-tailed test rejects null hypothesis if obtained t-value is extreme is one direction (you choose too high or too low) One-tailed tests are twice as powerful as two-tailed, but they are only focused on iden

8、tifying differences in one direction.,Compare one group (a sample) with a fixed, pre-existing value (e.g., population norms)E.g., Does a sample of university students who sleep on average 6.5 hours per day (SD = 1.3) differ significantly from the recommended 8 hours of sleep?,Single sample t-test,Co

9、mpares mean scores on the same variable across different populations (groups) e.g.,Do males and females differ in IQ?Do Americans vs. Non-Americans differ in their approval of George Bush?,Independent groups t-test,Assumptions (Independent samples t-test),IV is ordinal / categorical e.g., gender DV

10、is interval / ratio e.g., self-esteem Homogeneity of Variance If variances unequal (Levenes test), adjustment made Normality t-tests robust to modest departures from normality: consider use of Mann-Whitney U test if severe skewness Independence of observations (one participants score is not dependen

11、t on any other participants score),Do males and females differ in memory recall?,Paired samples t-test,Same participants, with repeated measures Data is sampled within subjects, e.g., Pre- vs. post- treatment ratings Different factors e.g., Voters approval ratings of candidate X vs. Y,Assumptions- p

12、aired samples t-test,DV must be measured at interval or ratio level Population of difference scores must be normally distributed (robust to violation with larger samples) Independence of observations (one participants score is not dependent on any other participants score),Do females memory recall s

13、cores change over time?,Assumptions,IV is ordinal / categorical e.g., gender DV is interval / ratio e.g., self-esteem Homogeneity of Variance If variances unequal, adjustment made (Levenes Test) Normality - often violated, without consequence look at histograms look at skewness look at kurtosis,SPSS

14、 Output: Independent Samples t-test: Same Sex Relations,SPSS Output: Independent Samples t-test: Opposite Sex Relations,SPSS Output: Independent Samples t-test: Opposite Sex Relations,What is ANOVA? (Analysis of Variance),An extension of a t-test A way to test for differences between Ms of: (i) more

15、 than 2 groups, or (ii) more than 2 times or variables Main assumption: DV is metric, IV is categorical,Introduction to ANOVA,Single DV, with 1 or more IVs IVs are discrete Are there differences in the central tendency of groups? Inferential: Could the observed differences be due to chance? Follow-u

16、p tests: Which of the Ms differ? Effect Size: How large are the differences?,F test,ANOVA partitions the sums of squares (variance from the mean) into: Explained variance (between groups) Unexplained variance (within groups) or error variance F represents the ratio between explained and unexplained

17、variance F indicates the likelihood that the observed mean differences between groups could be attributable to chance. F is equivalent to a MLR test of the significance of R.,F is the ratio of between- : within-group variance,Assumptions One-way ANOVA,DV must be: Measured at interval or ratio level

18、Normally distributed in all groups of the IV (robust to violations of this assumption if Ns are large and approximately equal e.g., 15 cases per group) 3. Have approximately equal variance across all groups of the IV (homogeneity of variance) 4. Independence of observations,Example: One-way between

19、groups ANOVA,Does LOC differ across age groups?20-25 year-olds40-45 year olds60-65 year-olds,h2 = SSbetween/SStotal= 395.433 / 3092.983= 0.128Eta-squared is expressed as a percentage: 12.8% of the total variance in control is explained by differences in Age,Which age groups differ in their mean cont

20、rol scores? (Post hoc tests),Conclude: Gps 0 differs from 2; 1 differs from 2,ONE-WAY ANOVA Are there differences in Satisfaction levels between students who get different Grades?,Assumptions - Repeated measures ANOVA,1. Sphericity - Variance of the population difference scores for any two condition

21、s should be the same as the variance of the population difference scores for any other two conditions (Mauchly test of sphericity) Note: This assumption is commonly violated, however the multivariate test (provided by default in SPSS output) does not require the assumption of sphericity and may be u

22、sed as an alternative. When results are consistent, not of major concern. When results are discrepant, better to go with MANOVA Normality,Example: Repeated measures ANOVA,Does LOC vary over a period of 12 months? LOC measures obtained over 3 intervals: baseline, 6 month follow-up, 12 month follow-up

23、.,Mean LOC scores (with 95% C.I.s) across 3 measurement occasions,1-way Repeated Measures ANOVA Do satisfaction levels vary between Education, Teaching, Social and Campus aspects of university life?,Followup Tests,Post hoc: Compares every possible combination Planned: Compares specific combinations,

24、Post hoc,Control for Type I error rate Scheffe, Bonferroni, Tukeys HSD, or Student-Newman-Keuls Keeps experiment-wise error rate to a fixed limit,Planned,Need hypothesis before you start Specify contrast coefficients to weight the comparisons (e.g., 1st two vs. last one) Tests each contrast at criti

25、cal ,TWO-WAY ANOVA Are there differences in Satisfaction levels between Gender and Age?,TWO-WAY ANOVA Are there differences in LOC between Gender and Age?,Example: Two-way (factorial) ANOVA,Main1: Do LOC scores differ by Age?Main2: Do LOC scores differ by Gender?Interaction: Is the relationship betw

26、een Age and LOC moderated by Gender? (Does any relationship between Age and LOC vary as a function of Gender),Factorial designs test Main Effects and InteractionsIn this example we have two main effects (Age and Gender)And one interaction (Age x Gender) potentially explaining variance in the DV (LOC

27、),Example: Two-way (factorial) ANOVA,IVsAge recoded into 3 groups (3)Gender dichotomous (2) DVLocus of Control (LOC)Low scores = more internalHigh scores = more external,Data Structure,Plot of LOC by Age and Gender,Age x gender interaction,Age main effect,Age main effect,Gender main effect,Gender ma

28、in effect,Age x gender interaction,Mixed Design ANOVA (SPANOVA),It is very common for factorial designs to have within-subject (repeated measures) on some (but not all) of their treatment factors.,Mixed Design ANOVA (SPANOVA),Since such experiments have mixtures of between subjects and within-subjec

29、t factors they are said to be of MIXED DESIGN,Common practice to select two samples of subjectse.g., Males/Females Winners/Losers,Mixed Design ANOVA (SPANOVA),Then perform some repeated measures on each group. Males and females are tested for recall of a written passage with three different line spa

30、cings,Mixed Design ANOVA (SPANOVA),This experiment has two Factors B/W = Gender (male or Female) W/I = Spacing (Narrow, Medium, Wide) The Levels of Gender vary between subjects, whereas those of Spacing vary within-subjects,Mixed Design ANOVA (SPANOVA),CONVENTION,If A is Gender and B is Spacing the

31、Reading experiment is of the typeA X (B) signifying a mixed design with repeated measures on Factor B,CONVENTION,With three treatment factors, two mixed designs are possible These may be one or two repeated measures A X B X (C) or A X (B X C),ASSUMPTIONS,Random Selection Normality Homogeneity of Var

32、iance Sphericity Homogeneity of Inter-Correlations,SPHERICITY,The variance of the population difference scores for any two conditions should be the same as the variance of the population difference scores for any other two conditions,SPHERICITY,Is tested by Mauchlys Test of SphericityIf Mauchlys W S

33、tatistic is p .05 then assumption of sphericity is violated,SPHERICITY,The obtained F ratio must then be evaluated against new degrees of freedom calculated from the Greenhouse-Geisser, or Huynh-Feld, Epsilon values.,HOMOGENEITY OF INTERCORRELATIONS,The pattern of inter-correlations among the variou

34、s levels of repeated measure factor(s) should be consistent from level to level of the Between-subject Factor(s),HOMOGENEITY OF INTERCORRELATIONS,The assumption is tested using Boxs M statisticHomogeneity is present when the M statistic is NOT significant at p .001.,Mixed ANOVA or Split-Plot ANOVA D

35、o Satisfaction levels vary between Gender for Education and Teaching?,ANCOVA Does Education Satisfaction differ between people who are Not coping, Just coping and Coping well?,What is ANCOVA?,Analysis of Covariance Extension of ANOVA, using regression principles Assess effect of one variable (IV) on

36、 another variable (DV) after controlling for a third variable (CV),Why use ANCOVA?,Reduces variance associated with covariate (CV) from the DV error (unexplained variance) term Increases power of F-test May not be able to achieve experimental over a variable (e.g., randomisation), but can measure it

37、 and statistically control for its effect.,Why use ANCOVA?,Adjusts group means to what they would have been if all Ps had scored identically on the CV. The differences between Ps on the CV are removed, allowing focus on remaining variation in the DV due to the IV. Make sure hypothesis (hypotheses) i

38、s/are clear.,Assumptions of ANCOVA,As per ANOVA Normality Homogeneity of Variance (use Levenes test),Assumptions of ANCOVA,Independence of observations Independence of IV and CV. Multicollinearity - if more than one CV, they should not be highly correlated - eliminate highly correlated CVs. Reliabil

39、ity of CVs - not measured with error - only use reliable CVs.,Assumptions of ANCOVA,Check for linearity between CV & DV - check via scatterplot and correlation.,Assumptions of ANCOVA,Homogeneity of regression Estimate regression of CV on DV DV scores & means are adjusted to remove linear effects of

40、CV Assumes slopes of regression lines between CV & DV are equal for each level of IV, if not, dont proceed with ANCOVA Check via scatterplot, lines of best fit.,Assumptions of ANCOVA,ANCOVA Example,Does Teaching Method affect Academic Achievement after controlling for motivation? IV = teaching metho

41、d DV = academic achievement CV = motivation Experimental design - assume students randomly allocated to different teaching methods.,ANCOVA example 1,Academic Achievement (DV),Teaching Method (IV),Motivation (CV),ANCOVA example 1,Academic Achievement,Teaching Method,Motivation,ANCOVA Example,A one-wa

42、y ANOVA shows a non-significant effect for teaching method (IV) on academic achievement (DV),An ANCOVA is used to adjust for differences in motivation F has gone from 1 to 5 and is significant because the error term (unexplained variance) was reduced by including motivation as a CV.,ANCOVA Example,A

43、NCOVA & Hierarchical MLR,ANCOVA is similar to hierarchical regression assesses impact of IV on DV while controlling for 3rd variable. ANCOVA more commonly used if IV is categorical.,ANCOVA & Hierarchical MLR,Does teaching method affect achievement after controlling for motivation? IV = teaching meth

44、od DV = achievement CV = motivation We could perform hierarchical MLR, with Motivation at step 1, and Teaching Method at step 2.,ANCOVA & Hierarchical MLR,ANCOVA & Hierarchical MLR,1 - Motivation is a sig. predictor of achievement. 2 - Teaching method is a sig, predictor of achievement after control

45、ling for motivation.,ANCOVA & Hierarchical MLR,Does employment status affect well-being after controlling for age? IV = Employment status DV = Well-being CV = Age Quasi-experimental design - Ps not randomly allocated to employment status.,ANCOVA Example,ANOVA - significant effect for employment stat

46、us,ANCOVA Example,ANCOVA - employment status remains significant, after controlling for the effect of age.,ANCOVA Example,Summary of ANCOVA,Use ANCOVA in survey research when you cant randomly allocate participants to conditions e.g., quasi-experiment, or control for extraneous variables. ANCOVA all

47、ows us to statistically control for one or more covariates.,Summary of ANCOVA,We can use ANCOVA in survey research when cant randomly allocate participants to conditions e.g., quasi-experiment, or control for extraneous variables. ANCOVA allows us to statistically control for one or more covariates.

48、,Summary of ANCOVA,Decide which variable is IV, DV and CV. Check Assumptions: normality homogeneity of variance (Levenes test) Linearity between CV & DV (scatterplot) homogeneity of regression (scatterplot compares slopes of regression lines) Results does IV effect DV after controlling for the effec

49、t of the CV?,Multivariate Analysis of VarianceMANOVA,Generalisation to situation where there are several Dependent Variables.E.g., Researcher interested indifferent types of treatmenton several types of anxiety. Test Anxiety Sport Anxiety Speaking Anxiety,IVs could be 3 different anxiety interventions: Systematic Desensitisation Autogenic Training Waiting List ControlMANOVA is used to ask whether the three anxiety measures vary overall as a function of the different treatments.,