Significance vs. Effect Size for One Way ANOVA using SPSS

Dr. Todd Grande
16 Aug 201709:51
EducationalLearning
32 Likes 10 Comments

TLDRDr. Grande's video offers a detailed tutorial on comparing significance to effect size using SPSS. The example uses fictitious data with an independent variable 'treatment' having two levels (psychodynamic and REBT) and a dependent variable 'depression' measured continuously. The video demonstrates how to conduct an ANOVA to test the null hypothesis and introduces the concept of effect size through partial eta squared, explaining its importance over mere statistical significance. It highlights the practical significance of differences between groups, the impact of sample size on statistical significance, and the comparison of effect sizes across studies. The tutorial also covers how to calculate partial eta squared and interpret its values, emphasizing the need to consider both significance and effect size in data analysis.

Takeaways
  • πŸ“š The video is a tutorial by Dr. Grande on comparing significance to effect size using SPSS.
  • πŸ” The example data in SPSS includes a continuous dependent variable 'depression' and an independent variable 'treatment' with two levels: psychodynamic and REBT.
  • 🧐 The video demonstrates how to conduct an ANOVA to analyze the significance of the difference between the two treatment groups.
  • πŸ“‰ The significance level (p-value) obtained from the ANOVA is 0.024, indicating a 2.4% chance of observing the results by random error if the null hypothesis were true.
  • ❌ Rejecting the null hypothesis with a p-value less than 0.05 (common alpha level in social sciences) suggests that the two treatment groups are significantly different in terms of depression.
  • πŸ” Beyond significance, the video emphasizes the importance of effect size to understand the practical importance of the observed difference.
  • πŸ“Š Effect size is measured by 'partial eta squared' in this context, which represents the proportion of variance in the dependent variable explained by group membership.
  • πŸ“ˆ The partial eta squared value of 0.106 indicates that 10.6% of the variance in depression scores is due to the treatment type.
  • πŸ”Ž The video explains that a statistically significant result does not necessarily imply a meaningful difference, which is where effect size plays a crucial role.
  • πŸ“˜ Effect size can also be used to compare findings across studies and is particularly important when sample sizes are large, as statistical significance can be easily achieved.
  • πŸ“ The tutorial also covers how to calculate effect size using SPSS output and Excel, providing a step-by-step guide for understanding the calculations.
Q & A

  • What is the purpose of the video by Dr. Grande?

    -The purpose of the video is to demonstrate how to compare significance to effect size using SPSS, specifically with a focus on ANOVA.

  • What type of data does Dr. Grande use in the video?

    -Dr. Grande uses fictitious data in the SPSS data editor, with an independent variable named 'treatment' having two levels (psychodynamic and REBT) and a dependent variable 'depression' measured at the interval level.

  • Why does Dr. Grande choose to perform an ANOVA instead of a t-test despite having only two groups?

    -Dr. Grande chooses to perform an ANOVA to compare significance and effect size, rather than just testing for statistical significance with a t-test.

  • What is the significance level (p-value) obtained in the ANOVA for the treatment variable?

    -The significance level (p-value) obtained for the treatment variable in the ANOVA is 0.024, which indicates a 2.4% chance of observing the results by random error if the null hypothesis were true.

  • What does the p-value of 0.024 suggest about the null hypothesis in this context?

    -The p-value of 0.024 suggests that there is a low probability (2.4%) that the observed difference in depression levels between the two treatment groups is due to random error, leading to the rejection of the null hypothesis that the groups are equal.

  • Why is it important to consider effect size in addition to significance?

    -Considering effect size is important because it provides information about the practical significance of the observed difference between groups, which mere statistical significance does not convey.

  • What additional statistic is provided when Dr. Grande re-runs the ANOVA with options for effect size?

    -The additional statistic provided is 'partial eta squared', which measures the proportion of variance in the dependent variable explained by the independent variable.

  • What is the value of partial eta squared obtained in the example, and what does it represent?

    -The value of partial eta squared obtained is 0.106, which represents 10.6% of the variance in depression levels explained by group membership, indicating the effect size.

  • Why might statistical significance not be enough to determine the importance of a result?

    -Statistical significance alone might not be enough because as sample sizes increase, it becomes easier to find statistically significant results that may not have practical importance, which is where effect size comes into play.

  • How can effect size help in comparing findings across different studies?

    -Effect size helps in comparing findings across studies by providing a standardized measure of the importance or magnitude of the difference between groups, allowing for a more meaningful comparison than significance levels alone.

  • What are the different thresholds for small, medium, and large effect sizes for partial eta squared?

    -For partial eta squared, small, medium, and large effect sizes are categorized as follows: small (1% to 6%), medium (6% to 14%), and large (14% or greater).

Outlines
00:00
πŸ”¬ Introduction to ANOVA and Effect Size in SPSS

Dr. Grande introduces a video on comparing significance to effect size using SPSS. The example uses fictitious data with an independent variable 'treatment' having two levels: psychodynamic and REBT, and a continuous dependent variable 'depression' measured at the interval level. The video demonstrates how to conduct an ANOVA to determine if there's a significant difference between the two treatment groups. The significance level (p-value) of 0.024 indicates a 2.4% chance that the observed difference is due to random error, leading to the rejection of the null hypothesis. However, the video emphasizes that significance alone doesn't convey the importance of the difference, which is where effect size comes into play.

05:01
πŸ“Š Understanding Effect Size and Its Importance

This paragraph delves into the concept of effect size, explaining its importance in determining the practical significance of observed differences between groups. It contrasts statistical significance with effect size, noting that large sample sizes can easily yield significant results that may not be meaningful. The paragraph introduces different measures of effect size, such as partial eta squared and Cohen's D, explaining that partial eta squared represents the proportion of variance in the dependent variable explained by group membership. The video provides a step-by-step guide on calculating partial eta squared using SPSS output and Excel. It also discusses the interpretation of effect size values, highlighting the thresholds for small, medium, and large effect sizes. The video concludes by emphasizing the importance of considering both significance and effect size in statistical analysis.

Mindmap
Keywords
πŸ’‘ANOVA
ANOVA stands for Analysis of Variance, which is a statistical method used to compare means of two or more groups to determine if there are any statistically significant differences between them. In the video's context, ANOVA is used to compare the significance of the effect of two types of treatment (psychodynamic and REBT) on the level of depression. The script mentions conducting an ANOVA to analyze the data and determine the significance level, which is a key step in understanding the impact of the treatments.
πŸ’‘Effect Size
Effect size is a measure that indicates the magnitude of the difference between groups, not just the presence or absence of a difference. It helps to determine the practical significance of the results beyond statistical significance. In the video, the concept of effect size is introduced to explain the importance of the difference between the two treatment groups in terms of their impact on depression levels, with a focus on the partial eta squared as a measure of effect size.
πŸ’‘Significance Level
The significance level, often denoted by the Greek letter alpha (Ξ±), is a threshold used in statistical tests to determine whether the results are statistically significant. In the video, a significance level of 0.05 is mentioned as a common standard in social sciences, which means that if the p-value (probability value) is less than 0.05, the null hypothesis is rejected, indicating a statistically significant result.
πŸ’‘Null Hypothesis
The null hypothesis is a statement of no effect or no difference that researchers test to disprove. In the context of the video, the null hypothesis is that there is no difference between the psychodynamic and REBT treatment groups in terms of their impact on depression levels. The video explains that rejecting the null hypothesis means that the observed difference is unlikely to be due to random error.
πŸ’‘Dependent Variable
A dependent variable is the outcome or response that is measured in an experiment. In the video, depression is the dependent variable, which is a continuous variable measured at the interval level. The script discusses how the level of depression is influenced by the type of treatment, making it the focus of the statistical analysis.
πŸ’‘Independent Variable
An independent variable is the factor that is manipulated or changed by the researcher to observe its effect on the dependent variable. In the video, the independent variable is the type of treatment, with two levels: psychodynamic and REBT. The script explains how these treatments are being compared to see their effect on depression.
πŸ’‘Psychodynamic
Psychodynamic treatment is a type of therapy that focuses on unconscious processes and early experiences to bring about therapeutic change. In the video, psychodynamic is one of the two treatment levels being compared against REBT to determine its effectiveness in treating depression.
πŸ’‘REBT
REBT stands for Rational Emotive Behavior Therapy, which is a type of cognitive-behavioral therapy that focuses on changing irrational beliefs and thoughts to improve emotional and behavioral outcomes. In the video, REBT is compared against psychodynamic treatment to evaluate its effectiveness in reducing depression levels.
πŸ’‘Partial Eta Squared
Partial eta squared is a measure of effect size that represents the proportion of variance in the dependent variable that is attributed to the independent variable, after controlling for other factors in the model. In the video, partial eta squared is calculated to determine the effect size of the treatment on depression levels, with a value of 0.106 indicating the proportion of variance explained by group membership.
πŸ’‘Cohen's D
Cohen's D is another measure of effect size that represents the standardized difference between two means. It is calculated by taking the difference between the means of two groups and dividing it by a pooled standard deviation. Although not directly calculated in the video, the script mentions Cohen's D as a popular effect size statistic for understanding the magnitude of the difference between groups.
Highlights

Introduction to the video by Dr. Grande on comparing significance to effect size using SPSS.

Use of fictitious data with an independent variable 'treatment' and a dependent variable 'depression'.

Explanation of the difference between interval and ratio level measurements for continuous variables.

Decision to conduct an ANOVA despite having only two groups to compare significance and effect size.

Description of the general linear model univariate analysis setup in SPSS.

Interpretation of the significance level (p-value) of 0.024 indicating a 2.4% chance of random error.

Discussion on rejecting the null hypothesis when p-value is less than the alpha value (0.05).

Importance of effect size in determining the practical significance of observed differences.

Introduction of partial eta squared as a measure of effect size in ANOVA.

Calculation and interpretation of partial eta squared value of 0.106 indicating 10.6% variance explained.

The difference between statistical significance and practical significance in research.

The impact of increased sample sizes on the ease of finding statistical significance.

Comparison of findings across studies using effect size as a common metric.

Different types of effect sizes, such as partial eta squared and Cohen's D, and their calculations.

Guidelines for interpreting effect sizes as small, medium, or large based on their values.

The importance of considering both significance and effect size in inferential statistics.

Conclusion emphasizing the different purposes of p-value and effect size in statistical analysis.

Transcripts

Browse More Related Video

Eta Squared Effect Size for One-Way ANOVA (12-7)

Pretest and Posttest Analysis with ANCOVA and Repeated Measures ANOVA using SPSS

P value versus effect size

Cohen’s d Effect Size for t Tests (10-7)

One-Way ANOVA with LSD (Least Significant Difference) Post Hoc Test in Excel

What is Effect Size? Statistical significance vs. Practical significance

Rate This

5.0 / 5 (0 votes)

Thanks for rating:

Related Tags
SPSS TutorialEffect SizeSignificance TestingANOVA AnalysisPsychology ResearchStatistical AnalysisData InterpretationCohen's DGroup ComparisonResearch Methodology