P value versus effect size

Social Psychology Explained
3 Sept 202003:31
EducationalLearning
32 Likes 10 Comments

TLDRThis video introduces 'Stats and Methods Shorts,' a new series focusing on statistical significance versus effect size. It explains that statistical significance measures the likelihood that observed data differences are due to chance, with a p-value < 0.05 typically considered significant. However, effect size is crucial as it quantifies the magnitude of an effect, indicating its practical importance. The video outlines common effect size measures like Pearson's r, r-squared, Cohen's d, and eta squared, emphasizing their importance in understanding the impact of findings, especially in large populations like the U.S.

Takeaways
  • 📊 The video introduces a new series called 'Stats and Methods Shorts' aimed at explaining statistical concepts.
  • 🔍 It discusses the difference between statistical significance and effect size, emphasizing their importance in data interpretation.
  • ⚖️ Statistical significance measures the likelihood that observed data differences are due to chance, not representative of the population.
  • 🎲 A p-value is calculated to determine statistical significance, with a common threshold of less than 0.05 indicating significance.
  • 📈 Effect size measures the magnitude of an effect and is crucial for understanding the practical impact of findings.
  • 🔢 Pearson's r is a correlation coefficient used to measure the strength and direction of a linear relationship between two variables.
  • 🧩 R squared (r²) indicates the proportion of total variance explained by the model, giving insight into the model's explanatory power.
  • 🌐 Cohen's d is used for comparing two groups, representing the mean difference as a proportion of pooled variance.
  • 📊 Eta squared and partial eta squared are used for comparing more than two groups, showing the proportion of variance explained by group differences.
  • 🌟 Even small effect sizes can be meaningful in large populations, as they can still impact a significant number of individuals.
  • 🛠 The video encourages viewers to suggest topics for future 'Stats and Methods Shorts' episodes and to engage in the comments section.
Q & A

  • What is the main focus of the new series 'Stats and Methods Shorts'?

    -The new series aims to regularly discuss topics related to statistics and research methods, starting with the distinction between statistical significance and effect size.

  • What does statistical significance indicate in the context of a study?

    -Statistical significance indicates the likelihood that the observed difference in data is not due to chance, with a common threshold of p < 0.05 representing a 95% certainty that the findings are not random.

  • What is a p-value in statistics?

    -A p-value is a probability that the observed findings are due to sampling error, and it is used to determine if the results of a study are statistically significant.

  • Why is it important to consider effect size in addition to statistical significance?

    -Effect size provides a measure of the magnitude of the effect being studied, which is crucial to understand how meaningful or impactful the findings are, beyond just being statistically significant.

  • What is Pearson's r and how does it relate to effect size?

    -Pearson's r is a correlation coefficient that ranges from -1 to 1, indicating the strength and direction of the relationship between two variables. While it shows the association, it does not directly indicate how much variability is explained by the correlation.

  • How is r squared calculated and what does it represent?

    -R squared is calculated by squaring Pearson's r for simple correlations or through a different method for complex models. It represents the percentage of total variance that the model explains.

  • What is Cohen's d and how is it used?

    -Cohen's d is a measure of effect size that signifies the mean difference between two groups as a proportion of their pooled variance, often used when comparing just two groups.

  • What are eta squared and partial eta squared, and when are they used?

    -Eta squared and partial eta squared are measures of effect size used when comparing more than two groups. They represent the proportion of total variance in the outcome of interest that is explained by the differences between the groups.

  • Why might a small effect size still be meaningful in a large population?

    -A small effect size can still be meaningful if the population is large, as even a small percentage improvement can translate to a significant number of people benefiting from the effect.

  • How can a small effect size be significant in a real-world application?

    -A small effect size can be significant if the intervention or treatment is inexpensive and easily accessible, allowing it to be applied to a large population, thereby helping a substantial number of people.

  • What does the speaker encourage the audience to do at the end of the transcript?

    -The speaker encourages the audience to like, share, and subscribe to the channel, and to provide suggestions for future topics or to share more thoughts on the discussed topic in the comments section.

  • Why is it essential to measure both statistical significance and effect size in a study?

    -Both statistical significance and effect size are essential because statistical significance tells us if the findings are likely not due to chance, while effect size tells us how large or meaningful the observed effect is, allowing for a more comprehensive understanding of the study's outcomes.

Outlines
00:00
📊 Understanding Statistical Significance and Effect Size

The video script introduces a new series called 'Stats and Methods Shorts' aimed at discussing important statistical concepts. The first topic is the distinction between statistical significance and effect size. Statistical significance is the probability that observed data differences are not due to chance, with a common threshold of p < 0.05 indicating a 95% confidence that the findings are not random. Effect size, on the other hand, measures the magnitude of the effect and its importance. The script explains different measures of effect size, including Pearson's r for correlation strength, r squared for the proportion of variance explained by the model, Cohen's d for the mean difference between two groups in terms of standard deviation, and eta squared and partial eta squared for the proportion of variance explained by group differences. The video emphasizes that even small effect sizes can be significant when applied to large populations, using the example of a treatment that benefits a small percentage of the U.S. population but still impacts millions due to the sheer size of the population.

Mindmap
Keywords
💡Statistical Significance
Statistical significance refers to the probability that the observed difference in data is not due to chance. It is a fundamental concept in statistics used to determine if a study's results are likely to be true for the entire population. In the video, the speaker explains that a finding is usually considered statistically significant if the p-value, which is the probability that the finding is due to sampling error, is less than 0.05. This means we are at least 95% sure that the observed effect is not due to random chance.
💡Effect Size
Effect size is a measure of the magnitude of an effect or the strength of a relationship in a study. It helps to understand how meaningful the results are, beyond whether they are statistically significant. The speaker in the video emphasizes that while statistical significance is important, effect size provides insight into the practical significance of the findings. For example, a small effect size could still be meaningful if applied to a large population, such as the entire population of the United States.
💡Sampling Error
Sampling error is the error that occurs due to the sample not being fully representative of the population. It is a concept that is directly related to statistical significance because it represents the possibility that the observed differences are due to the specific individuals who were sampled rather than a true effect. The video script mentions that the difference observed between group A and group B could be due to sampling error, which is why statistical significance is calculated to account for this.
💡P-Value
The p-value is a statistical measure that indicates the probability that the observed results occurred by chance if the null hypothesis were true. In the context of the video, a p-value less than 0.05 is used as a threshold to determine statistical significance. A lower p-value means there is stronger evidence against the null hypothesis, suggesting that the observed effect is not due to random chance.
💡Pearson's R
Pearson's R, or Pearson correlation coefficient, is a measure of the linear relationship between two continuous variables. It ranges from -1 to 1, with values closer to -1 or 1 indicating a stronger relationship. In the video, Pearson's R is mentioned as a common way to measure effect size in psychology, indicating how related the variables of interest are.
💡R Squared
R squared, or the coefficient of determination, is a measure that represents the proportion of the variance for a dependent variable that's explained by an independent variable or variables in a regression model. The speaker explains that squaring Pearson's R gives R squared, which indicates what percentage of the total variance the model explains. For instance, an R squared of 0.3 means the model accounts for 30% of the variance.
💡Cohen's D
Cohen's D is a measure of effect size used to indicate the magnitude of difference between two group means in terms of their pooled standard deviation. It is commonly used in statistical analysis to provide context for the size of the difference. In the video, a Cohen's D of 0.5 is given as an example, meaning that the two group means are half a standard deviation apart.
💡Eta Squared
Eta squared is a measure of effect size that represents the proportion of total variance in the dependent variable that is explained by the independent variable(s) in an ANOVA (Analysis of Variance) model. The script mentions eta squared in the context of comparing more than two groups, indicating the proportion of variance explained by the differences between groups.
💡Partial Eta Squared
Partial eta squared is a variation of eta squared that accounts for the proportion of variance explained by a specific independent variable when controlling for other variables in the model. Although not elaborated upon in the video, it is mentioned alongside eta squared as a measure of effect size for comparing groups in more complex models.
💡Population
In statistics, the population refers to the entire group that is the subject of a study. The video script discusses how a sample's representativeness of the population can affect the statistical significance of the findings. A small effect size can be meaningful when applied to a large population, such as the example of a new treatment affecting a small percentage of a large population like that of the United States.
Highlights

Introducing a new series called 'Stats and Methods Shorts' on the channel.

Discussing the difference between statistical significance and effect size.

Statistical significance is the likelihood that observed data differences are due to chance.

Sampling error can affect the representativeness of a study's sample.

P-value is a measure of the probability that a finding is due to sampling error.

A p-value less than 0.05 is generally considered statistically significant.

Effect size measures the magnitude of an effect and its importance.

Pearson's r is a correlation coefficient ranging from -1 to 1, indicating the strength of a relationship.

R squared represents the percentage of total variance explained by the model.

Cohen's d is used to measure the mean difference between two groups as a proportion of their pooled variance.

Eta squared and partial eta squared are used for comparing more than two groups.

Even small effect sizes can be meaningful in large populations.

An example given is a new treatment that helps a small percentage of the U.S. population but is accessible and affordable.

The importance of considering both statistical significance and practical implications.

Invitation for viewers to like, share, subscribe, and suggest topics for future videos.

Encouraging comments and further discussion on the topic.

Transcripts

Browse More Related Video

What is Effect Size? Statistical significance vs. Practical significance

Introduction to Effect Size

Eta Squared Effect Size for One-Way ANOVA (12-7)

Power & Effect Size

Effect size in meta analysis

What do my results mean Effect size is not the same as statistical significance. With Tom Reader.

Rate This

5.0 / 5 (0 votes)

Thanks for rating:

Related Tags
Statistical SignificanceEffect SizeResearch MethodsSampling ErrorP-ValueCorrelation CoefficientCohen's DEta SquaredPopulation ImpactData Analysis