The scandal that shook psychology to its core

Neuro Transmissions

24 Aug 202229:34

EducationalLearning

32 Likes 10 Comments

TLDRIn a riveting exploration of psychology's integrity, this video delves into the crisis shaking the field's foundations. Renowned psychologist Daryl Bem's controversial ESP study, published without formal funding, ignited debates over the scientific method's reliability. The reproducibility crisis emerges starkly when only 36% of 100 significant psychological studies could be replicated, questioning decades of accepted facts about the mind. The narrative unfolds, examining the dangerous blend of publication biases, statistical misinterpretations, and unethical research practices eroding trust in scientific findings. Amid this existential crisis, the video advocates for rigorous methodologies, transparency, and the vital role of replication in restoring confidence in psychology, suggesting a path toward a more robust scientific future.

Takeaways

🔍 The script discusses the replication crisis in psychology, highlighting the challenges in verifying past research findings and the implications for the field's credibility.
🧠 Daryl Bem's decade-long study on ESP (Extrasensory Perception) and its controversial publication in the Journal of Personality and Social Psychology sparked debates on the validity of psychological research.
📈 The Reproducibility Project attempted to replicate 100 studies with only 36% success rate, questioning the foundational research in psychology.
🎓 Publication bias is a significant issue, with 96% of published psychology studies showing positive results, potentially inflating the occurrence of false positives.
🚫 The 'file drawer problem' refers to the tendency to not publish studies with null or contradictory results, leading to a skewed perception of research outcomes.
📉 The overemphasis on positive results and the 'publish or perish' mentality can lead to questionable research practices and even outright fraud.
🔧 The term 'p hacking' is used to describe the manipulation of data and statistical analyses to achieve significant results, which can undermine the integrity of research.
📊 The script calls for increased transparency, rigor in study designs, and a shift in incentive structures to value null and negative results as much as positive ones.
🌐 Open science initiatives, such as making research data publicly available and pre-registration of study protocols, are encouraged to improve reproducibility and transparency.
📖 The script suggests that changes in statistical significance thresholds, larger sample sizes, and more representative studies can contribute to a more accurate understanding of psychological phenomena.
💡 The future of psychology is seen as brighter due to the ongoing efforts to address the replication crisis, emphasizing the importance of replication in the scientific process.

Q & A

What was Daryl Bem's pet project about?
-Daryl Bem's pet project was about investigating the existence of ESP (Extrasensory Perception), the ability to sense the future.
Why did Daryl Bem replicate his study nine times?
-Daryl Bem replicated his study nine times to ensure the validity of his findings and to eliminate any possibility of error, thus strengthening the credibility of his research.
What was the reaction to Bem's paper in the Journal of Personality and Social Psychology?
-The reaction to Bem's paper was explosive, as it challenged the foundation of psychological research by suggesting the existence of ESP, which had significant implications for the field.
What does the 'replicability crisis' in psychology signify?
-The replicability crisis in psychology signifies a widespread concern that many published studies cannot be replicated, leading to doubts about the validity of the research and the field's understanding of the mind and behavior.
What was the outcome of the Reproducibility Project Psychology?
-The Reproducibility Project Psychology found that only 36% of the 100 replicated studies yielded significant results, indicating a serious issue with the original research findings in the field.
What is the 'file drawer problem' in scientific research?
-The 'file drawer problem' refers to the tendency for studies with inconclusive or negative results to remain unpublished, as they are often left in researchers' file drawers and not submitted for publication.
What is the significance of the p-value in psychological research?
-The p-value is used to determine whether a study's findings are statistically significant. A low p-value, typically below .05, suggests that the observed results are unlikely to have occurred by chance, thus lending support to the study's hypothesis.
What is 'p-hacking' and how does it contribute to the replication crisis?
-P-hacking is the practice of manipulating data or statistical analyses to obtain a significant p-value. This can lead to false positives and contributes to the replication crisis by inflating the number of studies that appear to support a hypothesis when they may not be reliable.
What changes have been proposed to address the replication crisis in psychology?
-Proposed changes include moving towards open science practices, pre-registration of studies, requiring journals to publish more null results, changing the statistical significance threshold, and re-evaluating the incentive structures in research to value replication and negative results equally.
How does the replication crisis affect public trust in scientific findings?
-The replication crisis can undermine public trust in scientific findings, as it raises questions about the reliability of research. This skepticism can extend to other scientific topics, such as climate change and vaccines, where public confidence is crucial.
What is the role of larger sample sizes in improving psychological research?
-Larger sample sizes can help to limit statistical anomalies by averaging out results across more participants. This also helps to identify how specific or general the findings are, leading to more accurate and representative conclusions.

Outlines

00:00

🔍 The Replication Crisis in Psychology

This paragraph introduces the replication crisis in psychology, highlighting the controversial study by Daryl Bem on ESP (Extrasensory Perception) and the subsequent shockwaves it sent through the scientific community. It emphasizes the importance of replicability in scientific research and the crisis of confidence that has emerged as numerous studies fail to produce consistent results upon replication. The paragraph also discusses the pressure on psychologists to produce publishable results and the potential flaws in the foundational research that shapes our understanding of the mind.

05:02

📉 The Impact of Publication Bias and Questionable Research Practices

This section delves into the impact of publication bias, where positive results are favored over null or negative findings, leading to a skewed representation of scientific truth. It also explores questionable research practices (QRPs) such as p-hacking, where researchers manipulate data analysis to achieve significant results. The paragraph discusses the consequences of these practices, including the inflation of false positives and the undermining of scientific integrity. It also touches on the case of Dedrick Stapel, whose data falsification further exemplifies the severity of the issue.

10:03

🚫 The Pressure on Researchers and the Consequences

This paragraph discusses the immense pressure on researchers to produce publishable results due to the 'publish or perish' mentality in academia. It explains how this pressure can lead to a focus on quantity over quality and potentially to misconduct, including fraud and QRPs. The paragraph also highlights the need for more transparency and rigor in study designs to combat the replication crisis and improve the overall quality of psychological research.

15:06

🧐 Addressing the Replication Crisis: Changes and Solutions

The paragraph outlines various changes and solutions being implemented to address the replication crisis in psychology. It discusses the movement towards open science, including public data availability and pre-registration of research plans. The paragraph also suggests changes in the statistical significance threshold and the consideration of larger sample sizes to improve research quality. It emphasizes the need for journals to publish more null results and for funding to support replication studies, ultimately calling for a shift in the incentive structures in research.

20:07

🌟 The Future of Psychology and the Importance of Replication

In this concluding paragraph, the speaker expresses optimism about the future of psychology, seeing the replication crisis as an opportunity for growth and improvement. It underscores the importance of replication in the scientific process and the need to value null and negative results as much as positive ones. The speaker also promotes the platform 'Brilliant' as a resource for learning STEM topics, including statistics, and encourages researchers to embrace the learning process, including the possibility of being proven wrong.

Mindmap

Keywords

💡Replication Crisis

The replication crisis refers to the challenge within scientific research, particularly in psychology, where studies are not being successfully replicated, thus casting doubt on the validity of the original findings. This crisis undermines confidence in scientific research and has led to a push for more rigorous and transparent methodologies. In the video, the replication crisis is central to the narrative, illustrating the field's struggle with the reliability of its research outcomes and the need for reform.

💡Daryl Bem

Daryl Bem is a renowned psychologist mentioned in the video who conducted a controversial study on extrasensory perception (ESP) or the ability to sense the future. His research, which suggested the existence of ESP, created a stir in the scientific community due to its outlandish nature and the potential implications for psychological research. Bem's study is used as an example of the kind of research that, despite following strict methodological guidelines, contributed to the replication crisis by being difficult to replicate and therefore question the validity of similar research practices.

💡Publication Bias

Publication bias refers to the tendency of scientific journals to prefer publishing studies with positive or significant results over those with null or negative results. This bias can skew the scientific record by overrepresenting findings that are statistically significant, potentially at the expense of accuracy and reliability. In the context of the video, publication bias is identified as a contributing factor to the replication crisis, as it encourages a focus on positive outcomes and may lead to the suppression of less exciting but equally valuable research.

💡P-Value

A p-value is a statistical measure used to determine whether the results of a study are statistically significant or not. In psychology and many other sciences, a common threshold for significance is a p-value less than .05, meaning there's less than a 5% chance that the observed results are due to random chance. The video discusses how an overemphasis on p-values and statistical significance can lead to questionable research practices and contribute to the replication crisis, as researchers may manipulate data or analyses to achieve the desired p-value.

💡P-Hacking

P-hacking, also known as data dredging, is the practice of using various statistical techniques to manipulate data in a way that yields a significant p-value, even when there may not be a true effect. This questionable research practice can lead to false positives and undermines the integrity of scientific findings. In the video, p-hacking is presented as one of the methods researchers might use to obtain significant results, which can contribute to the replication crisis when these results cannot be replicated in subsequent studies.

💡False Positives

False positives occur when a study's results indicate a significant finding or effect when, in reality, there is none. This can be due to random chance or the result of questionable research practices. False positives are a significant concern in the replication crisis because they can lead to the publication and widespread acceptance of research that is not reliable or valid. The video highlights the prevalence of false positives in psychology and the need to address this issue to restore confidence in scientific findings.

💡Open Science

Open science is a movement within the scientific community that advocates for greater transparency, reproducibility, and accessibility in research. It involves practices such as making research data publicly available, pre-registering study protocols, and encouraging the publication of null results. In the video, open science is presented as a solution to the replication crisis, with the potential to improve the quality of research and restore faith in scientific findings.

💡Statistical Significance

Statistical significance is a measure used in research to determine if a result is unlikely to have occurred by chance. It is typically assessed using p-values, with a lower p-value indicating greater significance. However, the video argues that an overreliance on statistical significance can lead to misinterpretations and the acceptance of false findings, as it does not necessarily imply practical significance or a meaningful effect.

💡Effect Size

Effect size is a measure that indicates the magnitude of a phenomenon or the strength of the relationship between variables in a study. Unlike p-values, which only indicate whether an effect is statistically significant, effect sizes provide a sense of how large or important the effect is. The video suggests that focusing on effect sizes, rather than just statistical significance, can lead to a more accurate understanding of research findings and their practical implications.

💡Incentive Structures

Incentive structures in research refer to the systems and policies that influence the behavior of researchers, such as tenure decisions, funding allocations, and publication practices. The video argues that current incentive structures may encourage the publication of positive results and discourage replication studies, contributing to the replication crisis. Changing these structures to value null and negative results equally could improve research quality and transparency.

💡Brilliant

Brilliant is an online platform for learning STEM topics through interactive challenges and courses. In the video, Brilliant is mentioned as a resource for learning about statistics and probability, which are crucial for understanding and addressing issues like the replication crisis. The platform's courses are designed to help users gain a deeper understanding of complex topics, including those related to the statistical analysis and interpretation that are central to the discussion in the video.

Highlights

Daryl Bem's decade-long study on ESP (Extrasensory Perception) suggested its existence, causing a stir in the scientific community.

Bem's study was rigorously conducted, with nine replications and conventional analyses to ensure no mistakes.

The study's results were published in the Journal of Personality and Social Psychology, indicating a potential flaw in scientific research.

The replication crisis in psychology questions the validity of decades of research and foundational knowledge about the mind.

The Reproducibility Project Psychology aimed to re-run 100 studies, with only 36% showing significant results, highlighting the crisis.

A 2018 attempt to replicate 21 studies from prestigious journals like Nature and Science showed only 62% significant results.

The crisis of confidence in psychology raises questions about the reliability of research findings.

The field of psychology strives to become more scientific, with tools like the IQ test and the Skinner box.

The p-value is a critical number in science, used to determine statistical significance.

Publication bias in psychology, with 96% of published studies showing positive results, may inflate false positives.

The 'publish or perish' mentality in academia can lead to a focus on quantity over quality in research.

Questionable research practices like p-hacking can lead to false positives and are more common than outright fraud.

Lack of transparency and rigor in study designs contributes to the replication crisis.

The replication crisis affects multiple scientific fields, not just psychology, undermining public confidence in science.

The movement towards open science aims to make research more reproducible, accessible, and transparent.

Pre-registration and registered reports are gaining momentum to improve study quality and prevent p-hacking.

Changing incentive structures in research, such as valuing null results and replication, is crucial for improvement.

The future of psychology looks brighter with increased focus on accurate representation of findings and more replication.

Transcripts

Browse More Related Video

Is Most Published Research Wrong?

The Replication Crisis: Crash Course Statistics #31

Chapter 2.1: Thomas Kuhn, normal science

FoodGate: The Break-in, the Cover-up, & the Aftermath

Science Isn't Dogma, You're Just Stupid (Response to Formscapes)

This Scientist catches FRAUD in Harvard and Stanford Research

Related Tags

Psychology Crisis Daryl Bem ESP Study Replication Crisis Scientific Trust Publication Bias Statistical Significance Research Ethics Open Science Science Education