Meta-Analysis with CMA - Case Study: ADHD Treatment

This paragraph introduces a workshop on meta-analysis, focusing on standardized mean differences. It's part of a series and is recommended for those planning to work with mean differences. The video covers basic and advanced topics such as data entry, analysis execution, effect size estimation, heterogeneity understanding, results reporting, and plot creation. It also addresses subgroup analysis, meta-regression, and publication bias assessment using the Comprehensive Meta-Analysis (CMA) software. The example is based on a study of methylphenidate's impact on cognitive function in adults with ADHD, using data from a 2011 review. The presenter guides viewers on using CMA with a free trial and dataset from their website, metaanalysis.com.

The speaker demonstrates how to enter data into CMA using an Excel file formatted for a tutorial. The data includes study names, means, standard deviations, sample sizes for treated and placebo groups, and potential moderating variables. The process involves copying data with labels, pasting it into CMA, and using the software's features to identify column functions, such as study names, effect size data, and sample characteristics. The goal is to prepare the data for meta-analysis within the software, ensuring correct identification of each column's role in the analysis.

This section explains how to customize the display of data within CMA, focusing on the effect size computation. The speaker guides on identifying columns for treated and placebo group means, standard deviations, and sample sizes. It also discusses assigning descriptive names to groups and setting the effect direction, which is crucial for calculating the standardized mean difference (Cohen's d). The video shows how to automatically compute effect sizes and adjust the display to show standard errors and variance alongside the effect sizes.

The speaker details the process of running a meta-analysis in CMA, explaining the choice between fixed and random effects models. The random effects model is selected to account for study-to-study variance and to generalize findings to a broader universe of studies. The analysis results are presented, including the mean effect size, its confidence interval, and statistical tests for heterogeneity. The paragraph also clarifies misconceptions about interpreting heterogeneity statistics like I Square and the importance of understanding the prediction interval for effect size variation.

This part delves deeper into heterogeneity, explaining its relevance and how it's distinct from the confidence interval's precision. Heterogeneity is crucial for understanding the variability of true effect sizes across studies. The speaker refutes common misunderstandings about I Square and clarifies that it does not indicate the extent of effect size variation. Instead, the prediction interval is the statistic that provides insight into how much the effect size could vary in different populations.

The speaker discusses the importance of saving the analysis file in CMA and provides a step-by-step guide on annotating and exporting the results. Annotations can help clarify the interpretation of statistics like I Square and the prediction interval. The exported report can be used in other programs like Word or PowerPoint, facilitating the sharing and presentation of the meta-analysis findings.

Sensitivity analysis is introduced as a method to ensure the meta-analysis results' robustness. The speaker demonstrates how to assess the influence of individual studies on the overall effect size by examining the weight each study contributes to the analysis. Additionally, the 'one study removed' approach is used to test the stability of conclusions when excluding one study at a time. The robustness of the findings against potential publication bias is also discussed.

The paragraph explains the concept of publication bias and its detection using funnel plots in CMA. Funnel plots visualize the distribution of study effect sizes, with potential bias indicated by an uneven distribution, especially among smaller studies. The speaker guides through the process of generating a funnel plot, adjusting for the random effects model, and interpreting the plot to assess the likelihood of missing studies, which could suggest publication bias.

This section introduces the trim and fill procedure, a method for adjusting meta-analysis results to account for potential publication bias. The speaker demonstrates how to use CMA to conduct this analysis, which imputes and includes missing studies in the analysis to provide a more balanced view. The results before and after the trim and fill procedure are compared to assess the impact of potential missing studies on the overall effect size estimate.

The speaker discusses the importance of subgroup analysis and meta-regression in exploring heterogeneity and identifying factors that may explain variations in effect sizes. The process involves categorizing studies into subgroups based on specific variables, such as the inclusion or exclusion of substance abuse disorder (SUD) patients, and analyzing the impact of these variables on the effect size. The results of subgroup analyses can provide insights into the factors contributing to the observed heterogeneity.

This paragraph focuses on meta-regression analysis, which is used to examine the relationship between continuous moderators and effect sizes. The speaker demonstrates how to use CMA to conduct a meta-regression, including selecting the covariate, in this case, dosage, and interpreting the results. The analysis shows the predicted change in effect size with varying dosages, providing insights into the impact of dosage on the treatment's effectiveness.

The speaker concludes the workshop by summarizing the key findings from the meta-regression analysis. They emphasize the importance of understanding the relationship between dose and effect size, as well as the limitations of the analysis, such as the inability to draw causal conclusions from subgroup comparisons. The speaker also discusses the implications of the findings for future research and clinical decision-making.