MODERATING META-ANALYSIS
MODERATING META-ANALYSIS: “META-ANALYTIC HOMOGENIETY”
Joel C. Snell Kirkwood College,
Mitchell Marsh St. Elizabeth Medical Center
Introduction:
The authors over the years have expressed criticism about meta-analysis.
However, it could not be more popular. Recently a major news station reported the
use of meta-analysis in a new finding.
Discussion:
To try to reduce the errors in this statistical and methodological strategy, we have suggested that all data be nominalized and use chi-square as a goodness of fit study.
(Snell & Marsh: 2003) We thought that perhaps allowing for all the heterogeneity built into the methodical strategy that this may be more defensible. However, we retracted this position when more studies using numerous samples, methods, and analysis were
compiled into meta-analysis. (Snell& Marsh: 2009) (Begley: 2008)
Further, Wikipedia and other sources
have many supporters and statistical strategies to homogenize the research strategy.
There are now numerous books on the strategy, software, and related analysis that
appear to make it more valid and reliable. There are literally thousands of articles.
Our position is that this approach has metasized through out numerous disciplines like a virulent cancer. Thus, we suggest that the following be used by editors in journals
across the academic fields.
- Has the author used “best evidence meta-analysis” or has claimed to use this. That means weaker studies are deleted.
- Have they attempted to deal with the “file drawer effect” (only published studies are used)” “Simpson paradox” (two studies have opposing results, but when combined show one direction) and “coding bias.” (anecdotal, convenience samples, and related are combined together.)
- Acknowledge the controversial status of the research methodology and that
other studies have shown a difference of up to 35% in outcomes. (Ioannidis, J.
et. al.1998) when meta-analysis is compared to randomized controlled trials.
- Have they clearly stated the hypothesis? Used only studies where the test-retest or cross sectional methods are same or similar? Thus a study on clinical depression uses the same index or scale. As an example Hamilton Depression is used in all studies compiled as opposed to Langer’s Psychiatric Screen or Sroles Anomie Scale or numerous other measures.
In other words, we are really talking about the purest form a plural analysis of many different samples brought together in which a larger sample is formed and analyzed and is not sullied by different measures, choice of studies, and other analytic digressions. This means again that all studies that use the same research strategies and measurements differ
only by the various samples that have been collected. All else among the studies is the same. If that is the case meta-analysis is vastly improved. Or, for lack of a better term
“Meta-analytic homogeneity” is the next best strategy to controlled random samples. We believe that enough research in an area can generate 4 or 5 studies that all use the same
strategy and analysis and the samples can be collapsed.
Conclusion:
In this article, the authors have tried to indicate research problems and how they may be moderated using meta-analysis. We go beyond any strategies specified before. The term used to apply for this method is “meta-analytic homogeneity”
REFERENCES
Begley, S. (2008) Newsweek Whitewashing toxic chemicals, 5/12/39
Ioannidis, J. et. al. (1998) Meta-analysis and large randomized controlled trials. New England Journal of Medicine, 338: 59-72.
Snell, J. & Marsh M. (2003) Meta-Cognitive Analysis: An alternative to literature reviews for the sciences and the arts, Education, Winter, 364-367.
Snell, J. & Marsh M. (2009) Deconstructing meta-analysis, Psychology & Education:
An Interdisciplinary Journal, 36-39.
___________ (2009) Meta-Analysis, Wikipedia.org