MODERATING META-ANALYSIS: “META-ANALYTIC HOMOGENIETY”

Joel C. Snell   Kirkwood College,

Mitchell Marsh St. Elizabeth Medical Center

Introduction:

The authors over the years have expressed criticism about meta-analysis.

However, it could not be more popular. Recently a major news station reported the

use of meta-analysis in a new finding.

Discussion:

To try to reduce the errors in this statistical and methodological strategy, we have suggested that all data be nominalized and use chi-square as a goodness of fit study.

(Snell & Marsh: 2003) We thought that perhaps allowing for all the heterogeneity built into the methodical strategy that this may be more defensible. However, we retracted this position when more studies using numerous samples, methods, and analysis were

compiled into meta-analysis. (Snell& Marsh: 2009) (Begley: 2008)

Further, Wikipedia and other sources

have many supporters and statistical strategies to homogenize the research strategy.

There are now numerous books on the strategy, software, and related analysis that

appear to make it more valid and reliable. There are literally thousands of articles.

Our position is that this approach has metasized through out numerous disciplines like a virulent cancer. Thus, we suggest that the following be used by editors in journals

across the academic fields.

1. Has the author used “best evidence meta-analysis” or has claimed to use this. That means weaker studies are deleted.
2. Have they attempted to deal with the “file drawer effect” (only published studies are used)” “Simpson paradox” (two studies have opposing results, but when combined show one direction) and “coding bias.” (anecdotal, convenience samples, and related are combined together.)
3. Acknowledge the controversial status of the research methodology and that

      other studies have shown a difference of up to 35% in outcomes. (Ioannidis, J.

      et. al.1998) when meta-analysis is compared to randomized controlled trials.

4. Have they clearly stated the hypothesis? Used only studies where the test-retest or cross sectional methods are same or similar? Thus a study on clinical depression uses the same index or scale. As an example Hamilton Depression is used in all studies compiled as opposed to Langer’s Psychiatric Screen or Sroles Anomie Scale or numerous other measures.

In other words, we are really talking about the purest form a plural analysis of many different samples brought together in which a larger sample is formed and analyzed and is not sullied by different measures, choice of studies, and other analytic digressions. This means again that all studies that use the same research strategies and measurements differ

only by the various samples that have been collected. All else among the studies is the same. If that is the case meta-analysis is vastly improved. Or, for lack of a better term

“Meta-analytic homogeneity” is the next best strategy to controlled random samples. We believe that enough research in an area can generate 4 or 5 studies that all use the same

strategy and analysis and the samples can be collapsed.

Conclusion:

In this article, the authors have tried to indicate research problems and how they may be moderated using meta-analysis. We go beyond any strategies specified before. The term used to apply for this method is “meta-analytic homogeneity”

REFERENCES

Begley, S.  (2008) Newsweek  Whitewashing toxic chemicals, 5/12/39

Ioannidis, J. et. al. (1998) Meta-analysis and large randomized controlled trials. New England Journal of Medicine, 338: 59-72.

Snell, J. & Marsh M. (2003) Meta-Cognitive Analysis: An alternative to literature reviews for the sciences and the arts, Education, Winter, 364-367.

Snell, J. & Marsh M. (2009) Deconstructing meta-analysis, Psychology & Education:

An Interdisciplinary Journal, 36-39.

___________ (2009) Meta-Analysis, Wikipedia.org