From Wikipedia, the free encyclopedia
Jump to: navigation, search
          This article is of interest to the following WikiProjects:
WikiProject Psychology (Rated C-class)
WikiProject icon This article is within the scope of WikiProject Psychology, a collaborative effort to improve the coverage of Psychology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
C-Class article C  This article has been rated as C-Class on the project's quality scale.
 ???  This article has not yet received a rating on the project's importance scale.
WikiProject Statistics (Rated C-class, High-importance)
WikiProject icon

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

C-Class article C  This article has been rated as C-Class on the quality scale.
 High  This article has been rated as High-importance on the importance scale.
WikiProject Medicine (Rated C-class, Mid-importance)
WikiProject icon This article is within the scope of WikiProject Medicine, which recommends that medicine-related articles follow the Manual of Style for medicine-related articles and that biomedical information in any article use high-quality medical sources. Please visit the project page for details or ask questions at Wikipedia talk:WikiProject Medicine.
C-Class article C  This article has been rated as C-Class on the project's quality scale.
 Mid  This article has been rated as Mid-importance on the project's importance scale.


The list of pitfalls is far from complete a much more comprehensive look at the pitfalls of non-primary research/meta-anlysis needs to be included. --Hontogaichiban (talk) 05:24, 30 March 2015 (UTC)


More than one intercept might be a result of heterogeneity, but it is not the definition.—Preceding unsigned comment added by (talkcontribs) 22:23, 25 October 2006


It's not right to say that a metaanalysis can't be a scientific study just because it can't be double blind. It could be blind, in fact, just most of the time there isn't a point to it. It doesn't need to be double blind, because the study isn't going to change its behavior since someone is reading it, the same as a hard scientific study involving measuring the ph balance of a rock doesn't need to be double blind, since the rock won't change its ph based on who's wearing the lab coat. You could have a rater code a study with identifying information removed, and if you have multiple raters work on it, measure interrater reliability to help ensure quality. The first couple sentences in the weaknesses section have some pretty strong statements with nothing to back them up, especially getting into whether or not to include bad studies in a metaanalysis. What if the purpose of your metaanalysis is to prove that everyone is measuring a certain construct incorrectly? You have to include bad studies, since that's what you're looking for, regardless of what Slavin says, even though there's no citation to help us know what Slavin actually said. (talk) 21:14, 27 September 2010 (UTC)

First meta-analysis?[edit]

I've removed an edit referencing a conference talk. If this is true, it will be easy to cite better references. Tim Vickers (talk) 03:03, 10 September 2009 (UTC)

Found a source that refers to it, link, but this doesn't make much of this 1940s study. Tim Vickers (talk) 03:15, 10 September 2009 (UTC)
Firstly, the identification of the "ESP-60" study as the first meta-analysis is given by at least a reference - unlike other statements in this paragraph that simply give dates, a fact, but no reference; esp. "a meta-analysis of a medical treatment was not published until 1955". Why are these not deleted while one that is referenced is deleted? Secondly, the paper identiying ESP-60 as "the first meta-analysis in the history of science" is clearly part of edited convention proceedings, not simply what the editor describes as "a conference talk"; as for most academic papers, there is no free availability on the web, and it can be most readily acquired by contacting the author, or a library or a shop. I should think that trivializing a fact or source is not appropriate for WP editors to perform in place of a reasoned argument. Thirdly, while the editor is dissatisfied with only some description of the ESP-60 study in the linked article by O'Rourke, one also finds in the latter not even a mention of the alleged 1955 study referred to in the target article. Why does that point not therefore suffer deletion? Fourthly, it would be interesting to see a discussion of culling of referenced information before it is autonomously culled. --Rodgarton 05:54, 10 September 2009 (UTC)
There were earlier meta-analyses, we have a citation about that. That directly conflicts with the proposed source's title, and it's already not a very reliable source, as a paper delivered at a fringe science conference. Shoemaker's Holiday Over 204 FCs served 20:16, 10 September 2009 (UTC)
Yes, if a talk/conference abstract makes a surprising and verifiably-wrong statement, then I don't think I'd trust it as a source. Thankfully we don't have to cite it, since this source, which is indisputably reliable, discusses the point. Tim Vickers (talk) 20:54, 10 September 2009 (UTC)
That certainly is a very reliable source.Simonm223 (talk) 20:57, 10 September 2009 (UTC)
Indeed, but it also blows even the weakened claim for a 1940 parapsychology meta-analyses being first to cover a particular research issue out of the water with various discussions of agricultural meta-analyses from the 1930s. As well as Pearson's analysis of typhoid. We can't make a claim that Parapsychology was first using a poor-quality source that directly contradicts a high-quality one. Let's use the good source, and throw out the bogus claim. Shoemaker's Holiday Over 204 FCs served 21:17, 10 September 2009 (UTC)
The above respondents offer no answer to the points I have raised - particularly how unreferenced information can be maintained while referenced information is deleted. Instead, they seek to trivialize the referenced information by emotive descriptors, pushing their own opinion of the paper as self-evident truths. They also presume to critique the referenced paper by Bösch by way of its title and representation. The paper by O'Rourke certainly mentions the development and promotion of meta-analytical methods within other sciences; it does not even superficially conflict with the thesis of Bösch. Clearly, the above editors have had as their priority the elimination of the information on the basis that it involves a discipline which they do not like, and have then sought reasons to justify their elimination of it. Apart from the failure to address the substantive issues, to maintain unreferenced information, to make their points by hyperbole and partial reading rather than reasoned argument, this can be most readily seen by how they have eliminated the information: They have been so panicked to remove it, without attention to the details of the paragraph, that it now amusingly reads as: "The first meta-analysis was performed by Karl Pearson in 1904, in an attempt to overcome the problem of reduced statistical power in studies with small sample sizes; analyzing the results from a group of studies can allow more accurate data analysis. This encompassed a review of 145 reports on ESP experiments published from 1882 to 1939, and included an estimate of the influence of unpublished papers on the overall effect (the file-drawer problem)." The respondents are their own de-bunkers. I will not, however, restore the information as this culling is just another one by the advocates of marginal pseudo-skepticism on WP in their long-term campaign to rid WP of any responsible reference to psi-related information, by all irrational means available. It is better to keep the evidence of this irrationality and this campaign on WP up-front, and so to permit WP to self-demonstrate its unreliability.--Rodgarton 23:57, 10 September 2009 (UTC)

Please read WP:CIVIL.Simonm223 (talk) 13:24, 11 September 2009 (UTC)

Crank it up. What a sad reply. (talk) 15:21, 18 October 2009 (UTC)


This tool is presented in the article as a valid tool from deriving statistically significant data from small samples. Despite this there is no clear evidence that the tool in fact does so. As such I have concerns over the neutrality of the tone.Simonm223 (talk) 15:09, 10 September 2009 (UTC)

I second this. --Hontogaichiban (talk) 05:16, 30 March 2015 (UTC)


I have restored various deletions following WP:NOTCENSORED. Deleting refs to conference papers is unhelpful as many technical articles rely on them. The claim of "unrefereed" is not proven as many conference proceedings actually are refereed. Melcombe (talk) 17:07, 17 September 2009 (UTC)


Should anything be added perhaps to highlight the use of metaanalysis as a tool for identifying publication bias, particularly in the Pharma and related academic fields?? - Ann, 24/09/2009

Overly narrow definition excludes more general usage[edit]

Meta-analysis here is only discussed in the context of statistical studies. Such analysis can be done of more general less quantifiable arguments as well. This more general usage is not discussed at all in this article, nor is there a page that addresses this usage (to my knowledge). However, this sort of thinking is an important critical thinking skill. Specifically, it's definition should be in terms of its components. Meta, meaning transcendent, and analysis, meaning an examination, thus: a study at a higher level of abstraction. From this sort of broad definition, one might easily transition into the statistical context. Instead of immediately jumping into an editing war, I'm posting this as a discussion, because I have a feeling that this approach will be resisted from statistical practitioners, who will want to begin from the narrow definition. I do not believe this is settled usage or definition. Although Karl Pearson in 1904 used it in the statistical context [1], I would argue that it has great value as a philosophy technique [2]. For an example of more general usage (in an admittedly casually written article), see [3]. Aaronchall (talk) 06:10, 30 May 2010 (UTC)


  1. ^
  2. ^
  3. ^
I'd distinguish between 'meta-analysis', which is the statistical bit, and 'systematic review', which is the wider process of which meta-analysis is only a part. This distinction is by no means universal but it is the terminology used by the Cochrane Collaboration (see You can do a systematic review without a meta-analysis, and you can do a systematic review of qualitative studies. In this wider context I'd refer to the process of combining of evidence, with or without the use of statistics, as 'research synthesis' or 'evidence synthesis' rather than 'meta-analysis', which is typically reserved for statistical methods of combining evidence. The current article certainly needs to refer to systematic review early on in the body of the article, not just in the 'See also' section as currently. Qwfp (talk) 06:56, 30 May 2010 (UTC)
I have adapted some of this to add to the lead so that 'systematic review' now appears in the lead. Melcombe (talk) 09:21, 13 July 2010 (UTC)

Who is this article written for?[edit]

Most of this article is unreadable for someone without a statistics background. I can here for an overview because I've been seeing meta-analysis in some of the articles I'm researching for an MBA paper. I was trying to better understand the concept but was was quickly frustrated with the introduction section. Do you really think that people like me, people who use encyclopedias to get a quick overview of a topic, expect to see "MA(Pm, Dm =(Px,Tx), Tm)" in the second paragraph? I'm not saying formal definitions don't have a place at wikipedia, just put them in their place.

Perhaps a good question to ask is "what would an educated person encountering this concept for the first time expect to find here?"

For examples of what this looks like see Statistical hypothesis testing and other high quality mathematics articles here —Preceding unsigned comment added by (talk) 02:28, 11 July 2010 (UTC)

That second paragraph was added in February by an IP editor with no other contributions. I've just removed it as it was unsourced, overly technical and the end of it was pretty much incomprehensible. Qwfp (talk) 22:44, 11 July 2010 (UTC)

Dangers of agenda-driven bias--- Biased? and Unsourced[edit]

I found this section to be an "oversell" of this charge, especially since it lacks any sources indicating how frequently this problem occurs. Surely, most statisticians and researchers know that some sort of systematic method of choosing studies must be used, rather than just picking the one's one wants.

It should at least be flagged as needing a source. —Preceding unsigned comment added by (talk) 22:25, 23 July 2010 (UTC)

I would agree with the above comment. An agenda does not necessarily mean a bias exists. There can be all manner of a strong agendas within research that facilitates accurate scientific findings and outcomes. A agenda- driven meta-analysis only increases the likelihood or statistical possibility of bias, but is not in and of itself a bias due to an agenda, no matter how strong, being present. — Preceding unsigned comment added by Sateirti (talkcontribs) 17:08, 4 June 2013 (UTC)

Comparison of scientific method and meta-analysis[edit]

Regarding Melcombe’s removal of “Comparison of meta-analysis to the scientific method”. The editor didn’t give the reasoning that led to the removal. Perhaps the trouble was in one or more of three points the section made: a) That meta-analysis is retrospective; b) that the scientific method is prospective; or c) that the values of probabilities of events in the known past are restricted to zero and one. Maybe it was something else. I don’t know, but I’d like to find out because without a comparison to the scientific method the meta-analysis article is misleading. Gjsis (talk) 22:30, 15 February 2012 (UTC)

The reassoning given was "remove section as uncited conclusions and POV nonsense". Any opinions, as usual, need to be backed by reliable sources. Anything that says that meta-analysis is not or can not be part of "the scientific method" needs to be backed by citations to reliable sources. Since most people will think that meta-analysis is part of some version of a scientific method, you would need to be clear what abstract version you are talking about. But start by seeing Wikipedia:Verifiability and Wikipedia:Identifying reliable sources.
Melcombe (talk) 09:03, 16 February 2012 (UTC)

The principal conclusion of the “Comparison” piece was that a retrospective study does not test a hypothesis. A passage from Francis Bacon was quoted that said a retrospective study did not test a hypothesis. A passage from George Boole was quoted that said a retrospective study did not test a hypothesis. As to what version of the scientific method was intended, the three main steps in the method were spelled out in the passage from Bacon and in the passage from Boole.Gjsis (talk) 20:34, 2 March 2012 (UTC)

The section does look like WP:SYNTHESIS and WP:OR. What would be needed is a source which links Bacon and Boole ideas to Meta Analysis.--Salix (talk): 20:57, 2 March 2012 (UTC)
(this is a response to Gjsis, not Salix, which I had not seen) But there was no citations for anyone who has concluded that meta-analysis fails "scientific method" of any sort, or that it is always, sometimes or at all, a "retrospective study" ... so no citations for the strange conclusions that you are wanting to push as valid. Now some of this mighht actually be valid, but you would need to be more careful to specify what types of meta-analysis are being considered and even why considerations of some notional scientific method are more important than the need to make the best possible use of all of the limited amnount of information available. In modern terms, Boole was taking about Testing hypotheses suggested by the data (which has its own article) and about needing new data for testing such hypotheses. But meta-analysis is not implicitly concerned with such an activity, and the same warnings would apply to meta-analysis as they would to the rest of statistics. Melcombe (talk) 21:13, 2 March 2012 (UTC)

Per above, removing entire section as WP:SYNTH and WP:OR. —MistyMorn (talk) 00:00, 17 July 2012 (UTC)

Commentary on Rind et al[edit]

I have removed an editorializing comment on Rind et al., implying this is an example of an improper meta-analysis. Although Rind et al., is certainly controversial (as much for its subject matter and findings as for anything unique about its methodology), the statement here was simplistic and, I found, potentially biased. As noted in the wiki article on Rind et al., itself the controversy surrounding Rind et al. is very complex. Although it certainly has been criticized, those criticisms themselves have been criticized (such as by Scott Lilienfeld in American Psychologist). Thus the brief statement here was one-sided. To be clear, I have no "dog in this fight" regarding whether Rind et al is valid or not, but found the statement as worded to be misinformative and inaccurate. Avalongod (talk)

Longstanding NPOV concerns about the structure and content of this page[edit]

I've just tagged the prominent and poorly contextualized Disadvantages and weaknesses section [adding: which I've now provisionally retitled Pitfalls] as a "personal essay". I'm not sure this was the most appropriate tag to use, but attention does need to be drawn, imo, to the unencyclopedic approach which has characterized (eg [1]) this page and which, despite the recent work of several editors, continues to hamper NPOV presentation. Meta-analysis (along with systematic review) is key to the production of the highest level of evidence for the efficacy of medical interventions. But you would scarcely know that from reading this page, where the Cochrane Collaboration is treated primarily as a convenient terminological reference, and the Cochrane Library is mentioned in passing in a subsection titled Dangers of agenda-driven bias. In brief, a major NPOV intervention is needed, imo. Cheers, —MistyMorn (talk) 14:24, 8 November 2012 (UTC)

I read this and the section and, at least by the time I saw it (which granted was nearly a year after the tag) I didn't really see a POV problem. The issues that are mentioned in this section are, indeed, pretty well known problems with meta-analysis, and I'm not sure all scholars would agree that meta-analysis necessarily offers the highest level of evidence given its limitations. However, the comments about the Cochrane Collaboration are certainly well taken and there are probably ways to improve the section without necessarily tagging the whole thing as essay or POV. — Preceding unsigned comment added by (talk) 22:17, 3 October 2013 (UTC)
I completely disagree with this. Meta analysis is severely limited as a technique compared to primary research studies. The author of the above comment makes no reference to being aware of this. If anything, the article has strongly drifted towards a strong bias in favour of meta analysis without proper consideration of the drawbacks. Primary research is far any away preferable to meta analysis, I can't imagine any actual researcher would argue to the contrary or indeed any regulator. Admittedly meta analysis can be most useful in the absence of adequate primary research, though unfortunately it is all too often used when there is inadequate evidence even for a meta analysis. --Hontogaichiban (talk) 05:22, 30 March 2015 (UTC)

Dr. Callot's comment on this article[edit]

Dr. Callot has reviewed this Wikipedia page, and provided us with the following comments to improve its quality:

Far too long article with many vague statement. Discussion Stata and Excel methods have nothing to do here. Dubious research articles are mentioned in an inappropriate format, presumably to advertise their authors.

We hope Wikipedians on this talk page can take advantage of these comments and improve the quality of the article accordingly.

Dr. Callot has published scholarly research which seems to be relevant to this Wikipedia article:

  • Reference : Martin Paldam & Laurent Callot, 2010. "Natural funnel asymmetries. A simulation analysis of the three basic tools of meta analysis," Economics Working Papers 2010-01, School of Economics and Management, University of Aarhus.

ExpertIdeasBot (talk) 17:49, 6 July 2015 (UTC)

Dr. Bonache's comment on this article[edit]

Dr. Bonache has reviewed this Wikipedia page, and provided us with the following comments to improve its quality:

This has to be deleted : "The historical roots of meta-analysis can be traced back to 17th century studies of astronomy, while"

This has to be deleted : "Although meta-analysis is widely used in epidemiology and evidence-based medicine today, a meta-analysis of a medical treatment was not published until 1955." That should be deleted : "Important to note is that these are just a couple of methods that can be used, but several more exist." This is not clear : "Conversely, two-stage methods synthesize the AD from each study and hereto consider study weights." what is hetero ?

We hope Wikipedians on this talk page can take advantage of these comments and improve the quality of the article accordingly.

Dr. Bonache has published scholarly research which seems to be relevant to this Wikipedia article:

  • Reference : Bonache, Adrien & Maurice, Jonathan & Moris, Karen, 2010. "A Best evidence synthesis on the link between budgetary participation and managerial performance," MPRA Paper 20924, University Library of Munich, Germany.

ExpertIdeas (talk) 13:05, 5 August 2015 (UTC)

Thank you for your suggestions. However, I think most editors will agree that edits to the article be supported. See WP:CS. The sentences you specifically mention appear to be correct and properly referenced. juanTamad 06:14, 6 August 2015 (UTC)
First of all, we thank Dr. Bonache for his review of this article. For the first issue, I understand the skepticism, but I was able to find a source to back up the statement and added it to the article. For the second issue, I was unable to find any sources pointing to a meta-analysis of a medical treatment in 1955. Here are two papers that claim meta-analysis was not used in the study of medical treatments until the 1970's: [2], [3]. Without a source, the 1955 assertion probably needs to be removed. The third issue prose is vague and is unreferenced. I have removed it. For the fourth issue, I agree the prose is confusing. I rewrote it as "Two-stage methods first compute summary statistics for AD from each study and then calculate overall statistics as a weighted average of the study statistics." --Mark viking (talk) 02:35, 29 May 2016 (UTC)

IPD meta-analyses[edit]

While discussing sourcing for another article, I found multiple sources that say that the "gold standard" for meta-analysis is the kind using individual patient data (IPD).[4][5][6][7][8] This isn't mentioned in the article; shouldn't it be? --Middle 8 (tc | privacyCOI) 11:00, 26 October 2015 (UTC)

Dr. Ogundari's comment on this article[edit]

Dr. Ogundari has reviewed this Wikipedia page, and provided us with the following comments to improve its quality:

The sample size can be used as a weight as well as inverse of the standard error of the estimates. In fact, most meta-analysis uses effect size related to elasticity from the primary studies. It is also possible to focus on sign and significance of an outcome as effect size incase elasticity from the primary studies could not be used ( see: Card et al, 2010; Koetse et al., 2009)

Card. D., J. Kluve and A. Weber (2010). Active Labour Market Policy evaluation: A meta-analysis. The Economic Journal, Vol. 120: F452-F477.

Koetse.M.J., H.L.F. de Groot and R. J. G. H. Florax (2009). A meta-analysis of the investment-Uncertaity Relationship. Southern Economic Journal, Vol. 76(1): 283-306.

This is also a good reference:

Stanley, T. D. and Doucouliagos, C. (2012). Meta-regression analysis in economics and business. Routledge Advances in Research Methods.


While panel dimension of the meta-regression analysis has been encourages, but using weighted regression has been found to perform well than other form of estimation. See the paper below: Stanley, T. D. and Doucouliagos, C. (2015). Neither fixed nor random: weighted least squares meta-analysis. Statistics in Medicine 34: 2116-2127.

We hope Wikipedians on this talk page can take advantage of these comments and improve the quality of the article accordingly.

Dr. Ogundari has published scholarly research which seems to be relevant to this Wikipedia article:

  • Reference : Ogundari, Kolawole, 2014. "A meta-regression analysis of frontier efficiency estimates from Africa," 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota 165911, Agricultural and Applied Economics Association.

ExpertIdeasBot (talk) 16:09, 31 May 2016 (UTC)