Elsevier

The Lancet

Volume 351, Issue 9095, 3 January 1998, Pages 47-52
The Lancet

Series
Overcoming the limitations of current meta-analysis of randomised controlled trials

https://doi.org/10.1016/S0140-6736(97)08461-4Get rights and content

Summary

For a meta-analysis to give definitive information, it should meet at least the minimum standards that would be expected of a well-designed, adequately powered, and carefully conducted randomised controlled trial. These minimum standards include both qualitative characteristics—a prospective protocol, comparable definitions of key outcomes, quality control of data, and inclusion of all patients from all trials in the final analysis—and quantitative standards—an assessment of whether the total sample is large enough to provide reliable results and the use of appropriate statistical monitoring guidelines to indicate when the results of the accumulating data of a meta-analysis are conclusive. We believe that rigorous meta-analyses undertaken according to these principles will lead to more reliable evidence about the efficacy and safety of interventions than either retrospective meta-analysis or individual trials.

Section snippets

Protocol development

A well-designed RCT begins with identification of a medically important question. A protocol is developed before the trial begins, so that conduct of the trial follows a prospective plan. The design revolves around the primary and sometimes secondary hypotheses, which are usually parsimonious. These hypotheses drive the inclusion and exclusion criteria, the choice of specific treatment regimen, study outcomes, subgroups of interest, and analysis plan. All are generally defined in advance and

Key issues in conduct of a well-designed meta-analysis (panel 1)

For a meta-analysis to provide reliable information, it should meet, at least, the same qualitative and quantitative standards as a single well-designed large RCT. Several of the standards applied in the conduct of a well-designed RCT can be adopted, so that a more prospective approach to meta-analysis is taken.

Updating of meta-analysis

There have been recommendations for continuous updating of meta-analysis as each new RCT is published.8, 27 The principles of the OIS and application of formal monitoring guidelines provide a framework for examination of this issue. First, updating of meta-analysis should be periodic (not continuous, since this clouds the interpretation of any differences observed) when a significant increment in additional information is available—for example, when the increment in new information is at least

Disorders with few large trials

Many of the judgments and principles that govern the need for a large amount of information (from either RCTs or a meta-analysis of trials) are applicable to many specialties, not just cardiology, for which several examples are given. The development of both large trials that are affordable and meta-analysis, even in cardiology, is recent. Although a few moderate-sized (a few thousand patients and a few hundred events) RCTs in cardiology were completed in the 1970s, it has only been since the

Summary of standards and interpretation of meta-analysis

This paper provides guidelines for the conduct, analysis, and interpretation of meta-analysis, summarised in Panel 1, Panel 2. In general, we believe that meta-analysis should be at least as rigorous as a well-designed and adequately powered RCT. Despite reasonable assumptions, there may be good reasons to use more conservative statistical criteria for a meta-analysis than one would for an RCT. There are widely recognised potential biases inherent even in the best meta-analysis. Therefore, more

References (48)

  • A Sommer et al.

    Impact of vitamin A supplementation on childhood mortality: a randomised controlled community trial

    Lancet

    (1986)
  • R Peto

    Why do we need systematic overviews of randomized trials?

    Stat Med

    (1987)
  • CD Furberg et al.

    Lessons from overviews of cardiovascular trials

    Stat Med

    (1987)
  • S Yusuf et al.

    Why do we need some large, simple randomized trials?

    Stat Med

    (1984)
  • TC Chalmers et al.

    Meta-analysis of clinical trials as a scientific discipline—I: control of bias and comparison with large co-operative trials

    Stat Med

    (1987)
  • D Moher et al.

    Meta-analysis of randomized controlled trials: a concern for standards

    JAMA

    (1995)
  • I Chalmers

    The Cochrane Collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care

    Ann N Y Acad Sci

    (1993)
  • S Yusuf

    Obtaining medically meaningful answers from an overview of randomized clinical trials

    Stat Med

    (1987)
  • MJ Clarke et al.

    Obtaining data from randomized controlled trials: how much do we need for reliable and informative meta-analyses?

  • CB Begg et al.

    Publication bias: a problem in interpreting medical data

    J R Statist Soc A

    (1988)
  • N Victor

    The challenge of meta-analysis: indications and contra-indications for meta-analysis

    J Clin Epidemiol

    (1995)
  • RE Slavin

    Best evidence synthesis: an intelligent alternative to meta-analysis

    J Clin Epidemiol

    (1995)
  • Indications for fibrinolytic therapy in suspected acute myocardial infarction: collaborative overview of early mortality and major morbidity results from all randomised trials of more than 1000 patients

    Lancet

    (1994)
  • RP Byington et al.

    Reduction in cardiovascular events during pravastatin therapy: pooled analysis of clinical events of the pravastatin atherosclerosis intervention program

    Circulation

    (1995)
  • Cited by (360)

    View all citing articles on Scopus
    View full text