How to read a medical meta-analysis without being dazzled by the total

Begin before the pooled number

A systematic review uses explicit methods to find and synthesise research addressing a defined question. A meta-analysis is a statistical synthesis that a review may include. A review may reasonably avoid pooling studies when a common numerical answer would be misleading.

Write one sentence stating the population, intervention or exposure, comparator, outcome and time frame. Then compare it with the headline. A review of a short-term laboratory result cannot, by its title or size, become a review of long-term survival.

PRISMA 2020 asks authors to make their reporting transparent. It explicitly distinguishes reporting from assessing how well the review was conducted. Finding a PRISMA flow diagram is a useful start, not the end of your appraisal.

An original example: five papers are three studies

Imagine a folder with five reports. Study A enrolled 300 people and produced a primary paper, a follow-up paper on those same participants and a subgroup paper. Study B enrolled 200 different people and produced one paper. Study C enrolled another 100 people and produced one paper.

The folder contains five reports, three independent studies and 600 unique participants under these invented assumptions. Adding 300 + 300 + 300 + 200 + 100 would give 1,200, but would count Study A’s participants three times. The subgroup paper may provide useful detail; it does not create another independent sample.

This exercise is deliberately simple. In practice, overlapping recruitment or partially shared databases can be harder to recognise. Cochrane advises linking reports belonging to the same study using identifiers and details such as recruitment dates, settings and participants. Keep the reports; avoid treating them as independent studies.

Check what the search could have missed

Look for the search strategy, sources searched and last search date. Publication date and search date answer different questions: a newly published review can still contain an older evidence search. Eligibility rules and reasons for excluding studies should be inspectable. Trial registries and other sources can help identify research absent from journal databases.

For our invented folder, write “search coverage unknown” rather than assume these five reports are all that exist. A useful next task would be to look for the registration records of Studies A, B and C and check whether their planned outcomes are represented. Do not label a result missing merely because one familiar paper did not contain it.

Separate reporting, review methods and evidence certainty

AMSTAR 2 appraises methods of reviews of healthcare interventions. Its overall judgement depends on critical weaknesses; it is not meant to be converted into a simple points total. Our worksheet below is a reading aid, not an AMSTAR rating.

Certainty in the underlying evidence is another question. GRADE examines risk of bias, inconsistency, indirectness, imprecision and publication bias for each outcome. One review can therefore support different levels of confidence for different outcomes. A large participant count does not erase these concerns.

In our example, the fact that Study A supplies three reports says nothing by itself about whether its allocation, follow-up or outcome measurements were sound. More pages about the same participants do not repair a design flaw.

Read the forest plot as a set of comparisons

Identify the effect scale and the direction of benefit first. The no-difference value is 1 for ratios and 0 for differences. Inspect individual estimates and their intervals before the pooled estimate. A pooled average need not describe every study or patient group.

Heterogeneity concerns variation between study results beyond sampling error. A random-effects model does not make clinical differences disappear. I² is not a pass/fail quality score, and a low estimate does not guarantee comparable methods or unbiased studies. A prediction interval can help describe the spread of effects relevant to another similar setting, but depends on assumptions and may be unstable with few studies.

Treat missing evidence as a real uncertainty

Not every completed study or measured outcome becomes an available result. If availability depends on what was found, the visible evidence can be distorted. Look for checks against protocols or registrations and an explanation of what remains unavailable.

Funnel-plot asymmetry has several possible causes; it is not a diagnosis of publication bias. Conversely, a non-significant asymmetry test cannot certify its absence. Such tests often lack power with few studies; Cochrane gives at least ten studies as a rule of thumb, with further conditions.

Our three-study example would not justify reassurance from a quiet funnel-plot test. The most useful next step is to locate missing information, not to decorate the review with another plot.

Use the worksheet to write a proportionate conclusion

Suppose all we know is that our five reports concern the same broad topic. We cannot calculate a defensible pooled benefit from their participant counts. We still need study designs, comparable outcomes, effect estimates, uncertainty and a way to handle the repeated reports. “Three studies identified; benefit not yet assessed” is the accurate result of this exercise.

For an actual review, try a three-sentence note. First state the exact question and the studies contributing to that outcome. Next give the estimate with its uncertainty and absolute context when available. Finally state the main limitation that could change its interpretation. Avoid replacing that final sentence with a vague claim that more research is needed.

What better future reviews could make possible

A useful update to our imagined review would map every report to its study, explain any changed eligibility decisions and show exactly which new data altered the conclusion. If an eligible Study D later appears, readers should be able to tell whether it adds a new population, longer follow-up or a more reliable measurement.

AI could assist with organising candidate reports and flagging apparently duplicated identifiers, but a fluent summary is not proof that studies were selected or interpreted correctly. For this portal, the worthwhile ambition is a traceable evidence map: readers should be able to follow each claim back to the material supporting it. That is an editorial objective, not a claim that an automated system has already performed a systematic review.

A worksheet for the review you are reading

Original reading prompts, not a validated checklist, clinical recommendation or numerical quality rating. Use the source documents for formal appraisal methods.