Prediction Intervals: The Number Your Reader Needs
A confidence interval describes uncertainty around the average effect. A prediction interval describes what a new study might find.
A random-effects meta-analysis produces a pooled estimate and a confidence interval. Almost every review reports both, and almost none reports the interval that answers the question a reader actually has.
The confidence interval describes uncertainty around the average of the true effects. The prediction interval describes the range within which the true effect of a new study, in a new setting, would be expected to fall.
If you are deciding whether to implement an intervention in your own clinic, the second interval is the one that concerns you.
Why the two intervals diverge
In a random-effects model, effects are assumed to vary across studies, with between-study variance tau². The confidence interval around the pooled estimate narrows as studies accumulate, because more studies mean a better estimate of the mean.
The prediction interval does not behave that way. It incorporates tau² directly, so it reflects the spread of true effects and not just uncertainty about their average. Adding studies improves the estimate of tau² but does not reduce the underlying variability.
The result is the pattern that surprises people: with substantial heterogeneity, the confidence interval can be narrow and comfortably exclude the null while the prediction interval crosses it. Both statements are true and they are not in conflict. The average effect is real; the effect in the next study might not be.
Riley, Higgins and Deeks (2011) made this argument directly, and IntHout and colleagues (2016) followed it with the case for routine reporting.
What it looks like in practice
Twelve trials of an exercise intervention, pooled standardised mean difference of −0.42, 95% CI −0.61 to −0.23, tau² = 0.09.
The confidence interval is clean. The intervention works on average and the interval sits well clear of zero.
The 95% prediction interval runs approximately −1.04 to 0.20. It crosses zero.
Reporting only the first is not wrong, but it is incomplete in a way that matters for the reader deciding whether to act. Reporting both says: on average this helps, the size of the benefit varies substantially between settings, and a well-conducted trial in a new population could find nothing. That is the honest summary.
When not to report one
Prediction intervals depend on tau², and tau² is estimated poorly when there are few studies. With fewer than about five studies the interval becomes so wide as to be uninformative, and reporting it can mislead in the opposite direction by implying more is known about the spread than is warranted.
The reasonable approach is to prespecify a minimum number of studies for calculating prediction intervals, report the interval when that threshold is met, and state why when it is not. Some methodologists suggest a floor around 10 studies for a stable estimate. Whatever you choose, choosing it at protocol stage keeps the decision out of reach of the results.
Prediction intervals also make no sense under a fixed-effect model, which assumes a single common effect and therefore no distribution to predict from.
How it interacts with GRADE
A prediction interval crossing the null while the confidence interval does not is a strong signal for downgrading certainty for inconsistency. It is also a more defensible basis for that decision than pointing at I², because it is expressed on the scale of the outcome and connects directly to a clinical question.
This works in the other direction too. A prediction interval that stays entirely on one side of the null, even with high I², is a reasonable argument against downgrading. The effects vary in size but not in direction, which is a different situation from genuine inconsistency, and one that a rating based on I² alone cannot distinguish.
What to write when a reviewer objects
Reviewers rarely object to a prediction interval. What they object to is the discrepancy it exposes when you have reported one and the discussion continues as though the confidence interval settled everything.
If the prediction interval crosses zero, say so in the results, not just in the table, and reflect it in the conclusion. Something like: "The pooled effect favoured the intervention (SMD −0.42, 95% CI −0.61 to −0.23), but the 95% prediction interval (−1.04 to 0.20) indicates that in some settings the intervention may have no effect. Certainty was downgraded for inconsistency."
That paragraph is a stronger piece of writing than one asserting a benefit the data supports only on average, and reviewers notice.
References
IntHout, J., Ioannidis, J. P. A., Rovers, M. M., & Goeman, J. J. (2016). Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open, 6(7), e010247. https://doi.org/10.1136/bmjopen-2015-010247
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011). Interpretation of random effects meta-analyses. BMJ, 342, d549. https://doi.org/10.1136/bmj.d549
Common questions
- Which software calculates prediction intervals?
- In R, metafor reports them through `predict()` on a random-effects model, and the meta package provides them via the `prediction = TRUE` argument in forest plots. Stata's meta suite includes them, and RevMan supports their calculation. There is no reason to omit one for want of tooling.
- Should the prediction interval go on the forest plot?
- Yes, and most packages draw it as an extended bar beneath the diamond. It is the single clearest way to show a reader that the average and the range are different things, and it costs no space.
- Does a wide prediction interval mean my meta-analysis failed?
- No. It means true effects vary across studies, which is usually what a random-effects model was chosen to accommodate. A wide interval is a finding about the evidence, and the appropriate response is to investigate the variation through prespecified subgroups or meta-regression and to report the uncertainty rather than conceal it.