Why proper understanding of confidence intervals and statistical significance is important

To the Editor: In their medical education article on confidence intervals, Hemming and Taljaard1 describe an intuitively appealing but incorrect interpretation of confidence intervals, seeming to use a Bayesian interpretation in a frequentist paradigm.

They state: “Directive, yet not statistically significant results, can also arise when the confidence interval mostly overlaps with the values indicative of benefit (or harm), that is, when the interval covers treatment effects mostly in one direction.” This gives a confidence interval a property it does not have — that of a probability distribution. The true value of the population parameter, for which the confidence interval is providing an interval estimate, is fixed and cannot be more likely in one region of the confidence interval than any other. It is either in it or out of it.

Standard statistical texts routinely emphasise this point.2,3,4 Good and Hardin4 state: “In interpreting a confidence interval based on a test of significance, it is essential to realize that the center of the interval is no more likely than any other value.”

It would thus be mistaken to be directive in either direction (benefit or harm) if a confidence interval overlaps the value of no effect. All we can say here is that our data do not enable us to reject the null hypothesis, our results are inconclusive and more research may be necessary.