Health screening is a common sense approach applied to many medical conditions. It should be easy: create a sensible tool and then validate it. In the medical literature, this usually involves calculating performance measures such as the sensitivity, specificity, and predictive value of the tool. Calculating each of these is simple -- the risk resides in the interpretation of these numbers [1].

There are two rules of thumb that will protect anyone from mis-interpreting screening tool statistics. The first, I call the “Rule of 100”: If the sensitivity and specificity add up to 100, the tool is performing at chance level. The higher the sum (and closer to 200 maximum), the more accurate the tool is. If the sum adds to less than 100, then the performance is paradoxical (a positive result means the disease is less likely). In statistics parlance, when the sum of sensitivity and specificity is 100, then the likelihood ratio =1, which means the pre-test probability equals the post-test probability, which means the test result added no information.

The second rule is that you cannot interpret a screening test about disease probability without a key ingredient: the pre-test probability. This completes the Bayesian triad of ingredients, along with sensitivity and specificity, needed to interpret any screening test. The information gained by the screen can be viewed as the difference between the pre-test probability and the post-test probability (also called the “predictive value”). The predictive value can appear high because the pre-test probability was high, or because the test is very accurate. Even a random coin toss, where heads is positive for the disease, will have a “good positive predictive value” if applied to a population with a high probability of the disease.

Unfortunately, many publications miss one or both of these points. We could choose many examples, but today, a new publication will serve to demonstrate the problem [2]. The authors report one version of their screening tool for sleep apnea as having 24% sensitivity and 68% specificity. This violates the “rule of 100”, and in fact shows that the test is paradoxical. The supposedly improved versions of their screen showed sums of 100-103. The Rule of 100 tells us that the screen tool added virtually nothing – because it is performing near chance.

Bayes’ Theorem is all too often missing in medical testing. There are so many examples - one cannot publish letters to the editor for each (though I did recently for a particularly problematic example regarding the contentious topic of home sleep testing [3]). Yet there is hope: armed with just the Rule of 100, anyone can quickly glance at test performance and surmise the accuracy. And for those that pass this Rule, then the second rule will guide interpretation of the predictive value.

[1] Bianchi MT, Alexander BM (2006) Evidence based diagnosis: does the language reflect the theory?

BMJ. 333(7565):442-5. (full text at: https://www.ncbi.nlm.nih.gov/pubmed/16931846)

[2] Laratta et al (2016) Validity of administrative data for identification of obstructive sleep apnea.

J Sleep Res. doi: 10.1111/jsr.12465.

[3] Bianchi (2015) Evidence that home apnea testing does not follow AASM practice guidelines--or Bayes' theorem. J Clin Sleep Med. 15;11(2):189. (full text at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4298779/)

There are two rules of thumb that will protect anyone from mis-interpreting screening tool statistics. The first, I call the “Rule of 100”: If the sensitivity and specificity add up to 100, the tool is performing at chance level. The higher the sum (and closer to 200 maximum), the more accurate the tool is. If the sum adds to less than 100, then the performance is paradoxical (a positive result means the disease is less likely). In statistics parlance, when the sum of sensitivity and specificity is 100, then the likelihood ratio =1, which means the pre-test probability equals the post-test probability, which means the test result added no information.

The second rule is that you cannot interpret a screening test about disease probability without a key ingredient: the pre-test probability. This completes the Bayesian triad of ingredients, along with sensitivity and specificity, needed to interpret any screening test. The information gained by the screen can be viewed as the difference between the pre-test probability and the post-test probability (also called the “predictive value”). The predictive value can appear high because the pre-test probability was high, or because the test is very accurate. Even a random coin toss, where heads is positive for the disease, will have a “good positive predictive value” if applied to a population with a high probability of the disease.

Unfortunately, many publications miss one or both of these points. We could choose many examples, but today, a new publication will serve to demonstrate the problem [2]. The authors report one version of their screening tool for sleep apnea as having 24% sensitivity and 68% specificity. This violates the “rule of 100”, and in fact shows that the test is paradoxical. The supposedly improved versions of their screen showed sums of 100-103. The Rule of 100 tells us that the screen tool added virtually nothing – because it is performing near chance.

Bayes’ Theorem is all too often missing in medical testing. There are so many examples - one cannot publish letters to the editor for each (though I did recently for a particularly problematic example regarding the contentious topic of home sleep testing [3]). Yet there is hope: armed with just the Rule of 100, anyone can quickly glance at test performance and surmise the accuracy. And for those that pass this Rule, then the second rule will guide interpretation of the predictive value.

__References__:[1] Bianchi MT, Alexander BM (2006) Evidence based diagnosis: does the language reflect the theory?

BMJ. 333(7565):442-5. (full text at: https://www.ncbi.nlm.nih.gov/pubmed/16931846)

[2] Laratta et al (2016) Validity of administrative data for identification of obstructive sleep apnea.

J Sleep Res. doi: 10.1111/jsr.12465.

[3] Bianchi (2015) Evidence that home apnea testing does not follow AASM practice guidelines--or Bayes' theorem. J Clin Sleep Med. 15;11(2):189. (full text at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4298779/)

__Contributor__: Matt Bianchi MD PhD