Harold Pollack had a good post in 2012 on the math of screening and the importance of a priori observations:
since a test with greater than 99% sensitivity and specificity can still yield some pretty screwy results when it is deployed within the wrong population.
Suppose we use the above test in a pretty high-risk population in which the prevalence of unknown HIV infection is about 3%. That’s really high….In this population, the rapid test really seems to work well. It has high positive predictive value, and it has incredibly high NPV, too. Incidentally, you can now see why the commonsense term ”accurate” is a pretty sloppy way to discuss screening tests. Depending on the context and underlying prevalence, that word could denote any one of these four things: sensitivity, specificity, PPV, or NPV.
In any event, what happens if we apply the same test within a lower-risk population, such as the population of young sexually active straight couples likely to be a key market for this test. Within this population, HIV prevalence is more like 0.2% rather than 3%….
But the test doesn’t seem so “accurate” anymore. The positive predictive value drops to 20/40, which Austin tells me equals 50%. That’s a serious problem. Half of the positive test results will be false positives.
Austin Frakt at Academy Health digs into the same wellness effectiveness study that I mentioned a couple of months ago. As a refresher, Pepsi had a general lifestyle wellness program (lose some weight, eat more fruit etc) and a chronic disease management program that they allowed for a systemic evaluation. The results were aggregated as successful but the breakdown was that the disease management component drove all cost savings and health improvements.
The takeaway is a familiar story. When narrowly targeted wellness programs, like many other health interventions, can be beneficial–even cost saving (a rarity among health interventions). But, when more broadly implemented, they often are not. A focus on workers and dependents with specific diseases makes eminent sense. Not only are they the sickest–and in that sense deserving of greater focus–they are the most expensive to insure, offering a far greater opportunity for savings from disease and lifestyle management than a typical insured.
Prevention works when there is an a priori belief of elevated risk in a population. Screenings are the same thing. General screenings for unusual diseases have a hard time covering costs if the screenings cost more than a couple of dollars because too many people have to be screened to detect one case that otherwise would not have been detected. There are exceptions; most vaccinations are amazingly cost effective, clean water, bleach, and steam are amazingly cost effective but there are very few very low hanging fruit that a broad population can benefit from.