On January 2nd, 2022, the NYT ran an article “When They Warn of Rare Disorders, These Prenatal Tests are Usually Wrong“, which showed that current tests for rare prenatal diseases have very high false positive rates. For example, the false positive rate for the test for Prader-Willi syndrome (“a condition that offers little chance of living independently as an adult“) is 93%. The article was disturbing: the companies describe their tests as “near certain… reliable… highly accurate“, and the recommended follow-up test is sometimes not done, as it can cost thousands of dollars and has to be performed later in the pregnancy, sometimes past the time when abortions are legal in that state. “A 2014 study found that 6% of patients who screened positive obtained an abortion without getting another test to confirm the result“. So this is a big deal.

The article was NYT reporting at its best, imho, but it also left out a crucial piece of information. Using the Prader-Willi syndrome as an example, getting a positive test result does not mean that there’s a 7% chance that the fetus has the disease. This was not claimed in the article, but it’s the natural conclusion that patients and some doctors are likely to make. But the full picture is much more damning. What matters is the probability that the fetus has the disease, given that the test result was positive. Let’s call that quantity P(d|p). The key point is that P(d|p) depends heavily on the prevalence of the disease in the general population. Using the numbers given for the Prader-Willi syndrome in the article (namely, a 93% false positive rate, a prevalence of the disease in the general population of one in 20,000, and assuming that the test gives no false negatives, which is optimistic), gives the following result: while the probability that a randomly chosen fetus is free of the disease is 99.995%, the probability that a fetus that has tested positive for the disease, is actually free of the disease, is 99.9946%.

Would you want a test done that, if it turns out to be positive, changes the probability that the fetus is healthy, by only 0.0004 percent?

These kinds of tests were initially developed for Down syndrome and they worked very well. But their efficacy depends heavily on the prevalence of the disease. According to healthline.com, Down syndrome occurs in roughly one out of every 700 births in the U.S.

(Regarding the patient in the article: a further (invasive) test showed that the initial test result was wrong, and the woman now has a healthy 6-month old baby.)

To compute the above results you just need to use Bayes’ rule, which incorporates the frequency of the condition in the general population to get the correct answer. But you can see intuitively that just relying on the test results has to be wrong. Suppose the test was applied to one million people. With a prevalence of one in 20,000, we’d expect about 50 cases of Prader-Willi syndrome. Let’s suppose that there are actually exactly 50 cases. Then 999,950 fetuses are free of the disease, but at a false positive rate of 93%, the test would incorrectly flag 929,953 of them as possibly having the disease. That’s 929,953 unnecessary heartaches, invasive tests, and possible abortions.