False positives in testing

I was watching a "debunking via facts and science" video recently, and the author went off on someone he was responding to for calculating the percentage of positive PCR test results that were false positives. According to the author, this is a basic math error -- the false positive rate is calculated as a proportion of the negative results, not of the positive results. He was quite condescending about how people always make this mistake.

The reason it's traditionally calculated as a proportion of the negative results is not because that number is real-world useful, but because that number doesn't vary significantly with external factors. If you want to know if a revised test procedure is better, this number is nice and invariant, and if you can make it lower, your test is better. Other people can also repeat it and confirm results by getting similar numbers. It's useful for scientists doing development on testing procedures.

But here in the real world, it's not useful. What matters in the real world is indeed the rate calculated as a proportion of the positive results. It's answering the question "How many of these positive results are false?". If not many of them are false, that's a useful diagnostic test, as most of the positive results are genuine. If it's lots or most of the positive results that are false, that's not so useful test, and we should probably do something else.

The issue here is that the rate of false positives as a proportion of all positives varies wildly with how rare the condition is and how targeted the testing is.

The data that was being discussed had PCR tests showing a 0.3% false positive rate. If we're looking for a condition 3% of people have, only 9% of the positive test results will be false positives. This test would be very useful. If 0.3% of test subjects have the condition, 50% of the positive test results are false positives. While this might be handy as part of a test arsenal, it's probably not sufficient on it's own. If only 0.03% of the test subjects have the condition, 91% of the positive test results are false positives, and the test becomes almost useless at finding the condition. If no-one has it, then all the positive results are false, the number of positive results is entirely determined by how much testing you do, and the test is completely useless. Note that the test hasn't changed at all -- it's the same test throughout, only the prevalence of the condition you are searching for is unchanged.

If the 0.3% false positive rate applies to government run PCR tests too (I have no reason to suggest it wouldn't), then using the government's own stats, on the 12th of Feb 2021, the UK did 626,243 tests, with 12,200 positive results, of which 1,879 were likely false positives. That's 85% genuine positives, and the test is providing a useful diagnosis tool.

If we compare that to say 15th August last year, the UK conducted 180,197 tests, and had just 695 positive cases. 0.3% of 180,197 is 540 -- so 540 of those cases were likely false positives. Only 22% were actually genuine results. At that point, the test was not being all that useful.

If Coronavirus was Thanosed out of existence today, and we tested the entire UK (population 66.7 million) tomorrow, there would STILL be 200,000 positive test results -- all of which are false positives.

The take-home here is anyone expecting to test the entire population and come back with zero test results at any point ever is just as barking as anyone who thinks that PCR tests aren't valid at all.