Tests for COVID-19: Has the FDA said yes too many times?
There are many controversies about the coronavirus, but there is one point of consensus: We need testing, testing, and then more testing. But yesterday, in response to criticisms from chairs of two House health subcommittees, the FDA tightened their standards for antibody tests intended to identify people who were previously exposed to the coronavirus. Why?
Typically, the FDA approves tests based on evidence of accuracy. But, under the urgency of the pandemic, the FDA temporarily lifted the requirement that tests be validated before they are marketed. Until the new policy was announced, it was not clear when or if the FDA would review the accuracy of each test.
The FDA website shows that, to date, the FDA has temporarily approved coronavirus testing for 84 different labs and companies. There were 14 new approvals in the last week alone and more than 400 more applications are waiting for FDA’s review. Unfortunately, none of the tests currently available – not the 84 and not the other 400 — have a record of proven accuracy that can be independently verified.
There are two types of tests. One uses samples collected from the nose or throat and searches for the protein sequences that characterize the virus. The second looks for antibodies to the virus in blood serum. We don’t yet know if those antibodies protect against reinfection or whether those with the antibodies can still infect others.
The value of diagnostic tests is typically described in terms of sensitivity — does it accurately diagnose a disease that an individual has. The specificity of the test is defined as how often a test erroneously reports that an infected person is disease-free. The percent of infected patients who are diagnosed as not having the disease is referred to as false negatives.
The percentage of patients who test positive but are not infected are referred to as false positives. Usually, public health authorities are less worried about false positives because doctors are good at figuring out that a person is not sick, but that isn’t true with COVID-19. A positive antibody test signals potential immunity and if it is inaccurate, it could be used to allow a potentially infectious person back into sensitive jobs, such as first responders.
We have surprisingly little information about the percentage of false negatives or false positives in the 84 tests the FDA has granted emergency use authorization (EUA). Very few studies have appeared in the peer-reviewed literature. The most widely recognized research is based on just 205 patients in China, but in that study, nearly 40 percent of people known to be actively infected tested negative. Such a high false-negative rate would normally be considered unacceptable by the FDA. Are the new tests used in America more accurate? One evaluation suggested that a widely touted new test failed to detect infection in about 15 percent of the cases. Doctors across the country have reported many false negatives as well as questionable false positives.
There are real consequences of applying a test with so many false negatives, particularly for COVID-19. For example, if one percent of the population is infected, and the rate of detecting active cases is the same as in the Chinese study, about five healthy individuals will be incorrectly told they have the disease for each one person who is accurately detected as being ill. In other words, for each person that we justifiably quarantine, five would erroneously be told they have a life-threatening illness.
Even when the accuracy of the test is very high (95 percent sensitive, 97 percent specific), many people would get bad information. For example, if 250 million people are tested, about 67 thousand people who have the disease will be told they do not, while about 7.4 million who are not infected will have their lives upended by a positive test. We need testing, despite imperfections in the measures. But, the public also deserves the best tests possible.
What happens when hundreds of unvalidated tests flood the market? Monitoring a pandemic requires accurate, consistent information. With so many tests, we can’t know when cases are peaking, stabilizing, or decreasing in different communities — and therefore, when it is possible to loosen restrictions on social distancing.
We need access to more testing as soon as possible, but tests need to be as accurate as possible. The “let a thousand flowers bloom” philosophy that the U.S. government is allowing is leading to confusion and real harm. The FDA was created to protect the public from the “wild west” of snake oil cures. FDA must focus on ensuring the accuracy of fewer tests that have the best evidence of accuracy so that agencies can evaluate the spread of COVID-19 in each state and each county. That requires rigorous evaluation of tests, selecting only the best ones for use.
Robert M. Kaplan is a faculty member at Stanford University’s Clinical Excellence Research Center, a former Associate Director of the National Institutes of Health, former Chief Science Officer for the U.S. Agency for Health care Research and Quality, and author of “More than Medicine: The Broken Promise of American Health.”
Diana M. Zuckerman is the president of the National Center for Health Research. A former Capitol Hill and White House staffer, Zuckerman’s research focuses on products and policies that have the potential to improve public health and patients’ health.