The head of the Food and Drug Administration got into trouble with his colleagues by overstating the effectiveness of convalescent plasma against COVID-19. Dr. Hahn stated that this old-fashioned treatment strategy would save the lives of 35 out of 100 patients. This statement was misleading, because he confused the statistics of relative risk reduction with absolute risk reduction.
What Is Relative Risk?
Marketers love relative risk statistics because they are so much more impressive than the numbers on absolute risk. To explain this, instead of switching to apples or motorbikes, we’re going to stick with drugs. Back when Lipitor was a brand-new brand-name drug, the company advertised it as reducing the risk of heart attack by 36%.
Here’s how they got to that number.
The ad actually explained these statistics, in fine print under an *asterisk:
“That means in a large clinical study, 3 percent of patients taking a sugar pill or placebo had a heart attack compared to 2 percent of patients taking Lipitor.”
So, if you had 100 people taking atorvastatin for a year and another 100 people taking a look-alike inactive placebo pill for a year, at the end of the year 2 of the atorvastatin (Lipitor) takers had experienced heart attacks. Compare that to 3 of those swallowing placebo pills. What is the difference between them? 1 fewer person per 100 who had a heart attack in the Lipitor group. So the absolute risk reduction is 1 percent. Now, take that 1 person, divide it by the number of people who had heart attacks in the placebo group, and you get 1/3, or about 33 percent. That is the relative risk reduction. You can see why relative risk statistics are more appealing but also more confusing than absolute risk statistics.
What Did FDA Commissioner Hahn Get Wrong?
When FDA Commissioner Stephen Hahn said that convalescent plasma would save the lives of 35 people out of 100, he was presenting relative risk statistics as if they referred to absolute risk reduction. Convalescent plasma comes from the blood of people who have recovered from COVID-19. The plasma is rich in antibodies that can help fight off the coronavirus.
In a preliminary study, the absolute risk of death in people who got infusions was 3 to 5 points lower than in people who did not (medRxiv, Aug. 12, 2020). That is, after a month, 22 percent of people treated with plasma had died compared to 27 percent of those who did not get that treatment. (To us, that looks like an 18.5% relative risk reduction. We are not sure where 35% came from, but we won’t quibble about that.)
Dr. Hahn has acknowledged that his original statement was incorrect. We all need to pay close attention to statistics. When we confuse absolute and relative risk, we are too easily misled.
