A major ivermectin study has signs suggestive of scientific fraud

Today is ‘World Ivermectin Day,’ a day to celebrate one of the most important antiparasitic drugs used to treat neglected tropical diseases, and the controversial drug many claim is an effective treatment for COVID-19. So here is a crazy story for World Ivermectin Day: one of the largest randomized-controlled trials to date showing benefit of ivermectin for COVID-19 has signs suggestive of scientific fraud.

It looks like the data may have been purposefully altered or fabricated. At the very least, as it stands, it cannot be trusted as a valid scientific study.

This story was discovered and originally reported by Jack Lawrence, Gideon Meyerowitz-Katz, and Nick Brown; please check out their posts for full details.

Briefly, here’s the story. ‘Efficacy and Safety of Ivermectin for Treatment and prophylaxis of COVID-19 Pandemic‘ by Elgazzar et al. was a randomized controlled trial of ivermectin for COVID-19 from Egypt. It was never peer-reviewed and was posted on a preprint server in November 2020. The results claimed a huge benefit of ivermectin for COVID-19. But some of the details of the study (in particular the details of how patients were randomized) were incomplete, making the study overall poor quality or “high risk of bias.” If you are confused about why there is so much conflict over ivermectin, this essentially explains it right here: many who believe ivermectin works are looking at the promising statistical results reported by this and other studies, and those who remain skeptical are looking at the holes in the study methods and other technical issues, noting that these issues mean we can’t be confident in those promising statistical results.

That is where this story stood for several months: scientists looked at this and other ivermectin studies and said ‘well, that’s interesting, but there are some major issues in the study designs, so we need better designed studies before concluding ivermectin is a beneficial treatment for COVID-19.’ This is in agreement with the current NIH recommendations: we can’t say definitively that ivermectin does or does not work for COVID-19, and we need a high quality trial to answer this question. The FDA issued this warning against taking ivermectin for COVID-19.

But no one suspected fraud at all — many studies are “poor quality” without being fraudulent.

In a more recent version of the Elgazzar preprint, a link to the raw data used for the study was made available. But – that raw data (an excel file) was password protected, so it couldn’t be readily accessed by the public.

Recently, Jack Lawrence, a biomedical science masters student and independent journalist, was assigned the Elgazzar preprint as a part of an assignment for his masters degree, and noticed that parts of the paper appeared to be possibly plagiarized from other sources. He found this concerning, and started digging a bit. He found the password protected raw data linked in the new version of the preprint. He decided to make an attempt at guessing the password and succeeded: it was 1234.

With the raw data now available, the many, many problems with this study, in part hidden by the vague methods reported in the paper, became evident. First, data from different patients matched nearly identically and appeared to be duplicated, as if someone had simply copied and pasted data from one set of patients into another. Second, the raw data did not even match all the results presented in the study. Many patients were listed as being hospitalized and dying before the study was reported to have begun. There were numerical patterns that are suggestive of human interference (humans are bad at making up random numbers, and there are telltale signs when they try to do it). And there were suspect statistics reported in the paper itself. I won’t go into all the details here, but please check out these posts to learn more [1, 2, 3]. In summary, this is really, really, really bad.

This story is not new. Last summer when I started digging into hydroxychloroquine studies, I found that one fairly commonly cited study had some statistics that simply didn’t add up (citation #4 here). For example, one of the analyses compared outcomes of patients who had gotten hydroxychloroquine early versus later in their disease course. The p-value reported by the paper was p < 0.0001 (very significant). The raw numbers used to calculate this were not made explicit in the paper (which is somewhat unusual), but with a little high school algebra the values could be back-calculated, and running the statistics on those values yielded p = 0.15 (not significant). That’s not good. That study was also posted as a preprint (not peer-reviewed), yet it was being used to advocate that hydroxychloroquine be widely prescribed for COVID-19. While peer-review is not perfect (and is not designed to catch fraud), these examples illustrate how important it is to have people digging into the numbers, especially when the results of the study are being used to advocate drugs be prescribed to millions of people.

So, what does this all mean for ivermectin? It’s almost always impossible to say what happened for certain in these cases, but at the very least, with these data issues now apparent, it is clear we can no longer rely upon the results from the Elgazzar study. And this was the largest randomized trial to date on ivermectin, which was driving the results of many of the meta-analyses assessing ivermectin for COVID-19. Gideon Meyerowitz-Katz, an epidemiologist, re-ran the meta-analyses after removing this study (and including a new, recently published study) and found the previous meta-analysis results showing benefit of ivermectin for COVID-19 are dramatically reduced or disappear.

As has been said all along, many of the studies we have on ivermectin for COVID-19 to date are low quality, so to really know if there is or is not a benefit, we need a well-designed randomized trial (which are ongoing now). Some have asked why the medical community readily adopted dexamethasone as a treatment for COVID-19 after only one major study assessing it, but have not adopted ivermectin despite many studies assessing it. The answer is the RECOVERY trial (which assessed dexamethasone for COVID-19) was very well designed, so we can be confident in its results. It’s not necessarily the number of studies that matters, it’s also the quality of those studies that are taken into consideration. So we will have to wait until the results from a well-designed randomized controlled trial are available to know if there is any benefit of ivermectin for COVID-19.

But many have claimed that we already know ivermectin works and need to give it now (and some have argued that it can be used instead of vaccination.) The data does not support these claims. This whole saga feels like a repeat of last summer when hydroxychloroquine was touted as “the key to defeating COVID” based on extremely low quality evidence, and illustrates how important it is to have robust data backing medical decision making. This is the essence of evidence-based medicine: it is imperative that we know the drugs and treatments we give patients are actually helping them. That’s what separates medicine from wishful thinking.