Study shows 82% miscarriage rate among Covid-vaccinated women? Nope. Here's why.
A single table from the study is being seriously misinterpreted
There is a seriously erroneous claim making the rounds widely right now among vaccine skeptics: i.e., that a major study in the New England Journal of Medicine found that 82% of women who were vaccinated in the first or second trimester suffered a miscarriage.
This claim appears to have originated with this blog post by Steve Kirsch, which then got a big boost after Kirsch appeared on Bret Weinstein’s “Dark Horse” podcast. Disappointingly, despite having the error pointed out to him, Kirsch has not yet corrected it, which for me raises some serious questions which I address towards the end of this piece.
After reading the study, I understand how the critics concluded what they did. However, the claim is clearly based upon a serious misreading of a single (and somewhat confusing) table in the study. Understanding where the critics went so wrong requires diving into the weeds of the data, which I do below.
However, the upshot is that far from showing an 82% miscarriage rate, the data from the study actually shows a miscarriage rate of between four and nine percent (I explain how I calculate this range below). This is well within the range of normal (roughly between 10 and 20%). However, because the study was only ten weeks long, and because most of the women in the study (yes, including those vaccinated early in pregnancy) were still pregnant at the conclusion of the study, we really have no idea what the actual miscarriage rate is (although we can say with certainty that it's nowhere near 82%).
A confusing table
So how did so many people misread the study, and come up with a miscarriage rate that is so much higher than what I've just said?
In brief, the error comes down to treating a subset of the study participants (in this case, the 827 vaccinated women whose pregnancies were "completed" before the study ended), as if they represent the entire set of study participants (i.e. all 3958 vaccinated pregnant women who were registered participants in the study).
The 82% claim is based entirely upon a serious misreading of Table 4 (here) in the study. That table shows "Pregnancy Loss and Neonatal Outcomes in Published Studies." In plain English, the table records the outcome of pregnancies that were "completed" before the conclusion of the study.
(Here’s the table for reference. But, like I say, it’s a bit confusing without reading it in the context of the study text.)
A "completed" pregnancy as defined by the study authors can mean birth, but also miscarriage, induced abortion, and ectopic pregnancy. As the table shows, of the 827 women with a "completed" pregnancy at the end of the study, 712 gave birth, 104 suffered a miscarriage, and the remainder either had an abortion or suffered an ectopic pregnancy.
The confusion arises from the fact that the authors tried to calculate the safety of the vaccines by treating the 827 women who had completed pregnancies as a representative sample of all pregnant women who get vaccinated. Because - they argue - most of the women with completed pregnancies healthily gave birth, and comparatively few miscarried (104 out of 827, or 12.6%), then the vaccine is statistically "safe" for pregnant women.
To be frank, this strikes me as an odd and questionable move. But it is not questionable for the reason most of the critics are saying. I will return to why I think it is questionable later on. But first, let's examine the central point.
The critics' big mistake
Here's what the critics are saying about Table 4: Miscarriage in the third trimester is extremely rare. Therefore, in order to figure out what the real miscarriage rate is, you should remove the women from Table 4 who were vaccinated in their third trimester (700), and then calculate the miscarriage rate against the remaining women - i.e. those who were vaccinated in the first or second trimester (when miscarriage is more likely to occur). This leaves us with 127 women. If there were 104 miscarriages among those 127 women, this amounts to an 82% miscarriage rate. Horrible!
Here’s a visual representation of this argument that is being widely shared:
But here's what the people making this claim have failed to notice: There were far more than 127 women in the study who were vaccinated in the first and second trimesters (2,846 of them, to be exact), and according to the data, the overwhelming majority of those women were still pregnant at the conclusion of the study.
Let me repeat that: Most of the nearly 3000 women in the study vaccinated in the first and second trimester were still pregnant at the conclusion of the study. They had not miscarried. Their pregnancies were still ongoing. They were, so far as we know, healthy.
These women were not listed on Table 4 for the simple reason that their pregnancies were not yet "completed." Only women with completed pregnancies were listed on Table 4. Completed pregnancies includes miscarriages. In other words, Table 4 selects for the small subset of women in the study who miscarried. Given this, calculating the miscarriage rate from Table 4 is like saying: "Women who were likely to have miscarried were likely to have miscarried." Of course they were!
Here's another way to look at it: If (as the critics claim) Table 4 shows that 82% of women vaccinated in the first and second trimester miscarried, you might ask what happened to the other 18% of women? Did they have healthy pregnancies. No, they did not. Because Table 4 only includes women with "completed" pregnancies, the other 18% either had an induced abortion, ectopic pregnancy, or gave birth (presumably prematurely, given the short duration of the study). In other words, if you read the table the way the critics are reading the table, you would have to conclude that 100% of pregnant women vaccinated in the first or second trimester either had an abortion, ectopic pregnancy, miscarriage, or gave premature birth.
Obviously, that's nonsense. But that's how the critics would have to read the table, if they were being consistent.
Calculating the real miscarriage rate
So what happened with the rest of the thousands of women vaccinated in the first and second trimesters? Did those women miscarry, or were their pregnancies healthy?
The simple answer? We don't know. The study was only 10 weeks long. Pregnancy takes nine months. The only way to know for sure would have been to follow up with all the women nine months later, to find out how their pregnancies ended (birth/miscarriage/abortion/ectopic pregnancy). However, the goal of the study authors was only to capture an early snapshot of the effect of the vaccines on pregnancy, to see if there were any early warning signs. They wrapped up the study long before we could know for sure how all the pregnancies ended. That doesn't make their study useless. But it does mean that we have to be careful in interpreting their findings.
If, however, you did want to try to calculate the raw miscarriage rate, the best you could do is take the total number of miscarriages (104) and compare that to the total number of pregnant women who were vaccinated in the first or second trimester (2846). This gives us a miscarriage rate of 3.6%. That is, during the 10 week duration of the study, 3.6% of the women who were vaccinated in the first or second trimester miscarried.
Now, there is a case for restricting the sample further, only to women who were vaccinated in the first trimester, since most miscarriages happen in the first trimester. If you do that, you get 104 miscarriages for 1132 women. This gives us a miscarriage rate of about 9.2%. Still well within the range of normal.
Granted, this statistic must be taken with a grain of salt. Almost certainly (given statistical averages) more of the women would have gone on to miscarry after the study was completed. How many? Well, again, we don't know. We would have to wait nine months to know for sure.
But clearly, a huge majority of pregnant women who were vaccinated in the first and second trimester did not immediately miscarry. The preliminary data in the study give us no reason to believe that there was an unusually high miscarriage rate among vaccinated women. Neither, however, do they necessarily rule out that there might be a higher-than-normal miscarriage rate (although definitely not remotely as high as 82%). More data is needed.
But what about my criticisms of how the authors used Table 4? Basically, my concern is that the authors treated the subset of 827 women with "completed" pregnancies as if they were a representative sample of all pregnant women who complete their pregnancies. Based upon this, they concluded that because the ratio of women who had miscarriages to those who healthily gave birth was within the range of normal, then the vaccine is "safe" for pregnant women.
However, I don't see how you can use such a hodgepodge sample of women as a representative sample, and justify drawing the conclusion the authors did. The 3958 women in the study were all vaccinated at different stages of pregnancy, and at different stages of the study. Some early. Some late. The women also overwhelmingly worked in healthcare. In other words, the study sample is very messy and suffers from selection bias. Which is, I suppose, why the authors note in the study that their sample of women with "completed pregnancies" is not "directly comparable" to the general population of pregnant women. Indeed.
However, the study author's questionable math in Table 4 does not justify the far more questionable math of their critics. There may be problems with their study. But in no way were the study authors guilty of maliciously (or even unintentionally) covering up a cataclysmic miscarriage rate.
Why did so many vaccine skeptics accept the 82% miscarriage claim?
I fully understand why some pregnant women (or women who plan on getting pregnant) are hesitant to take the vaccine. This vaccine program is a massive global experiment. We lack a lot of the data we need to conclude with absolute certainty that the vaccine is safe for pregnant women. The picture is still evolving. Clearly, the fact that we only discovered the increased risks of myocarditis and blood clots after the vaccines were widely rolled out shows that the clinical trials did not capture the totality even of the immediate effects of these vaccines. The long-term effects are, by definition, even less known. Contrary to the media drumbeat, it is not unreasonable to be concerned, especially when the stakes are as high as the life of a baby. And especially in light of the growing number of reports of disruptions to women’s cycles following vaccination.
However, the fact that people like Steve Kirsch, and (according to this podcast) Yale's Dr. Harvey Risch, have endorsed the 82% miscarriage rate claim, and the fact that so many others have repeated the claim, including some who are well-credentialed, is something that has made me seriously question, more than anything else, what’s happening in the Covid vaccine skeptic community.
Bret Weinstein, for instance, is a man I have long admired. The care, precision, humility and expertise with which he approaches complex claims, carefully qualifying his remarks and identifying those areas where he could be mistaken, is an inspiration. Even when he is wrong, he’s wrong in a way that’s right (that is, that’s open to correction, and transparent with his arguments). And yet, Weinstein had Kirsch on his podcast, and so far as I can recall, at no point did he challenge him on the 82% miscarriage rate claim. Indeed, Weinstein repeatedly touted Kirsch’s blog post. How did Weinstein not notice, and challenge this highly suspect figure?
The fact that he did not, and that Kirsch has refused to correct the 82% miscarriage claim, despite having many commenters below his post point his mistake out to him, suggests to me either: a) the mental heuristics of some prominent vaccine skeptical figures are misaligned, or outright broken, or b) some of them are not acting in good faith. In either case, it suggests that they are not to be trusted.
If there were an 82% miscarriage rate out there among vaccinated pregnant women, it would be a medical apocalypse. Every single obstetric doctor and nurse in every country where vaccinations are taking place would be seeing a sudden and enormous influx of women miscarrying. It would be one of the worst medical calamities in history.
The idea that the authors of a study in the New England Journal of Medicine simply "buried" this catastrophic data in a chart, and that the medical system either has not noticed, or has ignored or covered up an 82% miscarriage rate, is preposterous. Anybody presented with that claim should immediately be suspicious that there's some serious funny business going on.
So, again I ask: why are so many vaccine skeptics repeating the claim?
The risks of being a renegade
Well, I suppose it’s because many of them found in the study what they expected to find. That is, what we’re seeing is a classic case of confirmation bias.
Those who accepted the 82% miscarriage claim, especially those who have the statistical and scientific training to read the study data accurately, likely did not notice the error because they already partially or wholly believed that the vaccines were causing huge numbers of miscarriages. The idea that the NEJM study authors covered up the damning data was, for them, simply the expected confirmation of what they already knew to be true. And, because they are now surrounded with people who think the same way as them, there was no one to prevent them from publishing their error to the world.
This is, of course, one of the dangers of being an ideological renegade.
Those in the Covid vaccine skeptic community have positioned themselves out of the medical and scientific mainstream. While there are possible advantages to this (i.e. the ability to speak hard truths to a group that has fallen prey to groupthink), there are serious dangers as well.
The scientific community and process were set up precisely to deal with the biases and blind spots of individual researchers. On our own, all of us, including (sometimes especially) the smartest among us, are prone to all manner of cognitive biases and mistakes. However, what we have learned over time is there is at least hope that we can ameliorate the worst effects of our individual biases by plugging into a community of trained individuals with rigorous processes in place for testing hypotheses and identifying error.
The other day, however, a Twitter account called “Vaccine Truth” set up by allies of Steve Kirsch (and possibly Kirsch himself) to promote his vaccine-skeptic views, replied to a complaint by journalist Claire Lehmann that Bret Weinstein should not be using podcasts to promote his vaccine-skeptic views. Instead, said Lehmann, he should be following the normative scientific process: writing up his findings in formal papers, and submitting them to reputable peer-reviewed journals.
In response to Lehmann, the “Vaccine Truth” account suggested that Bret needs to bypass the normative scientific process, because the peer-reviewed journals have an a priori alliance to “the narrative.” The evidence they used to support this claim is the fact that vaccine skeptics haven’t been able to get papers published exposing the 82% miscarriage rate.
However, as we've seen, the 82% miscarriage rate claim is transparently false. In other words, some of Kirsch's and Weinstein's supporters are arguing for the need to bypass the normative scientific process by pointing to an instance where the normative scientific process successfully weeded out a flagrantly erroneous claim - an error that Kirsch failed to detect because of his own cognitive biases.
Shortly thereafter, the Vaccine Truth page once again posted the claim about the 82% miscarriage rate.
That's what you call irony.
It should also be a wakeup call to all vaccine skeptics. On the one hand, I do believe that there is evidence that some medical and government institutions, and parts of the normative scientific apparatus, have been captured by special interests, or are driven by a desire to achieve certain a priori pragmatic aims. This means that there is reason to suspect that some of the data or talking points coming out on vaccines or the pandemic are not transparent or trustworthy.
We are in uncharted waters. And in a time of so many unprecedented firsts, we need determined and competent critics who are willing to risk their careers and reputations in order to hold powerful institutions and figures accountable, challenging unexamined claims and demanding transparency. Hence, my appreciation for Weinstein, Malone, Bridle, etc.
On the other hand, it is clear to me that a lot of questionable, or outright insane, claims are receiving airing among vaccine skeptics, and that at this point there is nothing that some confirmed vaccine-skeptics will not believe, without question, about how bad the vaccines are, or how malicious their creators and promoters are. In the worst cases, the vaccine skeptics know the truth even before they read the data or examine new claims. And the truth is that everything bad that is said about the vaccines is true, and anything good that is said about them is false.
And that, clearly, is an epistemological recipe for disaster.
I agree that it appears Steve was wrong on this point. But don't take him in bad faith. He spends his whole day trying to open doors of communication, and others have been pushing the false meme of the high percentage of spontaneous abortions. I suspect this is a case of proxy trust through two degrees.
What he is right about, and has done the leg work to talk with people at the highest levels of the FDA and other places about, is that there are significant risk signals that authorities aren't even bothering to examine---and that policy seems to have been put in place to block examination.
Ultimately, it is far more dangerous to be a true believer in new vaccine technology that might have a negative net risk-benefit than a vaccine skeptic with a subset of incorrect takes. Either way, we need to keep the doors of communication open. If Steve didn't hear this point well it's because he doesn't have 500 hours a day to hear everyone. But I can attest that when he hears new and better arguments, he does adjust.