Masked Science: Fact-checking Mask Studies
I saw this post on the interwebs citing a bunch of studies, suggesting that masks are ineffective at preventing infection and/or are unsafe. (And also suggesting there is some sort of cover-up of this science). Since there is still controversy around mask-wearing (and the post gave me PubMed ID’s! thank you!) I decided to dig into to see if there was any merit to what they were saying. Here is what I found.
Quote #1
“Preliminary Report on Surgical Mask Induced Deoxygenation During Major Surgery…Our study revealed a decrease in the oxygen saturation of arterial pulsations (SpO2)”
PMID 18500410.
Is this study real?
Yes (full text here).
Is the quote above actually in the study?
Yes, but it is missing the second half of the sentence. Full quote reads “Our study revealed a decrease in the oxygen saturation of arterial pulsations (SpO2 ) and a slight increase in pulse rates compared to preoperative values in all surgeon groups.”
What did the study do?
This study measured SpO2 (blood oxygen saturation) of surgeons before and after surgery. All the surgeons were wearing masks during the surgery. As a control group, they also measured SpO2 of individuals in the operating room who were observing (not participating in surgery) and who were not wearing a mask.
What did they find?
For short operations (< 60 min) there was no decrease in SpO2 after surgery. For longer operations (1-2h, 2-3h, 3-4h) there were subtle but statistically significant decreases in the SpO2 after surgery in the mask-wearing surgeons who were operating, but not the non-mask wearing observers. The largest drop in SpO2 was recorded after the longest surgeries (3-4h), but was still quite small (it dropped from ~97.6% SpO2 before the surgery to ~96% SpO2 after the surgery — these are both within normal range.)
What does this tell us about masks?
SpO2 stayed well within normal range for every study participant, indicating it is safe to wear masks for extended periods of time. There was a slight drop in SpO2 in surgeons after long operations, but unfortunately, this study doesn’t tell us whether this was due to the mask or due to operating. The control group tells us this decrease in SpO2 was not from standing for that period of time, so we can rule out that explanation out. However, given that surgery requires intense focus and many of us tend to hold our breaths when we are focusing on fine motor tasks, it’s possible this slight reduction in SpO2 is due to changes in surgeon breathing and/or other effects of performing surgery, and not due to the mask. Alternatively, it is possible that wearing a mask for several hours slightly decreases SpO2, but not to a level that is unsafe. The authors do acknowledge that they cannot attribute this drop to the mask: “This change in SpO2 may be either due to the facial mask or the operational stress, since similar changes were observed in the group performing surgery without a mask.” I think it’s unfortunate they chose a title that didn’t reflect this uncertainty.
Simple Conclusion:
This study indicates that surgeons in their study group had a slight drop in blood oxygen saturation after long surgeries, but that this drop is minimal, and does not present a health concern. This study does not provide evidence regarding the effect of masks on blood oxygen saturation, as there were significant confounding variables in the study design.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #2
“We know that wearing a mask outside health care facilities offers little, if any, protection from infection.”
PMID: 32237672.
Is this study real?
The article is real, but it’s not a study — it is a perspective/opinion piece. Here is the full text.
Is the quote above actually in the study?
Yes.
What did the article say?
This is an opinion piece discussing the pro’s and con’s of universal masking in hospitals during the COVID-19 pandemic, presumably written in March 2020 (published April 1, 2020). The quoted line above is the only mention of mask-wearing outside of hospitals, and the rest discusses whether every doctor/nurse/etc. needs to wear a mask in hospitals. The article does not question the effectiveness at proper mask use in hospitals, but discusses other factors that need to be weighed (supply availability, if people who wear a mask will touch their face more, etc.).
What does this tell us about masks?
They do not provide citations nor data backing the quoted claim above that wearing a mask outside health care facilities offers little protection, so this article does not provide any data on that topic. Overall, this piece is consistent with the thinking at the time it was written (the very beginning of the pandemic in the US) — it was not recommended that everyone wear masks at that point. (See below for discussion as to why.)
Simple Conclusion:
This article is an opinion piece written at the beginning of the pandemic that does not provide any data on whether masks are effective or not.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No, this is not a study, it’s outdated advice.
Quote #3
“.. both surgical and cotton masks seem to be ineffective in preventing the dissemination of SARS-CoV-2 from the coughs of patients with COVID-19 to the environment and external mask surface.” https://www.acpjournals.org/doi/10.7326/M20-1342.
Is this study real?
Yes (full text here), but it has now been retracted.
Is the quote above actually in the study?
Yes.
What did the study do?
They had 4 people with COVID-19 cough on petri dishes with and without masks (cotton and surgical, but not N95’s), and they measured the amount of virus they could detect with and without masks. The authors concluded that because they could not find a difference in the viral load on the petri dishes with and without masks and because they detected virus on the outside of the masks, surgical and cotton masks do not block SARS-CoV-2 when people are coughing.
Why was this study retracted?
Retraction means the study was so flawed that it can no longer be considered reliable — it is like “unpublishing” an article. The study was retracted because it had a Limit of Detection problem — the method they used to measure how much virus was present on the petri dishes could only measure viral loads above a certain threshold, and some of their samples were below that threshold. When samples are below the limit of detection, the experiment can’t tell the difference between no virus, a little bit of virus, and a little bit more virus. (This is kind of like squinting at a ruler from 20 feet away and trying to tell the difference between 0.4 inches and 0.33 inches… your eyesight is not reliable to detect subtle differences like that.) Unfortunately, the type of experiment they ran will still spit out numbers even below the limit of detection, even though those numbers aren’t reliable at all. It’s the scientists’ job to know when those numbers are reliable versus unreliable. These scientists made a mistake and thought their numbers were reliable, when in fact they weren’t.
Simple Conclusion
This study provides no evidence on the effectiveness or ineffectiveness of masks due to a problem in their methodology (there wasn’t enough virus to measure it accurately). Because of this issue, it was retracted.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #4
“Most healthcare workers develop de novo PPE (such as N95 face mask) associated headaches or exacerbation of their pre-existing headache disorders.”
PMID: 32232837
Is this study real?
Yes, full text here.
Is the quote above actually in the study?
Yes with a slight edit — the original quote reads “Most healthcare workers develop de novo PPE‐associated headaches or exacerbation of their pre‐existing headache disorders.” PPE stands for personal protective equipment.
What did the study do?
They gave questionnaires to 158 health care professionals working in high risk hospital areas in Singapore at the beginning of the COVID-19 pandemic. The questionnaires asked if they previously had headaches, if they had developed new headaches since the pandemic (and where the headaches were, how severe they were, and how often they had them), and what types of PPE they were wearing (at that hospital anyone caring for a COVID patient was required to wear goggles, close-fitting N95 masks, gowns, surgical gloves.)
What did they find?
The hospital workers wore goggles and face masks an average of ~6 hours per day. Over a 30 day period, 81% of participants reported sometimes having headaches when wearing the N95 mask with or without goggles, with the location of the headaches often corresponding to the points where the mask/goggles put pressure on the face. About half of the people attributed their headache to the mask, while half attributed it to the goggles. Most people graded their headache as mild and infrequent (1-4 days / 30 days). People who wore both N95 face masks and goggles for > 4 hours/day and those who had a pre-existing headache diagnosis were more likely to report headaches.
What does this tell us about masks?
Tight-fitting PPE like fit-tested N95 masks and goggles may cause mild headaches if worn for an extended period of time, likely due to the pressure from the straps/seal on the face.
Simple Conclusion:
Health care workers and other professions wearing tight-fitting masks and goggles for multiple hours may get mild headaches.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #5
“This study showed that the filtering efficiency of cloth face masks were relatively lower, and washing and drying practices deteriorated the efficiency.”
PMID: 31289698.
Is this study real?
Yes, here is the full text.
Is the quote above actually in the study?
Yes.
What did the study do?
The focus of this study was the effectiveness of cloth masks worn in developing countries on preventing inhalation of air pollution. They bought cloth masks from Kathmandu, Nepal and measured the pore size and filtering efficiency of air pollution compared to surgical masks. They found that cloth masks had larger pores and lower filtering efficiencies (63-84%) than surgical masks (94%). They also found that repeatedly washing cloth masks decreased the filtering efficiency further.
What does this tell us about masks?
While this study did not look at anything related to viruses or transmission of infectious diseases, it does tell us that cloth masks (at least those purchased in Kathmandu, Nepal) have larger pores than surgical masks, and that surgical masks are better at filtering small particles than cloth masks (though cloth masks still filter out a significant percentage of small particles in the air.) This is consistent with the idea that N95 > surgical mask > cloth mask > no mask.
Simple Conclusion
Surgical masks filter out more particles than cloth masks purchased in Nepal. Depending on the type of fabric, this may be true for cloth masks commonly used in the US as well.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #6
“None of the studies established a conclusive relationship between mask/respirator use and protection against influenza infection.”
PMID: 22188875.
Is this study real?
Yes, here is the full text.
Is the quote above actually in the study?
Yes.
What did the study do?
This study was a systematic review (a review of all studies on a particular topic) looking at whether or not the use of masks and/or respirators prevent transmission of influenza. One of the goals of this study was to figure out if it is worth it for people to wear masks every year to lower the transmission of seasonal influenza. They looked at 8 studies of masks use for preventing influenza, and 2 showed an effect (masks help prevent infection) and 6 did not show an effect (no difference in infection rates with and without masks.) Because there weren’t a lot of studies on influenza, they also looked at 9 studies of mask use during the first SARS epidemic. 8 out of 9 studies showed an effect (mask use was associated with reduced risk of SARS infection). However, the authors note that many of the SARS studies were suboptimal, and that the results can’t necessarily be generalized to influenza, since they are very different viruses. All together, they conclude that there wasn’t enough published evidence to support the idea of using masks to prevent transmission of the seasonal flu.
What does this study tell us about masks?
This study tells us that (at least as of 2012 when this study was published), there was not evidence to support wearing masks every year to prevent the seasonal flu. It does suggest that masks work against SARS, as 8/9 studies showed a protective effect, however better studies were warranted as there were some methodological flaws in those studies. Overall (as of 2012), the authors highlight that more research is needed.
Simple Conclusion
As of 2012, there was not enough evidence published to support the use of masks to prevent seasonal influenza. The authors want more (and better) research on the topic. There are quite a few studies indicating masks helped protect people in the first SARS epidemic, but those studies have some methodological weaknesses.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #7
“Face mask use in health care workers has not been demonstrated to provide benefit in terms of cold symptoms or getting colds.”
PMID: 19216002.
Is this study real?
Yes, abstract here. I couldn’t access the full-text so this is based off the abstract.
Is the quote above actually in the study?
Yes.
What did the study do?
A total of 32 health care workers were studied for 77 consecutive days — they were randomized to either wear masks or not wear masks, and the presence of cold symptoms was tracked. Only two people got colds, (one in each group). They also found that the group who wore masks were more likely to experience headaches.
What does this study tell us about masks?
This study doesn’t really tell us much about the effectiveness of masks at preventing infection because it didn’t study enough people — if only two total people got colds throughout the course of the study, if there was a difference in cold prevention we would need far more people to detect this difference. It does suggest wearing masks may increase risk of headaches in health care workers.
Simple Conclusion
This very small study suggests healthcare workers who wear masks daily may have increased risk of headaches. It does not give us reliable information on whether or not masks prevent colds. A bigger study is needed to address this question.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #8
“There is little evidence to support the effectiveness of face masks to reduce the risk of infection.”
PMID: 20092668.
Is this study real?
Yes, full text here.
Is the quote above actually in the study?
Yes, but it makes the conclusion sound simpler than it is. Here is the full paragraph:
“In conclusion there remains a substantial gap in the scientific literature on the effectiveness of face masks to reduce transmission of influenza virus infection. While there is some experimental evidence that masks should be able to reduce infectiousness under controlled conditions [7], there is less evidence on whether this translates to effectiveness in natural settings. There is little evidence to support the effectiveness of face masks to reduce the risk of infection. Current research has several limitations including underpowered samples, limited generalizability, narrow intervention targeting and inconsistent testing protocols, different laboratory methods, and case definitions.”
What did the study do?
This is a systematic review of studies that have looked at whether or not face masks, often in combination with other protective measures like hand washing, help protect against influenza infection. They looked at 12 different studies — some were underpowered and didn’t provide any useful information, some did not show a protective effect of mask use from influenza and/or cold symptoms, and some showed a protective effect of mask use or face shield. Sometimes this effect was only observed when combined with hand hygiene (hand washing). There were significant limitations in many of the studies, and the authors call for further research.
What does this tell us about masks?
As of 2010 when this review was published, there wasn’t strong evidence to support the idea that healthy people should wear face masks to protect against influenza infection. Many studies had limitations, and more research is needed. Remember not to confuse absence of evidence with evidence of absence — sometimes there is “not strong evidence” for something because we simply haven’t done the studies needed to find the evidence, and sometimes there is not strong evidence for something because we looked and there was not evidence to be found. (Check out this post for more on that.) Here is the conclusion provided by the authors: “There is some evidence to support the wearing of masks or respirators during illness to protect others, and public health emphasis on mask wearing during illness may help to reduce influenza virus transmission. There are fewer data to support the use of masks or respirators to prevent becoming infected. Further studies in controlled settings and studies of natural infections in healthcare and community settings are required to better define the effectiveness of face masks and respirators in preventing influenza virus transmission.“
Simple Conclusion:
As of 2010, there was not strong evidence to support mask-wearing by healthy people to prevent influenza infection in a non-pandemic setting. Some of the studies were pretty crappy, and more research is needed.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #9
“..laboratory-confirmed virus were significantly higher in the cloth masks group.. Penetration of cloth masks by particles was almost 97%.. This study is the first RCT of cloth masks, and the results caution against the use of cloth masks.. Moisture retention, reuse of cloth masks and poor filtration may result in increased risk of infection.”
PMID: 25903751.
Is this study real?
Yes, here is the full text.
Is the quote above actually in the study?
Yes, but it is somewhat taken out of context.
What did the study do?
This is a randomized-controlled trial looking at the effect of different types of masks on preventing infections in medical personnel in a Vietnam hospital. There were three groups: those who wore medical masks (surgical masks), those who wore cloth masks, and a control group, which, very importantly, was “usual practice, which included mask wearing.” So many in the control group were also wearing masks, which makes the results of this study very difficult to interpret (i.e. there is no “no mask” group). They looked at infections (respiratory illness, influenza-like lillness, and laboratory-confirmed respiratory virus infection) and found that the rates of infection were lowest in those who wore surgical masks and highest in those who wore cloth masks. The “control” group (which isn’t really a control group but more of a mystery group) was in the middle. Notably, a lot of people in the study didn’t actually wear the masks all the time like they were supposed to, which also limits the conclusions one can draw from this study.
What does this tell us about masks?
This study suggests that surgical masks may work better than cloth masks (at least those used by hospital workers in this study) at preventing influenza-like illness in a non-pandemic hospital setting. However many of the participants weren’t compliant with mask wearing, so I don’t put a lot of weight on the results of this study. The control arm is completely useless, so this study tells us nothing about the effectiveness of mask versus no mask.
Simple conclusion.
Once again, surgical mask > cloth mask (probably, but mostly this study was just messy).
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #10
“Respiratory acidosis develops when air inhaled into and exhaled from the lungs does not get adequately exchanged between the carbon dioxide from the body and oxygen from the air.” https://www.medicalnewstoday.com/articles/313110
Is this real?
Yes, but this is just a website about respiratory acidosis, and has nothing to do with masks.
What does this tell us about masks?
Absolutely nothing.
Simple Conclusion
This is just a medical definition.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No, this is not a study, it is a lesson in medical vocabulary.
Quote #11
“Conversely, surgical and hand-made masks, and face shields, generate significant leakage jets that have the potential to disperse virus-laden fluid particles by several metres. The different nature of the masks and shields makes the direction of these jets difficult to be predicted, but the directionality of these jets should be a main design consideration for these covers. They all showed an intense backward jet for heavy breathing and coughing conditions. It is important to be aware of this jet, to avoid a false sense of security that may arise when standing to the side of, or behind, a person wearing a surgical, or handmade mask, or shield.” /content/files/ftp/arxiv/papers/2005/2005.10720.pdf
Is this study real?
Yes (though it has not yet been peer-reviewed.)
Is the quote above actually in the study?
Yes.
What did the study do?
They used some fancy methods to track where a person’s breath goes when they breath and cough while wearing different types of masks/PPE (surgical mask, handmade mask, FFP1, FFP2, respirator, face shields.) They found that all masks (except the respirator, which is not designed to stop exhaled air) significantly decrease how far a person’s breath travels in the forward direction. They found that masks that seal (like FFP1, FFP2 masks) don’t have significant leakage to the side and back, while masks that don’t seal (surgical masks, cloth masks) do have leakage to the side and back. They caution people to remember that non-sealing masks can leak to the side and back.
What does this tell us about masks?
Non-sealing masks do not block everything and redirect some airflow to the side and back. This was based on a study of one person, so it should be verified in a larger sample size. It is not yet peer-reviewed, so there may be flaws that have not been caught yet.
Simple Conclusion
Non-sealing masks may leak to the side/back.
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No.
Quote #12
“Face masks should not be worn by healthy individuals to protect themselves from acquiring respiratory infection because there is no evidence to suggest that face masks worn by healthy individuals are effective in preventing people from becoming ill.” https://jamanetwork.com/journals/jama/fullarticle/2762694
Is this study real?
Yes, but it’s not a study — it’s a summary of information about masks published in early March 2020.
Is the quote above actually in the publication?
Yes.
What does this tell us about masks?
This article reflects the thinking at the time it was written — reserve masks for healthcare workers and people who are sick. It does not provide new data on the topic, only summary recommendations.
Simple Conclusion
At the beginning of the pandemic, the recommendation was that healthy people not wear masks. (More details on that below).
Does this study indicate that masks are unsafe or ineffective against SARS-CoV-2?
No, this is not a study, it’s outdated advice.
Summary: What did we learn from these studies?
- There is not enough evidence to suggest everyone wear masks every year during influenza season in a non-pandemic setting (and we don’t.)
- Surgical masks are probably better than cloth masks at preventing infection (though there may be variation on the type of cloth mask used). Both leak somewhat.
- Use of tight-fitting masks for long hours by health care workers may be associated with increased risk of mild headaches.
- While many of the quotes (as well as the title “masked science” with the crying masked person) make it sound like these articles provide evidence that masks are unsafe and/or don’t work against SARS-CoV-2, none of these articles actually provide any evidence to support this claim (and in some cases, provide evidence to the contrary.)
Was this post misinformation?
While none of the quotes or studies were fabricated, they were often taken out of context and/or presented in a misleading way. Read the quotes again now that you’ve gone through the details of the studies.
The quotes often avoid key details about the studies which results in misleading the reader.. for example, the studies on cloth masks were often compared to surgical masks, but you couldn’t tell that from the quotes. So reading these quotes you might assume they were comparing to not wearing masks at all, which changes the conclusion from “surgical masks work better than cloth masks” (what the studies actually showed) to “cloth masks don’t work” (what this post seems to be trying to imply). This is obviously very misleading, and most people would only catch it if they went and looked up the studies. Additionally, some of the quotes feel cherry-picked to reveal a specific point of view and don’t really reflect the conclusion of the authors of the studies. A lot of the studies are about influenza/common cold, but the way they’re presented makes it seem like these results should apply to SARS-CoV-2, which isn’t an accurate assumption (why? There’s probably many reasons, but a major one is SARS-CoV-2 is way more infectious than the seasonal flu). The random inclusion of a respiratory acidosis definition is just silly — thrown in there with all these quotes from studies, it would make many readers think there was a study that had shown respiratory acidosis was associated with mask use (which is a myth that has been circulating), when in fact this had absolutely nothing to do with masks. All together, this feels like it was put together by someone who was trying to convince people masks aren’t safe/don’t work against SARS-CoV-2 in spite of the evidence, not because of it. It is an important reminder that you can’t trust something just because it has lots of citations.
Are masks effective in slowing the transmission of SARS-CoV-2?
None of these studies actually looked at the effectiveness of masks for SARS-CoV-2 (except the retracted one, which is useless) — are there studies indicating that masks do protect against SARS-CoV-2 transmission? Yes, check out this meta-analysis of SARS-CoV-2, SARS, and MERS studies and this summary about cloth masks (note the study being asked about at the beginning of this summary is the problematic study from quote #9).
Why were we told not to wear masks at the beginning of the pandemic?
So if masks work against SARS-CoV-2, why were they not recommended for everyone at the beginning of the pandemic in the US? (They were recommended for healthcare workers and those who were showing symptoms). While I was not involved in making these recommendations (of course), here is my best guess. At the beginning of the pandemic, we were in the following situation: 1. we didn’t understand the virus, 2. we thought that the prevalence of COVID-19 in the US was quite low, 3. we didn’t have strong evidence that people could be infected without showing symptoms and 4. people were freaking out and panic-buying toilet paper. From that perspective, the risk of telling “healthy” people (people who weren’t showing symptoms) to wear masks outweighed the benefit… we weren’t sure how much it would help (we didn’t know so many “healthy” people were actually infected), and it would likely deplete mask supplies for healthcare workers if everyone started panic-buying N95s. Some argue that we should have told everyone to wear cloth masks back then as to not confuse everyone about the effectiveness of masks; in retrospect I think that would have been a good idea. Now, our situation is different: 1. we have data to show that wearing masks helps slow transmission, 2. the prevalence of COVID-19 in the US is way, way higher than it was in March (and we know a lot of cases are going undetected), 3. we know that many people are infectious without showing symptoms (meaning they can spread it when they think they are healthy), 4. we’re not as concerned about people panic-buying all the N95 masks, and 5. many places are trying to re-open, so masks provide a way to lower the rate of transmission without resorting to a full shutdown again. Given these changes, it makes sense for everyone to wear masks when around others, even if they aren’t showing symptoms, because they may be infected without knowing it.
