To mask or not to mask?

The recent publication of a Danish study into the efficacy of mask-wearing to prevent the wearer from becoming infected with the coronavirus has become very controversial.

Some have suggested that mask-wearing confers no benefit to the wearer. The question of whether “to mask or not to mask” is now a political one.

I’m going to look at the Danish study, and Prof Carl Heneghan’s response — the view that he said Facebook had censored (here).

The Danish study is here:

https://www.acpjournals.org/doi/10.7326/M20-6817

Prof Heneghan’s response in an article in the Spectator (you get a couple of articles free, otherwise ££) is here:

https://www.spectator.co.uk/article/do-masks-stop-the-spread-of-covid-19-#

Let’s look at what the Danish study says. The researchers’ objective was:

“To assess whether recommending surgical mask use outside the home reduces wearers’ risk for SARS-CoV-2 in a setting where masks were uncommon and not among recommended public health measures”.

They designed a randomised controlled trial of mask-wearing against not wearing a mask (“controls”); the setting was “Denmark”, and masks were to be worn in the community but not at home. At the time of the study, the community rate of infection was 2% as estimated by “the authorities”, and the number of participants was designed to show a reduction to 1%, a 50% reduction. 4636 participants would give the trial 80% power at 5% significance. They recruited over 6000 people, anticipating a 20% loss to follow up.

A small number of infections was detected in both groups, 42 in the mask group, 53 in the no-mask group. Covid infections in the home were reported by 52 people in the mask group, of whom 2 developed infection, and 39 in the no-mask group of whom 1 developed infection. This, the authors say, suggested “that the source of most observed infections was outside the home”.

The authors noted the limitations to the study:

“Inconclusive results, missing data, variable adherence, patient-reported findings on home tests, no blinding, and no assessment of whether masks could decrease disease transmission from mask wearers to others.” [Emphasis added]

Their conclusion was:

“The recommendation to wear surgical masks to supplement other public health measures did not reduce the SARS-CoV-2 infection rate among wearers by more than 50% in a community with modest infection rates, some degree of social distancing, and uncommon general mask use. The data were compatible with lesser degrees of self-protection.”

That is, the trial did not show that mask wearing outside the home reduced Covid infection in the studied individuals. No other question was asked or answered in this trial; the trial did not “give any answers”. In particular, the trial did not show that not wearing a mask is appropriate; no trial can answer a question that the trial didn’t ask.

Anyone who has ever tried to do a randomised controlled trial will know just how difficult it is. It is important to exactly specify a question beforehand, and to have a statistically sufficient number of participants. Often, the specification is little more than an educated guess. Here, a 50% reduction was the target, but other values, bigger or smaller, could have been used.

Further, trials often include assumptions — not always recognised — and any answers are dependent on statistical analysis.

The researchers report on the limitations of the trial as they saw them. I’m not clear what “the community” means in Denmark compared say to N Ireland. I’m thinking of public transport and getting to work, for instance. (The researchers may have included this detail in the Supplement, but the links to the pdf files don’t work for me.) It’s not clear if the study was in a rural or urban setting or a mixed environment.

What does “the home” mean in Denmark? More people live in flats there than in the UK, where homeownership is more common. Does, for example, “the home” include the communal halls and lifts in a block of flats?

The authors assume that most infections occurred outside the home. Can this assumption be justified?

The mechanics of statistical calculations have been relieved by computers; previously these were very tedious. It was important to discuss any trial with a statistical beforehand. Today, it’s not uncommon for researchers to present a computer program with data, and “torture the data” to get a valid result — I am emphatically not suggesting this was the case here.

Statistics do not give yes/no or black/white answers. Statistics deals in probabilities. In testing the hypothesis that mask-wearing reduces the wearers’ chances of becoming infected by 50%, we are employing the null hypothesis and Fisher’s method. This is where it all gets complicated with double negatives and the like, but bear with me.

The null hypothesis says that mask-wearing has no effect compared to the “controls”, and we then test our results against this, trying to disprove the null hypothesis. What this then shows is not the probability that the mask-wearing hypothesis is true, it tells us the probability of getting the survey results that were obtained if the hypothesis is true. The smaller the p value, the smaller the probability of getting the survey results if our hypothesis were true; this is not the same as showing that the mask hypothesis is true. The two are not the same.

There’s just a little more in that vein. We read about p values of 0.05 or 5%. The p is the probability that the null hypothesis can be rejected, and the smaller the value of p, the more likely we are to reject the null hypothesis. This value of p is the benchmark, the gold standard in medical research. It was introduced by Fisher in the 1930s. However, when he was asked for his rationale, he had to concede that there wasn’t any; the number was “convenient”. It is arbitrary.

This glimpse into the theory of statistics is, I’m afraid, more than a bit grim, the sort of thing that many would feel the need for a lie down afterwards. Nonetheless, it’s important to be aware of the limitations of statistics, as well as the difficulties in doing trials.

Professor Heneghan, in a Spectator article entitled Landmark Danish study finds no significant effect for facemark wearers discusses the results of the trial. The Spectator is a long-established weekly magazine about politics; it is owned by the Barclay brothers who also own The Daily Telegraph. It is generally small-c conservative, supporting the Conservative party. A previous editor was Boris Johnson.

Prof Heneghan began by asking:

“Do face masks work? Earlier this year, the UK government decided that masks could play a significant role in stopping Covid-19 and made masks mandatory in a number of public places. But are these policies backed by the scientific evidence?” [Emphasis added] He continued:

“In the end, there was no statistically significant difference between those who wore masks and those who did not when it came to being infected by Covid-19…it seems that any effect masks have on preventing the spread of the disease in the community is small.” [Emphasis added]

He continued: “When it comes to masks, it appears there is still little good evidence they prevent the spread of airborne diseases.” [Emphasis added] And then, “The only studies which have shown masks to be effective at stopping airborne diseases have been ‘observational’ — which observe the people who ordinarily use masks, rather than attempting to create a randomised control group.” [Emphasis in the original]

Thus, “This is why large, randomised trials like this most recent Danish study are so important if we want to understand the impact of measures like face masks.” He concludes:

“And now that we have properly rigorous scientific research we can rely on, the evidence shows that wearing masks in the community does not significantly reduce the rates of infection.” [Emphasis added]

Compare that with the researchers’ conclusion:

“The recommendation to wear surgical masks to supplement other public health measures did not reduce the SARS-CoV-2 infection rate among wearers by more than 50% in a community with modest infection rates, some degree of social distancing, and uncommon general mask use. The data were compatible with lesser degrees of self-protection.”

Remember, a trial is designed to answer one question; here this was about whether masks protect the wearer. The trial wasn’t designed to look at the prevention of airborne diseases, as Prof Heneghan says, for that is a different question. As above, no trial can answer a question that the trial didn’t ask.

So why is Prof Heneghan conflating the two questions — whether masks protect the wearer, and whether masks prevent or reduce the spread of infection in the community — with the unvoiced implication of spread to others? That is a very political question, and the Spectator is a political magazine. You might wonder if the Spectator tends to the right-wing libertarian view, where government “edicts” are frowned upon as a breach of liberty. Making mask-wearing compulsory is such a breach, for example. You might even wonder if Prof Heneghan is looking for articles that support this view.

It’s very clear, if you pay close attention to what the researchers described, and you understand how statistics “work”, you will come to entirely different conclusions to those described in Prof Heneghan’s article. But if you only read Prof Heneghan’s article you could come to the conclusions he does. This is another example of Fast and Slow Thinking in Politics (here) in action. Scientific papers by their very nature are turgid and indigestible; it takes effort to read them. Articles in the Spectator are for a general readership, where the readers do not stop to think or question things that seem so blatantly obvious. And so mischief and misunderstanding spreads.

To summarise; Prof Heneghan is saying that for mask-wearing in the community there is “still little good evidence they prevent the spread of airborne diseases”. The Danish study was about the effect of masks on the wearer in a setting where mask-wearing was uncommon, it was not about the airborne spread of disease in general or to others. As the trial authors say:

“The findings, however, should not be used to conclude that a recommendation for everyone to wear masks in the community would not be effective in reducing SARS-CoV-2 infections, because the trial did not test the role of masks in source control of SARS-CoV-2 infection.”

I appreciate that parts of this post are a much more difficult read than is usual on Slugger, but I hope it’s been worth the effort. There’s now an extensive literature about how to read a medical paper. If you want to know more, Prof Trisha Greenhalgh’s book is the reference (here).

My thanks are due to SeaanUiNeill for critically reading earlier drafts of this post.

Photo by coyot is licensed under CC BY-NC-SA