I really hate when people try to spread misinformation. My feelings are the same as Jeff's. This stuff has to be countered. People are dying.
Here are the facts that I've been able to put together from multiple critiques of the metareview.
1. This meta review is not “from Johns Hopkins”. One of the authors, Steve Hanke, is a professor of Economics at Johns Hopkins. The other two are Lars Jonung, a professor emeritus at Lund University in Sweden, and Jonas Herby, who is an economist at the Center for Political Studies in Copenhagen.
2. Steve Hanke is a member of the Cato Institute. This is a libertarian think tank. It is decidedly political. If you work for the Cato Institute, you’re not necessarily wrong, but you are coming to the issue with a strong libertarian ideology already.
3. Not a study. This is a working paper. It has not been peer reviewed. A fundamental part of research is peer review. They have gone around that process to get their opinion out into the media.
4. Of the 34 studies included in the review, 12 of them are other working papers. 14 are from economists. However, most of the actual research on this subject has been done by medical researchers.
5. All papers with modelled counterfactuals are excluded. Because this is the most common method used in infectious disease assessments, this has the practical impact of excluding most epidemiological research. The authors exclude many of the most rigorous studies, including those that are the entire basis for their meta-analysis in the first place.
6. The authors claim that they only include studies using a difference in difference approach. However, two of the studies, Chisadza et al. and Alderman & Hajoto, do not use difference in difference.
7. The 0.2% top line number in the news is based on only seven studies, listed in Table 3. This is a very small number of studies for a meta review.
8. Table 3 cites as supporting evidence seven papers that use a metric called the Oxford Stringency Index. However, the authors do not include the medRxiv paper by Thomas Hale et al. from July 6 2020, which produced the Oxford Stringency Index, on which all of these seven papers are based, and which actually estimated a massive reduction in deaths due to lockdown.
9. Their model weights papers by their standard error, which ends up giving one paper, by Chisadza et al 2021, 91.8% of all the weight. So to summarize, the -0.2% top line estimate of this paper is 91.8% based on this one paper, and 8.2% weighting is given to all other known studies in the entire universe.
10. The authors of the Chisadza et al. paper have publicly disagreed with this metareview, and accused the metareview authors of having a predetermined conclusion when writing the paper. “They already had their hypothesis. They think that lockdown had no effect on mortality, and that’s what they set out to show in their papers.”
11. Additionally, the authors of the Chisadza paper actually find “statistically significant non-linear associations on the number of deaths”. In other words, the effect of NPIs on preventing covid deaths does not increase in a simple linear way. This implies that less strict interventions can appear to have a weak effect on preventing covid deaths, but as the stringency of the intervention increases, the reduction in covid deaths decreases dramatically. The meta study by Hanke ignored this and just extracted a linear term.
12. In Table 5, of seven papers, most of the weighting is given to one paper by Aparicio and Grossbard 2021. It claims that this paper shows that lockdowns increase deaths by 2.6%. However, Professor Grossbard said that “the sign of the coefficient can not be meaningfully interpreted”. Meaning that, whether Hanke et al.’s model shows that lockdowns increase deaths (positive coefficient) or rather decrease deaths (negative coefficient), is impossible to interpret, given Hanke et al.’s flawed methods.
13. Another paper in Table 5 by Spiegel and Tookes 2021 found "strong evidence consistent with the idea” that mask mandates and business closures *reduced* future fatality growth. However, Hanke et al. disagree with Spiegel and Tookes about the results of their own paper, and claim that Spiegel and Tookes actually show that these interventions *increase* fatalities by 13.1%.
14. In sum: The authors have taken a number of papers, most of which found that restrictive non-pharmaceutical interventions had a benefit on mortality, and derive some mathematical estimate from the regression coefficients, indicating less benefit than the papers suggest. The actual numbers produced in the review are largely uninterpretable.
Sources for my information:
Gideon Meyerowitz-Katz, Postdoctoral epidemiologist at University of Wollongong.
https://twitter.com/GidMK/status/1489744749942620162
Andreas Backhaus, Affiliate Research Fellow at the Centre for European Policy Studies (CEPS) in Brussels.
https://twitter.com/AndreasShrugged/status/1488993038915489794
Bruce Y. Lee, Professor of Health Policy and Management at the City University of New York (CUNY) School of Public Health
https://www.forbes.com/sites/brucel...ineffective-against-covid-19/?sh=565aa8a1225b
Peter Hansen, Professor of Economics at Chapel Hill.
https://twitter.com/ProfPHansen/status/1489366511756058626
Mathias Heltberg, Postdoctoral Biophysics researcher at Niels Bohr Institute, University of Copenhagen
https://videnskab.dk/krop-sundhed/s...e-that-lockdowns-only-reduced-covid-deaths-by