Back in December 2022, the BBC trumpeted a staggering claim: AstraZeneca and Pfizer vaccines saved over 12 million lives in their first year, a figure drawn from Airfinity and rooted in an Imperial College London study estimating 20 million lives saved globally from December 2020 to December 2021. It's a headline designed to inspire awe, a testament to science's triumph over a deadly pandemic. But peel back the layers, and the numbers unravel. Even from a mainstream perspective, ignoring debates about vaccine harms, the methodology behind these claims, built on shaky mathematical models rather than hard clinical data, crumbles under scrutiny. For the millions who trusted the vaccine rollout, the truth matters more than feel-good stats, and the quiet pursuit of evidence deserves better than this house of cards.
The Imperial College study, published in The Lancet Infectious Diseases (June 2022), used a mathematical model to estimate that Covid vaccines averted 20 million deaths globally by assuming 88% protection against infection (post-second dose for mRNA vaccines) and reduced severity of disease. It leaned on all-cause excess mortality data, modelled by The Economist, and assumed uniform infection fatality ratios (IFRs) and reproductive number (Rt) trends across countries. For the UK, with 68% vaccine coverage, it claimed 507,200 deaths averted (52 per 10,000 vaccinated). Italy, at 73% coverage, supposedly avoided 491,300 deaths. The USA, with 61% coverage, was credited with preventing 1,902,000 deaths.
But these numbers strain credulity. In the UK, 2021 saw 667,479 deaths, down 22,150 from 2020's 689,629. The model implies that without vaccines, deaths would have skyrocketed to 1,174,679, a 76% jump, despite lockdowns, natural immunity, and a virus that was already mutating to less lethal variants like Omicron. Italy's 2021 deaths (701,346) would've hit 1,192,646 without vaccines, a 70% increase over 2020. In the USA, 2021's 3,464,231 deaths would've ballooned to 5,366,231, a 55% surge. These projections assume catastrophic death tolls that don't align with observed trends, especially as non-vaccine factors like herd immunity and improved treatments played significant roles.
Models aren't evidence, they're guesses dressed in maths. The Imperial study's assumptions are shaky: uniform IFRs across diverse populations, no detailed vaccination data for most countries, and an overreliance on excess mortality without accounting for non-Covid factors like healthcare disruptions or aging demographics. As The Spectator notes, models don't appear in evidence hierarchies like those of the Centre for Evidence-Based Medicine or SIGN, which prioritise Randomised Controlled Trials (RCTs) for therapeutic claims. Regulators like the FDA and NICE use clinical trial data, not projections, to approve drugs or vaccines. Yet the BBC ran with Airfinity's numbers, unchallenged, despite their reliance on a model that The Spectator calls "implausible" for inflating death tolls beyond reason.
Clinical trials, like those for Pfizer and Moderna, showed efficacy against symptomatic infection (95% and 94%, respectively) in controlled settings, but real-world data on preventing deaths is murkier. Observational studies, like those from Israel in 2021, suggested vaccines reduced severe outcomes, but they couldn't quantify millions of lives saved globally, especially as variants, natural immunity, and treatments like monoclonal antibodies shifted the pandemic's trajectory. The Imperial model ignores these complexities, assuming vaccines were the sole driver of lower mortality. It's like crediting a single chef for a feast cooked by a team.
For the average Briton, American, or Italian, these numbers aren't just stats, they're the backbone of trust in public health. Picture a London nurse, exhausted from 2020's chaos, taking the vaccine to protect her patients, only to hear later that the "lives saved" figures were hyped. Or a Chicago retiree, wary of mandates, wondering if the data justifying them was more hope than fact. The UK spent £4.7 billion on vaccines and boosters by 2022, per NAO estimates, while families faced lockdowns and economic strain. If the 12 million lives saved claim is a mirage, it erodes trust in institutions already battered by mixed messaging on masks, mandates, and boosters.
The human toll extends to policy. Governments, leaning on these models, justified sweeping measures, vaccine passports, workplace mandates, that reshaped lives. In New York, where trust in public health is fragile, inflated claims fuel scepticism, especially among communities hit hard by Covid's economic fallout. The quiet life, where people could make informed choices without fear of coercion, feels betrayed when science leans on speculation over evidence.
The BBC's uncritical reporting, echoing Airfinity's 12 million figure, reflects a broader failure. Journalists, as The Spectator laments, rarely check the math. Posts on X, like those from @DrAseemMalhotra, call out the "grossly exaggerated" claims, pointing to real-world data showing declining mortality before vaccines rolled out widely. The media's rush to amplify big numbers, without probing their source, drowns out the call for rigorous trials. It's not just sloppy, it's a disservice to the public, who deserve transparency over triumphalism.
If vaccines saved millions, why rely on models? Large, well-designed Random Controlled Trials (RCTs) could have tracked real-world outcomes, comparing vaccinated and unvaccinated cohorts across diverse populations. Instead, the Imperial study's GitHub data, openly available, but dense with assumptions, became the gospel. This isn't science; it's storytelling. As The Spectator puts it, "Models do not fit anywhere in the pathway for establishing effectiveness." When billions of lives and dollars are at stake, "bad guesses" won't cut it.
The claim that Covid vaccines saved 12 million lives in a year is a house built on sand. Even from a mainstream lens, ignoring debates about harms, the Imperial College model's inflated projections, 507,200 deaths averted in the UK, 1.9 million in the USA, defy real-world data and lean on unproven assumptions. Clinical trials, not mathematical guesswork, should anchor public health claims. For the nurse, the retiree, the taxpayer, the truth matters more than headlines. If vaccines were a triumph, let the evidence, not models, prove it.
https://www.spectator.co.uk/article/did-covid-vaccines-really-save-12-million-lives/
"The BBC reported that AstraZeneca and Pfizer are credited with together saving more than 12 million lives in the first year of Covid vaccination. To substantiate this claim, the BBC refers to Airfinity, a 'disease forecasting company'.
Models do not fit anywhere in the pathway for establishing effectiveness
Airfinity used an Imperial College London study, which calculated that Covid vaccines saved 20 million lives between December 2020 and December 2021. Using a mathematical model, the Imperial team assumed that vaccination conferred protection against Covid infection (mRNA vaccines were estimated to have given 88 per cent protection against infection after the second dose) and the development of severe disease requiring hospital admission. The team also assumed that those who develop infection are less infectious than unvaccinated individuals.
We downloaded the supplementary data from GitHub (where academics often store their computer code) to examine the country-specific estimates and assess the plausibility of the numbers for the UK, Italy, and the USA.
In the UK, with vaccine coverage of 68 per cent, 507,200 (382,200 to 789,900) deaths were estimated to be averted because of vaccination (52 deaths prevented per 10,000 vaccinated). In Italy, the modellers predicted that, with a coverage of 73 per cent, 491,300 (444,800 to 544,400) deaths were averted (53 per 10,000 vaccinated). In the USA, with 61 per cent coverage, 1,902,000 (1,737,000 to 2,069,000) deaths were averted (44 per 10,000 vaccinated).
The Imperial team fitted a model to all-cause excess mortality, using more modelling from the Economist. They also assumed that each country would follow the same time-varying reproductive number (Rt) trend since the start of the pandemic. Despite an absence of detailed vaccination data for most countries and an assumption that the relationship between age and the infection fatality ration was the same for each country, the BBC still used the numbers.
In 2021, there were 667,479 deaths in the UK, 22,150 fewer than the 689,629 deaths in 2020. By our reckoning, the modellers want us to believe that in the absence of vaccination, there would have been 1,174,679 deaths in the UK in 2021.
Similarly, in Italy, 701,346 deaths occurred in 2021; the model predicts the number would have been 1,192,646 and 452,329 more deaths than in 2020. In the USA, a total of 3,464,231 resident deaths were registered in 2021, 80,502 more than in 2020. The modellers assumed there would have been 5,366,231 deaths in the US in 2021 if vaccination had not been implemented.
Sadly, many journalists don't check their numbers or facts: many of the assumptions in the model are incorrect, and the estimated number of deaths averted by vaccines is implausible.
This isn't surprising. As in medicine, models do not fit anywhere in the pathway for establishing effectiveness. Regulators don't use them for approval, and decision-makers like the National Institute for Health and Care Excellence use economic models with reliable estimates of effect and credible costs. They do not appear in the The Centre for Evidence-Based Medicine levels of evidence or the Scottish Intercollegiate Guidelines Network system for grading evidence, as they are irrelevant to answering therapeutic questions. Clinical trials are the primary study type to determine the effectiveness of medicines or vaccinations; anything else is just a bad guess.
This post was written by two old geezers who will explain why observational data-based estimates of effect should be taken with a grain of salt. Models simply should not be used. Large, well-designed, well-reported, and data-accessible trials should be used for global public health interventions. So, why are we using models to justify decisions?