A Productivity Commission report finding was based on a fundamental misunderstanding of the nature of disease transmission dynamics, and now that error has itself been transmitted in the media and into the election campaign, write Nic Steyn, Mike Plank, Alex James, Rachelle Binny, Audrey Lustig and Shaun Hendy of Te Pūnaha Matatini.
On May 2, the Productivity Commission produced a report for Treasury titled A cost benefit analysis of five extra days at Covid-19 alert level four. Later released under the Official Information Act, the report used a cost-benefit analysis to argue that the government’s decision to extend alert level four by five days in April was not justified. The New Zealand Herald discussed the article under the headline What’s the real cost of staying safe from Covid-19?
Unfortunately, the commission’s report is based on a fundamental misunderstanding of the nature of disease transmission dynamics and, as a result, grossly underestimates the benefits of the level four extension. When case numbers are low, as they were in late April in New Zealand, disease spread is very stochastic, meaning that there is a probability an outbreak will grow but there is also a probability that an outbreak might fade out.
The commission’s report misses this point entirely. In simple terms, it assumes that the two options (to extend or not to extend) would initially lead us down different paths, but that these would quickly converge to the same endpoint. This takes them to the conclusion that the only benefits from the extension are the lives saved before the two paths join again. This simply isn’t true.
As the report says, at the time the decision to extend level four was made, it was uncertain whether the virus effective reproduction number, Re, would be greater than one or less than one after the move to alert level three. The reproduction number is the average number of people who go on to catch the virus from a single infected person. If Re was less than one at level three, the number of cases would continue to decline, albeit at a slower rate than under level four. But, the report argues, if Re was greater than one at level three, case numbers would rise and we would need a second lockdown sooner or later. The report argues that it was only the value of Re at level three that was important, rather than the time spent under the more restrictive level four.
This is wrong.
The commission didn’t understand that there was a third possibility: the extra time at level four could reduce the number of cases in the community to a level where they would fizzle out of their own accord. The longer you smother a fire, the more chance it will go out even after you turn the oxygen supply back on. What’s more, the benefits of this outcome, relative to the need to impose a second lockdown, are much, much greater than the benefits considered by the commission. This leaves the Productivity Commission’s analysis wildly off the mark.
To capture the benefits of elimination properly in a cost-benefit analysis the commission would have needed a completely different type of model for disease spread. Fortunately, the model we developed to inform government decisions at the time had the capability to do this. At the time our model showed that an additional five days at a higher alert level could increase the probability of elimination by up to 30 percentage points. On this basis, a cost-benefit analysis suggests that the five-day extension reduced economic damage by reducing the chance of another level four lockdown. In other words, the level four extension saved lives and money.
What is more, the extension likely allowed us to move through the subsequent alert levels faster. Treasury modelling showed that alert level four would reduce economic output by 40%, alert level three by 25%, and alert levels one and two around 10% compared to normal. Using these numbers, an extra day at alert level four is worthwhile if it reduces the time to elimination (and therefore level one) by at least one day. The commission suggests the extension could result in elimination five weeks earlier, so an extra day at alert level four might usher in alert level one an entire week earlier.
More recently, a paper from the United States has found that lifting lockdowns early had less economic benefit than expected: spending recovered no faster in places that opened up earlier than in places where lockdowns remained in place longer. It has turned out that if people are still concerned about the risks of catching the disease then they don’t go out and spend.
Another study from the University of Copenhagen found that while Denmark had much stricter restrictions than neighbouring Sweden, both countries have experienced similar economic costs. Sweden, tragically, paid the additional penalty of more than 5,000 fatalities. While this research wasn’t available at the time the government made the decision to extend level four, the benefits of additional certainty provided by the extension was a factor in cabinet’s decision making. This is all outlined in a blog post by … the Productivity Commission.
It is clear the global pandemic is going to be with us for a long time to come. We need robust ways of estimating the combined health and economic costs of different policies. But these need to be built on appropriate modelling assumptions for disease transmission and account for different potential long-term outcomes from different decisions. In this case, the commission should have sought advice or peer review from experts in disease dynamics. If the advice to cabinet had contained a misunderstanding like this it could have been catastrophic for New Zealand.
Unfortunately, the error has resulted in media coverage that misrepresents the social and economic consequences of the decisions made in April. This has the potential to undermine decisions in the future – indeed, the Act Party is already questioning the lockdown extension using the commission’s paper.
But as we have seen globally, resuming economic activity too early can quickly lead to a significant resurgence of Covid-19 cases with enormous health and economic costs. This is certainly much clearer in hindsight, but even at the time of the government’s decision it seemed likely to many people that an extension to level four would bring the benefits that New Zealand is currently enjoying.