Risk, Uncertainty, and COVID-19 Strategies
The current crisis has highlighted the risks associated with untamed uncertainty, as well as those associated with under- or overestimating the impact of measures intended to combat COVID-19.
Former World Bank President Jim Yong Kim recently argued that “[n]o one in the field of infectious disease or public health can say they are surprised about a pandemic.” And yet, the COVID-19 outbreak did take most policymakers very much by surprise. From their perspective, the situation was still one characterized by the kind of radical uncertainty highlighted by economists such as Frank Knight and George Shackle: Policymakers were simply unable to assess the possible consequences of action and inaction, and this made informed cost-benefit analyses of alternative (probabilistically assessed) outcomes impossible. One thing was, however, clear: The consequences of a runaway pandemic could be disastrous.
In such a situation, the precautionary principle tends to apply. As a prominent member of the Danish parliament told us in mid-March: “This is a natural disaster in slow motion. We basically know nothing. The only rational thing to do is to shut down entirely.” That was six weeks ago. At the time of writing, we are already in a very different situation. Now that many more data points are available, sophisticated cost-benefit analyses are emerging, and the curves depicting those who are hospitalized, in intensive care, or dying are flattening or even dropping in many places. We are moving from a situation of radical uncertainty to one that is in many respects more like a risky situation—one in which we can be fairly confident of what the likely outcomes of our actions and the associated costs and benefits will be.
This shift has occurred as a result of observing new developments and analyzing the data those developments produce—testing and re-testing infected individuals, recording and comparing hospital data across countries and regions, and so on. This gradual transformation of uncertainty into risk (and finally into certainty) is not a process that follows any predetermined path. Rather, it is a search for the right responses partly shaped by existing institutions and policies. We now know enough to identify some important challenges to this process, including some pitfalls, familiar from the literature on cognitive biases.
The dramatic early pictures from Lombardy, in particular, including footage of military trucks transporting corpses out of Bergamo, received extensive coverage on mainstream as well as social media and were widely cited by policy makers. So were the alarming outcomes predicted by the widely cited Imperial College simulation study (based on still undisclosed code written to simulate a flu epidemic 13 years ago). Focusing on these early reference points to the exclusion of other important considerations arguably sent information gathering down suboptimal paths, a problem compounded by political and institutional factors, and by poor risk literacy among decision makers as well as the public. This has hampered the sober assessment of costs and benefits of exit strategies from the current measures.
In what follows, we examine ways of taming the uncertainties associated with the crisis. We identify likely mistakes in the global search for responses, what we can learn from them, and how we can move forward to create viable and publicly accepted exit strategies.
Unknown unknowns, known unknowns, and known enough knowns
At a press briefing in 2002 as America prepared for the invasion of Iraq, US defense secretary Donald Rumsfeld famously drew a distinction between “known unknowns” and “unknown unknowns.” Rumsfeld was ridiculed in some quarters, but this insight into the nature of risk assessment (which was not original to him) is profound, and may be applied to our current situation. COVID-19 was initially an unknown unknown that quickly became a known unknown once the new disease and the novel coronavirus that caused it were identified.
As the death toll climbed rapidly in Italy, the possibility that similar catastrophes were about to unfold in other cities led governments to assume the worst. But the nature of their responses in this phase were very different. The Taiwanese authorities acted immediately, calling a crisis meeting at government level in December 2019, the same day the authorities learned of an emerging epidemic in Wuhan, China. By contrast, the Danish health authorities were still reluctant to issue warnings against going skiing in the Italian Alps by late February 2020. Retrieval biases might have played an important role in the response patterns: Taiwan had been hit by the SARS epidemic in 2003 (another contagious respiratory illness which also originated in China), while Denmark’s only recent experiences with epidemics were its annual bouts of influenza.
In any case, when the potential threat of the pandemic became clear, responses necessarily had to be made in circumstances of deep uncertainty—most of the costs and benefits of response measures could not be meaningfully assessed, because so much of the information required to do so was still missing or unclear. The early dynamics of the crisis were unpredictable because almost nothing was known about the epidemiological parameters of the novel virus, the possible role of super-spreaders, or the nature of contact patterns. The complex repercussions for the economy were also unknown and unknowable. As a result, optimal response strategies (whether in terms of saving as many life-years as possible or minimizing harm to GNP) could not be devised.
However, once COVID-19 became a known enough known, the knowledge accrued thus far could be employed to stabilize the global economy and political systems, to reduce the death toll, and to mitigate the economic and personal trauma caused by the effective shutdown of large swathes of societies.
Avoiding pitfalls in early crisis policy responses
When faced with a situation like the COVID-19 pandemic, political leaders and their advisors must embark on a rapid process of learning and discovery necessary to inform urgent decision-making. However, this process takes place under the impact of threat responses that are likely to disfigure decision-making in various ways. In the weeks after the dramatic pictures from Lombardy circulated the world, many epidemiological experts offered policymakers worst-case scenarios and proposed far-reaching measures to fight the pandemic. This is understandable—these experts were dealing with an unknown virus, and their responsibility at such times squarely lies in preventing a foreseeable medical disaster.
Less understandable is why policymakers relied exclusively on these experts and rarely formed interdisciplinary teams to think through related global health, economic, political, and social dynamics simultaneously. The response strategy that most countries followed was based on the following heuristic: Listen, almost exclusively, to epidemiological experts, and place great emphasis in decision-making on the scenarios that emerge from pandemic modeling. These tended to recommend shutting down societies in order to meet social distancing requirements intended to arrest the spread of the virus. This initial collective response to the COVID-19 outbreak relied on relatively few experts from a narrow range of disciplines and institutions. These included, notably, the simulation study provided by the response team at Imperial College London and the early fatality rate figures provided by the World Health Organization (WHO).
This approach, however, can be problematic. The Imperial College study has been criticized for focusing primarily on the benefits of the courses of action it examined, at the expense of their corresponding economic and social costs and consequences (which can also be lethal). The WHO, meanwhile, has been criticized for failing to perform in-population studies early on (some of which are now underway) that might have assisted a cost/benefit analysis of measures proposed to combat the virus. Consequently, two mistakes in the global policy response at this stage can now be identified:
- The early shutdown was used to shore up the capacity of the medical system, planning for the worst, but an opportunity was missed to manage the transition from a known unknown to a known enough known.
- The political dynamic of the crisis created a spiral of confirmation biases and escalating commitment. Absent was the management of attention needed for policymakers and the wider public to maintain a wider perspective, enabling them to compare the costs and benefits of different courses of action, or at least to work towards making such comparison possible.
The associated risk is that the second stage of the crisis response will now be driven by the pursuit of political advantage taking rather than the search for a response which is of greatest benefit to society.
In the wider public, the rationale behind the extreme shut-down measures (“flattening the curve”) was initially accepted. Indeed, graphs measuring new infections and deaths became important visual instruments for educating citizens in the logic of exponential growth. However, the public was not educated in elementary statistics and the dangers of sampling biases (for example, the fatality rates initially communicated in the media and by the WHO generally overestimated the number of infected people dying, because only the sickest patients were being tested). This effectively prevented them from being empowered to make their own risk assessments.
While epidemiological experts did try to educate the public and allocated ample time to this, they did not always manage to communicate the seriousness of the situation without slipping into counterproductive alarmism. When case numbers in Germany were still in three digits, the virologist Christian Drosten from Charité Berlin, one of the key advisors of Chancellor Angela Merkel, frightened the public by declaring “es wird schlimm werden.” (roughly, “It will be nasty.”) Merkel herself judged it to be the most severe crisis since the Second World War. This kind of language together with the logic of exponential growth of infections and deaths, arguably contributed to a widespread willingness to accept the drastic measures that were necessary in the early days of the pandemic. But it had unintended consequences, too.
First, frightening statements from leaders and health officials may help to impede logical thought which, in turn, can cause conspiracy theories to proliferate. On social media and in the blogosphere, doubts about the existence of COVID-19 circulate, while others link the ongoing public health crisis to the 5G network rollout. Others have taken to accusing Bill Gates of masterminding the pandemic as part of a global vaccine profiteering scheme, or to postulating that the shutdowns represent coordinated action to implement a lasting totalitarian political system. Alternatively, the public commitment to draconian measures may become too strong, causing citizens to resist their relaxation when it becomes safe to do so or tolerate wildly disproportionate enforcement. In Serbia, citizens have been jailed for several years for breaking lockdown rules. In general, the prevailing uncertainty during the lockdown weeks has given rise to many worrying political and moral responses from individuals. Some judge everyone who questions the measures to be selfish or lunatic. Others are losing their trust in democracy.
A more balanced and cooperative approach would be for those who are afraid to allow others to question the measures, and for those who are doubtful to obey to the measures anyway for the time being. A common understanding that we are all adapting to a rapidly developing situation that we need to think through ourselves may prevent many from expecting a political disaster. The question “Where will all this lead?” can, and should, be answered early on. After all, the current crisis is hardly a “black swan” event, and valuable lessons can be drawn from the dynamics of known precedents. Policymakers should be more forthcoming about prevailing uncertainty by replacing apocalyptic predictions with candid admissions that “We don’t know how bad it will be” and circumspect pronouncements such as, “Until time point X when we have data Y we will not know if there is a need for long-term measure Z.”
Taming uncertainties
With the benefit of hindsight, we can point to a number of policy failures (such as the failure to expedite widespread, regular testing to gauge the spread of the pandemic) or worse (the early attempts of Chinese authorities to suppress information relating to the coronavirus). However, decisions in the early days were being made in the dark, and similar epistemic conditions are likely to characterize decision-making situations when the next truly grave crisis occurs. Even though we are still operating in a highly uncertain situation, we have learned enough that a few suggestions for comparing and selecting exit strategies may be offered.
First, it is better to rely on simple rather than computationally sophisticated modeling. Simulation models are valuable when studying epidemics, but they are ill-suited to formulating policies while a pandemic is ongoing. Policymakers, leading experts, and an educated part of the wider population must be able to draw on a “common currency of understanding” from which to derive policy responses—it must be clear under what conditions certain scenarios prevail. Complex simulation models are opaque, but readily comprehensible datasets from testing create a common epistemic currency for anyone conversant in basic statistics. Of course, it will still be necessary to explain how the tests work, how reliable they are, if people who are infected can get reinfected, and so on. But by focusing on in-population random sampling, these points can be resolved in public debate. In sum, we need simple models that enable policymakers and the public to think. We should not outsource thinking to epidemiologists, who may be under pressures that prevent them from seeing the big picture.
Second, given some knowledge of population-wide effects, we can use simple statistical comparisons to check upper and lower bounds of impacts of policies. For instance, once the virus’s infection fatality rate is confirmed, likely projections of total deaths are straightforward to calculate. We can already deduce plausible lower bounds for the economic damage caused by suppression measures by carefully deriving lower bounds of expected unemployment, and by studying the known consequences of shut-downs and social distancing. That unemployed men have significantly higher suicide rates, for instance, is now a very well supported claim. We also know that deaths from alcoholism and/or nicotine addiction are correlated with economic recessions and confinement measures. The problems caused by COVID-19 policy measures are not likely to be very different.
Third, we need to be aware that what we are studying is both simple and complex. Exactly how COVID-19 is transmitted and exactly how it affects the human body are two complex questions that are unlikely to be answered soon. However, it would be a big mistake to say that such obstacles should stop us from simple in-population risk modeling. Current modeling attempts should focus on knowable aggregate effects, not the details of mechanisms. Even if a phenomenon is complex (like the transmission of the common flu), some aggregate properties are stable (like its in-population fatality rate given consistent conditions). By combining statistical knowledge about aggregate effects with counterfactual, hypothetical thinking, we may be able to learn even more. For instance, why has the Swedish experience so far not reflected that of Lombardy or Madrid, even though no tough measures were taken by the Swedish government? Once we have forwarded testable hypotheses in response to questions like these, we can try to support and falsify them.
Fourth, the rhetoric used to justify extreme measures needs to be very carefully chosen. While every policymaker wants to avoid the appearance of complacency or sleepwalking into a public health disaster like the one that befell Lombardy, comparisons with war-zones evoke fearful responses in publics that can run in unpredictable directions and become difficult to control. This in turn undermines the rational reflection that enables individuals to cooperate voluntarily. It is important to realize that, if we do not tame the uncertainties associated with COVID-19, the social, economic, and political risks associated with the countermeasures create unbounded risks. Not enough attention has so far been paid to the fact that institutions like the European Union, the postwar liberal order, and in some places, even democracy itself, are gravely threatened by this crisis.
Jesse Pelletier
Conclusions
Taken together, we would like to propose a preliminary heuristic for taming uncertainties and managing the crisis response as unknown knowns become known enough knowns that allow for policy formulation. For addressing known unknown situations with the capacity to cause great suffering and harm, a rough heuristic should include the following elements:
- After observing a threat (say, the situation in Lombardy in February and March), take drastic short-term measures. This may mean shutting down social life to buy time in which to take emergency measures. However, this phase should be limited to a very short period of, say, two weeks and must go hand in hand with a fully committed effort to adjust policy responses as more and better data become available, and countries move from situations of radical uncertainty to situations of informed risk.
- Just after the initial short period of drastic measures, communicate a preliminary set of questions that need to be answered to tame uncertainty, and communicate prospective dates by which they can hopefully be answered. Be cautious not to create common emotions of panic with inflammatory or alarmist comparisons (like “most severe crisis since the Second World War”); if it remains a known unknown, make comparisons based on a risk comparison, see point 4 below).
- Manage the transition from an emergency mode of policymaking to committed contingency planning. In particular, the emergency mode should be discontinued once remaining uncertainties are comparable to other known sources of uncertainty. Of course, COVID-19 could mutate, and/or create unknown harmful effects. But similar uncertainties apply, for instance, to new technologies.
- Balance long-term measures designed to mitigate the impact of COVID-19 with the need to address other and related sources of hardship and suffering that those measures may unintentionally exacerbate, such as deaths as consequences of loneliness, untreated other illnesses, unemployment, and so on.
In sum, the current crisis is a reminder that democratic, free societies require individuals who are empowered to form their own deliberate viewpoints, and cooperate to create and protect society and one another. Managing knowledge needed to inform policy responses and individual behavior is an important component of such empowerment. The current crisis has highlighted the risks associated with untamed uncertainty, as well as those associated with under- or overestimating the impact of measures intended to combat COVID-19.
