Edit: After speaking with a wide range of folks, including some senior data scientists who specialize in epidemiological models, I decided to write and publish an article about what I feel are the flaws in this model, and how it should not be used for planning purposes. All my concerns are summarized here:
Hey all, full disclaimer in advance, I am not an epidemiologist nor am I a particularly skilled statistical modeler. That being said I am extremely concerned about this model from the University of Washington. According to the former FDA commissioner, Dr. Scott Gottlieb, many people in Washington are using this model for planning and prediction. This is extremely troubling to me as the methodology appears to be extremely suspect, and if it is being used for planning, I fear it will lead to a complete misallocation of resources.
I would really appreciate it if people could take a look and reply with their thoughts. Thank you and stay safe.
They assume that this will be contained and that as long as we have 3 of the 4 following measures in place (close schools, restrict travel, shelter in place order, close nonessential businesses) that we will follow a trajectory similar to Wuhan. Wuhan took extreme measures including a full enforced lockdown, substantial use of technology and contact tracing, masks for all, and quarantining infected individuals in the same location. To me, this assumption appears to be completely unjustifiable.
They do not appear to consider the increased mortality rate from places where the healthcare system is overwhelmed. They estimate that NYC will need 11,439 ICU beds at peak, but only 718 will be available (including surge ICU beds). This leaves 10641 people who need ICU but won’t have it, and yet they predict only 798 people will die.
It looks like another professor at University of Washington, Carl Bergstrom, is also very concerned. Here are a few things he has posted:
So Dr. Birx, the White House’s Coronavirus Response Commissioner touted this model at Today’s White House Press Conference (38:22). It appears they are using it to plan their strategy. While she mentions they “assume full mitigation” she doesn’t mention that means that they are explicitly assuming we will follow the same trajectory as Wuhan and fitting a curve to that assumption.
If the government plans to allocate resources based on a peak that doesn’t come, it’s going to be devastating. I’d really appreciate someone looking at this and convincing me I’m wrong, or elevating it to someone who can do something about it.
As Bergstrom notes, it’s a model for what to expect with successful suppression. That’s OK if understood. But he notes, “the shaded regions are being interpreted almost everywhere as spanning the best and worst case scenarios”.
Also, it’s a curve fit. I lack the domain expertise to know if that’s more or less robust than a full SEIR model, given the poor quality of input data right now. Happily, it’s being updated daily. I was going to say that different kinds of models will converge over time, but that’s not necessarily true. As far as I can tell this model assumes one peak, and a logistic tail-off. Bad enforcement can lead to multiple peaks.
That said, all models agree things get much worse without successful suppression. And bad enforcement quickly exceeds capacity to cope.
The code is available on GitHub, so you could modify assumptions and re-run.
My concern is that if you project results based on the assumption of successful suppression, you will paint a picture that prevents the action necessary to actually achieve that suppression.
Again the whole model is founded on an insane assumption, and is being presented to the public without mentioning that assumption. The strongest term I’ve heard used publicly is that the model “assumes mitigation”, but the reality is that it assumes a level of mitigation on par with Wuhan. These are the actions Wuhan took, very early on, with a centralized outbreak:
Suspending all public transport, including airports
Public vehicles were barred from roads without special permit
Officials went door to door performing health checks and isolating the sick
Outings by citizens were strictly limited, and in some places curtailed entirely with only delivery for food
Masks were ubiquitous in Wuhan during this period.
Only residents of buildings were allowed into said buildings.
I haven’t seen this, do you have a link? The only github link I saw was to a curve fitting library they used.
Not a crazy concern, but paradoxically, it seems this projection is being used to justify more of a response than the more dire UK projections based on no intervention. (My ad-hoc observation; I’ve no real insight into the decision-making.)
Insane assumption: the open question is whether “3 of 4 interventions” is enough to get a similar response as in Wuhan. Epidemiologists? My naive understanding is the key is to get R0 < 1. Is 3 of 4 enough? To get a similar response? Or do you need to add masks, delivered food, and health checks?
Code: hmm… it’s specifically developed for COVID-19, but you’re right it doesn’t have the data right there. Argh. Wonder if they use the default assumptions in the code when fitting.