In the past 24 hours, there have been approximately 1,800 reported COVID deaths in the United States.
USA COVID-19 stats as of 08:30 AM on May 15, 2020
? Cases ?: 1,458,243 (+117.0)
☠️ Deaths ☠️: 86,942 (+5.0)
Updated GU,PR since 08:00 AM on May 15, 2020— USCovidDeathBot (@USCovidDeathBot) May 15, 2020
Today is May 15th.
The Council of Economic Advisors used an canned Excel tool to project that there would be virtually no deaths today just 10 days ago.
To better visualize observed data, we also continually update a curve-fitting exercise to summarize COVID-19's observed trajectory. Particularly with irregular data, curve fitting can improve data visualization. As shown, IHME's mortality curves have matched the data fairly well. pic.twitter.com/NtJcOdA98R
— CEA (@WhiteHouseCEA) May 5, 2020
Many people and sites, including Balloon-Juice did an immediate spit take of WTF-ery.
For there to have been no deaths in the past twenty four hours would strongly imply that there were no infections after the last couple of days of April. We know that COVID kills comparatively slowly. There is usually a significant gap between infection and testing, and then testing and hospitalization and finally between hospitalization and death.
So besides dunking on the CEA, what can we learn from this?
There are an incredible number of models out there. Some are trying to project infection rates. Others are attempting to predict hospitalization capacity. More are coming online to identify the safest way to re-open up limited physical interaction. Some of these models are going to be good. Some are going to be great at one specific task and useless for everything else, and others are going to be cubic fit dumpster fires.
How do you tell the difference?
The first thing we, as consumers of projections, need to do is know what a particular model is attempting to do. We need to know that a hospital bed projection model should be assessed on hospital bed demand first and foremost. A limited model should not be generalized past its own limits.
Secondly, we need to look into the assumptions that every model makes. Do those assumptions make inherent sense given what we have learned? We know that COVID is a slow killer. We know it is fairly easily spread when people are in prolonged contact in enclosed spaces. We know a lot. Do the assumptions violate what we know? Also do the assumptions make heroic assumptions about political and social behavior that are unlikely to be fulfilled? Does the model assume the US will go to Wuhan or Italian style sheltering in place and hold those orders for the entire summer? That is unlikely (the Italian version of opening up is the equivalent of the stricter US shelter in place orders).
Do the models reflect new information well? The approval of remisdivar is the equivilent of several hundred thousand hospital days avoided which is the equivilent of a few dozen hospitals and attendent workforces the size of Massachusetts General being magically constructed. Do the models adjust as we learn more and new technologies and knowledge change courses of treatment?
Do the models do well in predicting the short term? Tomorrow should be easier to predict than August. Does a model predict the near future well or are the modelers creating a bunch of epicycles to explain their misses? Is uncertainty expressed well? We are still on the OMG learning so much every day part of the knowledge curve right now. There is a tremendous amount of things that we just don’t know. Every model is going to be wrong, but does the model and modelers acknowledge that they are going to be wrong and give you parameters of plausible wrongness? Again, as a worker assumption, tomorrow is easier to project than August so confidence intervals should reflect that.
Finally, pulling back a golden oldie from the dawn of the blogosphere: Daniel Davies One Minute MBA
Fibbers’ forecasts are worthless. Case after miserable case after bloody case we went through, I tell you, all of which had this moral. Not only that people who want a project will tend to make inaccurate projections about the possible outcomes of that project, but about the futility of attempts to “shade” downward a fundamentally dishonest set of predictions. If you have doubts about the integrity of a forecaster, you can’t use their forecasts at all. Not even as a “starting point”. By the way, I would just love to get hold of a few of the quantitative numbers from documents prepared to support the war and give them a quick run through Benford’s Law.
Application to Iraq: This was how I decided that it was worth staking a bit of credibility on the strong claim that absolutely no material WMD capacity would be found, rather than “some” or “some but not enough to justify a war” or even “some derisory but not immaterial capacity, like a few mobile biological weapons labs”. My reasoning was that Powell, Bush, Straw, etc, were clearly making false claims and therefore ought to be discounted completely, and that there were actually very few people who knew a bit about Iraq but were not fatally compromised in this manner who were making the WMD claim. Meanwhile, there were people like Scott Ritter and Andrew Wilkie who, whatever other faults they might or might not have had, did not appear to have told any provable lies on this subject and were therefore not compromised.
Ignore known liars, charlatans and frauds.
Pooh
I come back to that blog post so frequently. Another maxim that I find stunningly useful is “Good ideas don’t need lies to sell.”
Jeffro
ie, don’t listen to Republicans or anything they’re promoting.
Another Scott
That Daniel Davies bit is great. Thank you.
Excellent post.
As I mentioned last night, in #28, I’m going to be keeping an eye on the ratio of Recovered Cases to Active Cases. Until that ratio is above around 1.0, it seems reasonable to conclude that we’re not really “over the hump”. (Of course, lots of caveats, etc., etc., but it seems a reasonable quick and dirty metric.) It doesn’t predict anything, it just asks “where are we now compared to countries that are doing well?”
Cheers,
Scott.
Cheryl Rofer
I’ve given up looking at the models in detail. I check out something about one or another occasionally, but I’ve been very disappointed in epidemiologists’ modeling capabilities.
They’ve all got their pet models, and none is transparent. The only model I’ve seen that has any possibility at all of pulling the effects of various factors out is the Imperial College model, and that seems to have slipped down the memory hole. In fact, I see few projections from any models any more. Could be because I’ve given up looking at the various websites.
Compared to climate models, say, these are not complicated models. Something that would give me more confidence in the modelers’ ability and good faith would be to see the modelers themselves comparing their results and trying to figure out why they differ. There are a couple of sites comparing or averaging model results, but those are outsiders and probably have only a little more understanding than I do of what the models are doing.
So I’ve got my own model in my head, and it says 100,000 US deaths by the end of May, with upticks around the country where we’ve had the gun nuts “protesting” and restaurants and bars packed. It says that we are going to have a very hard time controlling this until we have a government that wants to do that. It says that, unless a miracle happens with a vaccine, we are in for a very bad time for at least the next 2-5 years.
ETA: And yeah, don’t believe known liars.
Sab
@Cheryl Rofer: Thank you for saying the ugly stuff out loud. TV is all in for 12 to 18 months. I have been thinking 3 years. I think your 3-5 is more realistic.
ETA :It really will not be helpful if people are totally surprised when reality kicks in.
mapaghimagsik
I’ve been laying models on top of each other, and outliers tend to get discounted pretty quickly. I’m less in the predictive biz than trying to figure out what’s going on, and also try to figure out where the lies will break down.
I also tend to like data sources that tell you where they get their numbers from. I saw one, which operated high on the counts, that didn’t seem clear on where they were pulling their data from *mumble mumble John Hopkins* (I’m pulling the John Hopkins data, and their numbers were just a smidge higher, and then my feeds were filled with their links with BREAKING from various people)
The City of San Francisco has been really good with their data, and I’ve come to respect the Socrata API
I would love to get a timeline of events, maybe tied to location, to see if there are bumps. The day job kind of keeps me from doing that, it seems. I might want to start, though, and see what I can get.
MattF
Davies is an interesting case because he’s a finance guy and the financial industry is a rich source of errors and fallacies in modeling– ‘rich’ in the sense of plentiful as well as ‘rich’ in the sense of highly remunerative. The ‘technical’ modeling of stock market ups and downs that you see on teevee is patently, if not hilariously bogus, and yet you see it all the time.
Brachiator
I prefer charts which simply track the progress of the pandemic and don’t try to project future events.
I suppose someone could provide some fancy visual projections that allowed variation based on different assumptions (impact of opening up too soon, reduced social distancing, etc)
Projections are messy, and it’s not so much that they can be “wrong,” as that as with polling, some people (including the people doing the projections) confuse prediction with unerring prophecy.
New Deal democrat
I hope a reporter (preferably a young woman of color) confronts Trump with this today, particularly since Trump and his advisors were supposedly relying on this model.
Notice that the IHME model has (deservedly) also pretty much disappeared from the discourse.
While the models were very useful in March and early April – even IHME, which correctly called the peak in infections for early to mid-April – they are nearly useless now.
Why? Because now the decisive component is not the virus, but human behavior. And as I always point out, the humans always observe back, changing or amplifying their behavior based on the forecast of the model. If I had to guess, I would say there will be a renewed rise in infections over the next 4 weeks, leading some people in the most affected areas to go back to social distancing and self-quarantining. Beyond that, it is an utter chaos of behavioral feedback loops.
Redshift
@Cheryl Rofer:
I appreciate people attempting to do a 538-ish poll averaging on the models, but if there’s no way to evaluate the input models, it will just be garbage in garbage out. Even worse if they just average instead of doing something more sophisticated, since averages are sensitive to outliers.
Comparisons seem more useful.
MomSense
I’ve been operating under the assumption that we will all, or nearly all, get this virus. The point of staying at home was to slow the spread so we don’t overwhelm our hospitals and to give our government time to put a rigorous testing regime in place. Now we know the government had no intention or ability to put a testing regime in place so they decided to just pretend it’s safe to open. We also know they have no intention of supplying adequate PPE and funding. And now, thanks to Bright, we know that even if we create an effective vaccine we won’t have enough fucking syringes to administer it.
So what the fuck are we going to do?
Cheryl Rofer
@Redshift: Exactly
Cheryl Rofer
@MomSense: Vote for Biden.
The only hope I see for better management is when Biden becomes President. That means that a number of bad things will be baked in, although there may be some movement in the other direction, like when the gun fetishists start dying out.
There will be many more reports like this, and the media will stop paying attention to them in a month.
We can hope that scares like this and deaths will eventually convince the MAGAts that their god has feet of clay and they will start using masks and social distancing. If they don’t, I can see a half-million deaths by the time Biden is sworn in.
StringOnAStick
Wait, in your paragraph that starts with “secondly” it says ” we know it is not a aerosol spread”; um, typo?
Brachiator
@Another Scott:
This is very interesting, but I think that you should factor in the number of deaths as well.
But I remember reading your original post on this and the comparisons of the US and other countries seemed very reasonable.
Brachiator
@Cheryl Rofer:
A recent NPR Planet Money episode interviewed an American Chinese restaurant owner who was using tips from friends and relatives in China as he adapted his business for re-opening.
I think that during a phone call with one of the relatives in China, it was noted that restaurants had to note the name and other contact details of restaurant patrons. This would make it easier for officials to track the course of any subsequent outbreak and to contact people who might be infected.
I doubt that the potential invasions of privacy here would not be tolerated in the US, but maybe there is some middle ground?
From some of the dumb hostile reactions of people even objecting to wearing masks when in a store, I fear that a critical number of idiots will continue to be in deep denial or resist reasonable efforts at mitigation if there are any significant outbreaks of the virus.
StringOnAStick
I talked my former co-worker (dental hygienist; I quit rather than go back to such a high exposure job) yesterday; the office reopened this Monday. I am terrified for her and even more sure of my early retirement decision.
They get 1 N95 mask per day and were instructed to wear a plain mask over it to “preserve” it. Being properly trained and fitted to wear an N95, as required by OSHA, is not being done. She was told that OSHA has waived that requirement for now. There’s lots of other stuff, like how I was correct to predict that almost only right wing nutters would be coming in for cleaning appointments so you are treating people who think this is all a joke and aren’t taking proper precautions in their lives, and now they’re laying in your chair wanting you to spend 50minutes inside their mouth while they breathe on you. No thanks.
Also, the parking garage is closed for structural issues so employees have to park far away and take a crowded shuttle bus to get to the building. No, and fuck no.
Wag
@StringOnAStick: I think the distinction is based on droplet size. Aerosols are smaller than respiratory droplets, and stay suspended in the air for longer periods of time than larger droplets.
OzarkHillbilly
Die.
Amir Khalid
@Another Scott:
I just calculated the world’s recovered: active cases ratio at 0.68:1, the USA’s at 0.30:1 and Malaysia’s at 4.17:1. ETA: based on Worldometer numbers.
Brachiator
@StringOnAStick:
I had to go to the dentist yesterday for a serious tooth issue. The dentist mentioned some of the additional procedures his office was taking to try to protect themselves and patients. Lots of new protective equipment. And I heard him talking to one of the staff about the need to discard gowns? (he may have used another word, but I forget) regularly, not to put a gown on again once it had been taken off, and other procedures.
I also think he mentioned using ozone and ultraviolet light after hours on the office and equipment.
This is in Southern California. I don’t know if other dental offices are equally meticulous, or if there is a uniform standard that they are required or suggested to follow.
pat
@StringOnAStick:
That jumped out to me as well. I assume a typo.
Mike R
@MomSense: Put one foot in front of the other and take the proper safety steps and endure. Then vote these worthless scum out of office. We all die but we don’t have to give up, and succumb without a fight.
Mike R
@Brachiator: Had similar experience in Nebraska. Dentist allows one patient in at a time and they are cleaning and disinfecting between visits. Waiting room empty staff covered in ppe.
SFAW
I made the mistake of going to the CEA twitter feed. After the first couple of tweets, it was full of various luminaries (e.g., Ari Fleischer, Ivanka) doing variations of the “Trump is our Dear Leader, and only he can save us!!!” campaign bullshit.
Nice to know that a governmental group is shilling for the Murderer-in-Chief.
jl
” How do tell the difference? ”
Cannot tell unless you can state in plain English what you want to use the model for.
Do you want to passively observe a process to measure correlations?
Do you want to evaluate interventions made by others?
Do you want yourself to intervene and control the process?
Do you have a set of preferred causal models that can explain the correlations?
All statistics is curve fitting at some level, just depends on how much of the process you want to wrap up into a ball for the curve fitting. Where do you do the curve fitting in your model and why? What relationships will be preserved when you move your curve fits to other populations and forward into the future?
So, that is IMHO on it. May sound complicated, but in the stats business all complicated things are just a lot of very simple things strung together. And the most important thing is that you can state the items above in simple English.
ziggy
Yeah models for this don’t seem very effective, because the inputs are not predictable at all. You have human behavior, legislation, nut jobs, economic desperation. On the other hand you have the virus and it’s behavior, which is not completely understood. Throw in lack of availability of medical supplies, testing materials, medicines, it is a mess.
The virus has not exploded (yet?) in many areas (states and other countries) like we thought it would. It seems to me that there is a seasonal component that is tamping it down, and it just isn’t as virulent as it was even a month ago. Hospitalizations are falling even if case numbers aren’t. This is totally a non-scientific, fresh out of my a** observation, but also based on the behavior of the 1918 epidemic (which has so many parallels it is uncanny).
But also based on the 1918 epidemic, and what Rick Bright had to say, I am terrified of what could happen this fall. The virus has a good toehold on this country now.
ziggy
I appreciate these “stories from the front line”. To corroborate your observations–I have no intention for going in for teeth cleaning, any other procedure, until I feel it is absolutely imperative. I’ll just work harder at home dental hygiene. If dentists can hold on, they’ll have a ton of work in the future!
Just One More Canuck
@Another Scott:
That’s an interesting way of looking at it, and one that makes me feel somewhat better as I sit here in Canada. I’m in Ontario, and while things could be better, we are getting there. The problem here is in nursing homes – they have 11% of the cases in Ontario, but over 70% of the deaths -it’s even worse in Quebec.
laura
@Mike R: I had my orthodontist appointment yesterday that had been postponed 6 weeks. Very strict instructions to stay out of the office until 5 minutes of the start time. I showed up at the start time with my mask on. I was the sole patient. Waiting room cleared of reading material. Very limited seating and a giant air purifier – which I stood right next to. Called to the counter and received a big squirt of sanitizer before being handed the rights and advisements sheet. A couple of electronic signatures later I was escorted to the operating suite. Plastic coverings on the chair but first made to scrub hands with a foamy disinfectant and then a 1 minute rinse with a mouthwash peroxide combo. Orthodontist and assistant were gowned masked gloved and distant as he explained the safety measure being utilized. They have 8 operating suites off a corridor. I was at the end left. Only 4 appointments per day. No return to a suite after that day’s appointment. Air filters running in each suite and a deep clean after the appointment and 48 hours before that particular suite would be used again. Done and out in 15 minutes and 2 weeks to contact my DMD for the final implant build/install phase.
I was very grateful for the precautions and the appointment. It felt very safety and hygiene focused. Staff is being rotated and the fewest amount of people were there. I’m still completely freaked out that almost everyone else I saw was unmasked – including a construction crew working next door to the Otho office. That is my brief report from Sacramento.
Barbara
@Just One More Canuck: I see no realistic way of avoiding this outcome. At least in the U.S., we would need a lot more people prepared to serve as staff, and weekly or even daily testing to keep infected people out. It could be done, but it won’t be, at least not here.
Calouste
@Pooh:
Yep. One of my takes from that is that if something isn’t (also) sold via normal retail channels, it’s probably not worth buying.
Just One More Canuck
@Barbara: my wife’s stepfather is an Xray technician who goes into nursing homes to perform xrays – he was told by an administrator at one of the homes to not wear PPE because it might scare the residents.
how much of a soulless ghoul do you have to be to say that
daveNYC
I’m generally going to assume that any model that tries to cover the ISA on this is going to be useless because none of them are accounting for the level of stupid in regards to wanting to reopen, not wear masks, replace handshakes with face licking, etc.
Get me a model that assumes that a stupid large chunk of the population is actively trying to get this disease and I’ll start paying attention.
Victor Matheson
@Jeffro: Here’s what’s crazy – as recently as GW Bush’s term, the Republicans still hired rockstar economists. Greg Mankiw worked was GW’s chief economist in his first term. While I disagree with him politically, he is still a great economist and I use his textbooks to teach from. Yes, he can be a bit of a dick, but there is no question he was a solid economist. In fact, he coined the phrase “charlatans and cracks” for his first textbook to describe exactly the sort of clowns that are in the office now.
Victor Matheson
@jl: “Do you have a set of preferred causal models that can explain the correlations?”
For me it is this one. If you don’t have a plausible reason for why the model and data behaves like it does, then you are just Jackson Pollack throwing paint at a wall and you think you are finding correlations that just aren’t there.
That was always the problem with this COVID-19 model. There was never a plausible reason to explain why the deaths were going to fall to zero by now.
StringOnAStick
@Mike R: Most dental offices are following he recommendations from the state dental association,which are coming from the American Dental Association. As to if what is being recommended is adequate and effective, time will tell I guess. As an individual patient, you are probably safe enough and serious dental emergencies need to be taken care of promptly. As to being a worker in a dental office, that’s at least 8 hours a day of potential exposure. That’s more exposure the I’m willing to risk. YMMV
StringOnAStick
@ziggy: I completely agree with your assessment!
JR
I read that Hassett had to log transform the data before applying the cubic. If he hadn’t we would be looking at the dead coming back to life in a week or so.
catclub
@Pooh: another good part of that was ‘audit is important’,
i.e checking up on predictions, or statements.
The reporter who asked ‘exactly what string of burglaries existed that they said they were responding to’? And they were no burglaries.