Great Data Tool from U Dub

[keefe]

Perhaps people can put down their political axe and focus on some genuine empiricism...

http://covid19.healthdata.org/

[Frenns Liquor Depot]

This is pretty cool. 

[keefe]

This is pretty cool.

I know some of the guys who built both the db/visualization tool and did the analytics.

[keefe]

A genuine issue that front line intensivists will have to confront is how to manage the ethical aftershocks of making life and death decisions from having to triage scarce resources during the wave.

While common in natural disasters and combat, those scenarios are focused and short duration. Outbreak-driven shortages of critical resources, especially ventilators, will have greater duration than traditional triage scenarios.

[Hards Alumni]

A genuine issue that front line intensivists will have to confront is how to manage the ethical aftershocks of making life and death decisions from having to triage scarce resources during the wave.

While common in natural disasters and combat, those scenarios are focused and short duration. Outbreak-driven shortages of critical resources, especially ventilators, will have greater duration than traditional triage scenarios.

100%  Our healthcare workers will require an immense amount of psychiatric help by the time this is over.

[keefe]

100%  Our healthcare workers will require an immense amount of psychiatric help by the time this is over.

It is really about the magnitude - the shortfall in ventilators means that intensivists have ratios of dozens to one where they must choose the lucky one while effectively consigning dozens to death.

The impact is far different than rationing of care in traditional triage scenarios.

Fascinating case study for ethicists.

[Frenns Liquor Depot]

It is really about the magnitude - the shortfall in ventilators means that intensivists have ratios of dozens to one where they must choose the lucky one while effectively consigning dozens to death.

The impact is far different than rationing of care in traditional triage scenarios.

Fascinating case study for ethicists.

In all cases Keefe.  The country world as a whole is going to emerge from this shell shocked by the random carnage - with our healthcare workers at the top having seen the decisions occur (vs. the outcome).  We are at the beginning of a very long marathon.  I'm really pulling for the scientific minds around the world to blunt this as best as possible. 

[forgetful]

Perhaps people can put down their political axe and focus on some genuine empiricism...

http://covid19.healthdata.org/

I mentioned this in the previous thread, but if you know them, can you get them to fix their data analysis. Some very amateurish mistakes.

The main one being their calculation or ICU beds needed. They calculated them from the wrong peak. They should calculate the number of ICU beds needed from the peak in the ICU beds needed curve. They are not doing so, they are calculating the ICU beds needed from the peak of the all beds needed.

Great visualizations, but such an rookie mistake in data analysis makes one question the accuracy of their models.

[lostpassword]

Great visualizations, but such an rookie mistake in data analysis makes one question the accuracy of their models.

I looked at this yesterday.

In the last 24 hours, the "peak" in WI moved from May 22 to Apr 26.
In the last 24 hours, the "peak" in FL moved from May 14 to May 3.

Either the data or the models or both aren't to be trusted here... or are simply too sensitive to be useful.

[GooooMarquette]

It’s a nice tool on the surface, but the basic assumptions aren’t all correct. It says that MN has not implemented a shutdown of non-essential services. Not true. This was done last Friday at midnight.

If they correct the mistakes, it might be a very good tool.

[keefe]

Yea, the University of Washington and Fred Hutch are amateurs. Got it.

[Hards Alumni]

Yea, the University of Washington and Fred Hutch are amateurs. Got it.

Snowflake!

[GooooMarquette]

Yea, the University of Washington and Fred Hutch are amateurs. Got it.

Don't think anyone used that word. We have just pointed out mistakes.

The tool is only as good as the data on which it is used, and the methodologies it uses.

[forgetful]

Yea, the University of Washington and Fred Hutch are amateurs. Got it.

There is this thing called peer review. The guys at U-Dub and Fred Hutch are aware of it. They made some mistakes, they are easy to fix, and should want to fix them.

The ones I'm pointing out, look like they were implemented by a graduate student, who is likely newer to this type of data analysis. They probably have a minor coding error based on an incorrect, but reasonable assumption. It would take 5-10 minutes to recode that part of the script.

[wadesworld]

Yea, the University of Washington and Fred Hutch are amateurs. Got it.

Sheesh. A little sensitive there? For someone as alpha as you are, you sure are thin skinned.

You’d think people would want the most complete, accurate data out there. The best response probably would’ve been, “Maybe he was unaware. Thanks for that.  I’ll let my friend know.” (Sick name drop there, by the way.)

But that seems to be how this country is run. Instead of taking the information and updating whatever it may be to be correct, we’ll just dig our heels in and attack anyone who might suggest further data, questioning how they could ever know more than someone who is obviously smarter than they are.

SAD!

[GooooMarquette]

FWIW - the tool still says non-essential businesses aren't closed in MN, but they have been since Friday.

If you don't want to believe me, look at the map provided by ABCNews.com:

https://abcnews.go.com/Health/states-shut-essential-businesses-map/story?id=69770806

The researchers will help their credibility if they correct basic factual errors like this.

[BM1090]

Maybe I'm not fully understanding but they project 0 deaths in Wisconsin after the first couple weeks of June? I'd be shocked if that was even close to being the case. Hope it's accurate, though.

[GooooMarquette]

Maybe I'm not fully understanding but they project 0 deaths in Wisconsin after the first couple weeks of June? I'd be shocked if that was even close to being the case. Hope it's accurate, though.

The only way that might make sense is if they're assuming people stay at home indefinitely. Good luck with that.

[keefe]

Looks like the tool is actually spot on

[withoutbias]

Looks like the tool is actually spot on

...for now. You should know by now that these things change daily. We did miss you when the tool was looking way off though.

[jesmu84]

...for now. You should know by now that these things change daily. We did miss you when the tool was looking way off though.

Lol

[Coleman]

I've been keeping up with this tool for a while... while it is one of the best resources I have found, it is still a little puzzling

Projections have been all over the place for the country, jumping around from 80,000 deaths, to 93,000, down to 60,000. I get that data is updated but it seems to be moving quite a bit very quickly.

Also, there are inaccuracies with Illinois. Death numbers are off. I also have a hard time believing there will be ZERO deaths in Illinois after May 1. Will we be on a downward trajectory by then? Probably. But zero seems unlikely.

[GooooMarquette]

I've been keeping up with this tool for a while... while it is one of the best resources I have found, it is still a little puzzling

Projections have been all over the place for the country, jumping around from 80,000 deaths, to 93,000, down to 60,000. I get that data is updated but it seems to be moving quite a bit very quickly.

Also, there are inaccuracies with Illinois. Death numbers are off.

Agree - the tool and its methodology seem to be strong, but the data input has often lagged. When MN first implemented its stay at home order, it took them several days to change that parameter and adjust projected deaths.

[rocky_warrior]

Well, at this point, the header in the tool also says "COVID-19 projections assuming full social distancing through May 2020"

THROUGH May!  I mean - at this point, I know most people are hoping to "get back at it" sometime IN May - you know, after the Apr 30 deadline is gone.  But, that's not what the tool is using for it's projections.

Also, I'm amused at the uncertainty bands (not amused at the underlying data they represent...).  I get that's how statistics work - given limited data sets.  But they're um, huge. 

[GooooMarquette]

Well, at this point, the header in the tool also says "COVID-19 projections assuming full social distancing through May 2020"

THROUGH May!  I mean - at this point, I know most people are hoping to "get back at it" sometime IN May - you know, after the Apr 30 deadline is gone.  But, that's not what the tool is using for it's projections.

Also, I'm amused at the uncertainty bands (not amused at the underlying data they represent...).  I get that's how statistics work - given limited data sets.  But they're um, huge.

Yep. The result of limitations in testing. It's a shame the CDC dropped the ball so badly on test development, and left states and private entities to shoulder that load alone.

[Warriors4ever]

I've been following this, too, and I agree that the data posted today for Illinois makes no sense. Up until today it seemed pretty good.

[Frenns Liquor Depot]

Bill Gates just promoted this tool on LinkedIn

[forgetful]

https://www.foxnews.com/politics/coronavirus-model-estimates-us-deaths-down

Still concerned about this model. They updated it yesterday, and predict on the low end around 34,000 deaths. As of yesterday we already had 37,000. So either they think zombies are going to rise from the dead, or they aren't paying very close attention to the data.

Not to mention we currently have 2000 dying each day, so any model that would have less than 40-45k deaths is absurdly inaccurate.

Are they being pressured to have numbers that are revised down? I just don't believe their model would be so obviously incorrect. And it is suspicious that it would be inaccurately updated, and immediately be pushed by right-wing media as proof that the lockdowns are overly aggressive and we can reopen.

[Frenns Liquor Depot]

Am I missing something.  The tool predicts 60k deaths. 

[forgetful]

Am I missing something.  The tool predicts 60k deaths.

Apologies for not being clear. After updating yesterday the probability range for deaths is now:

34,063-140,381

The lower end of the range is 3,000 deaths below what we already had yesterday. It is massively incorrect, and implausible.

That error alone is likely more than enough to cause the overall model to lower the total death prediction from 68,841 to 60,308.

Clear and obvious error, so it is either intentional or extremely careless in terms of updating.

[Frenns Liquor Depot]

I think they take a day or two to put in actual results.  So it’s behind what happened.  There also has been revisions as states start counting differently with the cdc definition change.  My guess is it’s those items.   

I’ve been watching it for ct and it’s been directionally accurate.

[forgetful]

I think they take a day or two to put in actual results.  So it’s behind what happened.  There also has been revisions as states start counting differently with the cdc definition change.  My guess is it’s those items.   

I’ve been watching it for ct and it’s been directionally accurate.

I thought about that, and it generally makes sense. But then put a caveat on the page, instead of saying updated April 17. Similarly, don't go out and do interviews and report the numbers, which are no longer accurate.

The going out and doing interviews with obvious inaccurate numbers, is at least irresponsible. I'd say the same for the reporting. The way it is being reported is also knowingly inaccurate. It all gives these modelers a bad name, when they are already facing trust issues.

[lostpassword]

Not making any judgement on the source or those quoted but I've been seeing this model criticized with increasing regularity the past week.

https://www.statnews.com/2020/04/17/influential-covid-19-model-uses-flawed-methods-shouldnt-guide-policies-critics-say/

Others experts, including some colleagues of the model-makers, are even harsher. “That the IHME model keeps changing is evidence of its lack of reliability as a predictive tool,” said epidemiologist Ruth Etzioni of the Fred Hutchinson Cancer Center, who has served on a search committee for IHME. “That it is being used for policy decisions and its results interpreted wrongly is a travesty unfolding before our eyes.”

[forgetful]

Not making any judgement on the source or those quoted but I've been seeing this model criticized with increasing regularity the past week.

https://www.statnews.com/2020/04/17/influential-covid-19-model-uses-flawed-methods-shouldnt-guide-policies-critics-say/

Lots of criticism for this model, and as time has evolved, it is easy to see why.

The authors provide no fundamental methodology. So we have no idea how their model is constructed.

The plots look like they simply have a Gaussian distribution fit to current trends. That is going to lead to significant errors after the peak. Data from all nations shows (and this is what is normal for social distancing) that the disease curve, and death curve, more accurate follow a stretched exponential function. An exponential rise to the peak, then a long tailing off.

Unless the US is magically different, that means there will be a lot of mortality on the long tail. Leading to a substantially higher death count than the model predicts. That will lead to a lot of patting themselves on the back right now, looking pretty foolish.

[rocky_warrior]

Yeah, this could have been a great data tool - but I don't trust it at all now.  Not updated enough, and misleading figures when it is updated.

[Frenns Liquor Depot]

Yeah, this could have been a great data tool - but I don't trust it at all now.  Not updated enough, and misleading figures when it is updated.

That’s fair.  I defended it but the fact it is only actual data through the 16th makes it not useful

[rocky_warrior]

That’s fair.  I defended it but the fact it is only actual data through the 16th makes it not useful

Oh yeah - I like where they started.  And I still think they're trying to get it right.  Unfortunately,  pressure is now on them because it was promoted so early and so much.  It's researchers trying to create an accurate model - and now influential people and politicians are using as justification one way or another.  Unfortunately, the model is still developing - and given they're trying to encapsulate a fragmented data set (some staying at home, some not, and everywhere in between in the 50 states) - it's going to be difficult to be accurate!

[forgetful]

Oh yeah - I like where they started.  And I still think they're trying to get it right.  Unfortunately,  pressure is now on them because it was promoted so early and so much.  It's researchers trying to create an accurate model - and now influential people and politicians are using as justification one way or another.  Unfortunately, the model is still developing - and given they're trying to encapsulate a fragmented data set (some staying at home, some not, and everywhere in between in the 50 states) - it's going to be difficult to be accurate!

His own colleagues at U Dub have came out and criticized his methodology and have been quite harsh on its utility.

[Coleman]

We are well over 40,000 deaths, so I see no way this thing ends with only 60,000, which is what the tool is still predicting.

Could we stay under 100,000? Possibly, and hopefully. But if states start reopening sooner vs later that will be tough.

[GooooMarquette]

His own colleagues at U Dub have came out and criticized his methodology and have been quite harsh on its utility.

Yeah. I applaud the guy's intent and effort, but the execution and resulting accuracy have been lacking.

[Jockey]

Yeah. I applaud the guy's intent and effort, but the execution and resulting accuracy have been lacking.

Yesterday they revised and said deaths would be 45,000 - 65,000

I guess they figure today might be the last day that people die. Pretty flawed data.

[Lennys Tap]

Yesterday they revised and said deaths would be 45,000 - 65,000

I guess they figure today might be the last day that people die. Pretty flawed data.

I don’t think you’re reading it correctly. When I go to May 2nd, their projection is approximately 59,000 deaths. Range is 45,000 - 100,000.

[rocky_warrior]

I don’t think you’re reading it correctly. When I go to May 2nd, their projection is approximately 59,000 deaths. Range is 45,000 - 100,000.

I think his point is that there were already 45k dead in the US yesterday - and no statistical model should show zero people dying after today.  Though I guess an asteroid could hit the earth killing everyone - and thus zero more Covid-19 deaths!

[ZiggysFryBoy]

I think his point is that there were already 45k dead in the US yesterday - and no statistical model should show zero people dying after today.  Though I guess an asteroid could hit the earth killing everyone - and thus zero more Covid-19 deaths!

C'mon SMOD!

[Jockey]

I don’t think you’re reading it correctly. When I go to May 2nd, their projection is approximately 59,000 deaths. Range is 45,000 - 100,000.

You could be right - maybe I read it wrong. But that almost makes it worse. Their lower limit is fewer than the deaths as of April 22.

I would also be thrilled if the total is “only” 59,000 by May 2. That would mean we are making great progress at bringing down the daily death totals.

[Lennys Tap]

I think his point is that there were already 45k dead in the US yesterday - and no statistical model should show zero people dying after today.  Though I guess an asteroid could hit the earth killing everyone - and thus zero more Covid-19 deaths!

Actually (if anyone survived) we’d have a boatload more Covid deaths since everyone who die with it we are classifying as dying from it.

[TSmith34, Inc.]

Actually (if anyone survived) we’d have a boatload more Covid deaths since everyone who die with it we are classifying as dying from it.

If you are trying to say what I think you are trying to say, that we are overcounting deaths from COVID, then no--we actually undercounting them by about 50% according to a couple of statistical studies I've previously linked.

[rocky_warrior]

If you are trying to say what I think you are trying to say, that we are overcounting deaths from COVID, then no--we actually undercounting them by about 50% according to a couple of statistical studies I've previously linked.

(shouldn't have to explain...) He's being funny - saying if the asteroid hit, all positive covid cases (2M-ish) would count as covid deaths, not asteroid dealths...

[Lennys Tap]

(shouldn't have to explain...) He's being funny - saying if the asteroid hit, all positive covid cases (2M-ish) would count as covid deaths, not asteroid dealths...

Thank you, Rocky.

[ZiggysFryBoy]

(shouldn't have to explain...) He's being funny - saying if the asteroid hit, all positive covid cases (2M-ish) would count as covid deaths, not asteroid dealths...

Tsmith has a non-existent sense of humor.   He's like a kindergartner, who needs explaines to him in simple words.

[Frenns Liquor Depot]

This report actually shows the ihme model isn’t terrible and discusses many complexities of trying to model this.

https://www.nytimes.com/interactive/2020/04/22/upshot/coronavirus-models.html

[TSmith34, Inc.]

Tsmith has a non-existent sense of humor.   He's like a kindergartner, who needs explaines to him in simple words.

We all have our flaws I guess. I'm glad my isn't being a hypocritical douche.

[forgetful]

This report actually shows the ihme model isn’t terrible and discusses many complexities of trying to model this.

https://www.nytimes.com/interactive/2020/04/22/upshot/coronavirus-models.html

I'm pretty well versed in understanding the complexities in models, and am usually defending modelers because of these complexities.

For this model, I initially had minor concerns, where they were miscalculating the peak of their own data. This was almost assuredly a coding error that resulted from an incorrect assumption in rushing out a tool. But they never went back and corrected it. That is an issue.

Then, they have yet to publish the actual methodology. Another red flag.

Now, they are publishing data that is physically impossible. I know where this error is coming from, but not addressing it, is simply poor practice, and borderline unethical.

The last straw for me is that their published data now, violates what little we know about the methodology. The core methodology is that they refit the data using trends in other nations. All other nations are showing that there is a long tail after the peak, so the curve looks more like a stretched exponential than a gaussian.

Their fits do not follow the known stretched exponential character from other countries data. So either they are not doing what they claim, or they are doing it very poorly.

[TSmith34, Inc.]

This coronavirus model keeps being wrong. Why are we still listening to it?
https://www.vox.com/future-perfect/2020/5/2/21241261/coronavirus-modeling-us-deaths-ihme-pandemic

"On Wednesday, the US death count passed the 60,000 mark that the IHME model had said was the likely total cumulative death toll. The IHME on April 29 released a new update raising its estimates for total deaths to 72,433, but that, too, looks likely to be proved an underestimate as soon as next week. Even its upper bound on deaths — now listed as 114,228 by August — is questionable, as some other models expect the US will hit that milestone by the end of May, and most project it will in June.

<snip>

"But as the weeks have passed, it has become clear that the IHME’s projections have been too optimistic, and slow to adjust to reflect the fact that deaths have plateaued rather than rapidly decreasing to zero. The IHME has been regularly updating its model as new data comes in, but the updates have often been slow enough that the numbers are absurd by the time they’re changed in an update. For example, in late April the model still stated the expected total death toll was 60,000, even as the US was clearly only a few days from that milestone."

[Jockey]

When a model like this releases a possible range of deaths and the low end number is passed on the day it is released, there are serious issues with the model - enough to render the model meaningless.

[TSmith34, Inc.]

The model here takes into consideration individual state openings. Currently projection is for 161K deaths by August. Can anyone critique this?

https://covid19-projections.com/?fbclid=IwAR2HrpFdipbeNTjk55rYHFYxOkXjTB96Nkc7nhhQrqKvn483zNzRiyuEW1Y

[Frenns Liquor Depot]

Well it would tie to the 100K body bags that FEMA bought last week.

[GooooMarquette]

The model here takes into consideration individual state openings. Currently projection is for 161K deaths by August. Can anyone critique this?

https://covid19-projections.com/?fbclid=IwAR2HrpFdipbeNTjk55rYHFYxOkXjTB96Nkc7nhhQrqKvn483zNzRiyuEW1Y

I can't expertly analyze the stats...but common sense tells me 161k will be a heck of a lot closer than the 72,433 currently predicted by IHME. We are already at almost 69k and increasing by over 1k/day. And with the reopening taking place that 1k/day will almost certainly increase.

So 69k + (1k/day x 90 days) = 159k deaths. Pretty darn close to 161k.

But since the actual number of daily deaths has been more than 1k/day (between 1,100 - 1,600 lately), and since the numbers will likely increase as we reopen more, I suspect even that 161k might be conservative...unless states shut back down very quickly.

[cheebs09]

I’ve seen some posts/tweets showing the CDC cut their death count down to 37K and people are using that to say this isn’t that bad.

Looking closer at the numbers, I think that’s only due to the lag in data by a week. Anyone else see this?

[rocky_warrior]

I’ve seen some posts/tweets showing the CDC cut their death count down to 37K and people are using that to say this isn’t that bad.

Looking closer at the numbers, I think that’s only due to the lag in data by a week. Anyone else see this?

Huh?  They're showing 67k currently
https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/cases-in-us.html

[tower912]

Cheebs, I question the sources of your posters/tweeters.     

[rocky_warrior]

You'll also note the CDC is no longer using the IHME model for their forecasts.  You can still find it in their "Previous Forecasts" section.

https://www.cdc.gov/coronavirus/2019-ncov/covid-data/forecasting-us.html

[GooooMarquette]

You'll also note the CDC is no longer using the IHME model for their forecasts.  You can still find it in their "Previous Forecasts" section.

https://www.cdc.gov/coronavirus/2019-ncov/covid-data/forecasting-us.html

Probably a good call, given that we will pass their August 4 forecasted deaths in a day or two...

[forgetful]

Probably a good call, given that we will pass their August 4 forecasted deaths in a day or two...

They updated it again. Got rid of the stupid assumption of a perfect bell curve, now using a stretched exponential. Predicting deaths to be 134k.

[GooooMarquette]

They updated it again. Got rid of the stupid assumption of a perfect bell curve, now using a stretched exponential. Predicting deaths to be 134k.

They’re beginning to look like meteorologists who “predict” it’s going to rain after it starts to rain.

[cheebs09]

Cheebs, I question the sources of your posters/tweeters.   

I agree. Reading the fine print it looks like the data is based on death certificates which take some time. So it’s not as up to date. However, a few people I follow and a quick Twitter search shows people are taking and running with it.

This is why I don’t rely on sports journalists for my Covid takes.


https://www.cdc.gov/nchs/nvss/vsrr/covid19/index.htm

[withoutbias]

Keefe has gone radio silent. Why isn’t he around to tell anyone who dares question his friends that they’re morons and the experts know their chit?

[rocky_warrior]

They updated it again. Got rid of the stupid assumption of a perfect bell curve, now using a stretched exponential. Predicting deaths to be 134k.

Good to see.  I've never questioned that they *want* the model to be as accurate as possible.  It's just...they haven't succeeded to date.

[Hards Alumni]

Keefe has gone radio silent. Why isn’t he around to tell anyone who dares question his friends that they’re morons and the experts know their chit?

No need to point out the obvious, aina?

[Jockey]

Keefe has gone radio silent. Why isn’t he around to tell anyone who dares question his friends that they’re morons and the experts know their chit?

We're just too stupid to understand the complicated nature of the model.

[WarriorDad]

We all have our flaws I guess. I'm glad my isn't being a hypocritical douche.

Potting and kettling