One of the better ways to reduce errors is to admit what went wrong and then how to solve those problems. 538 and Nate Silver got the Republican race fundamentally wrong as they were applying a modified “The Party Decides” framework as a core assumption to their analysis. Nate Silver has a good tweet on how he is changing his thinking now that Donald Trump is the Republican nominee:
Updated post-Trump priors:
1. Voters are more tribal than I thought.
2. GOP is weaker than I thought.
3. Media is worse than I thought.— Nate Silver (@NateSilver538) May 1, 2016
This applies to hospitals and medical decision making errors.
Brigham and Women’s Hospital in Boston has an interesting approach about errors that are made at the hospital. They treat their errors as learning and process improvement opportunities. More importantly, they publish their errors. Here is one sample:
What Happened
Nurse A was caring for a patient, Susan, and administered a dose of cough suppressant, which could be taken on an as needed basis every 8 hours, at 7:00 AM. Susan’s next dose, if needed, could therefore be given at 3:00 PM. That afternoon, a second nurse (Nurse B) was assigned to care for the patient while Nurse A was off the unit. Nurse B administered the next dose of medication because Susan was coughing. When Nurse A, resumed care of Susan, she gave her what she believed was Susan’s second dose of cough suppressant at 2:30 PM. Nurse A administered the medication at 2:30 p.m. because Susan was scheduled to leave the unit for testing and would not be back by 3:00 PM when the dose was originally scheduled. The nurse received a message in the electronic health record that the dose was being given early, “Based on the ordered frequency, this medication is possibly being administered too close to another administration. Please review previous administrations to verify appropriateness.” The message wasn’t surprising since she was purposefully giving her the second dose 30 minutes early. What she didn’t know is that Susan had already received the second dose from Nurse B. Fortunately, Susan was not harmed by this additional dose.What We Are Doing
When Nurse A filed the safety report about this event, she included an idea for avoiding similar events in the future. She suggested that the warning box in the electronic health record that highlights the early medication time, also include the time the last dose was given. In this case, the warning would have alerted her to the time Nurse B administered the medication and prevented the duplicate dose.Based on this error, safety report and improvement suggestion, IS (Information Services) is adding a best practice alert (BPA) displaying the last administration time for all PRN (as needed) medications.
We learn by our mistakes. We only learn when we figure out why a mistake was made and we can then re-adjust either our mental model or our practical routines to incorporate that new and valuable information.
srv
I think it’s time to convene a death panel for pollsters.
aimai
Great idea by Brigham and Women. When Mr. Aimai was in Mt. Auburn hospital for uncontrollable back pain the nurse on duty at night was personally opposed to the medications his doctor had assigned to him and she simply cut them down or off without telling anyone. And, apparently, there was no pinging in the system to let the doctor know that the meds weren’t being given. In the morning, when I would come check on my husband, he would still be in pain and he would have been unable to rest. But because the nurse was off duty I never saw her to question her–and the doctor was unaware that the meds were not being administered as ordered.
Something that really surprised me is that there is no “floor level” nurse or doctor meeting, once or twice a day, to review the current patients on the floor and their needs or progress. Its all very fractured. People work the floor who don’t know each other or the doctors. People work tiny bits of the job (like giving injections or drawing blood) who never see the patient again. There’s just very little coherence to the treatment people are given and there is no supervisor who really knows and sees everything. The B and W system subsitutes a computer for a chief nurse function.
Matt McIrvin
Pollsters were not the problem. Nate Silver was ignoring what the polls were plainly telling him! It was completely contrary to the approach that had made him a celebrity in the first place.
lol
@srv:
Pollsters aren’t to blame here. It was the punditry reading those polls and going with their gut instead.
WaterGirl
@Matt McIrvin: Natel Silver lost his way a long time ago when he left his own site to join the big guys. It’s very sad.
Benw
A recent survey found that 9 out of 10 pollsters are against death panels for pollsters.
mac007
As a quality engineer, it’s really interesting to see the lean manufacturing principles I’ve used for years — such as continuous improvement — start to be utilized in non-manufacturing environments. I think it’s a good development, especially in healthcare. “Quality” is not just some nebulous concept — it could literally mean the difference between life and death in a hospital setting. In the old days, you’d just fire the nurse and continue with the status quo. Glad to see that’s changing.
Prescott Cactus
Richard,
Nuclear power industry a “lessons learned” method. They include a section in every work package (job or task) that points out previous screws up when a similar task was performed. Database is nation wide, so you can find some real winners if you look hard enough.
Great system if the workers have time to fully examine and read all info. The simpler the task, the higher the chance of a “grab & go” happening.
Robin G.
It’s easy to act now like Trump was always obvious, but he really wasn’t. High polling numbers in July 2015 are meaningless. Santorum, Huckabee, Cain, Bachmann… the last several GOP primary rounds were littered with “candidates” that went boom and bust the year before. Hell, Trump *was* basically one of those in 2012. Conventional wisdom is conventional for a reason; it’s usually based on *something*.
The point at which the thought process changed from some, and should have changed across the board, was after the first debate. Trump got up there and acted like… Trump, then had the Megyn Kelly thing, and if it were any other candidate that would have finished him. His poll numbers went up instead. That was, IMO, the canary in the coal mine. After that, Trump needed to be taken more seriously — though it’s still very forgivable to have continued to discount his chances until, say, October or November. At that point he’d just held too steady for too long.
Anyway. There’s going to be tons of Monday morning quarterbacking over this, but the reasons Nate listed for getting it wrong make a lot of sense (aside from the first — I don’t know how you miss the tribalism).
Roger Moore
And we only get full information when we consciously decide to encourage people to be open about their mistakes. If we treat them as something that can only be stamped out by punishing people for making them, we close off free exchange of information. You can choose to learn from mistakes or to punish the people who make them, but don’t expect to be able to do both.
Walker
This is why I moved from Nate to Sam Wang. Nate is trying too hard with the “secret sauce” in his predictive model.
gindy51
@mac007: They didn’t fire them per se, they let them find another position, even the ones who were murdering people like this gem: https://en.wikipedia.org/wiki/Charles_Cullen
Trollhattan
@Robin G.:
What you said.
Think most agree Trump started it as a vanity+self-promotion effort, figuring he’d have a few yucks and slap around some politicians and then go build a hotel and get a new teevee show. As Silver implies, the paper-mâché condition of the Republican Party and the utter incompetence of the “deepest bench evah” effectively made Trump an arsonist in a dry forest.
And here we are.
Richard Mayhew
@Prescott Cactus: Yep, people are amazingly creative at how things can get fucked up. It takes a true genius to find an innovative fuck-up. I have a bit of a background in applying statistical quality control methodologies to social services (first couple of years out of grad school) and it is an interesting experience applying manufacturing techniques to after school tutoring programs.
@Roger Moore: No blame, only learning is hard but the right way.
I’m pretty lucky, my company is pretty tolerant about honest fuck-ups. The saying is that you can tell what level a person is at when you hear about their greatest/most expensive fuck-up. Entry level <$10,000, early-career <$100,000, senior analysts <$500,000, experts $5,000,000, VP-level $10,000,000. It is tough to get promoted without an appropriate sized fuck-up
mac007
@gindy51: Yikes! Sure that wasn’t Ted Cruz’s dad? :-)
Dork
If Nate didn’t think to use Worst Case Scenerios for his 1, 2, and 3 points in that tweet when building his models, he’s far more obtuse and stupid than I thought. Does he never watch CNN or Fox? Is he not aware of the infallibility of Cleek’s Law?
Matt McIrvin
@Walker: Even Sam Wang found Trump hard to believe: in September he jumped on a little downturn in Trump’s poll numbers, mostly driven by a couple of outlier polls, and said it might be the end of the Summer of Trump. That didn’t last long, though.
2012 was really different. During the early clown-car period of 2012, one candidate after another took turns as the frontrunner, but Mitt Romney was always there running a strong second, ready to jump in when all that nonsense was over. The closest equivalent to those people in the 2016 cycle was Ben Carson, who never managed to top Trump’s poll numbers except in, again, one or two outliers that got a lot of attention. The closest Mitt Romney equivalents were lost down in the weeds, and the guy with consistent strong performance was Trump.
Face
@Dork: Someone ought to tweet/email Nate and ask him if he’s even aware of the effect of Cleek’s Law on voting proclivities.
Calouste
@Robin G.:
Trump’s high polling numbers were not just in July 2015, they were from July 2015. He went boom and never went bust. By November, if not October, it was obvious that he wasn’t comparable to the mayflies of 2012. At that point, I didn’t think he was 100% sure to win, but I did think that anyone who thought he wasn’t the favorite, and that included Silver, was not to be taken seriously.
FlyingToaster
@aimai: In my experience at Mt. Auburn, it’s the floor resident who keeps track of everything, and there’s a WIDE variation in their capabilities.
In HerrDoctor’s second-to-last surgery, the resident and hospitalist came by to talk about letting him go a day early, and met my then 5-year-old jumping on his hospital bed. They conferred about how much pain he still had (substantial, and this on a drip) and looked again at the jumping kid. And said to him, and me, “He should stay for another day, given his home situation involves a pre-schooler. And we’ll keep him on the IV painkillers until morning.”
And back when WarriorGirl was born, there was a clipboard that followed me all over South 5; everything that was done or administered got put on that thing, and my OB/GYN came and asked me questions about them. And bitched out the nurse who kept claiming that I had PID instead of PIH.
Jim
The military learned this lesson with aircraft accidents many years ago. They write up accidents and mistakes anonymously for general distribution to pilots, NFOs, and maintenance personnel. Pulled no punches. Discipline was on a totally separate track — if someone did something wrong, he/she would be disciplined, but it didn’t take away from the anonymous narratives. This program reduced crashes significantly.
shinobi42
My co workers get annoyed with me because every time a project goes completely off the rails I want to do a post mortem. BUT THIS IS WHY.
We need to look at what we did wrong so we can avoid it in the future. In all things.
jl
I’ve been drawn into interprofessional teaching for the health sciences schools at my main gig. Somewhere along in the session I can’t help put joke that the motto of US health care should be ‘Put good people in impossible situations and hold them strictly responsible!” And I can’t help but tell some stories about aviation, large engineering project safety to give them a hint about how horrid their situation is. I figure it is good buy in for the course.
In the sessions I see pics and sometimes videos of working conditions in clinics and hospitals and it is often a toxic abusive mess, IMHO. And I heard a news report about a study that asserted that iatrogenic conditions and medical errors are now the number one cause of disability and death in US hospitals. It will be interesting to follow that up and see how reliable it is.
I was surprised to see an attempt at an improvement that involved fiddling with an alert. US health care has a horrible problem with ‘alert fatigue’. In some places 20 to 25 percent of far ma sist and nurse actions trigger a warning alert. These warning alert systems have ridiculously high rate of false alarms (usually they mean noting important at all) so of course they are routinely ignored. It looks like this idea is, you cannot get rid of one, at least provide some information that allows the recipient to formulate a useful response. The usual is to just over ride the warning alert so people can attempt to get through their massively overloaded task schedule.
RSA
@Richard Mayhew:
In computer science, Tony Hoare is famous (among many other reasons) for describing what he called his “billion dollar mistake” in the design of an early programming language. He’s won the Turing Award, the equivalent of a Nobel Prize in computer science.
Roger Moore
@RSA:
He’s a piker compared to a recent President, who made a trillion dollar mistake of invading the wrong country. Unfortunately, the Republicans don’t seem to have learned very much from that one.
Redleg
That is a good story about reducing errors and learning from mistakes. Recent empirical research in my field (Organizational Behavior) shows that we are not nearly as good at learning from mistakes as we think we are.
giantslor
@Matt McIrvin: Neither pollsters nor Nate Silver was the problem. The primary season was too complex, with numerous candidates and multiple primaries and caucuses (some open, some closed) spread out unevenly over several months, for polls to predict who would be the nominee. Nate’s “Donald Trump’s Six Stages of Doom” article assumed that the GOP would consolidate around one not-Trump candidate, but they never did, and that’s why Trump won. If it had been just Trump vs Jeb, or Trump vs Rubio from a very early date, Trump would have lost.