Why AI Stumbles in Healthcare: The Devil's in the Data

Broken Compasses and Gut Feelings—How Inaccurate Data and Subjective Judgments Are Leading Us Astray

After I released my last video on AI, a colleague asked me why AI so often stumbles in healthcare. It’s a great question, and the three quotes I mentioned in that video connect directly to this issue.

Let’s dive into the first quote: “Any sufficiently advanced technology is indistinguishable from magic,” by Arthur C. Clarke. Remember that, to use AI, data is transformed into numbers, matrices, and vectors. But if that data is flawed, the resulting analysis is like a compass with a broken needle—it might look like it's pointing the way, but it’s pointing to and generating a false result.

I like to use the movie, Raiders of the Lost Ark, as a metaphor for this issue. The Nazis only had half the medallion, so they were working on incomplete information. As a result, the Staff of Ra was too long, and the sunlight pointed them to the wrong spot. In healthcare, inaccurate or incomplete data is our missing medallion, and it’s leading us to the wrong conclusions. Until we fix our data quality, we’ll keep digging in the wrong places.

The second quote is from Supreme Court Justice Potter Stewart: “I know it when I see it.” He used this phrase to tackle the tricky task of defining obscenity, but it’s surprisingly relevant to healthcare too. Those of us who work clinically know that many things fall into the "I know it when I see it" category—vague and subjective. The problem is that everyone sees things differently. If we rely on clinicians to input their "gut feelings," we’d need an army of them, constantly entering data, far more than they already do. Moreover, the data they enter will likely be inconsistent at best. And if you’re worried about clinicians' resistance to technology and burnout now, just imagine how much worse it would get by forcing something like this.

Let’s look at a simple recent example I have been personally been dealing with. We’ve been working to improve sepsis outcomes for a decade. Progress? Yes. But here’s one nagging problem that keeps coming up: no one can agree on when to “start the clock” for sepsis. Some say it’s when the patient spikes a fever, others when a blood culture is drawn, or when vital signs go haywire. Often, it’s just a judgment call. How can we improve if we can’t even agree on when the race begins?

Lastly, I mentioned that “it’s the same stuff we’ve always done... just faster and more.” AI can process data on a scale that’s unimaginable for humans, which is why it holds such promise in healthcare—or at least, that’s the theory. But we’re still stuck in old ways of thinking. We use AI to calculate myriad risk scores the same way we’ve always done, just faster. What we need is not just to process more data, but to rethink how we use it.

Prior authorization is a poster child for our legacy thinking. We’ve built AI tools that can gather and send huge volumes of information to payers at lightning speed. But instead of solving problems, we’ve just automated the red tape of a process that shouldn’t exist in the first place. Now, instead of one practice manager arguing with one insurance rep, we’ve got bots waging war over hundreds or thousands of requests and denials every minute. Is this really progress? Instead, we could use that data to build a value-based health dashboard, grading physicians on how well they manage patient populations. Just like the iPhone turned our phones into pocket-sized computers, we need to stop doing things the old dysfunctional way, just faster. There’s so much more we could achieve.

In the end, if we keep treating AI like a magic wand instead of a tool that needs quality data and fresh thinking, we'll keep getting Vegas magical results—fantastical, impressive, and completely fake.