AI Clinical Decision Support Tools in Practice
New AI CDS tools are spreading quickly partly because they are so accessible. Many are free or low-cost and are embedded directly into electronic health records or available on clinicians’ phones. Clinicians can use them at the point of care without toggling to another system. They can ask questions in plain language and get clarifying questions back in near-real time. As more clinical information is added, the tools synthesize it and update responses. “It will incorporate the previous questions into its new answer. It’s not static, but a dynamic process that tries to incorporate additional things as you go along,” notes Dr. Adeniyi.
For example, a clinician can take a photo of a rash and instantly compare it across a full spectrum of skin tones or ask for specific guidance like a tailored nutrition plan, including preferred foods, for a patient with diabetes. The ability for AI CDS to personalize medical advice is helpful for health centers that serve clients with diverse ethnic, cultural, and linguistic backgrounds. “Telling patients to stop eating tortillas, when they grew up on tortillas and ate them their whole life, is not a practical solution,” notes Dr. Adeniyi.
For CHCs facing persistent workforce shortages and rising patient complexity, these tools can feel like adding another colleague to the team. “It is like having a Ph.D. research assistant,” says Dr. Liz Powers, health services officer at Winding Waters Clinic in eastern Oregon. “It doesn’t give you a medical degree. But if you have a medical degree and understand what you’re asking and what you’re looking for, it is game-changing.”
Keeping Clinicians in the Driver’s Seat
Retaining clinician oversight is essential. Though some specific AI tools have shown empirical benefits for patient health, a recent evaluation of publicly available tools shows that they consistently perform poorly on tasks related to early diagnostic reasoning, a keystone of primary care practice. According to the study, “The risk is not just that LLMs are sometimes wrong but that their reasoning is brittle precisely where uncertainty and nuance matter most.” Even models built on trusted medical literature for clinician use produce odd or incorrect answers, especially when there isn’t good evidence available on a particular query.
To alleviate concerns about accuracy, Dr. Adeniyi says he tested the tool against his own clinical judgement. “I used [the tool] for several weeks beforehand just to see how it worked. I really tested it out, asking a ton of clinical questions to validate that it was something I could use. Then I introduced it to the other clinicians. I wanted them to experiment and see how it felt,” he says.
Still, some worry these tools will mean clinicians become less practiced at the slow, effortful diagnostic reasoning in which they were trained. “When it just gets fed to you, you stop doing the mental math of having to look things up and think things through and put things together,” says Dr. Jason Cunningham, chief executive officer at West County Health Centers. New evidence may substantiate this worry.
At the same time, Dr. Cunningham notes these tools can be an important support for advanced practice clinicians and newer clinicians that can make up a substantial portion of CHC staff. “It’s helpful for my right-out-of-school clinicians making really complex decisions. I want them to feel comfortable staying in this complex environment.”
New Oversight Challenges
Given the low-cost, accessible, real-time clinical guidance that AI CDS tools offer, use is growing rapidly, sometimes without an organization’s endorsement or awareness. Keeping abreast of the changing landscape, with new tools marketed directly to individuals rather than organizations, poses new liability and leadership challenges. Some systems are blocking access to unsanctioned websites, a difficult task when tools are increasingly available on individuals’ phones. “I’m sure [clinicians] are using things we don’t even know about, so it’s important we stay informed and can offer things we’d like them to use,” notes Alison Connelly-Flores, chief medical information officer at Urban Health Plan in New York City.
Community health centers need new approaches to govern AI clinical decision support tools. The American Medical Informatics Association highlights several key recommendations for further research, validation, and oversight to address significant concerns about patient safety and equity.
In this evolving landscape, Kimberly Beauchesne, vice president of digital strategy and innovation at CHAS Health, describes the importance of selecting a high-quality AI decision support vendor that is nonproprietary, transparent, and flexible. “We’re putting pieces of a puzzle together,” she says, “and the picture keeps changing. That’s just the reality we are in. . . . The strategy is flexibility and not committing until we see where the front-runners emerge. The list is getting shorter.”
