AI-Generated Health Advice
AI-generated health advice refers to information or guidance produced by large language models (LLMs) — for instance, ChatGPT or Claude — and other conversational AI tools in response to health questions. These tools are already becoming part of how people navigate their health, with about one-third of all U.S. adults using AI chatbots for health information. This includes a higher rate of adults who are uninsured, compared to those with insurance coverage. People may turn to chatbots for speed, privacy, or help when care is hard to afford or access. For those facing long waits, provider shortages, language barriers, or limited transportation, AI may feel like the most available source of support.
AI-generated health advice may help people better understand symptoms, prepare questions for clinicians, translate medical language into plainer terms, and feel more confident advocating for themselves. However, research suggests that many people may trust chatbot responses even when they are incomplete or inaccurate. Additionally, LLMs can provide different information in vulnerable subgroups or based on how questions are asked, creating serious risk for equity and safety.
As Medicaid cuts and other pressures make traditional care harder to reach, more people may supplement with chatbot advice, but it is unclear whether the tools will be safe, trustworthy, and equitable across subgroups. Public education on how to use these tools and their limitations, clear rules for developers and health care organizations, and support from trusted community groups could help people use AI-generated health advice more safely. This could include assistance from trained community health workers, local nonprofits, libraries, faith-based organizations, or other trusted community health organizations.
Care Management
Care management helps people with complex health and social needs stay connected to care, follow treatment plans, and avoid preventable health crises. It is especially important for people with Medicaid coverage and those served by safety-net providers, who may face barriers to health and health care such as unstable housing, transportation challenges, limited English proficiency, disabilities, or difficulty getting timely specialty care. Effective care management requires time, staff and coordination — resources that are often limited in the organizations caring for patients with the greatest needs.
AI and digital health tools could help care teams do this work. For example, remote monitoring and home-based tools can support people with chronic conditions who have difficulty traveling to frequent appointments. Generative AI may help translate discharge summaries or care instructions into plainer language. One study found that AI-generated discharge summaries were easier for patients to understand but also raised safety concerns related to omissions and inaccuracies. These tools should be used to support, not replace, care teams. If designed with equity in mind, they could help safety-net providers extend care management capacity while giving patients clearer, more continuous support.
Population Health Management
Population health management uses data (from electronic health records, claims, and/or reported directly from patients) to understand the needs of a defined group of people and can shift care from a reactive model to a proactive one, helping organizations target limited resources where they can have greatest impact. Predictive tools and natural language processing can help identify patients who may benefit from intensive care management or social needs interventions. For example, one study used an algorithm to identify hospitalized patients with both medical complexity and language access needs, helping care teams prioritize interpreter services.
The same tools can also reinforce disparities if they rely on incomplete data or use past spending and utilization as a proxy for need. To ensure fairness, population health tools should be transparent, regularly checked for biased performance across groups, and paired with input from the community.
Conclusion
AI and digital health technologies will not advance equity on their own. But if designed and implemented with underserved communities and safety-net health systems in mind, they can help make care more accessible, proactive, and responsive to the people who need it most.
