Rules Governing Human Oversight of AI in Health Care Could Shape Public Trust

State Laws and Professional Guidance Address Human Oversight of AI in Health Care

Oversight of AI was a common theme in AI legislation this year

State lawmakers and professional organizations are developing new approaches to responsible AI use, often focusing on human oversight and accountability for these tools. A new report from the Transparency Coalition, an AI-policy advocacy organization, identified 115 AI-related measures that legislatures in 30 states passed this year. As of mid-October, 108 had been signed into law and seven were awaiting a governor’s signature. The report ranked human oversight of AI in health care and AI disclosure requirements among the most common issues states addressed. Eight states, for example, enacted laws limiting health insurers’ ability to rely solely on AI or automated systems to deny or reduce coverage, including prior authorization for health insurance claims. At least five states have enacted laws this year restricting AI in mental health care, generally requiring therapy to be provided by a licensed human professional. California also enacted a law requiring health facilities to ensure clinicians retain independent judgment when using clinical AI tools. Most of these mental health laws are already in effect, while Rhode Island’s and California’s take effect in January 2027.

The results of the 2026 midterm elections could further influence state efforts to regulate AI and how they interact with federal policy. The Trump administration has previously urged Congress to adopt a national framework for AI regulation that would preempt existing state laws it considers unduly burdensome, but broad preemption may be less likely given Democratic control of Congress beginning in January. Many congressional Democrats have previously opposed overriding state AI laws without federal regulations in place. At the same time, some Democrats in the incoming Congress have indicated they will pursue new legislation related to AI, including requiring experts to evaluate AI safety and clarifying when technology companies can be held liable for harmful AI outputs. KFF will continue to monitor how state and federal officials approach AI regulation.

Medical boards and professional organizations are developing guidance for oversight of AI in clinical practice

Some professional organizations have also recently taken steps to develop guidance for the use of AI in clinical practice, with a similar focus on the role and responsibility that humans should maintain. Unlike state laws, however, these recommendations are voluntary and do not establish binding requirements. In October, the American Medical Association (AMA) released a governance framework for agentic AI outlining questions for AI systems that take on clinical functions, including what evidence supports their use, what oversight is appropriate, and who is accountable for their risks and outcomes. The AMA says these questions will guide its work on liability, licensure, and regulation. The framework does not set requirements for physicians or AI developers.

Separately, the Federation of State Medical Boards (FSMB) convened a workgroup in Washington in early October to begin developing recommendations for state medical boards overseeing the use of clinical AI tools. The group is focusing on systems that perform clinical functions, such as diagnosis and prescribing, with limited or no direct physician supervision. FSMB’s leadership has argued in the past that AI should be governed through existing medical accountability structures, with physicians remaining accountable for their use of AI. Draft recommendations are expected in the coming weeks or months and will be open for feedback before adoption. Once adopted, they will not be binding, though state boards and legislators could use them to shape policy.

Academic publishers also set new rules for AI use, human oversight, and disclosure

In October, the health policy journal Health Affairs issued a policy for AI use that reflects standards many academic journals have put in place since 2023. The policy prohibits authors who publish in the journal from listing AI as an author or using it in place of human judgment. It also requires authors to review AI-assisted work, remain responsible for its content, and disclose all AI use. The journal also recommends against using AI tools to manage citations and references, citing the tendency of some AI systems to generate inaccurate or nonexistent references.

Why This Matters

Although these efforts address different settings and establish different requirements, they reflect common questions about the role of human judgment and accountability. These questions have practical implications for clinicians deciding whether to rely on AI tools, researchers and publishers evaluating AI-assisted scientific work, and policymakers and medical boards developing standards for their use. They may also reflect public attitudes, with recent polling finding that human involvement and review are important to large shares of adults when AI is used in their own health care. How these standards are established and communicated may shape both professional accountability and public trust in AI-assisted health care and research.

Low Trust in Federal Health Agencies May Shape Perceptions of Vaccine Injury Initiatives

A series of actions announced by the Department of Health and Human Services (HHS) last month aims to expand research into and reporting of health problems following vaccination. These actions include establishing a vaccine injury clinic at the National Institutes of Health (NIH) Clinical Center and proposing a Medicare billing code for evaluating and treating suspected vaccine adverse effects, which HHS has described as reimbursing physicians for reporting to the Vaccine Adverse Event Reporting System (VAERS). The initiative also includes plans to modernize VAERS and require electronic health records to capture vaccine-related adverse events.

How do patient experiences shape trust in vaccine safety?

Federal health officials have framed the initiative as a response to people who say their reports of vaccine injury have been dismissed. In announcing the initiatives, officials said that the government must listen to these patients rather than dismiss them. Reporting has documented some of these experiences. A 2024 New York Times report on people who described health issues after COVID-19 vaccination, for example, shared that many felt dismissed by their health care providers and federal agencies. Research on vaccine conversations more broadly suggests these experiences can affect trust. A 2024 U.S. study of vaccine-hesitant patients and clinicians in the Veterans Affairs health system found that dismissive judgments about patients’ vaccine safety concerns could undermine communication and erode trust and empathy between patients and providers over time.

These findings suggest that when people feel their health concerns are not taken seriously, they may become less trusting of health care providers and public health institutions. Efforts to investigate reported injuries could help build trust among those who feel their concerns have been dismissed. Patients’ reports of health problems following vaccination warrant appropriate clinical evaluation and investigation, but the timing of symptoms alone does not establish that a vaccine caused them.

How does the U.S. currently monitor vaccine safety?

Serious adverse events following vaccination are rare, but federal agencies continue to monitor for potential vaccine safety concerns and investigate possible associations through several systems, including clinical trials conducted before vaccines are approved and safety monitoring after they are in use. One system of monitoring adverse events is VAERS, a passive surveillance program administered by the FDA and CDC. Anyone can submit a report to VAERS detailing a health problem that occurs after vaccination, and reports are unverified and do not, on their own, establish that a vaccine caused a particular health problem.

How might the new initiatives shape research and perceptions of vaccine safety?

The board overseeing the NIH Clinical Center has raised concerns about how the new initiative will be perceived. A statement from the board’s chairman, which the board unanimously approved, questioned how findings from the new vaccine injury clinic would be perceived given low public trust in federal health agencies, warning that the research might be dismissed regardless of whether it finds evidence of harm. The statement also raised concerns that the opening ceremony for the clinic, which included a discussion with people who said they were injured by vaccines, might give the impression that conclusions had already been reached before the research was complete. At the same time, the statement said serious vaccine injuries are rare but real and deserve rigorous research.

Some medical organizations, including the American Academy of Neurology and the Infectious Diseases Society of America (IDSA), have also expressed concerns about the proposed billing code. IDSA told CMS the code is not clinically necessary and that a separate payment for vaccine adverse effects could lead patients and the public to overestimate how common vaccine injuries are.

Public understanding of the research may also depend on what the center is designed to study. HHS framed it around vaccine injuries, while NIH leaders described a broader study that will also include people with lasting symptoms after infection.

Why This Matters

The announcement of these new initiatives could reinforce perceptions that vaccine injuries are common or have not been adequately investigated, even though multiple federal systems already track and investigate safety concerns. Given low confidence in federal health agencies, whether the new vaccine injury research strengthens or undermines public trust may also be shaped by how clearly its methods, limitations, and conclusions are communicated.

Health and Wellness Influencers Frequently Promote Products, While Citing Sources Is Rare

More than one-third (38%) of health and wellness influencers’ social media posts include a sponsorship or encourage viewers to use a specific product, according to a new Pew Research Center analysis. The report examined 535,512 posts on Instagram, TikTok, and YouTube from 6,334 health and wellness influencers. These influencers were individuals with at least 100,000 followers on one of those social media platforms who regularly posted health or wellness information. Among posts from these influencers that directly discussed health and wellness topics, nearly half included a product or service promotion. Nearly all influencers (92%) included promotions at least occasionally, and such content was particularly common in posts that mention nonprescription medication and supplements, men’s health, and physical appearance, though the analysis did not assess accuracy. At the same time, relatively few posts (9%) referred to an outside source like an expert, government agency, or research findings while making a health claim.  Citations were more common in posts that mentioned health policy or warned of a specific health harm.

Why This Matters

Social media users encounter health information from a mix of clinicians, researchers, people sharing personal experiences, and influencers whose expertise may be unclear. In KFF’s June 2026 Tracking Poll on Health Information and Trust, six in ten (61%) adults said they were “very” or “somewhat confident” in their ability to tell true from false when it comes to health information from social media, leaving about four in ten who were “not too confident” or “not at all confident.” While the Pew analysis did not assess whether the content was accurate, the frequency of promotional content, alongside the relative infrequency with which citations are given, may complicate efforts to evaluate health claims people encounter on these platforms.