VOLUME 41

How AI Can Both Detect and Enable Fraudulent Research


Highlights

Nearly 10% of cancer research papers showed signs of being fabricated by “paper mills” that sell manuscripts at industrial scale, with the share increasing exponentially over time, according to new research. The problem may intensify as generative AI becomes more sophisticated, prompting lawmakers to demand information from federal agencies about safeguards in place.

And, persistent claims that physicians are financially incentivized to promote vaccines may be contributing to vaccine hesitancy and declining trust, even as recent analyses show doctors typically break even or lose money when administering vaccines.


AI & Emerging Technology

Machine Learning Can Help Detect “Paper Mills,” Even as Generative AI May Contribute to Rise in Fraudulent Papers

What does new research show about the prevalence of fraudulent papers?

  • As generative AI makes it easier to produce fraudulent papers, researchers are turning to AI-powered detection methods in response. A study published in BMJ developed a machine learning model to identify cancer research papers with similarities to known “paper mill” publications that write and sell manuscripts at industrial scale. When applied to millions of cancer research papers published between 1999 and 2024, the model found that nearly 10% showed signs of coming from these paper mills, sharing textual characteristics with retracted publications.
  • The number of flagged papers increased exponentially over time, rising from about 1% in the early 2000s to over 15% of annual cancer research output by the 2020s. Flagged papers were not limited to low-impact journals, with the share of these papers in high-impact journals also increasing over time to over 10% in recent years.

Lawmakers demand safeguards

The study comes as trust in medical institutions, including scientific journals, becomes increasingly politicized, with officials questioning the legitimacy of leading medical journals. House Republicans sent oversight letters in early February to five federal agencies, demanding information on safeguards to prevent falsified or fraudulent studies from influencing federal grants and research. The letters specifically raised concern about paper mills linked to the Chinese Communist Party, arguing that pressures imposed on Chinese researchers have increased demand for fabricated research. The letters note that major publishers have retracted thousands of papers linked to paper mill activity, with some forced to shut down journal subsidiaries after discovering widespread fraud.

Why this matters

The findings suggest that paper mills represent a large and growing threat to research integrity, with generative AI potentially exacerbating the problem through automated text generation. As AI tools become more sophisticated and accessible, fraudulent paper mill activity may increase, requiring ongoing development of detection methods and stronger institutional safeguards to protect research integrity. Fabricated research entering the scientific literature can misdirect research funding and erode public trust in medical research at a time when confidence in scientific institutions is already declining.


What We’re Watching

Pediatricians Do Not Receive Illegal Financial Incentives to Vaccinate, Despite Persistent Claims

Claims that pediatricians receive illegal financial incentives to vaccinate children have continued to circulate by lawmakers in February, despite federal laws that prohibit pharmaceutical companies from paying providers to administer vaccines. Last month, Texas Attorney General Ken Paxton announced a formal investigation into “unlawful financial incentives” related to childhood vaccine recommendations, alleging that doctors are paid based on the number of vaccines they administer. Similar claims have circulated on social media for months and have been echoed by federal health officials, including Robert F. Kennedy Jr., who stated last summer that doctors were being “paid to vaccinate, not to evaluate.” While quality-of-care incentive programs from insurance companies do exist, these are legal programs, not offered by vaccine manufacturers, and are based on dozens of health metrics beyond vaccines. Recent analyses have shown that pediatricians typically break even or lose money when administering vaccines, particularly when serving people who are uninsured or who rely on Medicaid.

What To Watch Out For: Despite these persistent false claims, the KFF/Washington Post Survey of Parents found that children’s pediatricians were the most trusted source of vaccine information among parents, including across partisanship. KFF will continue to monitor whether the persistence of these unfounded claims may contribute to declining confidence in providers’ recommendations.

Pediatricians are the Most Trusted Source of Vaccine Information Among Parents, Far Fewer Trust Secretary Kennedy, Pharmaceutical Companies, Influencers

Claims that the Keto Diet Can Cure Mental Illness Draw Attention, Despite Lack of Robust Evidence

Health and Human Services (HHS) Secretary Robert F. Kennedy Jr. claimed earlier this month that the ketogenic diet, which emphasizes higher consumption of fats and proteins while limiting carbohydrate intake, could “cure” schizophrenia. Kennedy cited research from a Harvard researcher, who has since challenged the characterization. The researcher emphasized that he has not claimed to “cure” schizophrenia or other mental illnesses and does not advise patients to try the diet without close medical supervision. Kennedy’s claims overstate preliminary research into whether the diet might help people with mental health disorders. Early research has explored whether ketogenic diets may influence biomarkers and metabolic factors associated with severe mental health conditions, but the current body of evidence does not establish the diet as a cure for schizophrenia or other mental illnesses. A 2025 American Psychiatric Association (APA) policy paper describes the approach as controversial and lacking robust, evidence-based research. Kennedy’s comments were followed by a spike in online discussion about the ketogenic diet as people reacted to his statements and similar claims. KFF’s monitoring of social media (X, Reddit, and Bluesky) found that posts, reposts, or comments that contained variations of the phrase “ketogenic diet” along with “schizophrenia” reached the highest point of the past year in early February. The response to Kennedy’s comments in public discourse included both skepticism and belief in the claim, reflecting broader uncertainty that adults might have when facing false or misleading claims.

What To Watch Out For: The spike is decreasing, but the impact of these claims might linger; when senior health officials make unfounded claims that overstate or misrepresent early research, they risk undermining public understanding of how medical evidence develops. Patients with mental health conditions who encounter these claims may be confused about whether established treatments remain appropriate and whether these unproven approaches should replace evidence-based care.

There Was a Large Spike in Mentions of Ketogenic Diets and Schizophrenia at the Start of February

Older Adults See More Low-Quality Health Information Online, Study Shows

A new study published in Nature Aging found that, among the study’s participants, exposure to low-credibility health content online increased with age and was not solely driven by how often users viewed health-related content. The research showed that older participants consumed less content on YouTube overall, in line with KFF’s July 2025 Tracking Poll on Health Information and Trust, which showed that about half (49%) of adults ages 65 and over reported using social media to find health information and advice at least occasionally, compared to larger shares of adults ages 18-29 (76%). Despite lower overall use, the Nature Aging study found that a higher share of what older participants viewed came from low-credibility sources. Participants who believed false or misleading health claims were also more likely to encounter low-credibility health content, indicating a link between beliefs and exposure.

What To Watch Out For: As health information increasingly moves online, understanding these age-related disparities in exposure to and belief in false claims becomes more important for designing effective public health communication strategies.

About The Health Information and Trust Initiative: the Health Information and Trust Initiative is a KFF program aimed at tracking health misinformation in the U.S., analyzing its impact on the American people, and mobilizing media to address the problem. Our goal is to be of service to everyone working on health misinformation, strengthen efforts to counter misinformation, and build trust. 


View all KFF Monitors

The Monitor is a report from KFF’s Health Information and Trust initiative that focuses on recent developments in health information. It’s free and published twice a month.

Sign up to receive KFF Monitor
email updates


Support for the Health Information and Trust initiative is provided by the Robert Wood Johnson Foundation (RWJF). The views expressed do not necessarily reflect the views of RWJF and KFF maintains full editorial control over all of its policy analysis, polling, and journalism activities. The data shared in the Monitor is sourced through media monitoring research conducted by KFF.