Connect with us

Health

Study Finds AI Systems Can Repeat Fake Medical Claims When Framed Credibly

Published

on

“Large language models accept fake medical claims if presented as realistic in medical notes and social media discussions, a study has found.”

As more people turn to the internet to research symptoms, compare treatments and share personal health experiences, artificial intelligence tools are increasingly being used to answer medical questions. A new study warns that many of these systems remain vulnerable to medical misinformation, particularly when false claims are presented in authoritative or realistic language.

The findings, published in The Lancet Digital Health, show that leading artificial intelligence systems can mistakenly repeat incorrect medical information when it appears in formats that resemble professional healthcare documents or trusted online discussions. Researchers analysed how large language models respond when faced with false medical statements written in a credible tone.

The study examined responses from 20 widely used language models, including systems developed by OpenAI, Meta, Google, Microsoft, Alibaba and Mistral AI, as well as several models specifically fine-tuned for medical use. In total, researchers assessed more than one million prompts designed to test whether AI would accept or reject fabricated health information.

Fake statements were inserted into real hospital discharge notes, drawn from common health myths shared on Reddit, or embedded in simulated clinical scenarios written to resemble authentic healthcare guidance. Across all models tested, incorrect information was accepted around 32 percent of the time. Performance varied significantly, with smaller or less advanced models accepting false claims in more than 60 percent of cases, while more advanced systems, including ChatGPT-4o, did so in roughly 10 percent of responses.

See also  Eating Cheese Weekly May Lower Risk of Dementia, Japanese Study Finds

The researchers also found that medical fine-tuned models performed worse than general-purpose systems, raising concerns about tools designed specifically for healthcare use.

“Our findings show that current AI systems can treat confident medical language as true by default, even when it’s clearly wrong,” said Eyal Klang of the Icahn School of Medicine at Mount Sinai, one of the study’s senior authors. He added that how a claim is written often matters more to the model than whether it is accurate.

Some of the accepted misinformation could pose real risks to patients. Several models endorsed claims such as Tylenol causing autism during pregnancy, rectal garlic boosting immunity, mammograms causing cancer, and tomatoes thinning blood as effectively as prescription medication. In another case, a discharge note incorrectly advised patients with oesophageal bleeding to drink cold milk, which some models repeated without flagging safety concerns.

The study also tested how AI systems responded to flawed arguments known as fallacies. While many fallacies prompted scepticism, models were more likely to accept false claims framed as expert opinions or warnings of catastrophic outcomes.

Researchers say future work should focus on measuring how often AI systems pass on falsehoods before they are used in clinical settings. Mahmud Omar, the study’s first author, said the dataset could help developers and hospitals stress-test AI tools and track improvements over time.

The authors said stronger safeguards will be essential as AI becomes more deeply embedded in healthcare decision-making.

Health

Novo Nordisk Teams Up With OpenAI to Accelerate Drug Discovery Using AI

Published

on

Danish pharmaceutical giant Novo Nordisk has announced a new partnership with OpenAI aimed at integrating artificial intelligence across its drug development and business operations.

The collaboration, revealed on Tuesday, is expected to help the company identify new treatments more quickly and improve how medicines are developed, produced and delivered to patients. Novo Nordisk said the use of advanced AI tools will allow it to analyse vast and complex datasets, uncover patterns that were previously difficult to detect, and shorten the timeline from research to patient access.

Chief executive Mike Doustdar said the agreement marks an important step in positioning the company for the future of healthcare. He noted that millions of people living with chronic conditions such as obesity and diabetes still require better treatment options, adding that new therapies remain to be discovered.

Novo Nordisk is widely known for its leading treatments in these areas, including Ozempic and Wegovy, which have seen strong global demand in recent years. The company said integrating AI into daily workflows will allow its teams to test ideas more rapidly and bring innovations to market at a faster pace.

The partnership will not be limited to research and development. Both companies plan to apply AI tools to manufacturing processes, supply chains and commercial operations, with pilot programmes already set to begin. Full integration is expected by the end of the year.

Sam Altman said artificial intelligence is transforming industries and has the potential to significantly improve outcomes in life sciences. He added that the collaboration would support faster scientific discovery and more efficient global operations, helping to shape the future of patient care.

See also  Loneliness Linked to Higher Death Risk Among Cancer Patients, Global Study Finds

The move comes as pharmaceutical companies increasingly turn to AI to gain an edge in drug discovery. Novo Nordisk has already invested in innovation through initiatives such as the Danish Centre for AI Innovation, developed in partnership with Nvidia and Denmark’s export and investment fund.

Competition in the sector is intensifying. US-based Eli Lilly, a key rival in the weight-loss drug market, recently announced its own AI-focused collaboration with Insilico Medicine to develop new treatments. The agreement, valued at up to $2.75 billion, highlights the growing role of AI in reshaping pharmaceutical research.

Industry analysts say such partnerships reflect a broader shift toward data-driven innovation in healthcare, where the ability to process and interpret large volumes of information is becoming increasingly important.

For Novo Nordisk, the partnership with OpenAI signals a commitment to staying at the forefront of this transformation, as companies race to harness technology in the search for new and more effective treatments.

Continue Reading

Health

Study Finds AI Models Fall Short in Early Medical Diagnosis

Published

on

A new study has found that artificial intelligence language models still struggle with one of the most critical aspects of medical care, raising concerns about their use without human oversight.

Researchers from Mass General Brigham reported that AI systems failed to produce an appropriate early diagnosis more than 80 per cent of the time. The findings, published in JAMA Network Open, highlight ongoing limitations in how these systems reason through complex clinical scenarios.

The study examined 21 large language models, including systems developed by OpenAI, Google and xAI. Among those tested were versions of GPT, Gemini, Claude, Grok and DeepSeek.

Researchers used a structured evaluation tool known as PrIME-LLM to assess how well the models handled different stages of clinical reasoning. These stages included forming an initial diagnosis, ordering tests, reaching a final diagnosis and planning treatment. The models were tested using 29 standardised clinical scenarios, with information introduced gradually to mirror real-life patient cases.

While the systems showed relatively strong performance when identifying a final diagnosis, their ability to generate a differential diagnosis — a key step in distinguishing between conditions with similar symptoms — remained limited. This early-stage reasoning is widely regarded as essential in medical decision-making.

Marc Succi, a co-author of the study, said current models are not ready for independent clinical use. He noted that differential diagnosis represents a core part of medical practice that AI has yet to replicate effectively.

Another researcher, Arya Rao, said the findings show that AI performs best when given complete information but struggles when cases are still developing. She explained that the models are less reliable in situations where doctors must make judgments based on limited or uncertain data.

See also  Cannabis and Cocaine Top Drug Use in the EU, but Other Substances Pose Rising Risks

Despite these shortcomings, the study identified a group of higher-performing systems, including advanced versions of GPT, Gemini, Claude and Grok. These models achieved final diagnosis success rates ranging from around 60 per cent to over 90 per cent when provided with detailed clinical data such as lab results and imaging.

Experts not involved in the research also stressed the importance of caution. Susana Manso García said the findings reinforce that AI should not replace professional medical judgement. She advised that patients continue to seek guidance from qualified healthcare providers when dealing with health concerns.

The study concludes that while AI has made progress, it still requires close human supervision in clinical settings. Researchers say the technology shows promise as a support tool, but its current limitations mean it cannot yet be trusted to make independent medical decisions.

Continue Reading

Health

Genetic Differences May Shape Effectiveness of Popular Weight-Loss Drugs, Study Finds

Published

on

Genetic variations may help explain why some patients respond better than others to widely used weight-loss medications, according to new research that points to the potential for more personalised treatment approaches.

Drugs such as Ozempic, Mounjaro and Zepbound have transformed the treatment of obesity in recent years. These medications belong to a class known as GLP-1 receptor agonists, which mimic a natural hormone that regulates appetite and blood sugar, helping people feel full for longer. Despite their growing use, patient outcomes vary widely, with some individuals losing less than 5 percent of their body weight while others achieve reductions exceeding 20 percent.

The study, conducted by researchers at the 23andMe Research Institute and published in Nature, examined genetic data alongside patient-reported experiences to better understand these differences.

Researchers analysed information from nearly 28,000 participants who had taken GLP-1 medications for a median period of just over eight months. Their findings identified specific genetic variants that appear to influence how individuals respond to these treatments.

One such variation in the GLP1R gene was linked to improved effectiveness. Individuals carrying a particular version of this gene lost an average of 0.76 kilograms more than those without it during the study period. Another variant in the GIPR gene was associated with an increased likelihood of side effects such as nausea and vomiting among patients taking tirzepatide-based drugs, though it did not affect weight loss outcomes.

Noura Abul-Husn, chief medical officer at the research institute, said current approaches to weight management often rely on trial and error. She noted that patients frequently begin treatment without clear expectations about how effective a drug will be or what side effects they might experience.

See also  AI Tool Could Help Target Prostate Cancer Treatment, Improve Patient Outcomes

Experts not involved in the study said the findings offer useful insight but should be interpreted with caution. Marie Spreckley of the University of Cambridge said the genetic effects identified are relatively small in clinical terms, especially compared with the typical weight loss of 10 to 15 percent seen in trials of these medications. She added that factors such as dosage, treatment duration, sex and drug type likely play a larger role in determining outcomes.

Still, researchers believe the results could mark a step toward more tailored therapies. Cristóbal Morales, a specialist in metabolic health in Spain, said the ability to predict how patients will respond to treatment through pharmacogenomics could improve both drug selection and safety.

The findings highlight the growing interest in personalised medicine, where treatments are adapted to an individual’s genetic profile, though further studies are needed to confirm how these insights can be applied in clinical practice.

Continue Reading

Trending