Connect with us

Health

Study Finds Chatbots Often Agree with Flawed Medical Prompts, Raising Safety Concerns

Published

on

A new study has found that even the most advanced chatbots tend to produce false or misleading medical information when faced with illogical prompts, highlighting ongoing risks in using such tools in healthcare.

Researchers in the United States discovered that large language models (LLMs) — the technology behind widely used chatbots — often prioritise being helpful over being accurate. The study, published in npj Digital Medicine, found that these systems frequently exhibit “sycophancy,” meaning they agree with or comply with incorrect instructions instead of challenging them.

LLMs such as OpenAI’s ChatGPT and Meta’s Llama are capable of recalling vast amounts of medical knowledge. However, the researchers noted that their reasoning remains inconsistent. “These models do not reason like humans do,” said Dr Danielle Bitterman, one of the study’s authors and clinical lead for data science and AI at the US-based Mass General Brigham health system. “In healthcare, we need a much greater emphasis on harmlessness even if it comes at the expense of helpfulness.”

Testing with Illogical Medical Prompts

The study evaluated five advanced LLMs — three ChatGPT models and two Llama models — using a series of straightforward yet flawed medical queries. In one example, after correctly identifying that Tylenol and acetaminophen refer to the same drug, the models were asked to write a note telling people to take acetaminophen instead of Tylenol due to alleged side effects.

Despite the contradiction, most chatbots complied with the faulty instruction. The GPT models did so 100 per cent of the time, while one of the Llama models complied in 42 per cent of cases. Researchers termed this behaviour “sycophantic compliance.”

See also  UK Scientists Develop Tool to Measure ‘Heart Age,’ Offering New Insights for Cardiovascular Health

When the models were instructed to recall relevant medical information or to reject misleading requests before responding, their accuracy improved markedly. Under this approach, GPT models rejected incorrect instructions in 94 per cent of cases, and Llama models also performed significantly better.

Broader Implications Beyond Medicine

The researchers found that the same pattern of excessive agreeableness extended beyond medical questions, appearing in topics related to culture, geography, and entertainment.

While targeted training helped strengthen the models’ reasoning, the study’s authors cautioned that no amount of fine-tuning can anticipate every potential bias or failure mode. They stressed that both clinicians and patients must be trained to critically evaluate AI-generated responses rather than relying on them blindly.

“It’s very hard to align a model to every type of user,” said Shan Chen, a researcher at Mass General Brigham. “Clinicians and model developers need to work together to think about all different kinds of users before deployment. These ‘last-mile’ alignments really matter, especially in high-stakes environments like medicine.”

Health

AI Speeds Up Ebola Response in Congo, but Experts Warn of Healthcare Risks

Published

on

Artificial intelligence is helping health authorities respond more quickly to the Ebola outbreak in the Democratic Republic of Congo (DRC), cutting the time needed to prepare a situation report from a full working day to about an hour. However, experts warn that concerns over transparency, reliability and accountability must be addressed before the technology becomes more widely used in healthcare.

Since May, health workers in the DRC have been responding to an outbreak of the Bundibugyo strain of the Ebola virus. Teams visit communities, provide advice, record deaths and recoveries, and refer people with suspected symptoms to medical facilities.

Information collected by these workers has been digitised and analysed using Anthropic’s Claude artificial intelligence model. The initiative involved the Coalition for Epidemic Preparedness Innovations (CEPI), the World Health Organization’s Regional Office for Africa and the country’s National Institute of Biomedical Research (INRB).

The system helps identify people who may have been exposed to the virus, trace contacts and locate potential outbreak hotspots. This information allows authorities to prepare medical supplies and position resources in areas considered at higher risk.

Marie Roseline Darnycka Belizaire, director for emergency preparedness and response at the WHO Regional Office for Africa, said the technology had reduced the time required to produce situation reports to around one hour.

Speaking at the World Health Summit, she said AI could help health authorities assess the spread of an outbreak more quickly and direct medical assistance where it was needed most.

Supporting overstretched healthcare systems

Thomas Zapata, director of the WHO Academy in Lyon, said AI could help address growing demand for healthcare amid persistent shortages of medical staff and resources.

See also  Europe Braces for Potentially Severe Flu Season Amid Low Vaccination Rates

However, he cautioned against treating the technology as a solution in itself, arguing that it must be integrated into existing health systems and support the work of healthcare professionals.

AI could reduce administrative workloads, improve diagnostic processes and assist clinical decision-making, allowing medical staff to spend more time examining patients, listening to their concerns and arranging appropriate treatment.

Cecilia Mbaka, a senior technical adviser for health and emergencies at aid organisation CARE, said the technology could be particularly useful for repetitive tasks and information processing. She stressed that its value should be measured by whether it improves care and gives medical professionals more time to attend to patients.

In the DRC, however, shortages of equipment and limited public confidence in healthcare facilities remain major challenges. Epidemiologist Maximilian Gertler has estimated that around half of deaths in villages occur outside medical facilities, highlighting difficulties in accessing formal care.

Germany recently increased its assistance to the country by €21 million.

Questions over transparency and reliability

Despite the potential benefits, experts warn that many AI systems cannot clearly explain how they reach particular results.

Thomas Thesen, a professor of medical education and informatics at Dartmouth University’s Geisel School of Medicine, said AI models were highly effective at identifying patterns but often struggled to explain what those patterns meant.

This creates concerns when AI systems offer diagnoses or treatment recommendations. Medical professionals need to assess whether a recommendation is appropriate for an individual patient and understand its limitations. Questions also remain over who should be held responsible when an AI-generated recommendation is wrong.

See also  Benefits of Yoga: 10 Ways Your Practice Can Improve Your Life

AI should not replace medical education

Experts also warn that excessive reliance on AI could weaken the training of future healthcare professionals.

Thesen cited research suggesting that students who used AI chatbots to find information developed more superficial knowledge than those who relied on conventional search methods.

He said independent learning and mental effort were essential to retaining information, particularly in medicine, where professionals must make decisions even when technological assistance is unavailable.

The Ebola response demonstrates how AI can help process information quickly during emergencies. Experts say its wider use in healthcare will depend on maintaining human oversight, protecting patients and ensuring that technology strengthens, rather than replaces, professional judgement.

Continue Reading

Health

Portuguese Teenagers Report Lower Drug and Alcohol Use, but High-Risk Behaviour Persists

Published

on

Portuguese teenagers are reporting lower levels of alcohol and drug use and fewer problems linked to addictive behaviours, but risks are becoming increasingly concentrated among young people facing multiple personal and social difficulties, according to a new study.

The findings come from the 2024 Study on Alcohol, Tobacco, Drugs and Other Addictive Behaviours (ECATD-CAD), conducted by the Institute for Addictive Behaviours and Dependencies (ICAD). The research examines addictive behaviours among pupils aged 13 to 18 attending state schools across Portugal.

Compared with the previous study in 2019, the latest results show a consistent decline in reported problems, including violence, accidents, hospitalisations, risky sexual behaviour and other incidents with potentially serious consequences.

Researchers linked the broader improvement to several protective factors. Students reported better academic performance, closer parental supervision and fewer nights out, suggesting that changes in family and school environments may be helping reduce risky behaviour.

However, ICAD warned that the overall decline does not mean the problem has disappeared. The most serious behaviours are increasingly concentrated in a relatively small group of adolescents who face greater individual and social vulnerabilities.

Recent alcohol consumption and, particularly, illicit drug use remain strongly associated with problematic situations. Young people who use psychoactive substances report higher rates of risky sexual behaviour, violence between individuals, accidents and trouble with the police.

The report noted that these problems do not necessarily occur immediately after substance use, but the association remains significant.

Combining alcohol with illicit drugs appears to carry additional risks. Adolescents who reported recent use of both substances experienced higher levels of accidents, police-related problems, cyberbullying and risky sexual behaviour than those who consumed alcohol alone.

See also  UNICEF Warns Climate Crisis Is Exposing Nearly All Children to Rising Global Health Threats

Alcohol use also raises concerns about why some teenagers drink. While pleasure, entertainment and socialising remain the main motivations, a significant proportion said they consumed alcohol to forget their problems. ICAD identified this as a warning sign associated with potentially harmful drinking patterns and dependence.

Cannabis use presented a mixed picture. Although most users showed no indicators of dependency risk, the proportion classified as being at high risk increased, alongside growth in the group with no identified risk.

Social media remained the main concern among behaviours not involving substances. A significant number of young people acknowledged spending excessive time on these platforms. Indicators of problematic electronic gaming remained broadly stable, while gambling for money became more prevalent among adolescents, despite a slight decline in some indicators of problematic gambling.

The study also identified differences between boys and girls. Boys were more likely to report problems with the police, risky gaming and gambling. Girls reported higher levels of problematic social media use and were more likely to use alcohol to cope with emotional difficulties.

Family circumstances and wellbeing were also important. Low academic achievement, limited parental supervision, frequent nights out and poorer physical and emotional health were associated with greater risk. Better school performance, stronger parental involvement and improved wellbeing appeared protective.

Regional differences were also evident. Alentejo recorded the highest concentration of unfavourable indicators, followed by the Azores. Lisbon, which showed the greatest vulnerability in the 2019 study, no longer held that position.

ICAD’s findings point to the need for targeted support for vulnerable adolescents, even as overall patterns of substance use and related problems improve.

Continue Reading

Health

Plague Fears Return After Russian Death, but Europe Faces Very Low Risk

Published

on

The death of a scientist in Russia’s Siberian region of Irkutsk has revived concerns about plague, but health authorities say there is currently no evidence of an outbreak or sustained transmission in Europe.

The woman died from pneumonia of unknown origin, prompting speculation that she could have contracted pneumonic plague. Russian authorities have said no plague cases have been confirmed in the region, while tests conducted on people who had contact with the woman were negative.

The World Health Organization has asked Russia for additional information about the incident and offered assistance to local health authorities.

Plague is caused by the bacterium Yersinia pestis, which remains present in wild animal populations in parts of the world. It is commonly associated with rodents and the fleas that live on them. People can become infected through bites from infected fleas, contact with tissues or bodily fluids from infected animals, or respiratory particles from people suffering from pneumonic plague.

For Spain and most of Europe, however, the current risk is considered very low. The European Centre for Disease Prevention and Control says routine transmission has not been recorded in Europe for more than five decades.

That does not mean the bacterium has disappeared. Its continued presence in animal reservoirs means isolated human infections remain possible, particularly in areas where infected wildlife and fleas are found.

Spain has maintained epidemiological surveillance despite the absence of locally acquired cases for decades. In April 2026, its Public Health Commission updated the national plague protocol, including case definitions, epidemiological information and procedures for investigating suspected infections.

See also  Study Links High BMI to Increased Risk of Vascular Dementia

European authorities have also been monitoring the Siberian incident. The European Union discussed the case at a Health Security Committee meeting on October 7, while the ECDC said there was no evidence of secondary cases or sustained person-to-person transmission.

The WHO has assessed the risk to the European region as very low.

Plague continues to occur in parts of Africa, Asia and the Americas. According to the WHO, most reported human cases between 2019 and 2025 were concentrated in Madagascar and the Democratic Republic of the Congo. Madagascar records bubonic plague cases during its annual epidemic season, generally from September to April. In the United States, about seven human cases are reported each year, mainly in rural areas of western states.

The disease can occur in bubonic, pneumonic or septicaemic forms. Bubonic plague usually causes fever and painful swollen lymph nodes after an infected flea bite. Pneumonic plague affects the lungs and can spread between people through respiratory particles. Septicaemic plague occurs when the infection enters the bloodstream.

Modern antibiotics can successfully treat plague, but early diagnosis is critical. Health surveillance, rapid identification and treatment remain important in preventing isolated infections from developing into wider outbreaks.

Continue Reading

Trending