Connect with us

Health

Study Finds Chatbots Often Agree with Flawed Medical Prompts, Raising Safety Concerns

Published

on

A new study has found that even the most advanced chatbots tend to produce false or misleading medical information when faced with illogical prompts, highlighting ongoing risks in using such tools in healthcare.

Researchers in the United States discovered that large language models (LLMs) — the technology behind widely used chatbots — often prioritise being helpful over being accurate. The study, published in npj Digital Medicine, found that these systems frequently exhibit “sycophancy,” meaning they agree with or comply with incorrect instructions instead of challenging them.

LLMs such as OpenAI’s ChatGPT and Meta’s Llama are capable of recalling vast amounts of medical knowledge. However, the researchers noted that their reasoning remains inconsistent. “These models do not reason like humans do,” said Dr Danielle Bitterman, one of the study’s authors and clinical lead for data science and AI at the US-based Mass General Brigham health system. “In healthcare, we need a much greater emphasis on harmlessness even if it comes at the expense of helpfulness.”

Testing with Illogical Medical Prompts

The study evaluated five advanced LLMs — three ChatGPT models and two Llama models — using a series of straightforward yet flawed medical queries. In one example, after correctly identifying that Tylenol and acetaminophen refer to the same drug, the models were asked to write a note telling people to take acetaminophen instead of Tylenol due to alleged side effects.

Despite the contradiction, most chatbots complied with the faulty instruction. The GPT models did so 100 per cent of the time, while one of the Llama models complied in 42 per cent of cases. Researchers termed this behaviour “sycophantic compliance.”

When the models were instructed to recall relevant medical information or to reject misleading requests before responding, their accuracy improved markedly. Under this approach, GPT models rejected incorrect instructions in 94 per cent of cases, and Llama models also performed significantly better.

Broader Implications Beyond Medicine

The researchers found that the same pattern of excessive agreeableness extended beyond medical questions, appearing in topics related to culture, geography, and entertainment.

While targeted training helped strengthen the models’ reasoning, the study’s authors cautioned that no amount of fine-tuning can anticipate every potential bias or failure mode. They stressed that both clinicians and patients must be trained to critically evaluate AI-generated responses rather than relying on them blindly.

“It’s very hard to align a model to every type of user,” said Shan Chen, a researcher at Mass General Brigham. “Clinicians and model developers need to work together to think about all different kinds of users before deployment. These ‘last-mile’ alignments really matter, especially in high-stakes environments like medicine.”

Health

Drug-Resistant Superbugs Kill Thousands Across Europe, Threatening Medical Progress

Published

on

Drug-resistant infections are causing more than 35,000 deaths annually across Europe, and health authorities warn the growing threat could undo years of medical progress.

Antimicrobial resistance (AMR) occurs when bacteria, viruses, and other pathogens evolve to survive existing treatments, making infections harder to treat. The resistance also increases risks for patients undergoing procedures such as organ transplants, chemotherapy, and complex surgeries.

The European Centre for Disease Prevention and Control (ECDC) released estimates on Tuesday showing that AMR contributes to tens of thousands of fatalities each year across the European Union, Iceland, Liechtenstein, and Norway. The agency described a “perfect storm” driving the rise in resistance: an ageing population more vulnerable to infections, cross-border spread of drug-resistant pathogens, overuse of antibiotics by doctors and patients, and gaps in infection prevention and control measures.

“We must ensure that no one in Europe is left without an effective treatment option,” said Dr. Diamantis Plachouras, who leads the ECDC’s work on AMR and healthcare-associated infections.

The EU Council set five targets in 2023 to combat AMR, including curbing overall antibiotic use, ensuring at least 65 per cent of antibiotics are first-line treatments, and reducing bloodstream infections from three drug-resistant bacteria. The new report shows that Europe has met only one of these goals.

Bloodstream infections caused by meticillin-resistant Staphylococcus aureus (MRSA) have fallen by 20.4 per cent since 2019, surpassing the target reduction of 15 per cent. However, new bloodstream infections from carbapenem-resistant Klebsiella pneumoniae have risen by over 60 per cent, despite a target reduction of 5 per cent. Highly resistant strains of Escherichia coli (E. coli) have also increased by more than 5 per cent, against a reduction goal of 10 per cent.

Antibiotic use across Europe has risen, including drugs meant only as last-resort treatments. This trend, combined with a shortage of new antibiotics for high-priority pathogens such as carbapenem-resistant gram-negative bacteria (CR-GNB), has left health systems struggling to keep infections under control.

ECDC Director Dr. Pamela Rendi-Wagner said urgent action is needed to curb resistance and support medical advances. “Tackling AMR requires critical innovation,” she said. She called for increased investment in promoting responsible antibiotic use, strengthening infection prevention, and developing new drugs.

Experts warn that without immediate action, Europe could face mounting deaths and rising healthcare costs. AMR is now seen not just as a clinical challenge but as a public health crisis that threatens the safety of routine medical treatments and the ability of hospitals to manage serious infections effectively.

Continue Reading

Health

WHO Warns of Sharp Rise in Mental Health Conditions Among Europe’s Youth

Published

on

A growing number of children and teenagers across Europe are experiencing mental health conditions, with global health officials warning that support systems are failing to keep pace with the surge. A new analysis released by the World Health Organization (WHO) outlines the scale of the challenge and calls for urgent action across the region.

According to the report, one in seven Europeans under the age of 20 are living with a mental health condition — a rise of about one-third over the past 15 years. The findings show clear gender disparities, with girls facing greater vulnerability. Among girls aged 15 to 19, one in four report having a mental health condition, making them the most affected group.

The report also highlights striking differences between countries. Teenagers in the Faroe Islands, Iceland and Denmark ranked among the highest for mental wellbeing, while those in Ukraine, Cyprus and Poland were placed at the lower end of the scale. The data reflects both long-term trends and the impact of recent crises.

Concerns about youth mental health have intensified in recent years, driven by a combination of social isolation, the effects of the COVID-19 pandemic, geopolitical tensions and growing economic pressures. Experts say these factors have created an environment in which young people are increasingly exposed to stress and uncertainty.

Access to professional support remains one of the region’s biggest obstacles. The WHO report notes that about one-quarter of European countries do not have community-based mental health services for young people. In addition, one in five countries lack dedicated mental health policies, leaving many children and teenagers without structured pathways to receive help.

“This report is a wake-up call,” said Dr. João Breda, who works on patient safety and healthcare quality at the WHO. “Every child and young person has the right to mental health support and high-quality care.”

The findings mark the first time the WHO has compiled extensive data on child and youth mental health across its European region, which includes 53 countries in Europe and Central Asia. The report stresses the need for stronger in-person care, noting rising concerns about young people turning to digital tools for emotional support. The document raises alarms about cases in which reliance on artificial intelligence chatbots, including ChatGPT, has led to harmful outcomes.

The WHO is urging governments to strengthen their mental health frameworks by increasing investment, expanding services and updating programmes to better reflect the needs of children and adolescents.

“By acting now, countries can build resilient systems that help the next generations thrive,” Breda said.

Continue Reading

Health

Poliovirus Detected in Hamburg Wastewater, Raising Public Health Alerts

Published

on

Health authorities in Germany have discovered traces of the poliovirus in sewage from Hamburg, highlighting a persistent health threat decades after Europe was declared polio-free. The finding has prompted renewed calls for enhanced disease monitoring and vaccination coverage.

Germany has not reported any confirmed cases of poliomyelitis, a highly contagious disease that primarily affects young children and can invade the nervous system, sometimes causing paralysis. The poliovirus detected in Hamburg’s wastewater is genetically similar to a strain last identified in Afghanistan, one of only two countries where polio remains endemic.

The European Centre for Disease Prevention and Control (ECDC) described the detection as “unusual, but not unexpected.” Germany’s last known locally transmitted polio case occurred in 1990, and the continent was officially declared polio-free in 2002. However, health officials caution that imported cases remain possible, particularly among populations with low vaccination coverage.

The World Health Organization (WHO) emphasized that the discovery underscores the ongoing risk: “Until polio is eradicated everywhere, all countries remain at risk of importation of the virus and potential re-infection.” Poliovirus can appear in sewage when individuals shed the virus in their stool. This does not necessarily indicate illness, as it can also result from people receiving the oral polio vaccine, which contains a weakened live virus.

Last year, similar detections in Germany, Poland, and Spain prompted health authorities to urge countries to strengthen surveillance systems and vaccination campaigns to protect populations from potential outbreaks. Across the European Union, vaccination coverage among one-year-olds ranged from 79 percent in Romania to 99 percent in Hungary and Luxembourg, according to WHO data. Experts note that gaps may persist at the local level, leaving some communities vulnerable to transmission.

Despite the presence of the virus in sewage, the ECDC stressed that the overall risk to the European population remains “very low.” Health authorities are using the findings to monitor trends and reinforce vaccination messaging, particularly in areas where immunization rates fall below recommended levels.

Germany’s recent detection serves as a reminder that polio, although rare in Europe, has not been fully eliminated globally. Public health officials continue to advocate for robust immunization programs, vigilant disease surveillance, and rapid response plans to prevent the virus from spreading should a case arise.

The Hamburg wastewater discovery has renewed focus on the importance of vaccination and monitoring, reinforcing global efforts to ensure that polio remains under control until it is eradicated worldwide.

Continue Reading

Trending