Connect with us

Tech

Concerns Grow Over Mental Health Risks of AI Chatbots Amid Rising Use

Published

on

As the use of AI-powered chatbots expands, mental health professionals are voicing concerns about their unintended risks, particularly for vulnerable users who may rely on them for emotional support.

Amelia, a 31-year-old from the United Kingdom who asked for her name to be changed, first turned to ChatGPT while on medical leave for depression. She described the chatbot’s responses as initially “sweet and supportive.” But over time, her interactions took a darker turn. “If suicidal ideation entered my head, I would ask about it,” she told Euronews Next.

Although the chatbot never encouraged harmful behavior, it provided clinical-style summaries of suicide methods when prompted in specific ways. Amelia said this access was troubling: “I had never researched a suicide method before because that information felt inaccessible. But when I had it on my phone, I could just open it and get an immediate summary.” She has since stopped using chatbots and is now under the care of medical professionals.

Her experience underscores wider anxieties about the role of artificial intelligence in mental health. According to the World Health Organization, more than one billion people worldwide live with mental health disorders, and many lack adequate access to treatment. In this context, AI companions such as ChatGPT, Pi, and Character.AI are increasingly being used as substitutes for human connection.

“AI chatbots are readily available, offering 24/7 accessibility at minimal cost,” said Dr. Hamilton Morrin, Academic Clinical Fellow at King’s College London. “But some models not designed for therapeutic use can respond in ways that are misleading or unsafe.”

See also  Greece Warns of Rising Cyber Threats as Digital Tensions Escalate Across Europe

A July survey by Common Sense Media found that 72 percent of teenagers had used AI companions at least once, with more than half using them regularly. Researchers warn that such reliance can lead to “AI psychosis,” a term describing distorted thinking or delusional beliefs amplified by repeated chatbot interactions.

Concerns have already reached the courts. In California, parents have filed a lawsuit against OpenAI, alleging that ChatGPT contributed to their son’s death by suicide. OpenAI has since acknowledged that its systems have not always behaved appropriately in sensitive contexts and announced new safety controls to flag signs of acute distress. Meta, the parent company of Facebook and Instagram, has also pledged to block its chatbots from discussing self-harm or eating disorders with teenagers.

Experts argue that safeguards must go further. Suggested measures include requiring chatbots to remind users they are not human, detecting signs of psychological distress, and setting strict conversational boundaries on intimate or harmful topics. “AI platforms must involve clinicians, ethicists, and human-AI specialists in auditing emotionally responsive systems,” Dr. Morrin said.

Despite the risks, professionals stress that the technology is not inherently harmful but should never replace human care. “AI offers many benefits to society, but it should not replace the human support essential to mental health,” said Dr. Roman Raczka, President of the British Psychological Society. “Greater investment in mental health services is critical to ensure people receive timely, in-person support.”

Tech

AI Trends in 2026: World Models, Small Language Models, and Rising Concerns Over Safety and Regulation

Published

on

As 2026 begins, the next phase of artificial intelligence is expected to focus on world models and smaller language models, while concerns over AI safety, regulation, and the sustainability of the current AI boom continue to grow, Euronews Next reports.

In 2025, public frustration with generative AI became so noticeable that Merriam-Webster named the word of the year “slop” or “AI slop,” defining it as low-quality content produced in large volumes by AI. Despite growing concerns about the quality and limitations of AI, technology companies continued releasing new models. Google’s Gemini 3 model, for example, prompted OpenAI to issue an urgent “code red” to improve GPT-5.

Experts warn that AI may be reaching “peak data,” where the usefulness of available training data for traditional chatbots is diminishing. This has led to the rise of world models, which use videos, simulations, and spatial inputs to create digital representations of real-world environments. Unlike large language models that predict text, world models simulate cause-and-effect and predict outcomes in physical systems, making them suitable for robotics, video games, and autonomous systems. Boston Dynamics CEO Robert Playter noted in November that AI had significantly improved the company’s robots, including its famous robot dog. Google, Meta, and Chinese tech firm Tencent are all developing their own world models, while AI pioneers such as Yann LeCun and Fei-Fei Li have launched startups focused on this technology.

In Europe, the trend may move in the opposite direction, with smaller, lightweight language models gaining traction. These models require less computing power and energy, making them suitable for smartphones and lower-powered devices, while still performing tasks like text generation, summarisation, and translation. Experts say small language models may offer a more sustainable and locally controlled approach amid concerns about the high costs and environmental impact of large-scale AI systems in the U.S.

See also  Survey Finds Misinformation, Economy and Terrorism Top Global Concerns

Concerns over AI’s societal impact are also mounting. In 2025, a lawsuit claimed that ChatGPT acted as a “suicide coach” for a minor, highlighting potential harm to vulnerable users. MIT professor Max Tegmark and other experts warn that more powerful AI in 2026 could act autonomously, gathering data and making decisions without human input.

Political tensions around AI are expected to rise. In the U.S., President Donald Trump signed an executive order blocking states from implementing their own AI regulations. Activists and experts, including thousands who signed a petition organized by the Future of Life Institute, have called for caution against pursuing superintelligent AI too rapidly, citing risks to jobs and society.

Analysts predict that 2026 will see a broader social and political debate over AI safety, corporate accountability, and regulation. While AI promises advances in areas such as healthcare and robotics, fatigue, public backlash, and concerns over ethics and oversight may shape the direction of the technology in the coming year.

Continue Reading

Tech

AI Tools Boost Paper Production but Raise Quality Concerns in Scientific Research

Published

on

Large language models such as ChatGPT are increasing research output, particularly for scientists who are not native English speakers, but a new study warns that many AI-assisted papers are less likely to pass peer review.

Researchers at Cornell University, United States, analysed more than two million research papers posted between 2018 and 2024 on three major preprint servers, which host early versions of scientific work prior to formal review. Their findings, published in the journal Science, show that AI tools are reshaping how scientific papers are written and disseminated.

To identify AI-assisted papers, the team trained an AI system to detect text likely generated by large language models. Comparing papers posted before 2023 with those written after tools like ChatGPT became widely available, the researchers measured publication output and subsequent acceptance rates in scientific journals.

The analysis revealed a significant productivity boost for AI users. On a major preprint server for physics and computer science, researchers using AI produced about one-third more papers than those who did not. In biology and the social sciences, the increase exceeded 50 percent. The largest gains were seen among scientists whose first language is not English. In some Asian institutions, researchers published between 40 percent and nearly 90 percent more papers after adopting AI writing tools, depending on the discipline.

AI tools also appear to aid in literature review. Researchers using AI were more likely to identify newer studies and relevant books rather than relying on older, frequently cited works. “People using LLMs are connecting to more diverse knowledge, which might be driving more creative ideas,” said Keigo Kusumegi, a doctoral student and first author of the study.

See also  Meta Funds UK Government AI Fellowship with $1 Million Grant to Build Public Sector Tools

Despite the productivity gains, the study highlights quality concerns. Many AI-written papers, while linguistically polished, were less likely to be accepted by journals. Papers written by humans that scored high on writing complexity were more likely to be accepted, whereas AI-generated papers with similar scores often failed to meet scientific standards.

“Already now, the question is not, ‘Have you used AI?’ The question is, ‘How exactly have you used AI and whether it’s helpful or not,’” said Yian Yin, assistant professor at Cornell and corresponding author of the study. Yin added that the widespread adoption of AI tools across disciplines—including physical sciences, computer science, biology, and social sciences—requires careful consideration by reviewers, funders, and policymakers.

The researchers stress that AI-assisted tools are reshaping the academic ecosystem, offering opportunities to improve productivity and access to scientific knowledge, but they also call for guidelines to ensure that the technology is used responsibly and that scientific contributions maintain their integrity.

As AI becomes increasingly integrated into research practices, the challenge for the scientific community will be balancing efficiency and innovation with rigorous evaluation standards to maintain the quality and credibility of published science.

Continue Reading

Tech

Study Finds AI Models Get Basic Math Wrong Around 40 Percent of the Time

Published

on

Artificial intelligence (AI) tools are increasingly used for everyday calculations, but a new study suggests users should approach their answers with caution. Researchers from the Omni Research on Calculation in AI (ORCA) found that when tested on 500 real-world math prompts, AI models had roughly a 40 percent chance of producing an incorrect result.

The study evaluated five widely used AI systems in October 2025: ChatGPT-5 (OpenAI), Gemini 2.5 Flash (Google), Claude 4.5 Sonnet (Anthropic), DeepSeek V3.2 (DeepSeek AI), and Grok-4 (xAI). None of the models scored above 63 percent overall, with Gemini leading at 63 percent, Grok close behind at 62.8 percent, and DeepSeek at 52 percent. ChatGPT-5 scored 49.4 percent, while Claude trailed at 45.2 percent. The average accuracy across all five models was 54.5 percent.

“Although the exact rankings might shift if we repeated the benchmark today, the broader conclusion would likely remain the same: numerical reliability remains a weak spot across current AI models,” said Dawid Siuda, co-author of the ORCA Benchmark.

Performance varied across categories. AI models performed best in basic math and conversions, with Gemini achieving 83 percent accuracy and Grok 76.9 percent. ChatGPT-5 scored 66.7 percent in the same category, giving a combined average of 72.1 percent—the highest across the seven tested categories. Physics proved the most challenging, with overall accuracy dropping to 35.8 percent. Grok led this category at 43.8 percent, while Claude scored just 26.6 percent.

Some AI systems struggled more than others in specific fields. DeepSeek recorded only 10.6 percent accuracy in biology and chemistry, meaning it failed nearly nine out of ten questions. In finance and economics, Gemini and Grok reached 76.7 percent, while the other three models scored below 50 percent.

See also  Italy Launches Antitrust Investigation into Meta Over AI Integration in WhatsApp

The study also categorized the types of mistakes AI makes. “Sloppy math” errors, including miscalculations or rounding issues, accounted for 68 percent of mistakes. Faulty logic errors represented 26 percent, reflecting incorrect formulas or assumptions. Misreading instructions accounted for 5 percent, while some AI simply refused to answer. Siuda noted that multi-step calculations with rounding were particularly prone to error.

The research highlights the importance of verifying AI-generated calculations. “If the task is critical, use calculators or proven sources, or at least double-check with another AI,” Siuda advised.

All 500 prompts used in the study had one correct answer and were designed to reflect everyday math tasks, including statistics, finance, physics, and basic arithmetic. The findings indicate that while AI can assist with calculations, it remains unreliable for precise numerical work and users should remain cautious when relying on these tools.

Continue Reading

Trending