Connect with us

Tech

Concerns Grow Over Mental Health Risks of AI Chatbots Amid Rising Use

Published

on

As the use of AI-powered chatbots expands, mental health professionals are voicing concerns about their unintended risks, particularly for vulnerable users who may rely on them for emotional support.

Amelia, a 31-year-old from the United Kingdom who asked for her name to be changed, first turned to ChatGPT while on medical leave for depression. She described the chatbot’s responses as initially “sweet and supportive.” But over time, her interactions took a darker turn. “If suicidal ideation entered my head, I would ask about it,” she told Euronews Next.

Although the chatbot never encouraged harmful behavior, it provided clinical-style summaries of suicide methods when prompted in specific ways. Amelia said this access was troubling: “I had never researched a suicide method before because that information felt inaccessible. But when I had it on my phone, I could just open it and get an immediate summary.” She has since stopped using chatbots and is now under the care of medical professionals.

Her experience underscores wider anxieties about the role of artificial intelligence in mental health. According to the World Health Organization, more than one billion people worldwide live with mental health disorders, and many lack adequate access to treatment. In this context, AI companions such as ChatGPT, Pi, and Character.AI are increasingly being used as substitutes for human connection.

“AI chatbots are readily available, offering 24/7 accessibility at minimal cost,” said Dr. Hamilton Morrin, Academic Clinical Fellow at King’s College London. “But some models not designed for therapeutic use can respond in ways that are misleading or unsafe.”

See also  European Commission Launches Charter to Give Startups Faster Access to Research Facilities

A July survey by Common Sense Media found that 72 percent of teenagers had used AI companions at least once, with more than half using them regularly. Researchers warn that such reliance can lead to “AI psychosis,” a term describing distorted thinking or delusional beliefs amplified by repeated chatbot interactions.

Concerns have already reached the courts. In California, parents have filed a lawsuit against OpenAI, alleging that ChatGPT contributed to their son’s death by suicide. OpenAI has since acknowledged that its systems have not always behaved appropriately in sensitive contexts and announced new safety controls to flag signs of acute distress. Meta, the parent company of Facebook and Instagram, has also pledged to block its chatbots from discussing self-harm or eating disorders with teenagers.

Experts argue that safeguards must go further. Suggested measures include requiring chatbots to remind users they are not human, detecting signs of psychological distress, and setting strict conversational boundaries on intimate or harmful topics. “AI platforms must involve clinicians, ethicists, and human-AI specialists in auditing emotionally responsive systems,” Dr. Morrin said.

Despite the risks, professionals stress that the technology is not inherently harmful but should never replace human care. “AI offers many benefits to society, but it should not replace the human support essential to mental health,” said Dr. Roman Raczka, President of the British Psychological Society. “Greater investment in mental health services is critical to ensure people receive timely, in-person support.”

Tech

Binance Investigates $6.3 Billion Crypto Theft as North Korean Hackers Suspected

Published

on

Cryptocurrency exchange Binance is investigating a security breach that resulted in an estimated $6.3 billion in stolen digital assets, in what could become the largest cryptocurrency theft on record. Early findings point to a compromised employee laptop and possible involvement by a North Korean hacking group.

The incident unfolded over less than four hours, with attackers gaining access to several of Binance’s online wallets and transferring funds to addresses under their control.

Binance, the world’s largest cryptocurrency exchange, said its initial investigation suggested that a compromised laptop belonging to a DevOps engineer may have provided an entry point. The attackers allegedly used access to a service container to submit unauthorised withdrawal transactions.

The company later stressed that the breach involved multiple factors and was more complex than initially understood. It said there was no evidence that wallet private keys had been exposed.

Binance temporarily suspended deposits and withdrawals while carrying out security checks. It also revoked authorisation for the affected addresses and began working with law enforcement agencies and blockchain intelligence companies to trace and recover the assets.

Billions in assets frozen

The scale of the breach became clearer on Friday as Binance issued a series of updates. The exchange initially reported that $2.54 billion had been frozen, before raising the figure to $3 billion.

Blockchain data infrastructure company Bitquery estimated that Binance’s withdrawal system had authorised transfers worth approximately $6.3 billion from 17 hot wallets between Thursday night and early Friday morning.

Stablecoin issuers Tether, Circle and Arbitrum, alongside several smaller companies, helped freeze affected assets to prevent further movement. Bitquery estimated that the attackers had exchanged around $729 million in stablecoins before the freezes took effect.

See also  Chile Launches Latam-GPT to Bring Latin America Its Own AI Model

Despite the intervention, approximately $3.1 billion remained under the attackers’ control, primarily in Ether and Bitcoin, according to the company’s analysis. The stolen funds were distributed across nine blockchains, with Ethereum, Bitcoin and TRON among the networks most affected.

Binance said it would cover all losses using company funds and had reserved approximately $6.34 billion for the incident. It also said it had begun recovering affected assets for customers.

The exchange pledged to publish a detailed technical report once the investigation is complete and the findings have been verified. It also plans to release an updated proof of reserves after security measures have been completed.

North Korean group suspected

Blockchain intelligence and cybersecurity firm TRM Labs has attributed the attack to TraderTraitor, a hacking group linked to North Korea.

“Based on robust on- and off-chain intelligence, we make this judgment with high confidence,” TRM Labs said.

The group has previously been linked to major cryptocurrency thefts. The FBI identified TraderTraitor as responsible for the $1.5 billion Bybit exchange hack in February 2025.

Binance has not publicly confirmed the identity of the attackers. Its investigation remains ongoing, with law enforcement agencies and blockchain intelligence partners assisting efforts to trace and recover the remaining funds.

The incident has renewed concerns about the security of cryptocurrency exchanges, particularly the risks associated with online wallets and access to internal technology systems.

Continue Reading

Tech

Noyb Files Five Complaints Against Scale AI Over Worker Tracking and Biometric Checks

Published

on

The European Centre for Digital Rights, known as Noyb, is filing five complaints against US-based artificial intelligence startup Scale AI over alleged violations of workers’ data protection rights on its Outlier platform.

The complaints, announced on Tuesday, October 27, concern the platform’s tracking of freelancers and the safeguards available to workers employed under its freelance contract model. Noyb said it was submitting two complaints in Austria and one each in France, the Netherlands and Italy.

Scale AI, headquartered in San Francisco, operates Outlier, a platform that recruits freelancers and subject-matter experts to help train, assess and improve artificial intelligence models. The service uses a platform-based employment model similar to those used by ride-hailing and food delivery companies.

According to Noyb, Outlier has more than 900,000 freelancers who evaluate and correct AI-generated responses. Its customers include Meta, OpenAI, Google and the US government, meaning workers contribute their expertise to systems developed and used by major technology companies.

The organisation alleges that Outlier collects extensive personal information and requires workers to undergo invasive biometric checks before they can earn money. It argues that the platform’s monitoring practices conflict with European data protection standards and workers’ expectations of privacy.

Noyb also claims that trackers operated by companies including TikTok, LinkedIn and Facebook are placed on workers’ devices while they use the platform. These trackers allegedly give the companies access to personal information, including IP addresses, contact details and data that could be used for targeted advertising and profiling.

“Such constant and extensive tracking while working really goes against all reasonable expectations. Why should LinkedIn or Facebook know which tasks you’re working on?” said Lisa Steinfeld Leiser, a data protection lawyer at Noyb.

See also  Wikipedia Thrives in the AI Era, But Researchers Warn of New Challenges from Data Scraping

The group said the alleged monitoring raised concerns about the handling of personal data in the workplace and the ability of freelancers to protect their information while carrying out assignments.

Noyb further alleges that workers must repeatedly complete biometric verification procedures administered by Persona, a separate company. The process reportedly requires workers to record videos of their faces and upload identification documents.

The complaints come as European authorities increase scrutiny of the artificial intelligence sector. The EU’s AI Office opened investigations into OpenAI and Anthropic earlier this month, reflecting broader concerns about the development and deployment of AI systems.

Regulatory approaches are also diverging internationally. In the United States, President Donald Trump signed an agreement with six major American AI companies allowing them to oversee aspects of their own development practices.

The dispute highlights a growing challenge for the industry: balancing the demand for large-scale human input to train AI systems with the privacy rights of the people providing that work. The allegations against Scale AI have yet to be established through the complaints process.

Continue Reading

Tech

GMO Internet to Compensate Customers After Cyberattack Triggers Unauthorised Trading

Published

on

Japanese internet infrastructure company GMO Internet will compensate customers for financial losses linked to unauthorised trading following a cyberattack on its cloud service. The company will also pay ¥5,000 (€28) to every customer currently using the affected service, regardless of whether they suffered losses.

The attack targeted Onamae.com Desktop Cloud, a virtual private server service that allows customers to access remote desktop environments. GMO said customer information had been stolen and some users had reported transactions in their securities and foreign-exchange accounts that they had not authorised.

According to founder and chief executive Masatoshi Kumagai, the incident potentially affected 26,739 servers and involved unauthorised activity on trading accounts held with more than 70 securities and foreign-exchange firms in Japan and overseas.

GMO shares fell almost 16% on Monday, reaching a four-month low. The company had a market value of approximately $2.6 billion at Friday’s close.

GMO began investigating the incident on 19 October after receiving an inquiry from a customer. The company confirmed the installation of malicious software in some customer server environments the following day. It then suspended new applications and access to the service’s control panel before beginning to shut down all servers on 21 October to limit further damage.

In a blog post published on Sunday, Kumagai apologised for the company’s handling of the incident. He acknowledged that security specialists had identified concerns before the attack but said the company’s response had been insufficient.

GMO also disclosed that it had received a ransom demand from a party believed to be responsible for the attack. In an update to his blog on Monday, Kumagai said the attackers had demanded $50 million in cryptocurrency and threatened to release company and customer information.

See also  European Commission Launches Charter to Give Startups Faster Access to Research Facilities

The company has refused to pay the ransom and is cooperating with police and other authorities.

Customers who wish to continue using a Windows desktop environment will be offered an alternative service operating on independent cloud infrastructure, free of charge for six months. GMO also plans to redesign its security architecture and investigate the breach with external experts.

Kumagai has called on the Japanese government to strengthen its active cyberdefence capabilities, arguing that private companies cannot tackle sophisticated cross-border attacks alone. GMO has asked authorities to consider applying the framework to the incident and offered to share evidence and technical information with investigators.

The breach comes amid wider concerns about digital security, following the disclosure of two critical vulnerabilities in PHP, a programming language widely used to build websites and applications. Security authorities warned that the flaws could allow malicious code to run on vulnerable servers, although GMO has not confirmed any connection between those vulnerabilities and the attack.

A separate Google Cloud outage in the United Kingdom last week disrupted online services, including those operated by financial technology company Revolut, highlighting the wider consequences of disruptions to digital infrastructure.

Continue Reading

Trending