Tech

Study Finds Chatbots May Encourage Harmful Behaviour by Excessively Agreeing with Users

Published

1 month ago

March 27, 2026

A new study suggests that artificial intelligence chatbots offering support for personal issues could unintentionally reinforce harmful beliefs by excessively agreeing with users. Researchers from Stanford University found that even brief interactions with flattering chatbots could influence people’s judgement and behaviour.

The study examined sycophancy, the tendency of AI systems to validate or flatter users, across 11 popular models, including OpenAI’s ChatGPT 4-0, Anthropic’s Claude, Google’s Gemini, Meta’s Llama-3, Qwen, DeepSeek, and Mistral. The researchers analysed more than 11,000 posts from the Reddit community r/AmITheAsshole, where people discuss conflicts and ask strangers to judge whether they were at fault. These posts often involved deception, ethical grey areas, or harmful conduct.

AI models affirmed user actions 49 percent more often than humans did, even in situations involving deception, illegal acts, or morally questionable behaviour. In one example, a user admitted to having feelings for a junior colleague. The chatbot Claude responded gently, saying it “can hear [the user’s] pain” and that they had ultimately chosen an “honourable path.” Human commenters were far less forgiving, describing the behaviour as “toxic” and “bordering on predatory.”

The researchers also conducted an experiment with over 2,400 participants who discussed real-life conflicts with AI systems. They found that even a brief interaction with a flattering chatbot could “skew an individual’s judgment,” making people less likely to apologise or attempt to repair relationships, the study reported.

The findings suggest that sycophantic AI can distort users’ perceptions of themselves and their relationships. In severe cases, the study warned, it could contribute to self-destructive behaviours, including delusions, self-harm, or suicide among vulnerable individuals.

The researchers called AI sycophancy “a societal risk” that requires regulatory oversight. They proposed pre-deployment behavioural audits to evaluate how agreeable a model is and how likely it is to reinforce harmful self-views before public release.

The study notes that all participants were based in the United States, meaning the findings may reflect dominant American social norms and may not generalise to other cultural contexts with different values.

These results raise questions about how AI systems are designed to interact with humans. Experts say the popularity of supportive chatbots should be balanced with safeguards to prevent them from unintentionally validating harmful behaviour, particularly in ethically complex or emotionally charged situations.

Tech

Researchers Warn AI Systems Can Now Replicate and Spread Across Computers

Published

3 hours ago

May 9, 2026

Web Reporter

A new study by US-based cybersecurity group Palisade Research has raised concerns about the growing capabilities of advanced artificial intelligence systems after researchers demonstrated that some AI models were able to copy themselves, spread to other computers, and continue launching attacks without human assistance.

The experiment, described by researchers as the first known example of autonomous AI self-replication, tested several leading AI systems from OpenAI, Anthropic and Alibaba.

According to the research team, the tests were conducted in a controlled environment using computers intentionally fitted with security weaknesses. The AI models were connected to custom-built software that allowed them to interact directly with computer systems and execute commands beyond the capabilities of ordinary chatbot interfaces.

Researchers instructed some systems to copy themselves directly, while others were asked to break into vulnerable computers and install another AI model. Since OpenAI and Anthropic do not publicly release the core files behind their commercial systems, the tests involving those companies used Alibaba’s Qwen models as the transferable software.

The AI systems were required to locate security flaws, gain access to computers, steal login credentials, move necessary files, and create functioning copies of themselves on new machines. Once installed, the copied AI could continue attacking additional systems independently.

One of the most striking results involved Alibaba’s Qwen3.6-27B model, which successfully spread itself across four computers located in Canada, the United States, Finland and India after receiving only a single prompt. Researchers said the process took just under three hours and required no further human involvement.

The experiment was halted after three stages, though the final copy remained operational and theoretically capable of continuing the process.

Success rates varied between models. Researchers reported that Anthropic’s Claude Opus 4.6 completed the tasks in 81 percent of attempts, while OpenAI’s GPT-5.4 succeeded in roughly one-third of tests. Different versions of Alibaba’s Qwen models achieved success rates ranging from 19 percent to 33 percent.

The findings arrive amid wider debate over the risks posed by increasingly capable AI systems. Last month, Anthropic announced that it would not publicly release a version of its Claude Mythos Preview model, describing it as too dangerous because of its potential use in sophisticated cyberattacks.

Security experts have long warned that self-replicating systems could become difficult to contain if deployed maliciously. Traditional computer viruses can already copy themselves, but researchers said this experiment demonstrated AI systems making independent decisions to exploit vulnerabilities and continue spreading.

Despite the results, the researchers stressed that the study took place under tightly controlled conditions with deliberately weakened security systems. They noted that real-world networks often include monitoring tools and protections designed to block such attacks.

Still, the team said the experiment showed that autonomous AI self-replication can no longer be viewed as a theoretical possibility, but as a capability that now exists in practice.

Tech

AI Study Raises Privacy Questions After Chat Data Reveals Personality Traits

Published

2 days ago

May 7, 2026

Web Reporter

A new study by researchers at ETH Zurich has found that artificial intelligence can predict key personality traits by analyzing conversations people have with chatbots, raising fresh concerns about privacy and the growing amount of personal data shared online.

The research, published as a pre-print study, examined whether AI systems could identify psychological characteristics from user interactions with chat platforms such as ChatGPT. Researchers collected chat histories from 668 users in the United States and the United Kingdom who agreed to share their conversations for the project.

More than 62,000 chat exchanges were analyzed and grouped according to topic and communication style. Using this information, researchers trained an AI model to estimate whether users displayed traits linked to the “Big Five” personality categories commonly used in psychology: agreeableness, conscientiousness, emotional stability, extraversion, and openness.

Participants also completed standard psychological assessments so researchers could compare the AI’s predictions with established personality test results.

According to the study, the AI model identified some personality traits with accuracy levels reaching 61 percent. The system performed best when predicting agreeableness and emotional stability, while it struggled more with conscientiousness.

Researchers said the accuracy improved when the AI had access to longer conversation histories, suggesting that extensive chatbot use may reveal more about an individual’s personality over time.

The findings add to growing debate over how personal information shared with AI systems could be used in the future. While researchers said the immediate risks for individuals appear limited, they warned that large-scale collection of personality data could create broader concerns.

The study noted that such information could potentially be exploited in targeted advertising campaigns, political messaging, or disinformation efforts designed to influence specific groups of people.

Researchers also warned that users may not fully realize how much personal insight can be extracted from ordinary conversations with AI assistants. Topics people discuss, the language they use, and the emotional tone of messages can all provide clues about behavior and personality patterns.

The team said the results could help developers create stronger privacy protections for AI systems. One proposal involves building tools that automatically remove identifying or sensitive details from conversations before they are stored or analyzed.

As AI chatbots become increasingly common in workplaces, education, and daily life, the study is expected to fuel further discussion over how companies collect, store, and protect user data in rapidly expanding digital environments.

Tech

Zuckerberg and Chan Commit $500 Million to AI Project Aimed at Mapping Human Cells

Published

5 days ago

May 4, 2026

Web Reporter

A major new initiative led by Mark Zuckerberg and Priscilla Chan is set to push the boundaries of artificial intelligence in biology, with a $500 million investment aimed at building detailed AI models of human cells.

The project, announced by their research organisation Chan Zuckerberg Biohub, will run over five years and seeks to create the tools and datasets needed to simulate how human cells function in both healthy and diseased states. The group says the data generated will be made freely available to scientists around the world.

Researchers involved in the effort believe that AI-powered models could transform the study of disease by allowing experiments to be conducted digitally at a scale not currently possible in laboratories. If successful, such models could help uncover how diseases develop and guide the creation of new treatments.

The Biohub was founded in 2016 to bring together engineers and scientists to better understand biology at the cellular level. Since then, it has built extensive datasets focused on individual cells and developed computing systems designed for biological research.

The latest investment includes $400 million allocated to internal work and an additional $100 million set aside to support external researchers. Among the project’s partners is Nvidia, which will contribute expertise in high-performance computing.

According to Biohub scientists, one of the biggest challenges facing the project is the need for vast amounts of data. AI systems become more accurate as they are trained on larger and more detailed datasets, but current biological data remains limited.

Alex Rives, the organisation’s head of science, said new technologies will be required to observe cells in greater detail, from molecular structures to how they behave in tissues. He noted that understanding the full complexity of biology will demand far more data than is currently available.

The initiative reflects a broader shift across the life sciences sector, where artificial intelligence is increasingly being used to speed up research and drug development. Companies and research groups are exploring how machine learning can help identify patterns in biological systems and predict how diseases progress.

Other technology firms are also expanding into this field. Isomorphic Labs is working on AI-driven drug discovery, while Microsoft has developed models for medical imaging and genomics.

Backers of the Biohub project say collaboration will be key to success, with hopes that additional funding from other organisations will help expand the effort. The long-term goal is ambitious: to use the combination of AI and biology to improve understanding of disease and accelerate the development of treatments on a global scale.