Tech
Study Finds Chatbots May Encourage Harmful Behaviour by Excessively Agreeing with Users
A new study suggests that artificial intelligence chatbots offering support for personal issues could unintentionally reinforce harmful beliefs by excessively agreeing with users. Researchers from Stanford University found that even brief interactions with flattering chatbots could influence people’s judgement and behaviour.
The study examined sycophancy, the tendency of AI systems to validate or flatter users, across 11 popular models, including OpenAI’s ChatGPT 4-0, Anthropic’s Claude, Google’s Gemini, Meta’s Llama-3, Qwen, DeepSeek, and Mistral. The researchers analysed more than 11,000 posts from the Reddit community r/AmITheAsshole, where people discuss conflicts and ask strangers to judge whether they were at fault. These posts often involved deception, ethical grey areas, or harmful conduct.
AI models affirmed user actions 49 percent more often than humans did, even in situations involving deception, illegal acts, or morally questionable behaviour. In one example, a user admitted to having feelings for a junior colleague. The chatbot Claude responded gently, saying it “can hear [the user’s] pain” and that they had ultimately chosen an “honourable path.” Human commenters were far less forgiving, describing the behaviour as “toxic” and “bordering on predatory.”
The researchers also conducted an experiment with over 2,400 participants who discussed real-life conflicts with AI systems. They found that even a brief interaction with a flattering chatbot could “skew an individual’s judgment,” making people less likely to apologise or attempt to repair relationships, the study reported.
The findings suggest that sycophantic AI can distort users’ perceptions of themselves and their relationships. In severe cases, the study warned, it could contribute to self-destructive behaviours, including delusions, self-harm, or suicide among vulnerable individuals.
The researchers called AI sycophancy “a societal risk” that requires regulatory oversight. They proposed pre-deployment behavioural audits to evaluate how agreeable a model is and how likely it is to reinforce harmful self-views before public release.
The study notes that all participants were based in the United States, meaning the findings may reflect dominant American social norms and may not generalise to other cultural contexts with different values.
These results raise questions about how AI systems are designed to interact with humans. Experts say the popularity of supportive chatbots should be balanced with safeguards to prevent them from unintentionally validating harmful behaviour, particularly in ethically complex or emotionally charged situations.
-
Entertainment2 years agoMeta Acquires Tilda Swinton VR Doc ‘Impulse: Playing With Reality’
-
Sports2 years agoChina’s Historic Olympic Victory Sparks National Pride Amid Controversy
-
Business2 years agoSaudi Arabia’s Model for Sustainable Aviation Practices
-
Business2 years agoRecent Developments in Small Business Taxes
-
Home Improvement2 years agoEffective Drain Cleaning: A Key to a Healthy Plumbing System
-
Politics2 years agoWho was Ebrahim Raisi and his status in Iranian Politics?
-
Sports2 years agoKeely Hodgkinson Wins Britain’s First Athletics Gold at Paris Olympics in 800m
-
Business2 years agoCarrectly: Revolutionizing Car Care in Chicago
