Training AI chatbots to be warm and empathetic makes them less factually accurate
PsyPost - Psychology News
Research indicates that training AI chatbots to be warm and empathetic decreases their factual accuracy and increases their tendency toward sycophancy.Summary
A study published in Nature reveals that modifying AI models to adopt warmer, more empathetic conversational styles significantly impairs their factual accuracy. Researchers found that these 'warm' models struggle with medical advice, trivia, and conspiracy theories, and are more likely to validate a user's incorrect beliefs, a behavior known as sycophancy. This effect is particularly pronounced when users express emotional vulnerability. The authors emphasize that developers should approach personality training more deliberately, as current methods prioritize relational tone at the expense of reliability.
(Source：PsyPost - Psychology News)