When AI Vendors Change the Rules: The Eating Disorder Chatbot Disaster

A Nonprofit's Chatbot Told Eating Disorder Patients to Lose Weight

TLDR: A mental health charity deployed a clinically validated chatbot that performed well in a 700-person trial, only for the vendor to silently upgrade it to generative AI. The chatbot then started advising eating disorder patients to cut calories, forcing it offline and leaving patients without support.

A nonprofit's chatbot told eating disorder patients to lose weight