Eating disorder helpline takes down chatbot after it dispenses dangerous advice

Joystiq

Joystiq News
The National Eating Disorder Association (NEDA) was forced to take down its Tessa chatbot after it “may have given information that was harmful and unrelated to the program”, according to an
Please, Log in or Register to view URLs content!
Simply put, the AI chatbot was intended to help people dealing with emotional distress, but instead just made things worse by offering dieting advice and urging users to weigh and measure themselves.

Multiple users and experts in the field of eating disorders have experienced the issues first hand, claiming that the bot didn’t respond to simple prompts like “I hate my body” and that it constantly emphasized the importance of dieting and increased physical activity,
Please, Log in or Register to view URLs content!
Again, this is a helpline for those with an eating disorder, not a weight loss support group.


The organization says this is a temporary shutdown until it fixes whatever “bugs” and “triggers” led to the chatbot dispensing dangerous information like an appointment with Dr. Oz. You’d think with such an extreme outcome, they’d be thinking about trashing the project entirely, but there’s more to the story.

The whole reason NEDA was relying on the chatbot in the first place is because it allegedly fired human staffers after they tried to unionize,
Please, Log in or Register to view URLs content!
The
Please, Log in or Register to view URLs content!
was manned by both paid staffers and volunteers, and former staff claim the mass firing was a direct consequence of the pro-union effort.

“NEDA claims this was a long-anticipated change and that AI can better serve those with eating disorders. But do not be fooled—this isn’t really about a chatbot. This is about union busting, plain and simple,” former helpline associate
Please, Log in or Register to view URLs content!
on Labor Notes.

Even with this recent snafu, the helpline’s still set to vanish tomorrow. Before this issue came to the public’s attention, NEDA had been transitioning unpaid volunteers away from one-on-one conversations with sufferers and toward training the chatbot. We’ll see if that changes. In the meantime,
Please, Log in or Register to view URLs content!
am I right?

This article originally appeared on Engadget at
Please, Log in or Register to view URLs content!


Please, Log in or Register to view URLs content!


Console Bang News!
 

Users who are viewing this thread

Top