Instagram has apologized for a “mistake” that led its algorithm to promote harmful diet content to people with eating disorders.
The social network had been automatically recommending terms including “appetite suppressant,” which campaigners feared could lead vulnerable people to relapse.
Facebook, which owns Instagram, told the BBC that the suggestions were triggered by a new search functionality:
As part of this new feature, when you tap on the search bar, we’ll suggest topics you may want to search for. Those suggestions, as well as the search results themselves, are limited to general interests, and weight loss should not have been one of them.
The company said the harmful terms have now been removed and the issue with the search feature has been resolved.
[Read: The biggest tech trends of 2021, according to 3 founders]
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!
Content that promotes disorders is banned from Instagram, while posts advertising weight-loss products are supposed to be hidden from users known to be under 18.
However, the error shows that policies alone can’t control the platform’s algorithms.
Greetings Humanoids! Did you know we have a newsletter all about AI? You can subscribe to it right here.
Get the TNW newsletter
Get the most important tech news in your inbox each week.