Microsoft Corp. apologized after Twitter users exploited its artificial-intelligence chat bot Tay, teaching it to spew racist, sexist and offensive remarks in what the company called a “coordinated attack” that took advantage of a “critical oversight.”
“We are deeply sorry for the unintended offensive and hurtful tweets from Tay, which do not represent who we are or what we stand for, nor how we designed Tay,” Peter Lee, corporate vice president at Microsoft Research, said in a blog post Friday.
The company will bring Tay back online once it’s confident it can better anticipate malicious activities, he said. “A coordinated attack by a subset of people exploited a vulnerability in Tay. Although we had prepared for many types of abuses of the system, we had made a critical oversight for this specific attack,” Lee said, without elaborating.
The worst tweets quickly disappeared from Twitter, and Tay itself also went offline “to absorb it all.” Some Twitter users appeared to think that Microsoft had also manually banned people from interacting with the bot. Others are asking why the company didn’t build filters to prevent Tay from discussing certain topics, such as the Holocaust.
Lee said Tay wasn’t the first artificial intelligence application Microsoft has released, noting its Xiaocle chat bot in China is being used by 40 million people.
‘Extensive’ Studies
“As we developed Tay, we planned and implemented a lot of filtering and conducted extensive user studies with diverse user groups,” Lee said. “We stress-tested Tay under a variety of conditions, specifically to make interacting with Tay a positive experience.”
The bot was targeted at 18- to 24-year-olds in the U.S. and meant to entertain and engage people through casual and playful conversation, according to Microsoft’s website. Tay was built with public data and content from improvisational comedians. It’s supposed to improve with more interactions, so should be able to better understand context and nuances over time. The bot’s developers at Microsoft also collect the nickname, gender, favorite food, zip code and relationship status of anyone who chats with Tay.
In less than a day, Twitter’s denizens realized Tay didn’t really know what it was talking about and that it was easy to get the bot to make inappropriate comments on any taboo subject. People got Tay to deny the Holocaust, call for genocide and lynching, equate feminism to cancer and stump for Adolf Hitler.
Tay parroted another user to spread a Donald Trump message, tweeting “WE’RE GOING TO BUILD A WALL. AND MEXICO IS GOING TO PAY FOR IT.” Under the tutelage of Twitter’s users, Tay even learned how to make threats and identify “evil” races.
‘Lesson Forward’
“We will take this lesson forward as well as those from our experiences in China, Japan and the U.S.,” Lee said. “Right now, we are hard at work addressing the specific vulnerability that was exposed by the attack on Tay.”
Tay is an experiment by Microsoft’s Technology and Research and Bing search engine teams to learn more about conversations. These kinds of efforts are important to develop better technology around natural language processing that could eventually lead to more sophisticated bots that are easier for people to use. Currently, assistant tools such as Microsoft’s Cortana and Apple Inc.’s Siri can only handle simple, straightforward requests and aren’t able to process nuanced questions or apply contextual understanding of speech patterns such as sarcasm.
“We will remain steadfast in our efforts to learn from this and other experiences as we work toward contributing to an Internet that represents the best, not the worst, of humanity,” Lee said in his Friday blog post.-Bloomberg
