Microsoft Tay: What Could Possibly Go Wrong?


Machine learning has been all over the tech news for the past couple of months. We've had our share of discussion here in the All Analytics community, with talk about the future relationship between machines and humans in the workforce.

Google/Alphabet executive Eric Schmidt predicted huge growth for machine learning and startups that develop machine learning tools. Other executives highlighted the potential for cloud-based machine learning, and hundreds of celebrities and tech leaders, including Bill Gates, Elon Musk, and Stephen Hawking, have endorsed a letter calling for care in the use of artificial intelligence and machine learning.

Lack of care appears to have been behind one of Microsoft's ventures into machine learning, a project that led the company to step in the smelly substance that you should use care to avoid. That was the Tay chatbot mishap.

I think we've all learned over the years that there are two basic but vital questions that we have to ask when implementing any technology: What can this technology do for us? What can go wrong? I guess the second question could be rephrased to, "What can this technology do TO us?

Entrepreneurs specialize in the first question. I've interviewed hundreds of executives in startups, spinoffs, and new business units. Their brains are packed with ideas, all about the potential for their new tech. On the "glass is half full" debate, those folks take the approach that the glass is completely full, no matter how it appears. They see no reason to not be cock-eyed optimists.

The second question about what can go wrong is where corporate cybersecurity professionals, investors, and risk managers come into play. True, they are sometimes the Dr. No's of the tech world. Yet, they serve a vital purpose. They want to know what port might be exposed and who will have access to data, or, if they have a financial stake, what happens if people don't buy a product and if another company has a competing product. They keep our dreams honest.

Credit: Ars Technica
Credit: Ars Technica

So, what was Microsoft thinking when it unleashed Tay on the world? Maybe they weren't thinking, or didn't have enough of the checks the Dr. No's of the world provide.

The Tay chatbot -- designed to communicate like a 19-year-old woman -- learned, or at least parroted, all the wrong things by the end of its 24-hour lifespan. Its activity quickly degenerated into crude, racist, and myoginistic Twitter posts that espoused violence, as it was "trained" by the sick minds in the online world.

In exploring what went wrong, one article noted, "In its apology, Microsoft's Peter Lee, corporate vice president of Microsoft Research, writes that the company did test her under a range of conditions to ensure that she was pleasant to talk to. It appears that this testing did not properly cover those who would actively seek to undermine and attack the bot."

One challenge with technology such as a chatbot or an audio guide like Apple's Siri is that they can't necessarily add context to what they hear or say. For example, we discussed Siri's limitations when faced with queries about rape and suicide in Tech Airs Its Downside, and Some Good News. I don't believe the flaw is in Siri itself but how heavily we rely on technology. Machines can learn from us or from their own activity, but they are only tools, not human. With Tay, Microsoft provided a tool, but didn't set limits on how it could be used. One observer said that technologies like Tay, Siri, and IBM's Watson need to be programmed with an ever-evolving code of conduct that keep pace with our ability to do evil.

Machine learning could work wonders in helping us to accomplish tasks such as improving product maintenance or customer support. Machine learning offers an opportunity to comb millions of files to help find the best treatments for disease or strategies for dealing with environmental threats. Machine learning as a social tool? Maybe that was the first mistake.

In a world where someone, somewhere is just waiting to pounce on any vulnerability in a new technology the burden is on the shoulders of developers to find and close loopholes. They can be cock-eyed optimists but they also have to switch modes and play Dr. No for a while, particularly when we see how quickly something with a bit of promise can turn so bad.

James M. Connolly, Editor of All Analytics

Jim Connolly is a versatile and experienced technology journalist who has reported on IT trends for more than two decades. As editor of All Analytics he writes about the move to big data analytics and data-driven decision making. Over the years he has covered enterprise computing, the PC revolution, client/server, the evolution of the Internet, the rise of web-based business, and IT management. He has covered breaking industry news and has led teams focused on product reviews and technology trends. Throughout his tech journalism career, he has concentrated on serving the information needs of IT decision-makers in large organizations and has worked with those managers to help them learn from their peers and share their experiences in implementing leading-edge technologies through publications including Computerworld. Jim also has helped to launch a technology-focused startup, as one of the founding editors at TechTarget, and has served as editor of an established news organization focused on technology startups and the Boston-area venture capital sector at MassHighTech. A former crime reporter for the Boston Herald, he majored in journalism at Northeastern University.

Big Data Success Essentials: Tech, People, and Process

While there is an increasing focus on the role of people alongside technology in analytics initiatives, let's not forget that process -- business rules -- play an important role in big data success.

Why You Should Remember Equifax

The Equifax breach raises multiple concerns about how companies respond to hacks, but also how they handle third-party consumer data.


Re: Doom
  • 4/11/2016 8:50:58 PM
NO RATINGS

..

Not having had the pleasure of conversing with Tay, I searched for an example of the controversial behavior. From BBC.com, here's a sample of one of the exchanges with Tay:

Aside from the probability that this is not exactly what a major corporation would want an example of its technology to be saying to the public ... Uh, is this how typical 19-year-olds talk? I didn't think they even had words like "indeed" in their speaking vocabulary ...

 

Re: Doom
  • 4/6/2016 12:19:15 PM
NO RATINGS

@Jamescon In contrast to the big chains (and Walmart has also introduced self-checkout lanes, thogh they still have mostly human cashiers) my local grocery stores have all human cashiers, and most also have baggers (usually absent from chain stores). They manage to do this without having overall higher prices, though they don't necessarily offer the same promotions on big name products that the chains do.

Re: Doom
  • 4/6/2016 12:12:01 PM
NO RATINGS

<Personally, I have no problem with self-service checkout, at least for small orders that I can bag myself and don't need me to type in product codes. For bigger grocery orders I prefer the interaction with the clerk and bagger.>

@James I'm happy to do self-checkout at Home Depot because I usually have just a few items and nothing needs to be weighed, etc. But I prefer having human cashiers at the supermarket because things tend to get stuck on larger orders. Currently, my local Stop and Shop has 2 to 3 human cashiers down from around 7 or 8 before, so clearly the shift to self-service is intended to cut down on labor costs. Last time I was there along with a daughter who asked about it and I explained the motive for the store is really savings rather than service. A woman behind us said that she makes a point of using human cashiers not just for her own sake but to support jobs.

Re: Doom
  • 4/6/2016 12:00:34 PM
NO RATINGS

@Ariella. You're certainly right about things like human vs bot-based customer support. The reason you can have bots do it these days (and not do it well in many cases) is that so many companies have been using script-based customer support even when the "help" is human. Of course, the first thing that script does is try to convince the customer that their problem is their own fault.

Personally, I have no problem with self-service checkout, at least for small orders that I can bag myself and don't need me to type in product codes. For bigger grocery orders I prefer the interaction with the clerk and bagger.

Self checkout shouldn't eliminate jobs if the retailer is smart. Paying for a product often is the easiest part of the sales cycle, right? Where you want help as a customer is while you are cruising the aisles deciding what to buy or how to find the right product location (grocery or dry goods). So, putting at least some trained people out on the floor where they can enable or even increase sales makes good business sense.

Re: Doom
  • 4/6/2016 11:43:51 AM
NO RATINGS

@kq4ym. Right, tech leaders play up the positive and overlook, or at least downplay, the negative side of new tech. It's all about the marketing mandate: "Stay on Message".

That might have worked to some extent years ago when it was easier to do damage control if a product didn't work or if it caused problems (this applies to all types of products and services, not just tech). Today, however, a flaw in design or application of tech can be attacked as a vulnerability, and with social media there is no hiding the damage, even for a few hours.

Business leaders love to talk about transparency but still try to bury their mistakes. The problem is that it only takes one hacker or one dissatisfied customer to start a wave of public attention.

Re: Doom
  • 4/6/2016 11:12:21 AM
NO RATINGS

<Another possibility, replace that chat bot with an intelligent human.> @James I'd second that. I do so much prefer to speak with people but only when they actually pay attention to what I'm asking and really respond. So many live chats online just give out canned responses. Unfortunately, the trend is to automate more and hire fewer people. I see this even on the physical retail side where more and more self-checkout lanes crop up, eliminating jobs. 

Re: Doom
  • 4/6/2016 9:34:58 AM
NO RATINGS

A good point, as noted in the article, to pay attention more to what tech can do to us as well as the usual what can it do for us. This plays in reverse to the most usual "positive" spin in most discussions with tech leaders and may now insist we also look at the "negatives" or even the dark side of implementing new ideas and technology to prevent things going astray like Microsoft Tay.

Re: Doom
  • 4/4/2016 8:24:24 AM
NO RATINGS

If the intelligent chat bot could parse a natural language text message into an order and send a confirmation I think that's all that most people are looking for.  They don't want or need to be sold at that point, they are coming to you to buy and probably have a very good idea what they want.  Bots I can see being useful because there is nothing like being put on hold while phoning in an order.  A bot that can interface with a dozen customers at once could be very useful.  I can also see a bot for simple things like "what time do you close" or "are you open on Sunday?", I just don't see the usefulness of a 10 minute text conversation with a bot to order a pizza.  

Re: Doom
  • 4/1/2016 8:43:57 AM
NO RATINGS

Another possibility, replace that chat bot with an intelligent human.

Re: Doom
  • 4/1/2016 8:35:19 AM
NO RATINGS

I think we got a better idea of what Tay was supposed to be during the Build 2016 presentations.  Tay appears to be part of the Microsoft Bot framework, a simple chat bot connected to Twitter with less structure than the Dominos pizza chat bot.  The demonstration of learning from users and how to talk to them came back to bite Tay but it does show that with some controls those chats can be much less mechanical than days gone by.  My brain has a hard time understanding why someone would want to chat with a bot to order pizza rather than open a site, make some choices from a set of check boxes and hit an order button but I do see the usefulness when selling more complex items and not wanting to send a customer to a list of FAQs.

Page 1 / 4   >   >>
INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +