Bad Data Can Kill... Your Reputation


Deep in the data your business or organization collects everyday, a silent menace lurks, waiting to strike.

An unchecked quantity of low-quality data is all it takes to kill your organization's reputation -- a reputation that may require great effort to resuscitate.

And one big flub can do the damage, as Tom Redman, "The Data Doc" and founder of Navesink Consulting, discusses in his blog from earlier this week. In the blog, he explores the much-reported $2 trillion error by Standard & Poor's, which downgraded the US credit rating in a historical move.

"The error may not have mattered to the fact of the downgrade. But it can't help S&P's reputation," elaborated Redman in a follow-on AllAnalytics.com e-chat.

But more important than the S&P debacle itself, Redman added, is that the factors leading to the error are hardly unique.

"I want to build on this point," Redman said. "The most important questions in my blog pertain to other companies, not S&P. If they think they are not at risk, they should think again."

During the live chat, Redman cited one study that suggested 20 percent of all data records contain at least one error, enough to possibly cause considerable problems further on. Sadly, this fact is not news to most companies and organizations in that many simply accept or ignore the uncertainty and considerable risks this situation poses, he said.

"I find it a bit of a paradox here. Almost anyone I talk to readily agrees their data is bad, costs them big-time, and subjects them to risk. But somehow individual awareness does not translate into group action."

So, how can companies assess the threat posed by potentially bad data quality and begin to amass a better quality of data upon which they can base smarter decisions?

The answer involves a series of decisions by a company or organization about how best to manage its data.

"I've always just said we need to correct current data errors and prevent future errors,” said Danette McGilvray, principal of Granite Falls Consulting, and chat participant. "Of course, there is always debate about which of those two should be addressed."

However, McGilvray and Redman agreed on the first step. And that, they said, is for companies to decide why they want the data in the first place. The decision will help improve the data collection process and, quite possibly, simplify matters by eliminating collection of data for which there is no specific purpose.

"Understanding where the data flows after it is entered and the impact if the quality is poor can go a long way when training people, about not just the 'how' but the 'why,' " McGilvray said.

Everybody would do well to consider this comment McGilvray relayed from one of her clients: "If I had known years ago when I was entering data that this is how it was used, I would have been a lot more careful."

Shawn Hessinger, Community Editor

Shawn Hessinger is a community manager, blogger, social media and tech enthusiast, journalist, and entrepreneur based in Northeastern Pennsylvania. He serves as community manager and blogger for BizSugar.com, a business news and information Website, and contributes regularly to the online business news source, Small Business Trends. He is the founder of PostRanger.com, an online content and media community, and has provided blogging and social media services and consulting for companies all over the world. He researches and writes on a variety of business, Internet-related, and other tech topics including business intelligence and analytics. He is also keenly interested in computer-aided data management as it relates to his various online ventures. A newspaper journalist with more than 11 years experience as a reporter and then managing editor, Shawn began blogging in 2006 and now provides a variety of consulting and outsourcing services in Search Engine Optimization, Web development, and online marketing to companies large and small. He is a strong advocate for the use of BI and related computer data management in business decision making, whether using software as a service (SaaS), cloud, or other applications, and in the opportunity these technologies provide to transform small startups and larger established businesses alike.

BCBSNC, SAS Team on Advanced Analytics

The key to improving heathcare outcomes is to look at individual needs, the companies say.

Spoofing, Privacy Greatest Barriers for Biometrics

In Wednesday's e-chat, we discussed the analytics of identification and whether the technology might find a bigger role one day in marketing intelligence.


Re: Major rethink
  • 8/18/2011 11:55:35 PM
NO RATINGS

Hi Broadway,

If memory serves from the chat the study cited was referring to all data which, to my mind, would suggest that we are talking about an average here with some having more errors and some having less than 20 percent depending upon the company. My suggestion is that you check out the chat (follow the link in the post above) for more details.

Re: Major rethink
  • 8/13/2011 10:31:40 PM
NO RATINGS

You're right, cmophil, but it is far less costly to learn from mistakes that others make, rather than your own.

Re: Major rethink
  • 8/13/2011 11:27:27 AM
NO RATINGS

"20 percent of all data records contain at least one error"?

I don't have the research to back me up, but my gut tells me that the ratio is probably much much higher. I think the ratio also depends on what sort of data we're talking about here. What type of data records are we talking about here?

Re: Major rethink
  • 8/12/2011 8:26:34 PM
NO RATINGS

aaphil,

But it's the size and visibility of the mistakes that is at issue in this case...

Re: Major rethink
  • 8/12/2011 8:19:46 PM
NO RATINGS

Learning through mistakes sometimes is the best way. 

Re: Major rethink
  • 8/12/2011 11:56:59 AM
NO RATINGS

Hi Beth,

I think it's a real balancing act, as we discussed before. Try to collect too much and run the risk that poor quality data might be the result. Collect too little and you may be unprepared for changes that your data might otherwise have helped you predict by simply looking at things in a different way.

Re: Major rethink
  • 8/12/2011 11:53:38 AM
NO RATINGS

""If I had known years ago when I was entering data that this is how it was used, I would have been a lot more careful.'" Another instance of 20/20 hindsight?

Major rethink
  • 8/12/2011 8:44:36 AM
NO RATINGS

Shawn, thanks for recapping the chat for us. Data quality and the best ways to make sure data destined for analytics is clean are no doubt big ongoing challenges for most of our community members. They certainly were big topics at the SAS Power Series event I attended a few weeks back. One attendee, for example, related the frustration she felt when internal folks dismissed her department's analysis in favor of that from an outside firm. That her colleagues trusted an  outsider's data more than their company's own was quite distressing. This mistrust stemmed from sales folks who knew the half-hearted approach they themselves took to entering data in the CRM. So getting over the trust issue meant cleaning up that whole process, including minimizing the amount of data collected to ensure what is collected is clean. A good lesson learned and action plan. 

INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +