Big Data: Gospel or Myth?


For the past few years, many IT evangelists have been preaching the εὐαγγέλιον/gospel/”Good News” about Big Data. Big Data has been promoted in books, trade magazines, and conference papers.

Credit: iStock
Credit: iStock

The image projected concerning Big Data, has been salvific in nature. It represents deliverance to all those who are in bondage regarding the optimization their data assets. It represents freedom to all those who are oppressed regarding the range of their analytic options. It represents redemption to all whose bosses have questioned their database investments. For some, Big Data is the corporate deity; Big Data is the way, the truth and the life. Even the Pointy-Haired Boss (Dilbert by Scott Adams) has become a disciple. But let’s explore this belief system a little; let’s assess whether Big Data should be canonized as gospel or classified as a myth.

Big Data as “gospel” suggests that the people who walked in darkness have seen a great light. Those who could not make sense of the data at a granular level, can now see the big picture. Those who were limited to hypothesis testing, can now let the data tell its own stories. Those who had curbed their work to structured data, can now expand their options with unstructured data. Those who had to restrict the number of files for storage on their local workstations, can now work unrestrained on the cloud server. Big Data simplifies sentiment analysis. Big Data enables research. Big Data eases forensic accounting. Big Data is the gateway to information heaven.

Big Data as “myth” suggests that the firestorm surrounding this technology may produce more heat than light. It suggests that the concept is less of a divine inspiration and more of marketing strategy that projects abstract benefits vis-à-vis usable results. Big Data as myth means that prerequisites such as a solid platform, the ability to integrate with multiple applications and high data storage, fast data transmission and tight data security must be well-planned, financed, implemented and tested. It suggests that this is not a shrink wrapped/plug and play technical solution. It suggests that at the end of ETL, analysis, and visual analytics, the human element must still make a decision on if and how to use the information produced. Like the Farmer’s Almanac, Big Data is a decision support tool that is dependent upon human judgment.

So how does one discern whether Big Data is gospel or myth? One way is to ask three questions:

  1. Regarding core utility services (i.e., network connections, email, internet, printing, faxing, file sharing, etc.) does Big Data support those services or is Big Data supported by those services?
  2. Regarding mission support services (i.e., word processing, tabulations, analytics, report generation, human resources, finances, etc.), does Big Data support those services or is Big Data supported by those services?
  3. Regarding mission creation services (i.e., business development, generating or responding to RFPs, sales, marketing, maintaining a social media presence, etc.) does Big Data support those services or is Big Data supported by those services?

In short, knowing whether Big Data is a producer or a consumer of human, information, and capital resources will inform your opinion.

Big Data -- gospel or myth? Should we embrace its virtues literally or interpret them as allegory of potential uses? Is Big Data solving problems or just being promoted as a revenue stream for vendors? Are you a believer or a skeptic? Please share your thoughts.

Bryan Beverly, Statistician, Bureau of Labor Statistics

Bryan K. Beverly is from Baltimore. He has a BA in sociology from Morgan State University and an MAS degree in IT management from Johns Hopkins University. His continuing education consists of project management training through the ESI International/George Washington University programs. He began his career in 1984, the same year he was introduced to SAS software. Over the course of nearly 30 years, he has used SAS for data processing, analytics, report generation, and application development on mainframes, mini-computers, and PCs. Bryan has worked in the private sector, public sector, and academia in the Baltimore/Washington region. His work initially focused on programming, but over the years has expanded into project management and business development. Bryan has participated in in-house SAS user groups and SAS user group conferences, and has published in SAS newsletters, as well as company-based newsletters. Over time, his publications have expanded from providing SAS technical tips to examining the sociological, philosophical, financial, and political contexts in which IT is deployed. He believes that the key to a successful IT career is to maintain your skills and think like the person who signs your paycheck.

Regulatory Oversight vs. Crowdsourcing: The Best Approach for Quality

If you are looking for data quality, should you rely on professionals or passionate amateurs? Here are the pros and cons.

Work/Life Balance: Do Analytic Professionals Have It?

You use analytics in your career, but do you also apply the same principles and tools to your personal life?


Re: Hanky-panky with Big Data
  • 4/28/2016 3:22:15 PM
NO RATINGS

Big data and analytics can seem almost miraculous in providing good guidance for decisionmaking, but only if the data is fed into the process fairly, honestly, and without a preset agenda.

"When he lies, he speaks according to his own nature, for he is a liar and the father of lies." (John 8:44)

Re: The Potential of Big Data: Myth versus Reality
  • 4/27/2016 1:20:09 AM
NO RATINGS

@Ariella: So why not "Big Data Street" ... or "Big Data Sesame...Seed Bun"?

Re: The Potential of Big Data: Myth versus Reality
  • 4/26/2016 1:38:18 PM
NO RATINGS

@Bryant     Thanks for showing me this section.  I sometimes get blinders on and don't see the interesting and useful links on the right side of A2.    I really did enjoy "Big Data Archipelago".  Lot of great information which can be easily accessed.

 

I really enjoyed the characters and the humor.   The News parody makes a topic as weighty as Big Data easier to digest. 

Kudos to SAS for making the topic enjoyable to learn - I think this is a great marketing tool and effort.   

 

Others in this area should take note.

Re: The Potential of Big Data: Myth versus Reality
  • 4/25/2016 8:25:33 AM
NO RATINGS

The truth also tends to be presented as the truth someone prefers.  There are books full of "How to like with Statistics" examples.  Spinning the truth has become an art form lately.

Re: The Potential of Big Data: Myth versus Reality
  • 4/24/2016 9:29:59 PM
NO RATINGS

@Joe yes, plus a bit of nostalgia for those of us who recall Mr. Rogers' Neighborhood.

Re: The Potential of Big Data: Myth versus Reality
  • 4/24/2016 9:28:28 PM
NO RATINGS

The truth can be explained. It's just a matter of how well. And how much the listener wants to see that truth.

Hanky-panky with Big Data
  • 4/23/2016 11:42:04 PM
NO RATINGS

..

Louis writes


... companies either do not have the internal abilities to leverage Big Data or they are attempting to use it in ways that could be considered ethically questionable.

 Using large data stores to come to minute conclusions is always dangerous and Big Data is not exempted from this kind of abuse.


 

From personal experience I'd expand this observation to include public agencies. Take for example, public transportation planning, which has actually used big data and analytics for many decades (as I've written, this field can probably be considered one of the pioneers in this regard).

Here in Austin we had an example of this as recently as 2013, in the case of a "high-capacity transit" study that attempted to sell what was widely perceived as a faulty urban rail project, by brandishing analytics/big data-based results that were arrived at by skewing and manipulating the basic data inputs. A virtually classic case of GIGO.

Big data and analytics can seem almost miraculous in providing good guidance for decisionmaking, but only if the data is fed into the process fairly, honestly, and without a preset agenda.

 

Re: The Potential of Big Data: Myth versus Reality
  • 4/23/2016 6:22:42 PM
NO RATINGS

Yes, it's always interesting to see if we can discover the "truth" and even that quest is subject to what's in the eye of the beholder. Maybe the truth really can't be explained, but we'll have to rely on a concept of mostly true, most of the time.

Re: The Potential of Big Data: Myth versus Reality
  • 4/23/2016 9:57:49 AM
NO RATINGS

@Ariella: Plus, "Neighborhood" just sounds nicer -- friendly and acccessible.  And it evokes the notion of sharing.

Re: The Potential of Big Data: Myth versus Reality
  • 4/22/2016 9:18:31 AM
NO RATINGS

@SaneIT - yeah, I believe the cannonical reference (Matthew 7:5) is 'removing the pole from our eye before we remove the splinter from our neighbor's eye'. Yes - totally agree - good point!

Page 1 / 3   >   >>
INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +