REGISTER   |   LOGIN   |   HELP
Home  |  Blogs  |  Message Boards  |  Webinars  |  Resources |  By Channel
Noreen Seebacher

The Daunting Task of Defining Big-Data

NO RATINGS
View Comments: Threaded | Newest First | Oldest First
Ariella
User Rank
Blogger
lotta data
Ariella   11/15/2012 10:32:35 AM
NO RATINGS
I rather like the expression, "we got a whole lotta data." It almost sounds like a song.

Noreen Seebacher
User Rank
Blogger
Re: lotta data
Noreen Seebacher   11/15/2012 10:42:22 AM
NO RATINGS
LOL Ariella!

kicheko
User Rank
Blogger
Re: lotta data
kicheko   11/15/2012 4:35:55 PM
NO RATINGS
Ariella, - So true though what he says. Big-data is just lots of data in different terms. Funny though how as soon as someone found the shortest word to refer to lots of data, it became complicated. I believe that most of the difficulty in explaining big-data is in actual sense difficulty in understanding the tools of handling big-data and their dynamics

Ariella
User Rank
Blogger
Re: lotta data
Ariella   11/15/2012 4:41:58 PM
NO RATINGS
@kicheko, true.  A short description packs a lot of complications here.

Noreen Seebacher
User Rank
Blogger
Re: lotta data
Noreen Seebacher   11/16/2012 8:35:41 AM
NO RATINGS
Let's here some of your own definitions. If you had to make a dictionary entry, what would you write about big-data?

philsimon
User Rank
Data Doctor
the anti-definition
philsimon   11/16/2012 12:02:14 PM
NO RATINGS
1 saves
I like this one from The Register:

Big Data is any data that doesn't fit well into tables and that generally responds poorly to manipulation by Structured Query Language (SQL).

[T]he most important feature of Big Data is its structure, with different classes of Big Data having very different structures.

With that definition, we can start to look at examples. A Twitter feed is Big Data; the census isn't. Images, graphical traces, Call Detail Records  (CDRs) from telecoms companies, web logs, social data, RFID output can all be Big Data. Lists of your employees, customers, products are not.




Alexis
User Rank
Data Doctor
Re: the anti-definition
Alexis   11/16/2012 1:07:54 PM
NO RATINGS
Nice one Phil! It makes the distinction very clear.

Ariella
User Rank
Blogger
Re: the anti-definition
Ariella   11/19/2012 11:15:07 AM
NO RATINGS
@philsimon That's a good way to put it. I've seen other articles that identify "big data" as that which is not yet set for use, whether you call it structured or "cleaned," as suggested in Big v. Small Data – Why Marketers Today Need to Know the Difference

Noreen Seebacher
User Rank
Blogger
Re: the anti-definition
Noreen Seebacher   11/27/2012 10:16:07 AM
NO RATINGS


Noreen Seebacher
User Rank
Blogger
Re: the anti-definition
Noreen Seebacher   11/27/2012 10:16:46 AM
NO RATINGS


Noreen Seebacher
User Rank
Blogger
Re: the anti-definition
Noreen Seebacher   11/27/2012 10:19:29 AM
NO RATINGS


Ariella
User Rank
Blogger
Re: the anti-definition
Ariella   11/27/2012 10:21:08 AM
NO RATINGS
@Noreen love the crystal ball image! That suggests yet another name for the people who work with big data: see-ers.

Noreen Seebacher
User Rank
Blogger
Re: the anti-definition
Noreen Seebacher   11/27/2012 10:21:49 AM
NO RATINGS
The Definition of Enterprise Big Data

Big Data is emerging from the realms of data science projects to help companies understand exactly, make decisions, and act in real-time to better serve their customers and target markets. The IT techniques and tools to execute big data processing are new, very important and exciting.

Noreen Seebacher
User Rank
Blogger
Re: the anti-definition
Noreen Seebacher   11/27/2012 10:23:42 AM
NO RATINGS


Noreen Seebacher
User Rank
Blogger
Re: the anti-definition
Noreen Seebacher   11/27/2012 10:26:04 AM
NO RATINGS


scorellis
User Rank
Prospector
Re: lotta data
scorellis   11/18/2012 2:24:30 AM
NO RATINGS
We have spent the past yeart trying to define "big data."  In the end, it comes down to a couple of things.  A) people like to think they have big data, even when they don't.  Here is why I say this.  If you have a 32 GB 8 core SQL server and you have a database table with 250 GB of data in it. yeah, you might think you have big data.  If you have 900 GB of data, and 512 GB of RAM on your database server, you might think you have "big data." No, you don't,  B) If you have a 6TB table of data in your DB, and you have 2 TB of RAM, and you think you have big data, YES. YOU DO.  Well, at least according to my definition...my definition, constrained by current techological limitations, goes like this: If you have to make a call to disk for data, and the amount of time it takes to return that data to your users exceeds the amount of time that users are willing to accept, AND the ONLY WAY to alleviate this issue is to increase your bandwidth to disk because you have ALREADY MAX'D out the technologically available amount of RAM within which the database could reside, then you have "BIG DATA."   I hope this clarifies things. My team deals with this on a daily basis from some rather gigantic companies with rather large amounts of corpuscular data.  By "corpuscular," I mean single, extremely large chunks of data that can not be fragmented across multiple systems and MUST be treated symptomatically.  That is, the data is interrelated, and the users seek to examine it as a whole, and not as discrete entities. I really could go on and on about this....

DukeGanote
User Rank
Prospector
Re: lotta data -- 120 Years of the New Frontier
DukeGanote   11/18/2012 6:57:30 AM
NO RATINGS
"Big data" started at least 120 years ago-- even if the buzzword is more recent. Vincent McBurney wrote a superb blog entry about "big data", which I highlighted in my own here:

http://it.toolbox.com/blogs/data-ruminations/a-short-history-of-big-data-the-new-frontier-53878





Alexis
User Rank
Data Doctor
New Age
Alexis   11/15/2012 1:52:20 PM
NO RATINGS
Florissi predicted the era of big-data we will trigger changes similar to the ones that occurred during the Industrial Revolution,

It's really amazing to think of the changes big-data could usher in -- comparing it to the industrial revolution is mind-boggling!

mnorth
User Rank
Blogger
It's all relative
mnorth   11/15/2012 3:47:18 PM
NO RATINGS
@Noreen: Excellent topic.  Defining big data is something I pondered on while penning my recent musings on the definitions of other industry terms.  I like the statement about 'big data' being relative.  It's kind of like calling something 'new'.  Well, it's new now, but for how long?  And when it's not anymore, will we take the new off?  In my first analytics job, I worked on a SAS server that had 800 GB of disk space.  We thought that was a ton.  Now you can buy mulitple TB drives off the shelf at Staples.  Is 800 GB still 'big'?  Nope.  Exa-, zetta- and yotta- are big, for now, but for how long?

More Blogs from Noreen Seebacher
Everyone is talking about big-data as an HR solution, so why doesn't it seem like we're any closer to solving the people problem?
Even at a trade fair better known for seminars on information technology, big-data was too significant to ignore.
All Analytics readers have serious issues with the data hidden in digital photos.
The system we use to select American courtroom juries is riddled with delays, frustrations, and inefficiencies.
It was actually a little too easy to slip a cellphone past security at a federal courthouse.
Cartoon
Most recent post: Viewer beware!
CARTOON ARCHIVE
AllAnalytics House Ad
Quick Poll
AllAnalytics Video Blogs
We'll Be Your Eyes & Ears
We'll be on the scene at SAS Global Forum events in ...

2:15

0 comments
7 Tips for Deploying ...
We chat with Analise Polsky, a data visualization ...

33:15

0 comments
Top Big Data Platforms
All Analytics editors Beth Schultz and Michael ...

31:53

3 comments
Attention on Retail Shoppers
The retail store of the future will track customers ...

02:14

16 comments
Demand-Driven Forecasting
Charles Chase, chief industry consultant for the ...

02:22

1 comment
Intelligent Labels & LE ...
Andrew Dark, CEO of Displaydata, explains the ...

03:21

0 comments
Big-Data & In-Store Analytics
SAS Institute's Lori Schafer shares insights on ...

02:31

5 comments
Privacy Protection in 7 Steps
Gaurav Pant, SVP of Research and Principal Analyst ...

03:05

1 comment
Customer Insight Drives Retail
Lori Bieda, executive lead for customer ...

02:13

13 comments
NRF BIG Show Highlights
All Analytics Executive Editor Michael Steinhart ...

02:51

7 comments
Nurturing Analytics Talent
Sarah Gates, vice president of research for the ...

02:25

4 comments
Banking on Analytics
Capgemini Senior Manager Rex Pruitt explains how ...

02:39

1 comment
360-Degree Slam Dunk
Analytics helps the Orlando Magic score higher ...

01:50

3 comments
Securing Hadoop Data
Big data often contains sensitive or protected ...

06:20

5 comments
Powerful Analytics Ecosystems
Combining Hadoop with high-performance ...

02:04

1 comment
Digital Audio
Latest Archived Broadcast
Thought-leader Tom Davenport explains why you and your company need to recognize big data's importance.
April 29th 2pm EDT Tuesday
Readerboards
Have a question or topic but don't want to write a blog? Post it on our readerboards and get feedback from the community!
MORE READERBOARDS
Live Video
On-demand Video with Chat
As retailers evolve toward an omnichannel environment, much of their success will depend on how effectively they use big-data and analytics.
Upcoming Events
for the Business and IT Communities
Executive forums with additional hands-on learning opportunities offered around the world
Each ideal for practitioners, Business leaders & senior executives
SAS Health Analytics Virtual Conference
The Health care is rapidly transforming. And there has never been a greater need for analytics. We're tackling tough challenges like data transparency, care delivery, consumer engagement, and financial and clinical risk. And there are still numerous opportunities to use health data that we haven't even tapped into.
May 14, 2014
2014 VA Interactive Roadshow -- Houston
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
May 15, 2014
Houston, Texas
2014 VA Interactive Roadshow -- New York
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
June 19, 2014
New York, New York
2014 VA Interactive Roadshow -- Rockville, MD
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
July 17, 2014
Rockville, Maryland
2014 VA Interactive Roadshow -- Detroit
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Aug. 7, 2014
Detroit, Michigan
2014 VA Interactive Roadshow -- Chicago
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Sept. 16, 2014
Chicago, Illinois
2014 VA Interactive Roadshow -- Cary, NC
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Oct. 10, 2014
Cary, North Carolina
2014 VA Interactive Roadshow -- Boston
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Nov. 4, 2014
Boston, Massachusetts
2014 VA Interactive Roadshow -- Atlanta
The 2014 VA Interactive Roadshow will feature SASŪ Data Management and SASŪ Visual Analytics experts covering topics like prepping data for VA and VA integration with SASŪ Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Dec. 11, 2014
Atlanta, Georgia
Analytics 2014
The The Analytics 2014 Conference is a two-day educational event for anyone who is serious about analytics. This annual event brings together hundreds of professionals, industry experts, and leading researchers in the field of analytics. Register before April 30 for the early-bird discount.
June 4 & 5, 2014
Frankfurt, Germany
AllAnalytics on Twitter
AllAnalytics Twitter Feed
AllAnalytics Videos
Intro to Visual Analytics
Find a way to visualize your data and watch it come ...

1:58

0 comments
Visual Analytics, Instant ...
Analytics results delivered in visual form are ...

2:06

2 comments
Big Data, Fast Infrastructure
Big data calls for a high-performance analytics ...

3:35

6 comments
Red Hot Analytics
Jayson Tipp, Redbox VP of Analytics and CRM, ...

3:51

5 comments
Hotelier Checks Out Analytics
InterContinental Hotels Group has woven analytics ...

06:55

11 comments
Like Us on Facebook
Point/CounterpointBlog
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES

The Current Discussion

Visual Analytics: Who Carries the Onus?
The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question.
The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
MORE POINT/COUNTERPOINT BLOGS
About Us  |  Contact Us  |  Help  |  Register  |  Twitter  |  Facebook  |  RSS


Beth Schultz
We'll Be Your Eyes & Ears

3|20|14   |   2:15   |   (0) comments


We'll be on the scene at SAS Global Forum events in Washington, D.C., March 23 to March 25, glad to share what we learn with our community members.
Beth Schultz
7 Tips for Deploying Visualization

3|7|14   |   33:15   |   (0) comments


We chat with Analise Polsky, a data visualization thought leader on the SAS Best Practices team, about what you need to know before you deploy data visualization.
Michael Steinhart
Choosing a Big-Data Analytics Platform

2|19|14   |   31:53   |   (3) comments


The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.

Related posts:

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Keeping a Close Eye on Shoppers

2|13|14   |   02:14   |   (16) comments


At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.

Related posts:

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Real-Time Demand Drives Forecasting

2|11|14   |   02:22   |   (1) comment


The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Next-Gen Retail: Electronic Labels & Bluetooth Beacons

2|7|14   |   03:21   |   (0) comments


Electronic shelf-edge labels (ESLs) equipped with low-energy Bluetooth beacons enable retailers to deliver real-time customer interaction and execute dynamic pricing strategies. Andrew Dark, CEO of Displaydata, outlines the ESL architecture and explains how it integrates with backend management and analytics systems.

Related post:

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Retail Trends: Big Data Optimizes Sales & Operations

2|4|14   |   02:31   |   (5) comments


Retailers like Family Dollar and suppliers like Procter & Gamble are using big-data analytics to maximize efficiency and revenue across the entire supply chain. Lori Schafer, Executive Advisor for the SAS Institute Retail Practice, moderated a panel with executives from these companies at the National Retail Federation BIG Show in New York last month. Here, she shares insights on retail supply chain optimization and in-store customer tracking for targeted sales.
Michael Steinhart
7 Steps to Protecting Customer Privacy

1|31|14   |   03:05   |   (1) comment


EKN Research's "The Rising Importance of Customer Data Privacy in a SoLoMo Retailing Environment" report details the top challenges and opportunities that retailers face when embracing big data analytics. EKN SVP of Research and Principal Analyst Gaurav Pant explains the importance of data management and lays out seven steps that retailers can take to ensure customer privacy while reaping the benefits of big data.
Michael Steinhart
Integrating Customer Insight Across Retail Channels

1|29|14   |   02:13   |   (13) comments


Customer data is fueling a new phase of retail marketing across physical and online channels. Lori Bieda, executive lead for customer intelligence at SAS Americas, explains how integrated insight enables retailers to optimize offers and improve sales across product categories. She also shares some best-practices for leveraging analytics talent in retail.

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
National Retail Federation Show Highlights

1|27|14   |   02:51   |   (7) comments


This year's National Retail Federation BIG Show wrapped up on January 14. All Analytics executive editor Michael Steinhart reviews highlights of the conference and discusses trends around analytics, personalization, omnichannel, and retail security.

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Beth Schultz
Nurturing Analytics Talent

1|9|14   |   02:25   |   (4) comments


Sarah Gates, vice president of research for the International Institute for Analytics, shares advice on how to recruit and leverage analytics talent, whether your company is big or small.
Michael Steinhart
Analytics for Banking Regulation Compliance

1|7|14   |   02:39   |   (1) comment


In the wake of 2008's financial meltdown, banks are subject to strict regulations around the soundness of their loan portfolios. Capgemini senior manager Rex Pruitt explains how advanced transition matrices -- driven by SAS analytics tools -- help banks perform effective credit loss forecasting and meet their regulatory requirements.
Michael Steinhart
360-Degree Slam Dunk

12|24|13   |   01:50   |   (3) comments


David Bencs, assistant director of Insight and Analytics for the Orlando Magic, outlines different analytics projects and the benefits they're delivering to the NBA franchise. The team put demand-based pricing in place a few years ago, for example, and single-game ticket revenue grew 28% despite a disappointing season. Next up for the Magic is to combine social media activity, television viewership stats, and ticket sales data to achieve a 360-degree customer view.

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Encryption for Hadoop & Big Data

12|9|13   |   06:20   |   (5) comments


David Tishgart, senior director of marketing and alliances at security provider Gazzang, explains the importance of data encryption for companies that are rolling out Hadoop environments to leverage big data analytics.
Michael Steinhart
Hadoop's Place in the Analytics Ecosystem

12|5|13   |   02:04   |   (1) comment


At the Strata Conference / Hadoop World 2013, Samuel Kommu, technical marketing engineer at Cisco Systems, shares some of the benefits that Hadoop brings to analytics platforms that leverage next-generation hardware. Kommu looks at big data operations that required 3,500 nodes in 2009, 2,000 in 2011, and now require only 64 nodes.
Videos
Intro to Visual Analytics

6|5|13   |   1:58   |   (0) comments


With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Videos
Visual Analytics, Instant Insight

5|16|13   |   2:06   |   (2) comments


Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
Videos
Big Data, Fast Infrastructure

2|14|12   |   3:35   |   (6) comments


No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Videos
Red Hot Analytics

1|10|12   |   3:51   |   (5) comments


Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
Videos
Hotelier Checks In With Analytics

12|14|11   |   06:55   |   (11) comments


InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.