REGISTER   |   LOGIN   |   HELP
Home  |  Blogs  |  Message Boards  |  Webinars  |  Resources |  By Channel
Marshall Sponder

What to Do With Unstructured Data

NO RATINGS
View Comments: Newest First | Oldest First | Threaded View
Page 1 / 3   >   >>
ruehmkorf
User Rank
Prospector
Re: Graphic
ruehmkorf   6/1/2014 10:28:55 AM
NO RATINGS
Hi BritInBigD, 

as far as I can tell that is taken from a paper from The Data Warehousing Institute. See the corresponding article for more: BI Search and Text Analytics

BritInBigD
User Rank
Prospector
Graphic
BritInBigD   1/21/2013 8:45:31 AM
NO RATINGS
Hi Marshall,

I was curious to know the source of the graphic that accompanies your post (and specifically the indicative growth rates shown for the different data categories). Is that data from IDC? 

Maryam@Impact
User Rank
Blogger
Re: cleaning with integrity
Maryam@Impact   12/31/2012 6:56:50 PM
NO RATINGS
@Hospice I agree, but I don't know of any standard protocools because their is so much varaibaility with unstructured data depending on industry etc. Still a challenge to get standards.

kicheko
User Rank
Blogger
Re: Yikes - some misconceptions here that need to be addressed
kicheko   12/31/2012 8:50:05 AM
NO RATINGS
webmetricsguru, - One big challenge i see is analytics of sentiment oriented data even though the area has grown a great deal in the course of this year -- new apps and all. This kind of data at least for now may still need to be reviewed closer because its difficult to automate it in one given pattern without locking out new sentiments that are outside of that original set. However as intelligent systems learn this data, it will cut down what we have to look at even in sentiment analytics.

webmetricsguru
User Rank
Blogger
Re: Yikes - some misconceptions here that need to be addressed
webmetricsguru   12/30/2012 11:35:11 PM
NO RATINGS
If I understand you correctly, that's the bane of the PR / Marcom industry - that you can actually look at bunch of verbatim (maybe that's ok for 10-30) but what happens when you have thousands or more?

I think a discussion on just what cleaning data is and how to to best do it would be good for AllAnalytics.com personally.  I'd like to see what we come up with, and I bet a lot of others would too.

Hospice_Houngbo
User Rank
Prospector
Re: Yikes - some misconceptions here that need to be addressed
Hospice_Houngbo   12/30/2012 11:21:17 PM
NO RATINGS
"If we can find the essential information or pattern, we might not need to look at most of it"

I see. I suppose that those hand-written patterns can just be domain specific and will be difficult to generalize. I agree that extracting the most useful patterns might be enough in most cases, as it difficult to think of all possible patterns. One of drawback of such model is that human patterns are often low-recall, even if precision is high. 

Hospice_Houngbo
User Rank
Prospector
Re: Step 1
Hospice_Houngbo   12/30/2012 11:05:50 PM
NO RATINGS
It is true that some of the points you mentioned are debatable - like "Distribute the data in the cloud". But they are valid points to take into account when dealing with unstructured data. To the question how to clean unstructured data? I think that it depends on the shape and the model that has been defined.

webmetricsguru
User Rank
Blogger
Re: Yikes - some misconceptions here that need to be addressed
webmetricsguru   12/30/2012 11:05:17 PM
NO RATINGS
What I meant is that currently, people usually end up needing to look at the data to understand it (because it is un structured information) and attempts to use software to understand it, in my opinion, won't work, at least not today. What you can do, I think, and maybe our friends here can confirm or argue this, is cut down on what we have to look at. If we can find the essential information or pattern, we might not need to look at most of it - and hopefully the software created can help surface that information, and maybe that's the best we can hope for (big data hype or not). At any rate, this is an interesting discussion and I don't have all the answers - but I am wondering just what they are.

Hospice_Houngbo
User Rank
Prospector
Re: Yikes - some misconceptions here that need to be addressed
Hospice_Houngbo   12/30/2012 10:52:47 PM
NO RATINGS
@marshall,

"I define it as something a human needs to look at to fully process"

I still don't get it. Do you mean that there is the need for human intervention to figure out whether the data is unstructured or not? Won't that be time consuming and practically impossible for human to go through all the instances of the data due to it size? Maybe it is not what you mean?

Hospice_Houngbo
User Rank
Prospector
Re: Sifting through the sands of the unstructured
Hospice_Houngbo   12/30/2012 10:44:49 PM
NO RATINGS
@Jenn,

"So I guess, we need to find better search and organizing processes."

I think that is what the cleaning and storage processes are all about. Some data can fit in many categories depending on the search parameters. This may complicate the storage process as the same data will be duplicated - sometimes unnecessary.

Page 1 / 3   >   >>
More Blogs from Marshall Sponder
When the data we don't know is as important as the data we do, our analytics platform are all but guaranteed to fail us.
Segmentation, multichannel integration, and intelligent dashboard reporting are vital capabilities, yet many business analytics solutions fall short.
Social media is playing an important role in politics, but determining a victor based on what's happening out there isn't so easy.
Experts gathered at a conference to share the latest in this niche analytics technology.
Quick Poll
Digital Audio
Latest Archived Broadcast
In this A2 Radio episode, analytics thought leader Tom Davenport will take you into the worlds of business and sports, and talk about what one can learn from the other.
AllAnalytics House Ad
Readerboards
Have a question or topic but don't want to write a blog? Post it on our readerboards and get feedback from the community!
MORE READERBOARDS
AllAnalytics Video Blogs
Canada Post on Data Delivery
James Smith, lead of enterprise data governance for ...

02:53

2 comments
Big Data Checks In
Roger Ares, vice president of analytics at Hyatt ...

03:45

3 comments
Good Data, Smarter Travel
iJET International's Rich Murnane, Director of ...

2:50

4 comments
Healthcare Data Needs a ...
Tackling healthcare data management challenges ...

2:39

0 comments
In the Talent Sweet Spot
Gen Y shapes the new analytics workforce.

3:25

0 comments
A2 on the Road
What we found at last month's SAS Global Forum 2014.

2:28

0 comments
T-Mobile Hears Data's Call
Internal customer data provides an analytics ...

2:50

0 comments
A Is for Analytics in Academia
Professors and students agree that access to ...

4:11

0 comments
Advocating for Analytics ...
Analytics and business experts explore what it ...

5:51

0 comments
Analytic Myths: True or False?
Analytics experts give us their perspective on four ...

4:29

0 comments
We'll Be Your Eyes & Ears
We'll be on the scene at SAS Global Forum events in ...

2:15

0 comments
7 Tips for Deploying ...
We chat with Analise Polsky, a data visualization ...

33:15

0 comments
Top Big Data Platforms
All Analytics editors Beth Schultz and Michael ...

31:53

3 comments
Attention on Retail Shoppers
The retail store of the future will track customers ...

02:14

16 comments
Demand-Driven Forecasting
Charles Chase, chief industry consultant for the ...

02:22

1 comment
Live Video
On-demand Video with Chat
As retailers evolve toward an omnichannel environment, much of their success will depend on how effectively they use big-data and analytics.
Upcoming Events
for the Business and IT Communities
Executive forums with additional hands-on learning opportunities offered around the world
Each ideal for practitioners, Business leaders & senior executives
2014 VA Interactive Roadshow -- Detroit
The 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Aug. 7, 2014
Detroit, Michigan
2014 VA Interactive Roadshow -- Chicago
The 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Sept. 16, 2014
Chicago, Illinois
2014 VA Interactive Roadshow -- Cary, NC
The 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Oct. 10, 2014
Cary, North Carolina
2014 VA Interactive Roadshow -- Boston
The 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Nov. 4, 2014
Boston, Massachusetts
2014 VA Interactive Roadshow -- Atlanta
The 2014 VA Interactive Roadshow will feature SAS® Data Management and SAS® Visual Analytics experts covering topics like prepping data for VA and VA integration with SAS® Office Analytics. This year's events will keep presentations at a minimum and focus on giving attendees hands-on exposure to the latest version of VA.
Dec. 11, 2014
Atlanta, Georgia
AllAnalytics on Twitter
AllAnalytics Twitter Feed
AllAnalytics Videos
Intro to Visual Analytics
Find a way to visualize your data and watch it come ...

1:58

0 comments
Visual Analytics, Instant ...
Analytics results delivered in visual form are ...

2:06

5 comments
Big Data, Fast Infrastructure
Big data calls for a high-performance analytics ...

3:35

6 comments
Red Hot Analytics
Jayson Tipp, Redbox VP of Analytics and CRM, ...

3:51

7 comments
Hotelier Checks Out Analytics
InterContinental Hotels Group has woven analytics ...

06:55

11 comments
Like Us on Facebook
Point/CounterpointBlog
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES

The Current Discussion

Visual Analytics: Who Carries the Onus?
The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question.
The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
MORE POINT/COUNTERPOINT BLOGS
About Us  |  Contact Us  |  Help  |  Register  |  Twitter  |  Facebook  |  RSS


Beth Schultz
Canada Post on Data Delivery

5|12|14   |   02:53   |   (2) comments


James Smith, lead of enterprise data governance for the Canadian postal service, explains how and why the organization has embraced an enterprise data governance program.
Michael Steinhart
Big Data Checks In

4|30|14   |   03:45   |   (3) comments


The hospitality industry gathers massive amounts of customer data, and mining that data effectively can yield tremendous results in terms of improved CRM, better-targeted marketing spend, and more efficient back-end processes. Roger Ares, vice president of analytics at Hyatt Corp., discusses the ways he and his staff use big data.
Beth Schultz
Good Data, Smarter Travel

4|24|14   |   2:50   |   (4) comments


Charged with keeping track of travel assets, including employees, iJET International relies on data management best-practices and advanced analytics to keep its clients in the know on current and potential world events affecting travel, Rich Murnane, Director of Enterprise Data Operations & Data Architect, told All Analytics in an interview from the 2014 SAS Global Forum Executive Conference.
Beth Schultz
Healthcare Data Needs a Booster

4|23|14   |   2:39   |   (0) comments


W. Ed Hammond, Director of the Duke Center for Health Informatics, spoke from the recent 2014 SAS Global Forum Executive Conference about the data management challenges involved in healthcare today.
Beth Schultz
In the Talent Sweet Spot

4|23|14   |   3:25   |   (0) comments


Jason Dorsey, chief strategy officer for the Center for Generational Kinetics and keynote speaker at last month's SAS Global Forum 2014, describes how Gen Y professionals are enhancing the makeup of multigenerational analytics organizations.
Beth Schultz
A2 on the Road

4|22|14   |   2:28   |   (0) comments


From analytics talent development to the power of visual analytics, All Analytics found a variety of common themes circulating throughout the exhibition floor and session discussions at the 2014 SAS Global Forum and SAS Global Forum Executive Conference events held last month in Washington, DC.
Beth Schultz
T-Mobile Hears Data's Call

4|22|14   |   2:50   |   (0) comments


Talking with All Analytics live from the 2014 SAS Global Forum Executive Conference, Eric Helmer, senior manager of campaign design and execution for T-Mobile, discussed the importance of customer data -- starting internally -- in devising the mobile operator's marketing plans.
Beth Schultz
A Is for Analytics in Academia

4|21|14   |   4:11   |   (0) comments


Interviewed live at SAS Global Forum 2014, professors and students agree that access to enterprise analytics software in academic programs better prepares graduates for their careers.
Beth Schultz
Advocating for Analytics Culture

4|21|14   |   5:51   |   (0) comments


Speaking at the recent SAS Global Forum Executive Conference, analytics executives, business experts, and SAS insiders explore what it means to build an analytics culture.
Beth Schultz
Analytic Myths: True or False?

4|21|14   |   4:29   |   (0) comments


At the recent 2014 SAS Global Forum Executive Conference, five analytics experts give us their perspective on whether four common myths about IT and analytics are true or false.
Beth Schultz
We'll Be Your Eyes & Ears

3|20|14   |   2:15   |   (0) comments


We'll be on the scene at SAS Global Forum events in Washington, D.C., March 23 to March 25, glad to share what we learn with our community members.
Beth Schultz
7 Tips for Deploying Visualization

3|7|14   |   33:15   |   (0) comments


We chat with Analise Polsky, a data visualization thought leader on the SAS Best Practices team, about what you need to know before you deploy data visualization.
Michael Steinhart
Choosing a Big-Data Analytics Platform

2|19|14   |   31:53   |   (3) comments


The big-data analytics market can be a confusing place. Among the vendors vying for your dollars are traditional database management providers, Hadoop startup services, and IT giants. In this video, All Analytics editors Beth Schultz and Michael Steinhart sit down in a Google+ Hangout on Air with Doug Henschen, executive editor of InformationWeek. Henschen discusses use cases for big-data analytics, purchase considerations, and his recent roundup of the top 16 big-data analytics platforms.

Related posts:

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Keeping a Close Eye on Shoppers

2|13|14   |   02:14   |   (16) comments


At the National Retail Federation BIG Show last month, All Analytics executive editor Michael Steinhart noted a host of solutions for tracking and analyzing customer activity in retail stores. From Bluetooth beacons to RFID tags to NFC connections to video analytics, retailers must find the right combination of tools to help optimize the shopper experience, streamline operations, and boost revenues.

Related posts:

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Michael Steinhart
Real-Time Demand Drives Forecasting

2|11|14   |   02:22   |   (1) comment


The days when historical shipment trends and gut feelings were enough to forecast retail demand accurately are long over. SAS chief industry consultant Charles Chase outlines the benefits of pulling real-time sales information from point-of-sale and product scanner systems, then flowing that data into dynamic forecasting tools from SAS.

— Michael Steinhart, Circle me on Google+ Follow me on TwitterVisit my LinkedIn pageFriend me on Facebook, Executive Editor, AllAnalytics.com

Videos
Intro to Visual Analytics

6|5|13   |   1:58   |   (0) comments


With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Videos
Visual Analytics, Instant Insight

5|16|13   |   2:06   |   (5) comments


Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
Videos
Big Data, Fast Infrastructure

2|14|12   |   3:35   |   (6) comments


No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Videos
Red Hot Analytics

1|10|12   |   3:51   |   (7) comments


Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
Videos
Hotelier Checks In With Analytics

12|14|11   |   06:55   |   (11) comments


InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.