It would be trite to say that when Matt Turck talks about big-data, people listen. But it's true, so I'll say it anyway.
Turck is a managing director at Bloomberg Ventures, the incubation and investment arm of Bloomberg LP, and founder of the NYC Data Business Meetup, the main monthly big-data event in Manhattan.
In addition to organizing meetups and pursuing investment and partnership opportunities for Bloomberg, he helps operate Bloomberg Ventures-incubated businesses. Most recently, he spearheaded the launch of the Bloomberg Institute, which wants to become the world's leading online education and recruitment business for finance professionals.
Before joining Bloomberg Ventures as a principal in 2008, he was a senior director at Oracle. He joined Oracle in 2005, when the company acquired TripleHop Technologies, a venture-backed enterprise software company that Turck co-founded and where he served as president and COO.
Did I mention he started his career as a securities lawyer?
But none of that may be as impressive to big-data pros as the fact that Turck and a colleague recently attempted the near impossible by mapping the rapidly evolving big-data ecosystem. They made a first attempt in June. After more interviews, research, and feedback from people who have a stake in big-data, they released a second version of the map in Ocotber.
"It's still a work in progress and will presumably always be. That's the nature of the beast," he told me. The second version "is even more crowded than the first time around, which reflects the incredible vitality of the big-data space."
I was so impressed with the chart that I decided to ask Turck for even deeper insights -- a state of the union on big-data, if you will. Here are the top five things he thinks you should know.
We're a bit ahead of ourselves in the cycle. As of now, there's a bit of a gap between all the excitement (in the press, conferences, startups, and venture capital circles) and the reality of what most large companies are doing with big-data technologies (still often just starting to dip their toe in the water, in the form of pilots or lab experiments.)
Large companies are often right to be careful with adopting big-data technologies. A number of products are still very new and fairly immature, and a variety of early-stage companies are still largely "science projects," so it's important to evaluate those technologies carefully and thoroughly.
Things are evolving rapidly, however. Big-data (together with cloud computing) is a long-term trend that will play out over a number of years. The needs are real, the technologies to address the needs are maturing quickly, and the excitement draws a lot of smart minds to solving big-data problems, making it a self-fulfilling prophecy that big-data will have a profound impact -- in some ways, a revolutionary impact -- on business and society.
What's truly exciting is that we have barely started scratching the surface. A lot of progress has been made at the infrastructure and analytics level, but the world of big-data applications (whether enterprise or consumer-facing ones) is just starting. Take a look at the chart above for examples of companies doing great things in those categories.
Even though the spotlight is on the startups that build new big-data technologies, ultimately the real beneficiaries of big-data will be large companies that will fully leverage those new technologies, along with the customers they serve.
What do you think of his list? Can you amplify any of his points?
A problem of big-data projects is that they are of complex nature and require reasonable technical and human resources and oversight if they are to be executed effectively. Also the problem is that most companies lack experience in big data projects therefore there are many unforeseen setbacks. The solutions are to plan well or hire consultants who are experts in the field. I am commenting very vaguely and generally. Ofcourse there are many things to look at and also, some companies might think that big-data projects are easily manageable and of simple nature.
Reading between the lines is a good point - IT and anyone with tech in their responsiblity scope are sometimes approached with a light-switch mentality. In other words, decisions and information are shared as if the team can be started or stopped instantaneously. That can be a costly decision-making process if management does not apply good time management techniques. Managing technical resources is critical to project success and ensuring that poorly performing projects have a minimal cost to resources.
" IT professionals have seen it time and time again. It may be a while before any signs of burn out are revealed"
It may even be difficult for people like us to get aware of the burn-outs as many corporates wont disclose their failure to us i.e. if they realize that their big-data projects are a failure. What the executives will reveal to the press is that big data concept is new to the information industry and there is lot to be learnt. You just have to read between the lines and figure that it is an announcement of failure or criticism on the people responsible in their organizations.
It is true that big data, if taken as an industry to evaluate, has not gone beyond the immature or experimental phase. In case of many companies, it is just now that they are beginning to even think that they need data analysts in their HR pool for the task as specialists and the all-rounder financial analysts are too less-knowledgeable for handling the big quantum data in an effective manner.
To be honest I didn't see anything in the list that I was not aware of but I guess I understand big data issues a little better than the "average person", that aside as with any new area of study it will take time for theory to become reality but it is IMO a certainty because we as a society are moving towards ever more data not less.
So whether businesses want to or not - big data is an issue they must confront eventually.
Over-hyped doesn't mean Big Data won't change how we work and live. Most everything that makes a difference goes through stages of over-optimism, followed by over-pessimism before settling into reality.
For Big Data, the expected timeline is shorter than most of the other technologies that are at the same stage of the hype cycle - Big Data is projected to be mainstream in just 2 to 5 years:
Alexis, it's a strong possibility that the big data may be hyped passed the actionability busineses can take. IT professionals have seen it time and time again. It may be a while before any signs of burn out are revealed. I think a determining factor is the amount of actionable results that are announced related to big data. To me it is in the same boat at Facebook commerce - it will come but not without some trial and error before a strategy an be widely used.
The gap between excitement and reality will change as more companies discover economic value from their big data effort. Until then, the hype will outstrip the reality, though I don't think anything will be solely flights of fancy.
Tech Marketing 360 The only event dedicated to technology marketers. Discover the most current and cutting-edge innovations and strategies to drive tech marketing success. Hear from and engage with companies like Mashable, SAS, Dun & Bradstreet, ExactTarget, Google+, IDC, Microsoft, LinkedIn, Oracle Eloqua, Leo Burnett, Young & Rubicam, Juniper Networks and more – all in an intimate, upscale setting. Register at http://www.techmarketing360.com with priority code CMANALYTICS14 to save $100.
SAS Global Forum Executive Conference 2014 The Executive Conference is held in conjunction with SAS Global Forum, a SAS users technology event. Investing in thought leadership and technical training are two of the best moves a successful company can make so take advantage of the world-class speakers, sessions and discussions around Analytics, Big data, Risk, Fraud and Data management.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
David Tishgart, senior director of marketing and alliances at security provider Gazzang, explains the importance of data encryption for companies that are rolling out Hadoop environments to leverage big data analytics.
At the Strata Conference / Hadoop World 2013, Samuel Kommu, technical marketing engineer at Cisco Systems, shares some of the benefits that Hadoop brings to analytics platforms that leverage next-generation hardware. Kommu looks at big data operations that required 3,500 nodes in 2009, 2,000 in 2011, and now require only 64 nodes.
Wayne Thompson, manager of SAS Data Sciences Technologies, delivers a fascinating preview demonstration of SAS Visual Statistics, a tool that enables fast and flexible modeling against massive datasets on the fly. Visual Statistics will be made generally available in March, but you can see it here first.
At Strata/Hadoop World 2013, Cloudera CEO Tom Reilly discusses the new Enterprise Data Hub offering, explaining how it works with Hadoop, how it creates a single repository of full-history and full-fidelity data, and how it exposes that data to all users interested in exploratory analytics.
At this year's Strata Conference/Hadoop World 2013, SAS big data vice president Paul Kent presented a session on setting up Hadoop clusters for advanced analytics. We caught up with several audience members and recorded their impressions of the presentation.
In hearing directly from a doctorate-level Hadoop specialist, a healthcare data analyst, and a marketing executive, it's clear that big data analytics is a burgeoning field that cutting-edge companies are eager to explore.
At this year's Strata Conference/Hadoop World 2013 event, SAS VP of Big Data Paul Kent presented several sessions about modernizing and deploying advanced data analytics infrastructures based on Hadoop. In this video, he talks about the state of Hadoop adoption among enterprises today and looks out to the big data-driven applications of the future.
Companies that use SAS analytics tools for their traditional databases are looking to derive even more value by mining unstructured data. Data management platforms like Hortonworks enable that relationship by delivering an enterprise-ready Hadoop framework.
In this video, Shaun Connolly, vice president of corporate strategy at Hortonworks, explains how companies can incorporate Hadoop into their data analytics streams.
At the SAS Premier Business Leadership Series in Orlando, Manuel Sanchez, CRM Manager for Club Premier Aeromexico, explains the challenges and opportunities of transaction data. Using dozens of data sources among participating airlines and merchants, Club Premier creates robust customer profiles and works to maximize benefits for members and business partners alike while protecting individual privacy.
At SAS's October Premier Business Leadership Series (PBLS) in Orlando, attendees from the corporate and academic worlds joined thought leaders and analytics professionals to share insights and strategies around big data.
Will Hakes, CEO and co-founder of Link Analytics and keynote speaker at the SAS Analytics 2013 conference in Orlando, Fla., last month, talks candidly about the challenges that large enterprises face as they explore advanced analytics solutions. He also shares some practical tips for smoothing the transition.
At the SAS Analytics 2013 conference in Orlando, Bob Gladden, vice president for decision support and informatics at the Ohio nonprofit health insurance provider CareSource, explains how his company uses advanced analytics to keep administrative costs down and to identify at-risk patients for targeted healthcare initiatives.
At the Analytics 2013 conference in Orlando, Fla., two analytics experts from Dell -- global decision sciences manager Natalie Kortum and senior credit risk consultant Jack Chen -- share their real-world advice for analysts who want to sell their project ideas to business executives.
At the SAS Premier Business Leadership Series in Orlando, Fla., Lousiana State Representative Chris Broadwater outlined the state's success with analytics-driven fraud detection and shared his vision for streamlined processes at the DMV, the healthcare system, and even the department of corrections -- all delivered via a centralized repository of rich customer data.
Organizations that are ready to leverage big data need to move beyond buzzwords and approach the challenges with a business focus. Peter Guerra, principal at Booz Allen Hamilton, shares his insight and experience in helping clients transition to Hadoop and embrace new decision support platforms.
At this year's Strata Conference / Hadoop World 2013, Michael Steinhart chats with Rackspace Product Marketing Manager Sean Anderson about Hadoop, cloud computing, and how the two come together for companies that want to undertake a "proof of value" project.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.