We've all read the articles and blogs. Many of us have experienced the issues directly -- the demand for deep analytical skills is outpacing the supply.
As evidence of this, in a period of economic slowdown, where we read that 50 percent of college graduates can't get a job, college graduates with degrees remotely aligned with applied analytics have multiple offers in advance of graduation. Academic training in applied (versus theoretical) statistics is helpful -- and mitigates some of this talent gap at the entry level. Nonetheless, we all know it's insufficient to meet the growing demand for what we now know as the data scientist.
A scan of offerings at universities across the country shows that none will result in a single degree program in data science. As a professor, I can see at least three reasons why this is so:
Universities historically are not massive ivory towers, but groups of towers called colleges or departments. Cross-discipline degrees aren't a strength of university curricula. Pity the engineering student who wants to minor in history or the student who wants to cobble together a degree in public policy of architecture and ecological science. Ask any student who has tried to cross colleges within a university to create a targeted degree -- it is the worst of bureaucracy, outdated registration systems, and academic-elite egos all wrapped in a Gordian knot. And yet this is exactly what data science is -- the intersection of mathematics, statistics, and computer science, combined with some potential area of content application like finance, biology, or sociology.
The data tsunami washing over all companies, not just data-driven ones, is a fairly recent phenomenon. Professors who teach statistics and computer science, in particular, are recognizing that the traditional skills we have comfortably taught for years or even decades don't work in this environment. Concepts like p-values to derive significance are meaningless when you have a billion rows of data. Professors are being challenged to teach skills that many of them don't have. Some are rising to the challenge, and some (think tweed jackets with leather elbow patches) are just hoping it all goes away, which, of course, it won't.
We need your data. Remember the datasets you saw in the classroom? They had 100 observations, three variables, and no missing values. Everything was significant in its raw form. Welcome to textbook data. We do our students an immense disservice by using this kind of dataset to teach analytics. But, believe it or not, in a sea of data, we are dying of thirst. Universities need massive, complex, unstructured, messy data with missing and (mis)coded values for use in the classroom. Ultimately, we can't teach data science skills without big-data.
I encourage people within the public and private sectors to partner with universities and in particular with professors who have recognized these issues and are trying to pivot their curricula to meet the needs of the marketplace. Sit on advisory boards. Provide real datasets (scrubbed as needed). Offer to speak in the classroom of your experiences with big-data -- everyone's story is the same, but different. Partnerships with universities in this area are particularly important and mutually beneficial. You can help us train your future data scientists.
Do you agree that change is needed if universities are to educate the data scientists that businesses increasingly need? Share your thoughts below.
Very useful suggestion about companies offering partnerships with universities. This will be a win-win situation for both. Companies will get future data scientists from the universities which will help them reduce data scientist shortages and universities will get both structured and unstructured big data to play with and executives who could come and share their experiences with data in practical life. Whats needed is a forum where universities can meet such companies so that both can discuss their needs.
I wont speak about US but certainly in many countries, Asian countries esp, we dont have universities offering such courses which help us become data scientists even if we want to. A tech geek college student who is doing majors in applied statistics may go towards that path however it is not yet close enough. May be the universities need to not only introduce the course but also increase the awareness that such a course exists. It wont be easy.
" The fact is that analytics, like technology, does not exist in a vaccuum. It is a powerful tool for all disciplines."
@mnorth Excellent point and I couldn't agree more ! I sometimes feel the focus is too narrowly focused on business needs, but as you mention analytics is used in every area of society.
And thank you Jennifer for exposing some issues that stand in the way of effective training of future Data Scientist, I am not sure where to start - I am sure there will be much fine tuning of curriculum and approach for years to come.
@Cordell, somebody sure pulled the wool over your eyes!
But seriously, you raise an interesting point about developing something that was practically meaningful but not statistically meaningful. If something is not statistically meaningful is it OK to put it to practical use?
@bulk: Unfortunately, I tend to spread myself too thin a lot, I think it's in my nature. You really can't do too much tower crossing without compromising quality, so I try to pick one or two interdisciplinay projects to participate in each year, depending on the amount of work expected for each project. It has to be an intentional and planned approach or you can find yourself with way too many irons in the fire.
As an academician, I completely agree that crossing towers is needed. Who needs analytics more: a biologist or a sociologist? A psychologist or an economist? The fact is that analytics, like technology, does not exist in a vaccuum. It is a powerful tool for all disciplines. We ought to be stretching out across fields of study, across the boundaries of colleges or departments, and helping one another accomplish real, valuable work using the tools at our disposal. Where I teach, the only way that's happened has been for me to take the initiative to work one-on-one with colleagues in other departments. When they have projects on a health epidemic, urban sprawl, poverty, teen pregnancy, etc., their projects almost always generate data, both structured and unstructured. If I'm willing, there is no end to the opportunities to offer my analytics expertise to their work, but I must be willing to embrace interdisciplinarianism!
It really can be a pain to cross between colleges, my first attempt at college left me feeling locked out when I tried to grab a minor to go with my computer science major. There was just no way I could make it work and no one in either college was very helpful. 15 years removed from that situation I can say it worked out for the best, but at the time it did seem so.
Tech Marketing 360 The only event dedicated to technology marketers. Discover the most current and cutting-edge innovations and strategies to drive tech marketing success. Hear from and engage with companies like Mashable, SAS, Dun & Bradstreet, ExactTarget, Google+, IDC, Microsoft, LinkedIn, Oracle Eloqua, Leo Burnett, Young & Rubicam, Juniper Networks and more – all in an intimate, upscale setting. Register at http://www.techmarketing360.com with priority code CMANALYTICS14 to save $100.
SAS Global Forum Executive Conference 2014 The Executive Conference is held in conjunction with SAS Global Forum, a SAS users technology event. Investing in thought leadership and technical training are two of the best moves a successful company can make so take advantage of the world-class speakers, sessions and discussions around Analytics, Big data, Risk, Fraud and Data management.
LEADERS FROM THE BUSINESS AND IT COMMUNITIES DUEL OVER CRITICAL TECHNOLOGY ISSUES
The Current Discussion
Visual Analytics: Who Carries the Onus? The Issue: Data visualization is an up-and-coming technology for businesses that want to deliver analytical results in a visual way, enabling analysts the ability to spot patterns more easily and business users to absorb the insight at a glance and better understand what questions to ask of the data. But does it make more sense to train everybody to handle the visualization mandate or bring on visualization expertise? Our experts are divided on the question. The Speakers: Hyoun Park, Principal Analyst, Nucleus Research; Jonathan Schwabish, US Economist & Data Visualizer
David Tishgart, senior director of marketing and alliances at security provider Gazzang, explains the importance of data encryption for companies that are rolling out Hadoop environments to leverage big data analytics.
At the Strata Conference / Hadoop World 2013, Samuel Kommu, technical marketing engineer at Cisco Systems, shares some of the benefits that Hadoop brings to analytics platforms that leverage next-generation hardware. Kommu looks at big data operations that required 3,500 nodes in 2009, 2,000 in 2011, and now require only 64 nodes.
Wayne Thompson, manager of SAS Data Sciences Technologies, delivers a fascinating preview demonstration of SAS Visual Statistics, a tool that enables fast and flexible modeling against massive datasets on the fly. Visual Statistics will be made generally available in March, but you can see it here first.
At Strata/Hadoop World 2013, Cloudera CEO Tom Reilly discusses the new Enterprise Data Hub offering, explaining how it works with Hadoop, how it creates a single repository of full-history and full-fidelity data, and how it exposes that data to all users interested in exploratory analytics.
At this year's Strata Conference/Hadoop World 2013, SAS big data vice president Paul Kent presented a session on setting up Hadoop clusters for advanced analytics. We caught up with several audience members and recorded their impressions of the presentation.
In hearing directly from a doctorate-level Hadoop specialist, a healthcare data analyst, and a marketing executive, it's clear that big data analytics is a burgeoning field that cutting-edge companies are eager to explore.
At this year's Strata Conference/Hadoop World 2013 event, SAS VP of Big Data Paul Kent presented several sessions about modernizing and deploying advanced data analytics infrastructures based on Hadoop. In this video, he talks about the state of Hadoop adoption among enterprises today and looks out to the big data-driven applications of the future.
Companies that use SAS analytics tools for their traditional databases are looking to derive even more value by mining unstructured data. Data management platforms like Hortonworks enable that relationship by delivering an enterprise-ready Hadoop framework.
In this video, Shaun Connolly, vice president of corporate strategy at Hortonworks, explains how companies can incorporate Hadoop into their data analytics streams.
At the SAS Premier Business Leadership Series in Orlando, Manuel Sanchez, CRM Manager for Club Premier Aeromexico, explains the challenges and opportunities of transaction data. Using dozens of data sources among participating airlines and merchants, Club Premier creates robust customer profiles and works to maximize benefits for members and business partners alike while protecting individual privacy.
At SAS's October Premier Business Leadership Series (PBLS) in Orlando, attendees from the corporate and academic worlds joined thought leaders and analytics professionals to share insights and strategies around big data.
Will Hakes, CEO and co-founder of Link Analytics and keynote speaker at the SAS Analytics 2013 conference in Orlando, Fla., last month, talks candidly about the challenges that large enterprises face as they explore advanced analytics solutions. He also shares some practical tips for smoothing the transition.
At the SAS Analytics 2013 conference in Orlando, Bob Gladden, vice president for decision support and informatics at the Ohio nonprofit health insurance provider CareSource, explains how his company uses advanced analytics to keep administrative costs down and to identify at-risk patients for targeted healthcare initiatives.
At the Analytics 2013 conference in Orlando, Fla., two analytics experts from Dell -- global decision sciences manager Natalie Kortum and senior credit risk consultant Jack Chen -- share their real-world advice for analysts who want to sell their project ideas to business executives.
At the SAS Premier Business Leadership Series in Orlando, Fla., Lousiana State Representative Chris Broadwater outlined the state's success with analytics-driven fraud detection and shared his vision for streamlined processes at the DMV, the healthcare system, and even the department of corrections -- all delivered via a centralized repository of rich customer data.
Organizations that are ready to leverage big data need to move beyond buzzwords and approach the challenges with a business focus. Peter Guerra, principal at Booz Allen Hamilton, shares his insight and experience in helping clients transition to Hadoop and embrace new decision support platforms.
At this year's Strata Conference / Hadoop World 2013, Michael Steinhart chats with Rackspace Product Marketing Manager Sean Anderson about Hadoop, cloud computing, and how the two come together for companies that want to undertake a "proof of value" project.
With today's advanced visual analytics tools, you can stream data into memory for real-time processing, provide users the ability to explore and manipulate the data, and bring your data to life for the business.
Dynamic data visualizations let analysts and business users interact with the data, changing variables or drilling down into data points, and see results in a flash. Advance your use of data visualization with tools that support features like auto-charting, explanatory pop-ups, and mobile sharing.
No doubt your enterprise is amassing loads of data for fact-based decision-making. Hand in hand with all that data comes big computational requirements. Can traditional IT infrastructure handle the increasing number and complexity of your analytical work? Probably not, which is why you need a backend rethink. Big data calls for a high-performance analytics infrastructure, as Fern Halper, a partner at the IT consulting and research firm, Hurwitz & Associates, discusses here.
Redbox's bright-red DVD kiosks are all but ubiquitous these days, located in more than 28,000 spots across the country. Jayson Tipp, Redbox VP of Analytics and CRM, provides an insider's look at how the company has accomplished its phenomenal nine-year growth.
InterContinental Hotels Group (IHG), a seven-brand global hotelier, has woven analytics into the fabric of its operations. David Schmitt, director of performance strategy and planning, shares IHG's analytics story and his lessons learned.