Comments
Automation Will Change Data Science Beyond Recognition
View Comments: Newest First | Oldest First | Threaded View
Page 1 / 2   >   >>
It is possible to automate some data mining steps
  • 4/21/2013 9:24:26 AM
NO RATINGS

I think that steps such as data preparation and modeling can be automated (to a certain extent). As discussed on my post (http://www.dataminingblog.com/can-we-automate-data-mining/) I think some other steps such as business understanding are not easy to automate.

Re: Automation: Not The Ultimate Answer To Big Data
  • 10/26/2012 10:18:19 AM
NO RATINGS

Data/Decision Sciences has been a speciality-skill for decades because this function has been staffed with very skilled data analysts, modelers, scientists, and engineers. Today, with the digital exhaust and the volume of information logged with VRUs, web, e-mail, mobile, and other structured and unstructured data (as "Big Data"), there is a need for effectiveness of insights as well as efficiency (in terms of speed and costs.) The Data Scientists among us (the esteemed group) help to get the insights and predict outcomes through decision sciences and modeling. However, there is increasingly a need to develop a disciplined and rigorous process that is operationally-sound, repeatable and predictable that can make these algorithm- and decision sciences-enabled processes more efficient -- faster with compute power and cheaper with process automation.

Data Scientists will remain the innovators but they shouldn't be underutilized by mundane, routine processes that can be automated. With advances in event procesing, enabling software, and systems integration, more automation is possible today than we could a few years back. Watch this space... it will get more automated to make it more manageable.

Limits to Automation
  • 10/16/2012 6:39:02 PM
NO RATINGS

Beth,

I think that one can only automate that which is unlikely to change.  Within every information system, there are static and dynamic components. You can automate file transfers, exports, screening and editing (based on rules), and basic analysis.  But just because you have an automated card shuffler does not mean that you should use automation for playing black jack - some decisions - even mathematically based, need human intervention.  For example, you can automate a system that predicts producer prices, but there will still be a need for human intervention to adjust the analysis if an earthquake or flood wiped-out a resource producing country.  People are smart and politcal.  Automation may gradually assume more tasks - especially if we build knowledge on knowledge, but as long as there are unknown or unpredictable events, data scientists will have a role - even if they create jobs for themselves.  How big a role, or how many will be needed in the labor market?  Don't know.  But if something happens that makes the technology unavailable, then good old fashioned know how will be needed. Case in point - our dryer stopped working and my teenage daughters were at a loss to figure out how to dry the clothes.  With my wife and I being baby-boomers, we worked past the automation collapse by hanging clothes lines in the basement.  Hence, automation is great and should be progressive, but when it fails or a new situation arises - people power will save the day.  Besides, automation is only as good as the people who build it - that is restrictive in itself.

Re: Data Science
  • 10/1/2012 7:46:16 AM
NO RATINGS

Good point Wagas

Re: Data Science
  • 9/30/2012 9:07:39 AM
NO RATINGS

@ noreen Data science requires a lot of judgemental decision making as a lot of data involved is unstructured. What we are looking for through automation is AI which is difficult to get reach the perfection stage.

Lack of clarity
  • 9/30/2012 9:01:43 AM
NO RATINGS

I don't know about how things are in US but overall there is a lot of confusion as to what certification helps an individual fulfil the criteria of a data analyst and therefore such a confusion is bound to widen the demand supply gap even more. Somtimes I feel that to become a data analyst, one or two certifications are not enough and experience is a prerequisite as well.

Re: Automation: Not The Ultimate Answer To Big Data
  • 9/29/2012 1:11:34 PM
NO RATINGS

@Hospice    I agree.  Just one more reason we will need Data Scientist and plenty of them .

 

Re: Automation might mean more jobs..... maybe
  • 9/29/2012 12:37:41 PM
NO RATINGS

@Kichoko,

"Someone has to develop them and he will need direction from the data scientist."

You are correct, but when the technology becomes more mature, human intervention will be minimun. But as I said in an earlier post, we are not there yet.

Re: Automation: Not The Ultimate Answer To Big Data
  • 9/29/2012 12:24:15 PM
NO RATINGS

@Louis,

I don't think we already have the technological knowledge that is needed to build accurate data automata, that could seaminglessly work without human intervention in every domain.  So a 100% automation is not yet possible as per today's state of the art in data science.

Automation: Not The Ultimate Answer To Big Data
  • 9/28/2012 7:29:54 PM
NO RATINGS

Wow, this certainly an interest argument.  Because we don't have the man or woman power now we should just automate Big Data processes or lose this opportunity ?  

Well If I understand this argument correctly - I certainly don't agree. I think automation has it's place of course but I think it is a bit early to worry about Big Data going to waste because we don't have the workforce to mine it - that was the issue even before Big Data became Big Data.

Page 1 / 2   >   >>


INFORMATION RESOURCES
ANALYTICS IN ACTION
CARTERTOONS
VIEW ALL +
QUICK POLL
VIEW ALL +