Forty-one percent of data mining professionals polled for Rexer's 4th Annual Data Miner Survey reported working in the CRM/marketing field. Rexer said 735 data miners from 60 countries participated in its 50-question survey, conducted in early 2010.
CRM/marketing has remained the top field of respondents since the polling began four years ago. Other sectors employing large numbers of data miners include financial, academic, insurance, and telecommunications.
Other conclusions from the survey:
- Most data miners use decision trees, regression, and cluster analysis, though in the latest survey, 22 percent of respondents reported using ensemble models, which made their first appearance in the Rexer survey.
- Data miners tend to prefer models with fewer variables. A third of the respondents reported building final models with 10 or fewer variables, while only 28 percent built models with more than 45 variables.
- The majority of data miners reported using open-source tools and working on laptops or desktops with data stored locally.
- The biggest challenges for data miners remain dirty and difficult-to-access data, as well as the ability to explain their work to others effectively.
- Data quality and analytics capabilities continue to prove problematic. Only 13 percent of data miners rated their company’s analytical capabilities as excellent, and only 8 percent had confidence in the strength of company data.
Despite the challenges, the survey clearly shows an increase in the demand for data mining, a cause for general optimism in the field. More than 75 percent of data miners anticipate more projects at their companies in the near future. These are over and above increases reported in the previous survey.
In addition, Rexer said it has measured the use of the term “data mining” in a variety of online job ads and found strong requirements.
Have you seen an increase in the number of data mining projects and a rise in skill requirements in your company or organization? What fields do you believe make the greatest use of data mining today?