The human mind, on its own, cannot possibly fathom the gargantuan datasets such as those holding geophysical data for oil exploration. Machines augment human intelligence by doing the grunt work of data crunching and finding the patterns, which humans can use to tell a story from the data.
New machine learning techniques going by the moniker of deep learning, commonly referred to as neural network algorithms, find the clusters in the data and the links between them to make sense of the data. It is a lot like solving a crossword puzzle -- the answers are found by looking at the relationships in the clues.
There are two types of deep learning -- supervised learning and unsupervised learning. Supervised learning begins with an understanding of the categories that define the data while unsupervised learning does not presume any knowledge of the categories.
Supervised deep learning is analogous to searching for an undersea destination like an oil well with the knowledge of the coastline alone. It reads the relationships in the geophysical data in the layers underneath the seashore to reach, progressively, the oil well. Unsupervised learning first establishes whether a relationship exists between the contours of the coastline and the subterranean topography.
I spoke with Dr. Charles H. Martin, a long-time expert in machine learning and the founder of Calculation Consulting, about the prospects for enterprise applications of supervised and unsupervised deep learning. “Many in the business world recognize the vast potential of applications of deep learning and the technology has matured for widespread adoption,” he said. “The most hospitable culture for machine learning is scientific and open to recurring experimentation with ideas and evolving business models. The legacy enterprise fixation on engineering and static processes is a barrier to its progress,” he underscored.
Unstructured data abounds, and the familiar methods of analyzing it with categories and correlations do not necessarily exist. The size and variety of such databases can elude modeling. These unstructured databases have valuable information like social media conversations about brands, video from traffic cameras, sensor data of factory equipment, and trading data from exchanges that are akin to finding a needle in a haystack. Deep learning algorithms find the brand value from positive and negative remarks on social media, elusive fugitives in the video from traffic cameras, the failing equipment from the factory data, or the investment opportunity in the trading data.
“Unsupervised deep learning helps in detecting patterns and hypothesis formulation while supervised deep learning is for hypothesis testing and deeper exploration,” Martin said. “Unsupervised deep learning has proved to be useful for fraud detection and oil exploration -- anamolies in the data point to cybercrime and oil respectively,” he explained. “The prediction of corporate performance using granular data such as satellite imagery of traffic in the parking lots of retail companies is an example of second generation of supervised deep learning.”
Early detection of illnesses from medical imaging is one category of problems that deep learning is well suited to address. Citing the example of Chronic Obstructive Pulmonary Disease (COPD), Dave Sullivan, the CEO of Ersatz Labs, a cloud-based deep learning company based in San Francisco, told me, “The imaging data shows nodules and not all of them indicate COPD. It is hard for even a trained eye to tell one from another. Deep learning techniques evolve as they are calibrated and recalibrated (trained) on vast volumes of data gathered in the past, and they learn to distinguish with a high degree of accuracy for individual cases.”
Clarifai has democratized access to its deep learning with its API, which allows holders of data to analyze and benefit from the insights. I spoke with Matthew Zeiler, the CEO and Founder of Clarifai, to understand how its partners use the technology. One of them is France-based I-nside, a healthcare company, which uses smartphones to conduct routine examinations of the mouth, ear, and throat to generate data for diagnosis. “In developing countries where doctors are scarce, the analysis of the data points to therapies that are reliable,” Zeiler said. “In developed countries, the analysis of the data supports the judgment of doctors, and they have reported satisfactory results.”
Enterprise is not the only place where deep learning has found a home. Consumer applications like Google Now, Microsoft’s Cortana, and Assistant are available in the market. Folks are often anxious and distracted, at work or play, when they are unable to keep track of critical events that could affect them or their family. Home surveillance watches pets, the return of young children from school, elderly relatives falling, the arrival of critical packages and more. What matters is an alert on an unusual event. Camio uses the camera of a handheld phone or any other home device like a computer to capture video of happenings at home. If-This-Then-That (IFTTT) alerts are sent when something irregular happens. Deep Learning algorithms separate significant events from the routine.
Unstructured data presents unique problems with its size, variety, and apparent formlessness. Crucial insights lurk in that data. Deep learning helps to find the meaningful information in the clutter of the data.