Intelligence Moves to the Edge of the IoT

The fusion of the physical world with the digital skews at the edge, where numerous activity streams in factories, campuses, healthcare facilities, and vehicular traffic benefit from optimization. The dispersion of intelligence to edge nodes, particularly algorithms, paves the way for a new generation of applications development for systems or sub-systems.

Sensor-enabled monitoring of individual activities at the edge has been widespread for a generation. Emerging applications for systems or sub-systems at the edge streamline the flow of a series of inter-dependent activities at the edge with help from algorithms.

System or sub-system-wide applications compound the information value of multiple sources of data, and their processing, to consumers of actionable intelligence. Traffic data, for example, is immeasurably more valuable if data on events such as the Fukushima nuclear disaster and the concurrent tsunami could be used to redirect cars to safety.

At the edge, it is possible to merge information flows at several different levels to parse data for decision-making. It could start with data streams from sensors at an edge pole, converging in-stream and the intelligence displayed locally on a hub like a smartphone. Alternatively, the information flows could converge at a local gateway -- the host for the analytical engine. Another possibility is edge-to-edge communications for services to serve a region. Finally, information from individual edge points or several of them feeds data to a central cloud for processing.

I spoke to Aditya Yadav, the founder and Chief Executive Officer of London-based Automatski, about the solutions at the edge and the technological strategies to achieve them. “Eighty-percent of IOT solutions should be for the edge, 15% for the fog and only the remaining 5% should be for the cloud. Currently, the ratios are roughly the converse,” he said.

The shift can only happen with technological solutions that adapt to the constraints of memory, processing power and storage characteristics at the edge. “Our platform exposes a dozen algorithms as microservices to operate in the constrained environment at the edge,” Aditya Yadav explained. “The microservices enable distributed deep learning with parallel data processing at multiple edges, instead of sequentially, to speed up the mining of data for real-time service delivery to users.”

“An edge cloud mediates the communication between multiple edge poles for collaborative solutions,” Yadav added. The flow of data between edge points aggregates the data for a neighborhood, district, city or region. “A typical application is a vehicle-to-vehicle service for decongestion of traffic that aggregates data across multiple edges to predict choke points and redirects cars away to less congested roads,” he said.

The granularity of the data from the edge uncovers patterns that predictive models can use for new services. Weather data is widely available to drivers but is short on details of hyper-local data that is predictive of risks in rapidly changing inclement weather. Unbeknownst to drivers, roads could become dangerously slippery in parts of their route as a result of ice formation, or pools of water could accumulate, and they have no way of knowing where. One fifth of vehicle crashes are weather related and occur when drivers are caught off-guard by fast changing conditions on roads.

Weathercloud and Fathym have collaborated to deploy a solution for real-time prediction of local weather conditions in Colorado and Alaska, and trials are underway in several other states. Their solution crowdsources data from sensors in vehicles. One of the sensors, attached inside of windshields and pointed at the sky, reads the type and rate of precipitation, among other indicators. The other one, affixed to the bumper, has an infrared sensor directed at road surfaces to sense pavement temperature. Fathym’s data fusion algorithms combine the data from the infrared sensor with the data on ambient temperature, humidity, and pressure from other sensors on the bumper as well as current precipitation data from the windshield to infer road conditions such as ice formation and snow. I spoke to Matthew Smith, co-founder and CEO of Fathym, to learn more about the solution. “The analytics on the data is done partly in the smart hub that is a smartphone and some in-stream for near real-time results,” he told me. “Microservices enable the management of the entire application including the analytics component,” he explained.

Sensors generate detailed information at the edge, and it is easy to lose the devil in the clutter of information, much of which is noisy. Sprawling supply chains are devilishly difficult to monitor and are susceptible to fraud. Recent reports have documented increasing incidence of counterfeit drugs slipped into pharmaceutical supply chains while the more valuable legitimate ones are diverted for illegal sales. I spoke to Mark O’Neill, vice president of innovation for Axway, which has a platform with APIs for the development of applications for the IoT edge. “The provenance of pharmaceutical drugs is tracked, at each shipping point, with help from the data generated by the RFIDs that identify the ownership of the drugs. Smartphones serve as hubs and display reports to confirm the authenticity of the drugs transported. The Smartphone applications use the APIs to access data,” O’Neill said.

Innovation at the edge has accelerated as the barriers that previously hamstrung applications development are superseded by new technologies. Legacy devices had wired connections with networks and were hard to control remotely. “Legacy devices can now be equipped with embedded modules that enable seamless wireless connections for real-time monitoring and control," said Alex Glaser, director for Harbor Research Inc., an industry analyst firm specializing in pervasive computing. “Companies like Predixion and Predikto use algorithms to mine data at the edge to predict preventive maintenance needs for railways and other assets for implementation in real-time.”

The complexity of system-wide applications that span hundreds and thousands, if not millions of devices, not to mention dozens of analytical tools precludes real-world testing of applications before a full launch. For most customers, the cost of failure would be forbidding. “Simulations of applications using device logic instead of actual devices, including training of algorithms on real data, lowers the cost of development and testing of new applications at the edge,” Automatski's Yadav told me.

Security of the microdevices at the IoT edge is a priority, as physical systems are at risk. It is hard to accommodate the overhead of security software with the limited resources of tiny devices. “API gateways provide alternative ways to secure microdevices -- they can create perimeters for a bunch of them and write rules for roles and attributes-based rights access,” O’Neill said.

The technical architecture for the IOT edge is beginning to crystallize laying the foundation for pervasive computing in the physical world. Now is the time for visionaries to address long neglected use cases for solutions development.

Kishore Jethanandani, Market Researcher

Kishore Jethanandani's current professional focus is content development and communications of complex subject matter for clients in the technology and financial services industry in the San Francisco Bay and Boston area. His career has evolved from industrial economic research with an accent on policy reform to a business journalist while he was in India to marketing writing and research in the USA. A technology buff and a futurist, he has an instinct for spotting emerging technology and market trends and their likely impact on business strategy and competition.  His personal slogan is "A hedgehog who senses the future". My website can be found at

Mesh Networks Promise to Tap a Wealth of IoT Data

The Internet of Thing's last mile holds rich data seams that can be opened for data mining with mesh networks.

What You Need to Know About Prescriptive Analytics

Even though companies have just scratched the surface with adoption of prescriptive analytics, some are getting real-world benefits, based on better decision making.

Re: More memory and database choices
  • 6/30/2016 8:57:36 PM

Interesting anecdote, Pierre DeBois. What changed by doing this at the edge?

Re: More memory and database choices
  • 6/30/2016 7:19:21 PM

I am amazed at how that storage at the edge creates new options. I started using Meteor.JS which includes MongoDB.  The complete stack makes an app easier to maintain from the developer perspective.

Re: More memory and database choices
  • 1/5/2016 1:18:48 AM

Kishore writes:

Several different options are now available for storage which includes not just the cloud but the gateway or local hubs. Intelligence itself is distributed to numerous different points.

I agree. There will be more options as the future unfolds.

Re: More memory and database choices
  • 1/1/2016 1:24:05 PM

Several different options are now available for storage which includes not just the cloud but the gateway or local hubs. Intelligence itself is distributed to numerous different points. 

Re: More memory and database choices
  • 12/31/2015 10:14:29 PM

This is an awesome idea, and clearly more work needs to be done for the edge of the IoT really devlop, but solutions like this are not going to be the fight fit for every solution. Many of the sensors deployed just are not going to have the resources for this. But where it can work, it will be awesome. 

More memory and database choices
  • 12/31/2015 7:59:05 PM

The number of in-memory options will influence how edge calculation is handled.  It will also influence data management decisions -  which data should be centralized and which should be accessible for a device.

Near endless possibilities
  • 12/31/2015 7:56:04 PM

Excellent article! The limitations are set by the imagination. We will see benefits in our daily activities from work to recreation with greater optimization and safety.