Machine Learning Changes the Way We Forecast in Retail and CPG

Machine learning is taking a significant role in many big data initiatives today. Large retailers and consumer packaged goods (CPG) companies are using machine learning combined with predictive analytics to help them enhance consumer engagement and create more accurate demand forecasts as they expand into new sales channels like the omni-channel. With machine learning, supercomputers learn from mining masses of big data without human intervention to provide unprecedented consumer demand insights.

Predictive analytics and advanced algorithms, such as neural networks, have emerged as the hottest (and sometimes controversial) topic among senior management teams. Neural network algorithms are self-correcting and powerful, but are difficult to replicate and explain using traditional multiple regression models.

For years, neural network models have been discarded due to the lack of storage and processing capabilities required to implement them. Now with cloud computing using supercomputers' neural network algorithms, along with ARIMAX, dynamic regression and unobserved components, models are becoming the catalyst for "machine learning-based forecasting."

According to an article in Consumer Goods Technology magazine, through pattern recognition there will be a shift from active engagement to automated engagement. As part of this shift, technology (machine learning) takes over tasks from information gathering to actual execution. Compared to traditional demand forecasting methods, machine learning-based forecasting helps companies understand and forecast consumer demand that, in many cases, would be otherwise impossible. Here are several reasons why:

Incomplete versus complete information and data. Traditional demand forecasts are based on time-series forecasting methods (Exponential Smoothing, ARIMA, and others) that can only use a handful of demand factors (e.g., trend, seasonality, and cycle). On the other hand, machine learning-based forecasting combines learning algorithms (ARIMAX, dynamic regression, neural networks and others) with big data and cloud computing to analyze thousands -- even millions -- of products using unlimited amounts of causal factors simultaneously up and down a company’s business hierarchy.

Traditional demand forecasting and planning systems are restricted to only the demand history, while machine learning-based forecasting can take advantage of limitless data, determining what’s significant, then prioritize available consumer insights (demand sensing) to influence future demand using “what if” analysis (demand shaping). Compared to traditional time-series forecasting systems, machine learning-based forecasting solutions identify the underlying demand drivers that influence demand, uncovering insights not possible with traditional time-series methods. Additionally, the self-learning algorithms get smarter as they consume new data and adapt the algorithms to consumer demand.

Holistic models using multiple dimensions versus single dimension algorithms. Traditional forecasting systems are characterized by a number of single-dimension algorithms, each designed to analyze demand based on certain data-limited constraints. As a result, much manual manipulation goes into cleansing data and separating it into baseline and promoted volumes. This limits which algorithms can be used across the product portfolio.

Machine learning-based forecasting takes a more sophisticated approach. It uses pattern recognition with a single, general-purpose array of algorithms that adapt to all the data. They fit many different types of demand patterns simultaneously across the product portfolio up/down the company’s business hierarchy without data cleansing handling multiple data streams (e.g., price, sales promotions, advertising, in-store merchandising and many others) in the same model -- holistically -- without cleansing the data into baseline and promoted volumes.

For example, traditional forecasting systems have a specific purpose leading to multiple inconsistent forecasts across the product portfolio. With machine learning-based forecasting, the same algorithm is useful for multiple processes including pricing, sales promotions, in-store merchandising, advertising, temperature, store inventory, and others creating one vision of a realistic integrated forecast.

Partial versus complete use of item history. When creating demand forecasts, traditional demand forecasting and planning systems analyze the demand history for a particular product/SKU, category, channel and demographic market area. Machine learning-based forecasts leverage history for all items, including sales promotions, to forecast demand for every item at every node in the business hierarchy simultaneously.

Many feel the next generation of machine learning will also include cognitive computing where the supply chain becomes self-healing. This would improve upon machine learning by going beyond predictions to making decisions to automatically correct for anomalies in the supply chain.

Do you see machine learning-based forecasting supporting the next-generation demand management? Will it eventually lead to cognitive learning creating an autonomic self-healing supply chain; or are you still relying on cognitive dissonance to justify and maintain judgmental harmony within your current demand forecasting and planning process?

You can follow Charlie on:

This content was reposted from SAS Voices. Go there to view the original.

Charlie Chase, Industry Consultant, SAS Retail/CPG Global Practice

Charles Chase is the co-author of Bricks Matter: The Role of Supply Chains in Building Market-Driven Differentiation and the author of Demand-Driven Forecasting: A Structured Approach to Forecasting, as well as over 50 articles in several business journals on demand forecasting and planning, supply chain management, and market response modeling. He is the principal industry consultant for the SAS Retail/CPG global practice, and writes a quarterly column entitled, "Innovations in Business Forecasting" in the Journal of Business Forecasting.

Machine Learning Changes the Way We Forecast in Retail and CPG

Large retailers and consumer packaged goods companies are using machine learning and predictive analytics to improve customer experiences.

The Demand Signal Management Proposition...

While the availability and collection of data are compelling companies to invest in demand signal management solutions to support their planning processes, many have not gotten the return on their investment. However, others are realizing that the real value is in visualization, exploration, and predictive analytics.

Re: Wow.
  • 11/8/2016 9:29:45 AM

@Lyndon_Henry I did a search and what I found out was that Amazon offers a machine learning service. I have no doubts they're already using it internally.

"Amazon Machine Learning is a service that makes it easy for developers of all skill levels to use machine learning technology. Amazon Machine Learning provides visualization tools and wizards that guide you through the process of creating machine learning (ML) models without having to learn complex ML algorithms and technology. Once your models are ready, Amazon Machine Learning makes it easy to obtain predictions for your application using simple APIs, without having to implement custom prediction generation code, or manage any infrastructure."

Re: Wow.
  • 11/2/2016 9:07:45 PM


Magnetic writes

The efficiencies that machine learning can offer just amazes me. I do think, though, that online retailers will jump into this sooner than brick-and-mortar stores, though I'd say the latter need it more than the former.


I wonder if Amazon or other big online retailers like Costco or Walmart have implemented machine-learning technology (or even some elements of it) for their demand forecasting.


  • 10/31/2016 6:57:47 AM

I've never worked in consumer goods or retail, so supply chain tech is only vaguely familiar to me. The efficiencies that machine learning can offer just amazes me. I do think, though, that online retailers will jump into this sooner than brick-and-mortar stores, though I'd say the latter need it more than the former.