Three important data sources used in demand forecasting and why AI needs them

Data holds the answer to a lot of questions but it’s also the source of many challenges for retailers and, as AI has reached a new level of maturity and efficiency, data is critical. Underestimating the importance of the sources of data used in demand forecasting and their effect on the functioning of AI only undermines and cripples the real power of AI. Read our paper on Demand Forecasting and AI for more in-depth information.

Top French grocery retailer, Intermarché recently conducted a test to compare AI to its existing systems. We executed the test across multiple warehouses – the smallest containing frozen products and the largest managing nearly 30,000 SKUs. AI performed beyond expectations, resulting in a forecast accuracy rate of 95% and a reduction in forecast errors of 75%. While many factors contributed to this success, a key enabler was the availability of quality data in large volumes.

AI is data-hungry. In order to optimize results from an AI model built for demand forecasting, several data sources are required. We can define these sources by the following three categories: 

What has happened: establishing a baseline for AI interpretation with historical data

AI needs access to a sufficient amount of historical data in order to achieve Intermarché’s level of accuracy. In Intermarche’s instance, it was able to call upon three years of raw sales and shipment data. Raw data is the key to success for AI. Manual manipulation or interpretation of data essentially “contaminates” it, rendering it unsuitable. AI will not be effective if it’s making inaccurate decisions based on outside influences.

What is happening: how contextual data and AI take history to a new level

Factors such as holidays, weather, school calendars and events greatly influence demand across categories, clusters and individual stores. AI-enabled systems can map these and other significant influencers at local, regional or national levels against the raw sales and shipment data. They can then examine contextual data in relation to historical data to discover and understand previously undetectable patterns of behavior. Thus, enabling demand forecasts to respond in ways that were impossible via manual manipulation.

Where it’s happening: evaluating all products and categories no matter how big or small

Retailers must contend, daily, with a huge volume and diversity of products across many categories which creates challenges in maintaining the consistency and quality of data. They also risk running into visibility gaps for certain products that have been introduced to specific clusters or stores, have no established seasonality or are too new to have gathered sufficient historical data. AI can use item descriptions, item attributes or cluster or store locations to predict the behavior of specific products based on the patterns gleaned from other stores with similar attributes in similar circumstances. 

How does all of this relate to the case of Intermarché?

Intermarche’s test began with the objective of automating forecasts to match or exceed those produced by a statistical forecasting model. Initially, we applied three years of historical data to 2 of Intermarché’s warehouses. To increase the complexity of the data being analyzed, we later expanded the test to a total of 12 warehouses. By expanding, we were able to include products believed to be very difficult to forecast. One such product is the “salt” that is used to deice roads and walkways in the winter. AI was able to forecast this product, which was believed to be impossible using statistical models, with ease and with an acceptably low error rate.

The test, overall, was a clear success, resulting in 95% forecast accuracy – a 15% uplift – and reducing errors by 75%. We observed these gains across all products, categories and warehouse sizes. As a result, Intermarché was able to identify previously undetected patterns that offered new growth opportunities for the business.

There is a wealth of productivity gains to be achieved from AI-enabled demand forecasting for activities including promotions and management of fresh products. However, like any other learning-based model, AI requires constant and sufficient data to enable it to grow. But, it goes beyond that. While AI can cleanse data that’s fed into it, the data must be healthy and integrated before it can be used to drive improvements in demand forecasting. 

AI and machine learning demand forecasting solutions are incredibly powerful but, in order to use them to their full potential, it’s essential to understand the data required to feed them. 

Learn More:

  • If you’re just considering the role of AI, read our paper on Demand Forecasting.
  • To read more about Intermarché and the benefits they have achieved from AI enable demand forecasting, click here. (article is in French)