Rolling Forecasts

4 Demand Forecast Accuracy KPIs You’ll Actually Use

5 mins
If demand forecasting has ever felt like glorified guesswork, this blog will help. This article will cover demand forecast accuracy KPIs that are actually helpful. So, buckle up, and dive into the metrics that turn wild guesses into slightly less wild guesses.

Demand Planning vs Demand Forecasting

Demand Planning is arguably the most important part of Supply Chain Management. It is the first step in the SCM process. Only after the demand planning process is done, people from production can start working on their operations and plan for raw materials, and logistics can focus on details of transportation planning and various warehouse activities.

Also it is worth mentioning that demand planning ≠ demand forecasting. Demand planning is the process of forecasting demand for a product or service and executing an operational strategy across the supply chain in order to meet it. Demand forecasting refers only to the process of predicting demand/sales numbers, and as such is a part of the demand planning process.

About Demand Forecasting KPIs

You can only improve what we can measure. Demand planners are always on the lookout for THE metric that will help them improve forecast accuracy and solve one of the most persistent problems for any supply chain company – increasing the visibility across the supply chain. There are numerous metrics in use in this context, but this post will focus only on demand forecast accuracy KPIs that people most commonly use to track and improve forecast accuracy:

 
  • Forecast Accuracy/Error

  • Bias (Mean Forecast Error)

  • Mean Absolute Percentage Error (MAPE)

  • Weighted Mean Absolute Percentage Error (WMAPE)

 
No matter what metrics you use, your priority is choosing the right forecasting method/model. There is no “one size fits all KPI”. In other words, you have to be able to make quality assumptions about your business by using the perfect mix of qualitative and quantitative demand forecasting methods. More about those in another post.

Forecast Error

The most straightforward of all demand forecast accuracy KPIs. It provides insight into the difference between the actual value (Dt) and the forecast value (Ft) for the given period.

Forecast error demand forecasting KPI formula

How to calculate:


Forecast Error: 1 – [ABS (Dt – Ft) / Dt]

Dt – The actual observation or sales for period t
Ft – The forecast for period t

Forecast error numbers range from 0 – 100%.

100% forecast accuracy is perfect, obviously, but if your data is right, you won’t be seeing it very often, and this is OK. Depending on the selected period and other operational factors, anything north of 70% is acceptable. This goes without saying, but using the Forecast Error as the only KPI can be problematic for two reasons: it only provides information about the absolute value of the error, and as it’s limited when working with multiple time series, which, let’s face it, is always.

Bias (Mean Forecast Error)​

Bias is a simple demand forecasting KPI providing the information about the tendency of forecasts to persist in one direction – over or under-forecasting. It is a fairly reliable way to check if your forecasting model works the way it is supposed to. In theory, when the bias is zero, forecasts are not biased. If the bias is greater than 4, for the period of 24 observations, it is safe to say that your forecasting model is on the side of under-forecasting. Naturally, when the bias is less than -4, the model is biased toward over-forecasting.

Bias-mean forecast error demand forecasting KPI formula

How to calculate:

 

Bias: [∑ (Dt – Ft)] / n

Dt – The actual observation or sales for period t
Ft – The forecast for period t
n – The number of forecast errors
 
As mentioned earlier, a highly biased model indicates that something is wrong with the way you forecast, but the metric itself is not enough to precisely evaluate the precision forecast.

Mean Absolute Percentage Error (MAPE)​

Another great demand forecasting accuracy KPI. You will probably use MAPE most of the time. And for a good reason: it is simple, everyone understands it, and there are only a few accuracy tradeoffs when using it.

Simply put, MAPE is a percentage of relative error. It expresses forecast errors as a percentage of actual observations.
MAPE demand forecasting KPI formula

How to calculate:

 
MAPE: ∑ |Et / Dt |/n * 100
Dt – Actual observation or sales for period t
Et – the forecast error for period t
n- the number of forecast errors
 
The tricky thing with MAPE is that it is often miscalculated (and misinterpreted) and that it does not provide the information on how exactly the products and moments in time differentiate. Have a look at a table below:
MAPE-calculation

As you can see, when the sale numbers are lower, MAPE can get deceivingly high. To avoid this, the reccomendation is to to use WMAPE.

Another problem you will probably encounter at some point are the time periods where the actual observation is 0 – no products are sold. Dividing by zero is problematic, to put it mildly, so the formula doesn’t actually work in those instances. When this happens, you can use scale invariant metrics such as MASE, or simply ignore these periods, which is my preferred choice.

Weighted Mean Absolute Percentage Error (WMAPE)​

This demand forecasting accuracy KPI – WMAPE values weights more than MAPE does. What this means is that if demand planners know that some products and moments in time are more important to predict properly (for example Monday in the example below), they can attribute them more weight when calculating the error.

WMAPE demand forecasting KPI formula

How to calculate:

 
WMAPE: ∑(|Dt-Ft|) / ∑(Dt)
Dt – The actual observation for period t
Ft – the forecast for period t
 
Let’s have a look at the table once again:
WMAPE-calculation

In this case, we decided that Monday carries 80% of the importance. Consequently, this made more sense, and made our weighted error more precise – it went from 36.7% down to 9.1%.

 
More often than not, you will work with multiple time series and create forecasts for the next twelve months, which means that it is impossible to manually weight the periods, so you will need to find a way to automate this. Otherwise, WMAPE becomes pretty much useless.
Click on the image to read more about how to handle inventory replenishment.

Conclusion

There is no perfect measure to solve every forecasting problem. Choosing the right one depends on your specific situation and use case. Your data will never be perfect, and some information will always be missing, but using the combination of these demand forecasting KPIs and deep understanding of your business processes should help you increase your forecast accuracy. When in doubt, it is best to stick with MAPE, though.

Solving Demand Forecasting and Supply Chain problems is easier with an integrated supply chain planning, forecasting, and reporting software tool that helps automate your forecasts and build complex models in the blink of an eye.

FROM THE BLOG

Related articles

Budget vs actual cover image

Budget vs Actual Explained: How Does It Affect Your Business?

06 December 2024
the definition guide - margin analysis cover image

The Definitive Guide to Margin Analysis

06 December 2024
accounts receivable blog cover image

The Ultimate Guide to Accounts Receivable

04 December 2024