Mean absolute percentage error explained

The mean absolute percentage error (MAPE), also known as mean absolute percentage deviation (MAPD), is a measure of prediction accuracy of a forecasting method in statistics. It usually expresses the accuracy as a ratio defined by the formula:

MAPE=100

1
n
n
\sum\left|
t=1
At-Ft
At

\right|

where is the actual value and is the forecast value. Their difference is divided by the actual value . The absolute value of this ratio is summed for every forecasted point in time and divided by the number of fitted points .

MAPE in regression problems

Mean absolute percentage error is commonly used as a loss function for regression problems and in model evaluation, because of its very intuitive interpretation in terms of relative error.

Definition

Consider a standard regression setting in which the data are fully described by a random pair

Z=(X,Y)

with values in

Rd x R

, and i.i.d. copies

(X1,Y1),...,(Xn,Yn)

of

(X,Y)

. Regression models aim at finding a good model for the pair, that is a measurable function from

Rd

to

R

such that

g(X)

is close to .

In the classical regression setting, the closeness of

g(X)

to is measured via the risk, also called the mean squared error (MSE). In the MAPE regression context, the closeness of

g(X)

to is measured via the MAPE, and the aim of MAPE regressions is to find a model

gMAPE

such that:

g_\mathrm(x) = \arg\min_ \mathbb \Biggl[\left|\frac{g(X) - Y}{Y}\right| | X = x\Biggr]

where

l{G}

is the class of models considered (e.g. linear models).

In practice

In practice

gMAPE(x)

can be estimated by the empirical risk minimization strategy, leading to

\widehat_\text(x) = \arg\min_ \sum_^n \left|\frac\right|

From a practical point of view, the use of the MAPE as a quality function for regression model is equivalent to doing weighted mean absolute error (MAE) regression, also known as quantile regression. This property is trivial since

\widehat_\text(x) = \arg\min_ \sum_^n \omega(Y_i) \left|g(X_i) - Y_i\right| \mbox \omega(Y_i) = \left|\frac\right|

As a consequence, the use of the MAPE is very easy in practice, for example using existing libraries for quantile regression allowing weights.

Consistency

The use of the MAPE as a loss function for regression analysis is feasible both on a practical point of view and on a theoretical one, since the existence of an optimal model and the consistency of the empirical risk minimization can be proved.[1]

WMAPE

WMAPE (sometimes spelled wMAPE) stands for weighted mean absolute percentage error.[2] It is a measure used to evaluate the performance of regression or forecasting models. It is a variant of MAPE in which the mean absolute percent errors is treated as a weighted arithmetic mean. Most commonly the absolute percent errors are weighted by the actuals (e.g. in case of sales forecasting, errors are weighted by sales volume).[3] Effectively, this overcomes the 'infinite error' issue.Its formula is:[4] \mbox = \frac = \frac

Where

wi

is the weight,

A

is a vector of the actual data and

F

is the forecast or prediction.However, this effectively simplifies to a much simpler formula: \mbox = \frac

Confusingly, sometimes when people refer to wMAPE they are talking about a different model in which the numerator and denominator of the wMAPE formula above are weighted again by another set of custom weights

wi

. Perhaps it would be more accurate to call this the double weighted MAPE (wwMAPE). Its formula is:\mbox = \frac

Issues

Although the concept of MAPE sounds very simple and convincing, it has major drawbacks in practical application,[5] and there are many studies on shortcomings and misleading results from MAPE.[6] [7]

At<Ft

than on positive errors.[9] As a consequence, when MAPE is used to compare the accuracy of prediction methods it is biased in that it will systematically select a method whose forecasts are too low. This little-known but serious issue can be overcome by using an accuracy measure based on the logarithm of the accuracy ratio (the ratio of the predicted to actual value), given by \log\left(\frac\right) . This approach leads to superior statistical properties and also leads to predictions which can be interpreted in terms of the geometric mean.[5]

e\mu

where as it is MAPE optimized at
\mu-\sigma2
e
.

To overcome these issues with MAPE, there are some other measures proposed in literature:

See also

External links

Notes and References

  1. de Myttenaere, B Golden, B Le Grand, F Rossi (2015). "Mean absolute percentage error for regression models", Neurocomputing 2016
  2. Web site: Understanding Forecast Accuracy: MAPE, WAPE, WMAPE.
  3. Web site: WMAPE: Weighted Mean Absolute Percentage Error.
  4. Web site: Statistical Forecast Errors .
  5. Tofallis (2015). "A Better Measure of Relative Prediction Accuracy for Model Selection and Model Estimation", Journal of the Operational Research Society, 66(8):1352-1362. archived preprint
  6. Hyndman, Rob J., and Anne B. Koehler (2006). "Another look at measures of forecast accuracy." International Journal of Forecasting, 22(4):679-688 .
  7. Kim, Sungil and Heeyoung Kim (2016). "A new metric of absolute percentage error for intermittent demand forecasts." International Journal of Forecasting, 32(3):669-679 .
  8. Kim . Sungil . Kim . Heeyoung . A new metric of absolute percentage error for intermittent demand forecasts . International Journal of Forecasting . 1 July 2016 . 32 . 3 . 669–679 . 10.1016/j.ijforecast.2015.12.003 . free .
  9. Makridakis, Spyros (1993) "Accuracy measures: theoretical and practical concerns." International Journal of Forecasting, 9(4):527-529