Traditionally, the 2 x 2 contingency table method has been used for qualitative evaluation of forecasts. However, the conclusions drawn based on this method could be misleading because it does not account for the direction of the turning or no turning points. A 4 x 4 contingency table which overcomes this weakness and which gives more information on the qualitative performance of the forecast is suggested.Several qualitative and quantitative measures have been used in the literature to evaluate the performance of alternative forecasting techniques. Though the usefulness of these measures depends on the purpose for which the forecast is made, both qualitative and quantitative measures can be important in guiding economic agents in making decisions. The most commonly used measures of quantitative evaluation of forecasts are mean forecast error, mean absolute error, mean squared error, root mean squared error, percentage root mean squared error and goodness of fit. As far as qualitative evaluation is concerned, researchers have used directional change, binomial probability functions, and the turning point method. It is the purpose of this note to demonstrate a limitation to the now popular turning point method of qualitative evaluation, which potentially can lead researchers to incorrect interpretation of their results. We will then offer an alternative qualitative measure which can provide more insight into the forecast evaluation.
Qualitative MeasuresSeveral researchers (Hee; Kulshreshtha, Spriggs, and Akinfemiwa; Leuthold; Meyers, Havlicek, and Henderson) have used directional change as a measure of qualitative forecast evaluation. Usually, this is a matter of Gopal Naik and Raymond M. Leuthold are, respectively, a graduate student and a professor, Department of Agricultural Economics, University of Illinois.This study was a part of Hatch Project 05-381 of the University of Illinois Agricultural Experiment Station.Review was coordinated by Rulon Pope, associate editor.simply noting whether the forecasted variable is higher or lower than the current actual variate and if the subsequent move of the actual variable corresponded. The number or percentage of correct forecast directions is usually noted. Hee and Meyers, Havlicek, and Henderson used a binomial probability function in order to compare the number of correct directional changes the forecast has predicted with the number of correct directional changes that could be obtained purely by a chance factor. Kulshreshtha, Spriggs, and Akinfemiwa used the ratio of correct directional moves forecasted to total directional moves over the entire forecast period. The ratio is compared with a random correct directional move of .5. Theil suggested using the number of over or under predictions and the turning point method to evaluate the qualitative performance. This turning point method is now a popular qualitative measure frequently used by researchers Brandt 1979, 1981;Bourke;Bessler 1981, 1984;Harris and Leuthold;Hudson and Capps;Kost;Kulshreshtha and Rosaasen). This m...