Quantifying survey expectations: What’s wrong with the probability approach?

Abstract

Provided in Cooperation with Quantifying survey expectations:What's wrong with the probability approach? AbstractWe study a matched sample of individual stock market forecasts consisting of both qualitative and quantitative forecasts. This allows us to test for the quality of forecast quantification methods by comparing quantified qualitative forecasts with actual quantitative forecasts. Focusing mainly on the widely used quantification framework advocated by Carlson and Parkin (1975), the so-called "probability approach", we find that quantified expectations derived from the probability approach display a surprisingly weak correlation with reported quantitative stock return forecasts. We trace the reason for this low correlation to the importance of asymmetric and time-varying thresholds, whereas individual heterogeneity across forecasters seems to play a minor role. Hence, our results suggest that qualitative survey data may not be a very useful device to obtain quantitative forecasts and we suggest ways to remedy this problem when designing qualitative surveys.JEL-Classification: C53, D84, G17

A new approach for the quantification of qualitative measures of economic expectations

“…By means of simulation-based expectations, Löffler (1999) and Terai (2009) Breitung and Schmeling (2013) corroborated the importance of introducing asymmetric and dynamic indifference parameters, but found that individual heterogeneity across respondents plays a minor role in forecast accuracy. On the other hand, Lahiri and Zhao (2015) have recently found strong evidence against the threshold constancy, symmetry, homogeneity, and overall unbiasedness assumptions of the probability method.…”

Quantification of Survey Expectations by Means of Symbolic Regression via Genetic Programming to Estimate Economic Growth in Central and Eastern European Economies

“…Abberger (2007) uses probit analysis to estimate a quantitative threshold for employment expectations that allows to differentiate between a decrease and an increase in actual employment. Several refinements of the probabilistic approach have been proposed in order to reduce the measurement error introduced by restrictive assumptions (Lahiri and Zhao, 2015;Breitung and Schmeling, 2013;Łyziak, 2013;Müller, 2010;Mitchell, Mouraditis and Weale, 2007;Claveria et al, 2003Claveria et al, , 2006Weale, 2002, 2005;Löffler, 1999;Berk, 1999;Smith and McAleer, 1995;Dasgupta and Lahiri, 1992;Kariya, 1990;Batchelor and Orr, 1988;Seitz, 1988;Batchelor, 1986;Toyoda, 1979). states by means of panel vector autoregressive models, finding no significant differences in both groups.…”

A Data-Driven Approach to Construct Survey-Based Indicators by Means of Evolutionary Algorithms

“…Carlson and Parkin (1975) applied the method by using a normal distribution together with symmetric and constant threshold parameters across respondents and over time. Extensions of this framework are mainly focused on reducing the measurement error introduced by incorrect assumptions (Lahiri and Zhao 2015;Breitung and Schmeling 2013;Mitchell et al 2002;Löffler 1999;Berk 1999;Smith and McAleer 1995;Dasgupta and Lahiri 1992;Kariya 1990;Batchelor and Orr 1988;Seitz, 1987;Pesaran 1984;Batchelor 1982Batchelor , 1986Toyoda 1979). By means of Monte Carlo simulations, Terai (2009) andLöffler (1999) Matching firm-level responses to quantitative realizations, several authors have also developed extensions of the probability approach.…”