The Ombudsman: Reaping Benefits from Management Research: Lessons from the Forecasting Principles Project

Abstract

It is often claimed that managers do not read serious research papers in journals. If true, this neglect would seem to pose a problem because journals are the dominant source of knowledge in management science. By examining results from the forecasting principles project, which was designed to summarize all useful knowledge in forecasting, we found that journals have provided 89 percent of the useful knowledge. However, journal papers relevant to practice are difficult to find because fewer than three percent of papers on forecasting contain useful findings. That turns out to be about one useful paper per month over the last halfcentury. Once found, the papers are difficult to interpret. Managers need low-cost, easily accessible sources that summarize advice (principles) from research; journals do not meet this need. To increase the rate of progress in developing and communicating principles, researchers, journal editors, textbook writers, software developers, web site designers, and practitioners should make some changes. Some examples: Researchers should directly study forecasting principles. Journal editors should actively solicit papers -invited submissions were about 20 times better than standard submissions at producing useful findings that were often cited, and does so at a lower cost. Web-site and software developers should provide practitioners with low-cost ways to use principles. Practitioners should apply the principles that are currently available.

Findings from evidence-based forecasting: Methods for reducing forecast error

“…In addition, Armstrong and Pagell (2003) estimated that only 42% of the papers that contribute to forecasting are found by key word searches in the title or topic. Thus, there may have been about 35,000 papers relevant to forecasting (that excludes the Science Citation Index, which has substantially more forecasting studies than the SSCI).…”

“…Perhaps there is an additional one percent of multiple hypotheses studies related to other aspects of forecasting such as assessing uncertainly or gaining acceptance. Using a much different approach, Armstrong and Pagell (2003) estimated that only 3% of the papers in forecasting assessed which methods contribute to the development of forecasting principles.…”

Donor patterns: a modular structure for sharing knowledge

“…Evidence was obtained from published studies and collected in a large book, each chapter provided by experts in their field. The way in which the evidence was assembled is described in the book and by Armstrong and Pagell (2003). While the format follows that of a conventional text key advice is made plain as in this example from the chapter on role play:…”

Making progress in forecasting

“…This strategy of inviting papers worked well. As we found later (Armstrong & Pagell, 2003), invited papers have had about 20 times as strong an "impact" as those accepted via traditional channels.…”

“…This strategy of inviting papers worked well. As we found later (Armstrong & Pagell, 2003), invited papers have had about 20 times as strong an "impact" as those accepted via traditional channels.This impact factor was based on the number of ISI citations and also on whether the paper had been identified as useful in the development of forecasting principles. Invited papers are also less expensive to process -for authors, editors, and reviewers.…”