A comparison of two methods for estimating the effect of a countermeasure in the presence of regression effects

Abstract

The report studies the problem of estimating in a non-experimental before-and-after investigation the effect of a countermeasure on the number of traffic accidents at road junctions. The accidents are assumed to occur according to a Poisson process with different intensities at different junctions. The junctions studied in this investigation are assumed to have been selected with the blackspot-technique, i.e. junctions with high numbers of accidents during the before-period have been chosen for the investigation. In the mathematical model this has the consequence that the number of accidents occurring during the before-period at a selected junction has a truncated Poisson distribution. During the after-period the number of accidents has a Poisson distribution (without restrictions), so that the number of accidents on the average decreases between the periods even if the countermeasure has no effect. The magnitude of this regression effect is studied in the report. The observed numbers of accidents during the before and after period are used to estimate the pure effect of the countermeasure both with an intuitive method and with the maximum likelihood method. The characteristics of the two methods of estimation are illustrated with the aid of simulation studies. In general the maximum likelihood method appears preferable, mainly because it produces estimates with higher precision.

Examining the effects of site selection criteria for evaluating the effectiveness of traffic safety countermeasures

“…In fact, this topic has been researched thoroughly over the last 30 years (Hauer, 1980a(Hauer, , 1980b(Hauer, , 1997Abbess et al, 1981;Hauel et al, 1983;Danielsson, 1986;Wright et al, 1988;Davis, 2000;MirandaMoreno et al, 2009;Maher and Mountain, 2009;Rock, 1995;Hamed et al, 1999), where researchers have developed and applied various methods to minimize known biases associated with crash data. Developing precise and reliable methods to evaluate countermeasure effectiveness is crucial, since erroneously measuring the safety effects could have important consequences both in terms of lives saved and wasted funds.…”

Addressing the worst cases

“…It arises because by selecting those localities with the higher values in the 'before' period, one is essentially taking a sample from a 'truncated' probability distribution, while in the 'after' period, the probability distribution is no longer restricted. This issue was first noted in the injury field by Hauer (1980) who called it 'bias-by-selection,' and later it was called 'regression effect' by Danielsson (1986). This note describes and illustrates the phenomenon so as to provide a better understanding of how one must account for it in proper estimates of effectiveness of an intervention in 'before-after' interventions.…”

“…Both Hauer (1980) and Danielsson (1986) provide ways to estimate the regression to the mean effect under various assumptions about the distribution of Y. …”

Estimation of the effects of countermeasures on different types of accidents in the presence of regression effects

“…The conditional distribution for the total number of accidents at a selected junction is thus what can be termed truncated Poisson [see Danielsson, 1986;Hauer, 1980b;and Hauer and Persaud, 1983]…”

“…In an earlier paper [Danielsson, 1986], we described the problem of the regression effect which can occur in before-and-after studies [for detailed descriptions see Briide and Larsson, 1982;Hauer, 1980aHauer, , 1980bHauer, 1986;and Hauer and Persaud, 1983]. In the paper, we proposed a method of purely estimating the effects of countermeasures despite the presence of regression effects.…”