by Mark L. Burkey
Department of Economics and Transportation/Logistics
NCA&T State University
1601 E. Market Street
Greensboro, NC 27411
The first two pages serve as a brief response to the false allegations by the IIHS (via Richard Retting and Sergey Kyrychenko). Details and documentation for each statement follow this brief introduction.
February, 2005
The following is the two page brief response. For a copy of the full response, click here.
Brief Response
It is surprising that an institution such as the IIHS resorts to name-calling when a project reports data that go against their deeply-held beliefs. If there are valid limitations of our work, we made every effort to point them out in the report. We welcome any additional valid criticism and suggestions. However, Retting and Kyrychenko (R&K)’s comments admit to one of only two explanations: 1) They are purposefully distorting our methods, or 2) They did not read and understand the methods and careful robustness checks that were done. They bring up three main objections to our work:
1) “…Burkey and Obeng treated data from intersections with and without cameras as if the cameras had been randomly assigned to their locations. In fact Greensboro officials installed cameras at intersections with higher crash rates…” Status Report 2005
FALSE. It is impossible for our report to give this impression. We dealt with this issue at length, and modeled it in two specific ways. First, we included many variables that account for these differences in crash rates. Second, in order to make sure that we had accounted for any additional unobserved heterogeneity, we ran a Fixed Effects model. This method tracks each intersection individually, allowing the Red Light Camera (RLC) variable to pick up only the effect of the RLC placement relative to the accident rate at these same intersections (see last page of the report). The model R&K use with contrived data is in no way related to our methods, and the data does not reflect the data in the tables in our report. Their suggestion of a RLCGROUP variable is unacceptable, and is demonstrated to give incorrect results in this document. Additionally, Greensboro officials installed cameras at both high and low accident locations. Many high accident locations did not receive RLCs. We discuss the non-random, non-experimental nature of our data at length in the report, and discuss its shortcomings.
2) “…Publicity and media coverage generally make drivers aware that a city is using red light cameras, not specifically which intersections have cameras… By ignoring the spillover effect, the authors could obtain only a biased (low) estimate of red light camera effectiveness.” R&K 2004
FALSE. NCGS ยง 160A-300.1b: Any traffic control photographic system installed on a street or highway must be identified by appropriate advance warning signs conspicuously posted not more than 300 feet from the location of the traffic control photographic system. In other words, drivers in North Carolina know exactly where the cameras are located. Additionally, we did not ignore the possibility of spillover effects, and discussed this in the report. The spillover effect is well-known, but far from well-documented. Indeed, the primary effects of RLCs are still not well-understood. The IIHS often cite two of Retting’s studies as evidence of a spillover effect, which looked at 5 intersections for 24 hours to look for spillovers. We geocoded Greensboro’s intersections preparing to explicitly test for spillover effects using spatial correlation over time. If spillover effects exist, they should be stronger at intersections closest to the RLC sites since they are clearly marked. However, when many different (and appropriate) modeling techniques failed to show a benefit, measuring the spillover effects of the nonexistent benefits appeared to be a moot exercise. However, we tested for spillover effects several ways, and found nothing.
3) Additionally, they state that our conclusions were not reviewed by peers. They say that the purpose of peer review is to provide a “seal of competence” and that it tells us “These findings are worth paying attention to.” The purpose of peer review is not to review conclusions, but methods and clarity. Science is not about whether or not one is happy with the answer; rather, it is about trying to discern whether or not the investigator is honestly and intelligently searching for the truth. Of course, peer review is not a guarantee of this, but should be encouraged.
Our work has not been peer reviewed simply because enough time has not elapsed since finishing the initial report. In my profession we generally issue reports, get important feedback and comments, present our work at conferences, and then submit our work for formal “peer review” and publication. The process of peer review through publication can take many years, and lack of peer review is not a valid criticism of anyone’s recent work. Our methods are now being peer reviewed. Attempts by the IIHS staff to subvert this process are unprofessional.
In our study, the accident rates at intersections without RLCs went down much more than those with RLCs, continuing a long-term decreasing trend in accidents in Greensboro, NC. We address how this cannot reasonably be attributed to spillover effects in our report. We believe that we reasonably concluded, “At a minimum, we can say that there is no evidence that the RLC program is decreasing accidents. Additionally, the data shows that the sites with RLCs are not benefiting from the overall [long-term] decreasing trend in accidents in Greensboro.” (p. 48)
From looking at the data, the question should not simply be whether RLCs work, but when they may work and when they may not. From Table 4.1 in the report, simple before/after comparisons of accident rates at the 18 RLC sites in Greensboro show anywhere from a 36% decrease in accidents at one intersection, to a 57% increase in accidents at another. We need to design careful studies to examine whether these differences are random, or if some intersections can really benefit from RLCs. The IIHS wants you to believe that RLCs are always appropriate and will always reduce accidents at all intersections, including those without RLCs. If you question this conclusion the IIHS will label your work “incompetent junk science”. Real scientists who are objectively looking for the truth do not behave in this way.
There are flaws in all studies on the efficacy of RLCs, including ours. But the ones invented by Retting and Kyrychenko are simply not true. We continue to call for more careful studies of RLCs, because most scientists are simply not convinced either way. If we truly care about reducing accidents, we will continue to plan and execute more careful studies. Until we truly understand what will happen when a RLC is placed, we should be cautious about using them.
Please read the details that follow if you want additional documentation of each of the statements above. Consult researchers who do not have an agenda. Read any of the good comprehensive reviews of RLC literature by McFadden and McGee (1999), Maccubbin, Staples, and Salwin (2001), McGee and Eccles (2003), or Milazzo, Hummer, and Prothe (2001). Come to your own conclusions. Demand convincing evidence from people who demean those who disagree with them, and from anyone who wants to convince you that there is a clear and simple answer.
Mark L. Burkey, Ph.D.
Assistant Professor
Department of Economics and Transportation/Logistics
North Carolina A&T State University