In the normal course of developing a study design with the goal of testing a hypothesis, the first step would usually be identifying effects that could confirm or negate the hypothesis. In the case of the USGS-Austin study, the hypothesis was that sealcoat was contributing some (large) percentage of PAHs to the local sediment load. Testing PAH concentrations of suspended particle load is interesting data to have, but of dubious relevance. The hypothesis is about PAHs in sediments. If the researchers lacked the resources to sample sediments at the time of the simulated rainfall sample collections, the City of Austin had many PAH analyses of sediments available in monitoring reports such as Geismar (2000) which could have been and, even if of lower quality than data produced by the USGS labs, should have been used as part of hypothesis testing. This begs the question: why were such data not used? It is impossible to ignore that, as shown in the DeMott and Gauthier (2006) double ratio plot, available sediment data did not yield a result that matched the preferred policy of the USGS researchers and Austin city staff. Even Austin city staff noticed, as made explicit in Scoggins et al. (2007). Yet, that is precisely was what was not done.
It is tempting to speculate that the many readily available bed load sediment PAH data and the available suspended particle load from stream sheds where the wash off studies were conducted were not used because they did not sufficiently match the PAH ratios of RTS-related samples. Conversely, it is tempting to speculate that the suspended particle data from Williamson Creek and the three Fort Worth streams were used because their PAH ratios were sufficiently similar. We need not speculate – Dr. Mahler herself provides the answer. In an email dated July 19, 2004, Dr, Mahler wrote:
When the Williamson Creek suspended sediment data was plotted on the same graph, they tended to group with the sealed parking lots as opposed to with the unsealed (asphalt pavement or cement) parking lots. Suspended sediment data from three small urban watersheds in Fort Worth were similar.
Thus Dr. Mahler confirms that it was precisely because PAH ratios of the 8 samples from Williamson Creek and the 12 from Fort Worth most closely matched RTS that they were chosen to represent Austin stream sediments. Data from other locations – including streams located next to parking lots that were part of the study – must not have matched as well, and so were not used. To suggest the results supported their hypothesis, the authors drew circles around the data on PAH double ratio plots to indicate that this urban background signal is more similar to RTS than it is to some other materials. The authors repeated this strategy in Van Metre et al (2009), presenting a double-ratio plot showing difference between urban sediments and a handful of PAH containing materials while suggesting this indicates the role of RTS. Highlighting these plots, while misstating their meaning, is evidence of advocacy research. Any other choice would have, from the USGS point of view, been inconclusive at best or, at worst, unsupportive of the USGS’ preferred policy.
And that preferred policy was then and remains today to ban the use of RTS.