White Hat Bias in the Environmental Sciences (SETAC 2015)

At the 2015 annual meeting of the Society of Environmental Toxicology and Chemistry (SETAC) in Salt Lake City, Anne LeHuray (PCTC’s Executive Director) gave an invited presentation titled  White Hat Bias in the Environmental Sciences.  The abstract is available here and the presentation slide deck is posted here.

To summarize, scientists working in the field of obesity research have found that some research in their field shows evidence of White Hat Bias (WHB), which they defined as

bias leading to the distortion of information in the service of what may be perceived to be righteous ends.

 As described in the seminal WHB paper, the bias is towards putting a thumb on the scale weighing scientific evidence to further a desired outcome:

WHB bias may be conjectured to be fuelled [sic] by feelings of righteous zeal, indignation toward certain aspects of industry or other factors (Cope & Allison, 2010).

In the course of evaluating and testing the hypotheses generated by USGS scientists B. Mahler and P. Van Metre and others since 2005, a pattern of skewing data in different ways needed to promote the USGS hypotheses has been found across many published papers. Of the many possible examples of WHB, the presentation focuses on the details of the first USGS paper on RTS, Mahler et al. (2005).

Attempts to reproduce the graphic analysis of PAH ratios that Mahler et al. (2005) presented as diagnostic of RTS in urban sediments have not succeeded. Why the diagnostic graph could not be reproduced remained a mystery for 8 years, because the data used to represent urban sediment in Mahler et al. (2005) remained unknown.  Then, in 2013, PCTC wrote a letter to the Ethics Committee of the American Chemistry Society (ACS – the organization that publishes the journal Environmental Science & Technology). Within a few weeks, the USGS responded directly, providing the data with a list of 20 samples of suspended sediment that were not related in space, in time, or in connected water bodies to the parking lot study described in Mahler et al. (2005). With the data in hand, the question then became “why were these 20 samples used to represent PAHs in urban sediments?”

PCTC did not have to guess at the answer, which was provided by Dr. Mahler herself in an email to her “cooperators” (that’s the word the USGS uses when it has a paid consulting relationship with another government agency). In the email dated July 19, 2004, Dr, Mahler wrote:

When the Williamson Creek suspended sediment data was plotted on the same graph, they tended to group with the sealed parking lots as opposed to with the unsealed (asphalt pavement or cement) parking lots. Suspended sediment data from three small urban watersheds in Fort Worth were similar.

Thus it is now known with certainty that it was precisely because the PAH ratios of the 8 samples from one location in Williamson Creek (which is in another part of Austin, TX) and the 12 from three locations in Fort Worth most closely matched RTS that they were chosen to represent urban sediments. Selecting data because it supports a hypothesis is confirmation bias. Selective use of data to support a hypothesis is not a scientific method.

The SETAC presentation concludes with an overview of the tools available that the USGS, as a federal agency, could have used to avoid this instance of published WHB.

Leave a Response