Data Torturing at the CPSC

James Mills of the National Institute of Child Health and Human Development lamented in an article in the New England Journal of Medicine back in 1993: “‘If you torture your data long enough, they will tell you whatever you want to hear’ has become a popular observation in our office. In plain English, this means that study data, if manipulated in enough different ways can prove whatever the investigator wants to prove.”

Government regulators will resort to such data torture to justify an activist regulatory agendas if they can’t do it with good data and sound science. One approach includes selective use of data—excluding years or datasets that might change the conclusions of a risk assessment. The Consumer Product Safety Commission’s recent Chronic Hazard Advisory Panel (CHAP) report on the chemical class known as phthalates offers one new example of excluding inconvenient data.

In short, the CHAP report is being used to justify a proposed rule that would essentially ban the use of certain chemicals for toys that children might mouth or chew. These chemicals make plastics soft and pliable, suited for such things as a plastic version of a “rubber duckie.” For background on this issue, see my other blog posts here and here. 

In addition, in the absence of any compelling body of data that any individual phthalate is the cause of human health effects, the panel relied on the possibility that the cumulative effects of phthalates as a class pose risks. Accordingly, they needed data on human exposure from all sources.

The panel developed a “cumulative risk assessment” that they maintained justified regulations. But pharmacologist Christopher J. Borgert, Ph.D., observes in a review of the CHAP report that the panel’s cumulative risk assessment: “failed to recognize obvious inconsistencies with human experience and clinical evidence”; “overstates the accuracy of its cumulative risk methods and conclusions”; and “appears to have grossly overestimated chemical potencies.” In other words, the panel failed to properly apply the available data and research.

To make matters worse, they used old and irrelevant data for their human exposure assessments even though more accurate and recent data was available. Former and current CPSC commissioners have noted that had the panel used the most recent data, their risk assessment would have produced the opposite result. This issue raises the prospect that the panel members were intentionally “selective” in their use of data because they desired to generate a particular conclusion, as appears to be the case with their selection of studies that they also reviewed. 

For more perspective on this issue, former CPSC Commissioner Nancy Nord explains:

The CHAP found that “food, rather than children’s toys or child care articles, provides the primary source of exposure to both women and children….”  Nevertheless, the CHAP expressed its concern “that toys and child care articles may contribute to the overall exposure.” (See staff briefing package, “Prohibition of Children’s Toys and Child Care Articles Containing Phthalates”, page 13, emphasis added.)

Cumulative risk assessments can be a useful analytic tool in certain circumstances where risks come from identified multiple sources. However, in this instance, it is very clear that the CHAP had issues about how to do the risk assessment and then how to use it.  Cutting through the scientific jargon, the CHAP report and the CPSC’s proposed rule based on it address a potential health risk by proposing to ban a speculative contributor to the risk. The notion that this rule will make the marketplace safer is belied by the fact that the CHAP report describes the many other and more primary exposure routes from the other products that contain phthalates—most of which are outside the jurisdiction of the CPSC and many of which are not being used by children. …

These concerns are amplified by the fact that the CHAP based its findings on stale data, when there is ample evidence that had it used the most recent data available to it, the analysis may well have reached a different conclusion.  For the CPSC, which prides itself on being a “data-driven agency”, to acquiesce in such an inexplicable use of flawed data, much less base a proposed rule on it, is puzzling.  It might lead a cynic to wonder if this was a politically driven decision rather than a scientifically driven one.

Nord’s comments offer an excellent summation of this issue. That is: The CHAP should not have drawn conclusions from studies that are largely inconclusive, nor did they have the “strong science” they claim to have demonstrated “cumulative risk.” And finally, it is unacceptable for the agency to rely on old exposure data when better data is available. The only time that an agency would pursue such “science” appears to be when it’s is political rather than hard science.

For more on this topic, see my comments to the CPSC on the proposed phthalates rule.