Systematic Failures of Chemical Safety Research at Environmental Protection Agency
“If at first you don’t succeed, try, try again” could be the motto of one of the key research programs at the U.S. Environmental Protection Agency known as the Integrated Risk Information System (IRIS). EPA staff have been working for nearly a decade to improve the scientific process of this program, but have yet to get it right.
IRIS has proven controversial because of its history of relatively low productivity and sloppy research. Despite reform efforts during the past decade, the Government Accountability Office (GAO) continues to list IRIS on its “High Risk List” of programs that are vulnerable to “fraud, waste, abuse and mismanagement.”
EPA officials created IRIS administratively in 1984 to provide information on chemical risks to aid other EPA programs that needed such data for regulatory purposes. IRIS first produced a database of chemicals and relevant data. The office eventually went on to conduct risk assessments for various chemicals, but there are no legal guidelines for such assessments. In 2000, the Office of Management and Budget began to provide some oversight of the program. The GAO raised concerns about IRIS’s productivity and its procedures more than a decade ago, and since then IRIS reform has remained a hot topic in Washington, subject of more GAO reports, an Inspector General report, and congressional hearings.
A 2011 review of the IRIS risk assessment for formaldehyde turned up the heat for reform as a National Academies of Sciences review panel and report pointed out serious methodological failures and IRIS’s “recurring problems with clarity and transparency of the methods.” The NAS committee made a number of recommendations for improving the IRIS program, including the recommendation that it apply literature review procedures referred to as “systematic review.”
Originally designed for review of medical research, systematic reviews go beyond simple narrative literature reviews, employing procedures to ensure a more a comprehensive review of the literature in an attempt to reduce researcher bias involved with selecting studies. NAS also urged IRIS officials to operate in a more transparent way, among other recommendations.
Of course, operating in a systematic and transparent way should be a no brainer for anyone interested in producing solid and useful science, and EPA should have changed its procedures quickly. Yet three years after the first NAS review, EPA was still trying to make its research systematic when the NAS conducted yet another review, indicating that the agency made some improvements but still had work to do.
In January 2017, as IRIS staff were still trying to figure out how to be systematic and transparent, the Obama EPA selected new leadership for the program, placing former Environmental Working Group senior scientist Kristina Thayer at the helm. Shortly before leaving office, the Obama EPA also selected Tina Bahadori to head the National Center for Environmental Assessment (NCEA), which houses IRIS within EPA’s Office of Research and Development. Supposedly, Thayer and Bahadori picked up speed with improvements.
Yet concern about IRIS’s slow progress and past excesses advanced efforts to cut its budget and maybe even eliminate it all together. Earlier this year, President Trump’s budget proposed significant budget cuts and the Senate Omnibus reconciliation bill proposed IRIS elimination.
IRIS elimination is no longer only driven by the program’s inadequacies; IRIS has become much less relevant now that Congress reauthorized and expanded the Toxic Substances Control Act in 2016. This reform vastly expanded the EPA’s power to collect data from industry and use it for chemical safety assessments, and EPA has already begun setting priorities for this program. Those efforts will be extensive and governed by congressionally mandated, rigorous scientific standards, which is something IRIS lacks.
Seeing the writing on the wall, Thayer and Bahadori apparently hatched a plan to save IRIS by calling for yet another NAS review. For a day and a half last February, Thayer and Bahadori briefed the NAS panel on all the procedural reforms and activities they were implementing at EPA. NAS released this review on April 11, and this time EPA got what it had been trying to achieve for many years: praise for its reforms. The NAS review allegedly shows that the IRIS has finally made some real progress, but a closer look suggests that the NAS report is little more than a rubber stamp.
EPA achieved this end by keeping the scope of the NAS review extremely narrow, just calling on the committee to assess whether the “current trajectory of the program agrees with past recommendations of the National Academies” in 2011 and 2014. Note the review only assessed if EPA was on the right path, not whether it had reached its goals. The NAS report explains further that the committee “was not asked to evaluate the overall value of the IRIS program,” and it “was not tasked with conducting a comprehensive review of the IRIS program.” In other words, all the NAS was asked to do was validate the efforts EPA has undertaken to improve the process, not whether it finally achieved its goals or whether it had substantially improved its assessments.
Accordingly, EPA got what it paid for, as the new NAS report appears very positive but actually says almost nothing. Of the 130 pages, about 90 are simply a reprint of EPA’s presentations and posters, and much of the rest are background information, title pages, and appendices. In about 12 pages the NAS provided a brief history of the issue and offered a few sentences of praise for some procedural reforms, but it also pointed out that IRIS has still not achieved some very basic goals. For example, IRIS has not even finalized a handbook outlining the process, which NAS asked for in 2014. IRIS staff promised this would be done in 2018, but one must wonder why on earth it’s taking so long.
This pat on the back from the NAS may help IRIS program staff generate good press, but what does it really mean? It begs the question as to whether such process reforms have actually translated into sound assessments or improved productivity. We don’t know because EPA didn’t bother to ask the committee those questions. Essentially, EPA successfully used the NAS review as yet another vehicle to avoid responsibility. Arguably, IRIS suffers from a flaw not adequately addressed by the National Academies: its excessively cautious approach, which will be the subject of a subsequent post.
It’s time to scrap the IRIS program because not only has it failed to achieve reasonable goals, it’s also no longer needed. We have now a much better alternative, EPA’s newly reformed chemicals program under the Toxic Substances Control Act. I have expressed concerns about TSCA reform because the standards in the prior law were strong and change can be risky. But the revised law still contains some good language that demands application of the “best available science,” weight of the evidence considerations, and transparency requirements. And these standards have the force of law, which IRIS completely lacks.
Clearly, it’s time for a change to bring some more rational and scientific approaches to these issues. There’s no reason not to migrate these functions to the TSCA office where there is a higher probability that the risk assessments will be more science-based.