Scientific Integrity Policy Draft for Public Comment

February 23, 2024

Docket ID: EPA-HQ-ORD-2023-0240; FRL-10973-01-ORD

Submitted via Regulations.gov.

Comments Submitted by the Competitive Enterprise Institute, Energy & Environment Legal Institute, Domestic Energy Producers Alliance, Rio Grande Foundation, Caesar Rodney Institute, The Heartland Institute, Heartland Impact, Committee for a Constructive Tomorrow (CFACT), Frontiers of Freedom, Science and Environmental Policy Project, American Commitment, The Committee to Unleash Prosperity, Cornwall Alliance for the Stewardship of Creation, Americans for Prosperity, Independent Women’s Forum’s Center for Energy and Conservation, Scientific Integrity Institute, Eagle Forum, Institute for Energy Research, American Stewards of Liberty, Roughrider Policy Center

To Michael S. Regan, Administrator, Environmental Protection Agency:

On behalf of the undersigned organizations, we are pleased to provide comments to the Environmental Protection Agency (EPA) on its draft “Scientific Integrity Policy,” hereafter referred to as the “draft SI policy.”  We have a longstanding interest in promoting transparency and the proper use of science in agency actions, including rulemaking.

1. Vague Definitions and DEI Language Undermine EPA’s Commitment to Scientific Integrity

EPA’s draft SI policy contains more than 30 defined terms, many of which are concerning for their vagueness or the degree to which they grant discretion to agency civil servants. The lack of clarity opens the door to arbitrary, inconsistent, and politicized application of science by EPA staff.

Several key definitions are impermissibly broad or vague:

• The definition of the word “delay” includes language like “take longer than reasonably expected” and “unreasonable,” which lack clear boundaries delineating when legitimate inquiry might temporarily slow research and when actions might cross the line into suppression violations.

The definition of “scientist” as “anyone who collects, generates, uses, or evaluates scientific data, environmental information, analyses, or products” is so broad it could refer to nearly anyone, including bloggers, journalists, activists, and even casual followers of the news. By this definition nearly any EPA employee is a “scientist.”

• The definition of “inappropriate influence” as “the attempt to shape or interfere in scientific activities, or the communication about or use of scientific activities or findings, against well-accepted scientific methods and theories without scientific justification.” may discourage agency heads and their political subordinates from directing changes in how EPA staff frame, communicate about, or contextualize scientific work. It may also suppress minority voices within the community of scientists by discouraging perspectives that stand “against well-accepted scientific methods and theories.” In fact, questioning well-accepted methods and theories is central to the scientific process.

• The term “ethical behavior” introduces ideological concepts unrelated to scientific integrity. By including “equity and inclusion,” the definition elevates diversity, equity, and inclusion (DEI) considerations. Later in the document, the draft SI policy categorizes DEI issues as “integral to the scientific process.” Yet, characteristics like race and gender and normative aspirations like equity and inclusion have no inherent connection to the collection, analysis, or communication of empirical facts.

• The term “Indigenous Knowledge” is not defined (even though it is capitalized). At the same time the draft SI policy elevates DEI criteria, it is strangely silent on the meaning of “Indigenous Knowledge,” which appears as an item listed under EPA efforts to “protect the integrity of the scientific process.” EPA declares a policy to use “Indigenous Knowledge” upon consent in decision-making, as well as to protect it from disclosure.

The conflation of social justice concerns with science politicizes what could otherwise be objective scientific practices. The diversity criteria invite abuse of the allegation procedures intended to reduce suppression of science, as these definitions presuppose that scientists who do not actively prioritize the ideas of certain groups defined by arbitrary diversity criteria behave unethically and compromise integrity.

Moreover, EPA’s expansive definition of scientist suggests that EPA is intentionally trying to institutionalize partisan political biases rather than protect objective science from political interference. EPA’s definition essentially treats all career staff as scientists beyond question by political appointees. With EPA career employees known to be overwhelmingly politically liberal, this conveniently discourages future conservative administrations from influencing policy, delineating their activities as “political” with career staff activities as “science,” despite the fact that political appointees also often have scientific backgrounds. The danger is that these vague, political, and in some cases missing definitions will enable selective, ideological interpretation of scientific integrity violations, with the result being a chilling effect that will discourage legitimate oversight activities.

2. Science and Policy Conflated with Regard to Economic Analyses

A key area of concern in EPA’s draft SI policy is the treatment of economic analysis, specifically benefit-cost analysis. The draft SI policy states that “The Agency’s economic analyses, including benefit-cost analyses, are scientific products intended to inform the decision-making process…[and]…should not be changed except as needed to correct technical errors.” Furthermore, it states that “the decision of whether and how to quantify and value the benefits and costs of a policy option are scientific decisions.”

These statements fail to recognize the inherent mix of science and policy judgments involved in an economic assessment of regulations. By its very nature, benefit-cost analysis involves both scientific decisions regarding empirical facts and cause-and-effect relationships and policy decisions regarding whose costs and benefits to consider, over what timeframes, and through what economic measures. For example, the specification of which policy alternatives to consider in a regulatory impact analysis does not reflect objective criteria but rather a subjective framing of priorities about which policy options are worthy of consideration. 

Another example of a policy judgment embedded in economic analysis is the “standing” decision, which determines whose costs and benefits “count” in the analysis. While in theory, perhaps every individual’s welfare should count, in practice, analysts usually limit the scope of analysis to count some individuals and not others. Similarly, intergenerational discounting choices reflect policy choices about how much the welfare of different generations should count relative to one another. Equity weighting schemes likewise reflect policy judgments about the analyst’s preferred distribution of wealth.

Beyond these factors, valuation metrics used to “monetize” benefits and costs in economic analysis can also often be value-laden. For instance, most estimates of monetized benefits implicitly assume partial equilibrium outcomes in certain individual markets are “efficient.” This efficiency assumption is what allows agencies to employ a “benefit transfer” approach, whereby estimates of consumer or worker willingness to pay in one context (e.g. mortality risk-wage tradeoffs in the labor market) are applied in other contexts (e.g. air pollution benefits). The assumption of partial equilibrium efficiency is a policy judgment (one that is not very realistic in most instances). Thus while, in theory, cost, benefit, and other estimates in an economic analysis might have objective, and therefore theoretically “correct,” values associated with them, in practice, debatable assumptions about economic conditions and the rationality of consumer decisions and behavior underlie most valuation techniques.  Also problematic is that the draft SI policy prohibits “directing economists, analysts, and other scientists to change the quantification and valuation of benefits and costs based on internal or external policy or political concerns.” This prohibition is so expansive it would seem to prohibit EPA staff from implementing changes requested by the Office of Information and Regulatory Affairs (OIRA).

OIRA review enables White House officials, both career staff and political appointees, to seek changes to agency regulatory analysis.

While there is no doubt a purely technical aspect to OIRA review, the process is also inherently political, and this is not necessarily a bad thing. The political nature of OIRA review helps ensure federal agencies are acting in a manner consistent with presidential priorities. Yet changes to regulatory analysis as part of the OIRA review process would likely be disqualified as suppression activity under EPA’s draft SI policy. The same may even be true of changes made to economic analysis in response to the public commenting process, a process that is also inherently political as it involves clashing values, opinions, and interests.  Failing to consider comments would violate the the Administrative Procedure Act and deprive the EPA of information and feedback that would improve agency rules. 

The application of benefit-cost analysis can also have legal implications, as seen in Michigan v. EPA, in which the U.S. Supreme Court rejected the EPA’s decision to exclude consideration of costs in a rulemaking. On legal questions, the EPA General Counsel, a political appointee, will certainly have opinions on agency compliance with the law. The draft SI policy may create problems if legal interpretations of political appointees on the proper use of benefit-cost analysis are inconsistent with the views of career staff.  

When it comes to the relationship between science and policy, the state of present knowledge is such that economic analysis sits much closer to the “risk management” end of the spectrum than the “risk assessment” end. Risk management decisions involve policy choices, since deciding what risks to prioritize and how to evaluate tradeoffs are value-laden enterprises. Treating benefit-cost analysis as objective science beyond questioning removes healthy scrutiny of the analysis and privileges certain policy preferences embedded in the analysis over others.

If economic analysis, like benefit-cost assessment, enjoys privileged status in EPA decision making, it necessarily should have input from political appointees. By prohibiting changes to cost-benefit analysis based on internal or external policy feedback, EPA allows unelected career staff to enshrine their own policy preferences into analysis, leaving political officials bound to adhere to those preferences. Yet, policy oversight of economic analysis is essential to ensuring consistency of policy with voter priorities, not to mention accountability of agency staff.

Science can only provide an answer as to “what is the case.” It can never provide an answer as to “what should be done” about it. EPA’s draft SI guidance claims to protect science from politics but could seriously damage both in practice. To the extent possible, the solution lies in clearly delineating the scientific and policy dimensions of economic analysis, not treating the whole exercise as sacrosanct objective science. 

3. Reproducibility and Transparency Are Crucial to Scientific Integrity

A glaring oversight in EPA’s draft SI policy is the lack of priority afforded to the ability of outside researchers to independently validate pivotal research underlying EPA’s major regulations and technical documents. Without an opportunity for a full auditing and reproduction of agency methods and findings by independent experts, EPA essentially expects the public to rely on its scientific outputs on faith.

EPA’s draft SI policy makes no mention of the wider “reproducibility crisis,” whereby a majority of published research conclusions across a variety of fields have failed subsequent validation attempts. Nor does the draft contain a strategy to increase agency reliance on independently verifiable studies. Additionally, there is no effort to further incorporate sensitivity analysis into EPA technical documents, whereby model outputs are tested under varied inputs to clarify relationships and illuminate uncertainties.

While the draft SI policy pays lip service to balancing expert viewpoints and accommodating differing scientific opinions, EPA failed to practice this ideal in its 2022 update of the social cost of greenhouse gases. The estimate excluded reference to any of three published sensitivity analyses by scholar Kevin Dayaratna. Neglecting dissenting scientific literature stands against the principles of balance and objectivity EPA claims to uphold. Furthermore, the EPA’s modelling is not transparent. The agency failed to explain why its Social Cost of CO₂ more than triples even though EPA is now projecting less than one-third the CO₂ emissions projected by the Interagency Working Group (IWG) on the Social Cost of Greenhouse Gases as recently as February 2021. Note that EPA is a member of the IWG, and therefore presumably agreed with the IWG when it published its most recent “interim estimates” on the Social Cost of Greenhouse Gases.

The EPA draft SI policy also puts substantial faith in the peer review process as a safeguard for quality and integrity of research. Peer review is identified in the definition of “conduct of science,” and EPA further states that its policy is to “[e]nsure EPA decisions are based on or informed by science that has completed independent peer review and has been finalized.”

Yet, the economic guidelines from OIRA shaping EPA’s benefit-cost analyses were not rigorously peer reviewed in the sense of having input from reviewers seriously inform the final product. In 2023, OMB released controversial updates to its Circular A-4 economic analysis guidelines for regulations. While these went through a pro forma public comment and peer review process, OMB chose not to make changes based on most of the criticisms made. OMB also chose not to make changes based on criticisms made in a letter to the agency signed by fifteen past presidents of the Society for Benefit-Cost Analysis. Rather than revise the guidelines in response to criticisms, as would occur under normal academic peer review procedures where the editor has independence from the author, the agency simply finalized the guidance with remarkably few changes made from the draft version.

Examples of criticisms made during the commenting process that were neglected relate to discount rates, equity weighting of costs and benefits, and societal risk aversion assumptions, just to name a few. One peer reviewer went so far as to publish an op-ed in the Wall Street Journal, titled “Biden’s OMB Politicizes Cost-Benefit Analysis,” outlining some of the many deficiencies with the updated guidance.

The lack of meaningful peer review and revision to OMB Circular A-4 suggests these guidelines should not bind future EPA policymaking. Indeed, it might be fair to say that any EPA economic analysis shaped by the revised OMB Circular A-4 document is compromised from a scientific integrity standpoint, based on the peer review principles outlined in the draft SI policy.

Unfortunately, even if OMB Circular A-4 had been adequately peer reviewed, peer review is unlikely to be sufficient to guarantee the reliability of scientific research. Peer review can facilitate conformity and groupthink, for example. Academia is currently struggling with its own scientific integrity crisis of sorts, as presidents from both Stanford and Harvard have resigned in the last year amidst accusations of data manipulation and plagiarism, respectively. These controversies, at a minimum, highlight the inadequacy of the peer review process at bringing to light shortcomings in scientific research. Hence, reproducibility and data transparency are essential.

4. EPA Actions Diverge from Rhetoric

In addition to the missing definitions discussed earlier, EPA’s draft SI policy also lacks reference to

several other important policies and details. One is the omission of any reference to the EPA’s Information Quality Guidelines under the Information Quality Act. This law and accompanying agency guidance established government-wide standards for information quality and correction processes. EPA’s draft SI policy fails to discuss how scientific integrity principles relate to or support these legally-mandated data standards.

Also of interest to the agency should be the 2021 EPA Science Transparency Rule, which was vacated by a court in response to a lawsuit (after the current administration sought a motion to vacate and remand the final rule). However, that rulemaking upheld ideals similar to those of the draft SI policy, especially as pertains to transparency. The draft SI policy states that the definition of transparency is “ensuring all relevant data and information used to inform decision making or actions are visible, accessible, and easily usable by affected parties to the extent permitted by law.”

The draft SI policy notes that “EPA should request scientific data from registrants, permittees or coregulators.” In ordinary language, EPA plans to request data from companies registering pesticides, companies applying for permits, and state counterpart agencies, but EPA does not appear to plan to request data from epidemiological researchers. The draft SI policy therefore will do little to increase transparency in the development or use of regulatory science.

The EPA should promote a culture of openness in which researchers are more amenable to sharing their data, including with qualified independent reviewers. If EPA can request data from registrants, permittees, and coregulators, it should also request data from the authors of pivotal dose-response research.

EPA will request scientific data from authors of pivotal studies it considers citing in support of its rulemakings. EPA will identify the studies whose authors decline to make their data available for independent validation. If EPA cites such a study in a rulemaking, the agency will briefly explain why it has confidence in the study’s results despite the inaccessibility of its data.

The EPA’s actions with respect to its scientific advisory boards also exhibit contradictions. For example, the EPA states “Federal Advisory Committees (FACs) are an important tool for ensuring the credibility, quality, and transparency of Agency science, and enhancing the transparency of the peer review process.” Yet, upon taking office in 2021, EPA political officials took the unprecedented step of removing members of EPA’s Science Advisory Board and Clean Air Science Advisory Committee that were appointed by the previous administration. These committees provide scientific peer review and advice to inform agency decisions and rulemaking. Despite claiming to want to shield these scientific bodies from political interference, the current administration has engaged in precisely the kind of aggressive political intimidation that it states it wants to prevent.

The selective enforcement of the principles outlined in EPA’s draft SI policy, even before the policy is implemented, raises serious questions about EPA’s commitment to sound science. Scientific integrity does not occur in a vacuum, divorced from self-reflection. In multiple areas, EPA’s lofty scientific integrity rhetoric is unmatched by actions that would provide meaningful transparency, enable independent validation, reveal sensitivities, and incorporate diverse expert opinion. If the agency will not walk the walk, the credibility of its scientific integrity commitments remains in doubt.

5. Indigenous Knowledge is Useful but so is Other Local Knowledge

A further concerning inclusion in the EPA’s draft SI policy is the incorporation of “Indigenous Knowledge” as a form of scientific evidence for agency decision-making. The draft SI policy states that the EPA will “ensure that, as appropriate, EPA consults and collaborates with Tribal Nations and Indigenous peoples to include Indigenous Knowledge in decision making.” This statement no doubt stems from recently issued Biden administration guidance promoting its use, and it follows a more general trend across federal agencies to incorporate indigenous knowledge into policymaking.

While some knowledge of indigenous peoples may be of a scientific nature, typically what people mean when they discuss indigenous knowledge relates to folklore, i.e., cultural, traditional, or other information passed on over time across generations, which accumulates into a body of established wisdom among indigenous people. While this knowledge can sometimes be useful, it generally does not constitute science. Consistent with this interpretation, the authors of a recent article in Science Magazine noted that “We do not argue that Indigenous knowledge should usurp the role of, or be called, science” (emphasis added). While indigenous observations and perspectives can provide localized insights that supplement scientific data—and therefore be extremely useful—categorizing collection of such knowledge as part of ensuring

“scientific integrity” risks equating that knowledge with science itself. There is clearly a danger that by elevating indigenous insights to the level of scientific evidence, the EPA risks introducing subjectivity, bias, and mystical beliefs into what should be an objective weighing of facts.

The draft SI policy captures the ambiguity around what constitutes indigenous knowledge by failing to define the term at all. Presumably, it encompasses the worldviews, cultures, practices, and oral traditions passed down by certain indigenous communities. However, such information is better thought as part of the political process of incorporating information from various stakeholders. The draft SI policy strangely singles out native knowledge over other types of localized insight without offering any coherent justification. By doing so, the EPA’s draft SI policy appears to be making presuppositions about the validity of certain kinds of information based on social categories, rather than assessing the integrity of the particular knowledge itself on a case by case basis.

Proponents may argue that principles of “equity and inclusion” support the privileged status of indigenous knowledge as a way to redress historical harms against native communities. However, righting past wrongs falls outside the scope of ensuring objective assessment of facts, which is the heart of what science entails. Even from a reconciliatory perspective, favoring one category of previously marginalized peoples over another unfairly elevates knowledge from some groups and devalues others.

What ultimately matters is the validity of the knowledge itself, assessed by whatever reasonable methods its integrity can be evaluated, not the innate characteristics of those who share it. Indigenous knowledge can certainly contribute usefully to environmental decision-making. But by promoting indigenous insights as uniquely valuable, EPA risks depriving its decisions of the benefit of other forms of local knowledge.

EPA should either a) include standards of assessment of indigenous knowledge (including ways to adjudicate between competing forms of indigenous knowledge that may be in conflict with one another, as well as a list of other groups beyond indigenous ones for consideration) or it should, more preferably, b) remove discussion of indigenous knowledge altogether from the draft SI policy, in which case the agency can clarify that indigenous insights may usefully inform decisions, but that consideration of such knowledge falls outside the scope of scientific integrity (except, of course, in those cases where the indigenous knowledge constitutes objective science).

Conclusion: EPA Must Resurrect Lost Trust

General confusion between objective science and subjective policy pervades the EPA’s update to its scientific integrity policy. If the EPA aims to rebuild trust in its commitment to science-backed policy, it must remedy the deficiencies outlined in this comment. The blurring of science and policy, as well as the blatant inconsistencies between EPA rhetoric and EPA actions, creates the appearance that partisan political considerations rather than scientific impartiality motivate this update. While some provisions of the draft SI policy plausibly aim to shield career analysts from political pressure, others seem intended to empower an unaccountable bureaucratic elite to sabotage administration initiatives on ideological grounds. All told, the draft SI policy undermines credibility in the EPA’s espoused ideals of scientific integrity. Before attempting to claim the scientific integrity high ground, EPA should remedy the glaring shortcomings in its draft SI policy, and furthermore aim to practice what the agency preaches.

Mere words detached from EPA’s track record cannot resurrect the public’s lost trust in the scientific basis of federal environmental policy.

James Broughel, PhD
Senior Fellow
Competitive Enterprise Institute

Craig Richardson
President
Energy & Environment Legal Institute (E&E Legal)

Jerry R. Simmons
President/CEO
Domestic Energy Producers Alliance

Paul Gessing
President
Rio Grande Foundation

David T. Stevenson
Director, Center for Energy & Environment
Caesar Rodney Institute

James Taylor
President
The Heartland Institute

Cameron Sholty
Executive Director
Heartland Impact

Craig Rucker
Co-founder and President
Committee for a Constructive Tomorrow (CFACT)

George Landrith
President
Frontiers of Freedom

Thomas P. Sheahen
Chairman
Science and Environmental Policy Project

John Droz, Jr.
Physicist & Founder
Alliance for Wise Energy Decisions

Phil Kerpin
President
American Commitment and the Committee to Unleash Prosperity

E. Calvin Beisner, PhD
President
Cornwall Alliance for the Stewardship of Creation

Faith Burns
Energy Policy Fellow
Americans for Prosperity

Gabriella Hoffman
Director
Independent Women’s Forum’s Center for Energy and Conservation

James E. Enstrom, PhD, MPH, FFACE
President
Scientific Integrity Institute

Kristen A. Ullman
President
Eagle Forum

Tom Pyle
President
Institute for Energy Research

Margaret Byfield
Executive Director
American Stewards of Liberty

Bette Grande
CEO and President
Roughrider Policy Center