The report finds an economically significant increase in the cost of credit in the absence of arbitration clauses. The average customer will find their cost of credit increase by 3.43 percent—significant rate hike for anyone. While the results are not statistically significant, that simply means there is a small chance of rates going down. That is a statistical issue, rather than an economic one. For instance, while there is a 12 percent chance of rates going down, there is 42 percent chance of rates going up by more than 4 percent.
The CFPB used the lack of statistical significance to justify its assertion that there is no evidence that consumers might be harmed. In doing so, it ignored the careful conclusion of the author of the study underlying the OCC analysis that “I cannot rule out an economically significant response.”
The Bureau’s defenders will surely try to ridicule the OCC study on the grounds of statistical significance. In doing so they ignore the far more relevant point of economic significance. Rejecting a hypothesis that may be true on the basis of a “P value” (the measure of statistical significance) is what is known as a “Type II error.” As economist Deirdre McCloskey has long argued, failing to reject the null hypothesis (the hypothesis that there is no effect) does not mean that the effect in question is meaningless or unimportant.
As McCloskey notes, when the effect being considered is meaningfully large—as a 3.4 percent increase in the cost of credit would be for millions of Americans—the matter becomes one of practical rather than statistical importance, and the risks of Type II error are therefore much more significant. The CFPB seems quite clearly to be running a huge risk of Type II error.
The OCC’s mission is “[t]o ensure that national banks and federal savings associations operate in a safe and sound manner, provide fair access to financial services, treat customers fairly, and comply with applicable laws and regulations.” As such, economically significant effects are of great importance to it. The OCC was right to issue this report, right to warn that the CFPB had failed to establish any lack of consumer harm as a result of its rule, and right to call for Congress to disapprove of the rule.