Government’s Chinatown Bus Shutdowns Based on “Statistical Malpractice”

Reason’s Jim Epstein has an article up that does a nice job debunking a National Transportation Safety Board study, prompted by a 2011 bus crash in the Bronx that killed 15 people, that led the Federal Motor Carrier Safety Administration to shut down a number of supposedly unsafe small bus companies:

In 1997, Chinese-born entrepreneurs began regularly scheduled long-distance bus services that picked up passengers on the street. Tickets were priced so low that it was hard to figure how the operators could be breaking even, much less making a profit. Faced with declining market share, Greyhound and Peter Pan imitated the Chinatown model by teaming up to create a new venture called BoltBus. Then Coach USA got into the game with Megabus. Today, “curbside” buses—lines that begin and end their routes at the sidewalk as opposed to a traditional station—make up the fastest growing form of intercity travel in the U.S.

But over the past two years, the government has forced 27 bus companies based in Chinatown to close. The regulatory clampdown was fueled by a government study that found curbside carriers were disproportionately killing their passengers. Released by the National Transportation Safety Board, a federal agency, the study concluded that curbside bus companies were “seven times more likely to be involved in an accident with at least one fatality than conventional bus operators. That finding was reported by The New York Times, the Los Angeles TimesBusinessweekUSA Today, the New York Daily NewsWNYC, and Reuters, among others. Although the study did not single out Chinatown bus companies the headline in Businessweek read, “Chinatown Buses Death Rate Said Seven Times That of Others.”

The study is bogus. Not only is the “seven times” finding incorrect, the entire report is a mangle of inaccurate charts and numbers that tell us virtually nothing meaningful about bus safety. There’s no evidence that curbside or Chinatown buses are any less safe than any other kind of bus.

How did the study authors figure curbside bus companies are “seven times” more prone to fatal accidents? For starters, they counted 37 accidents during the study period involving curbside buses in which there was at least one fatality. When I rebuilt the study data and contacted the companies involved, I found that, in 30 of those 37 accidents, curbside buses were not involved. In fact, 24 of those 30 misclassified cases involved Greyhound’s conventional bus fleet. (Greyhound’s curbside subsidiary BoltBus had no fatal accidents during the study period.)

The National Transportation Safety Board denied my requests for the study data, even though it was a taxpayer-funded report with an impact on policy. After my Freedom of Information Act request also failed to return the information following a six-month wait, I began reconstructing the study data from other sources.

Proceeding on the time-honored hunch that people who are hiding something have reason to do so, I generated a list of the 37 fatal crashes using a database obtained from a federal contractor that collects nationwide accident data. I analyzed that data with help of Aaron Brown, a quantitative analyst with the hedge fund AQR Capital Management. Brown was the first to point out major flaws in the NTSB’s methodology in an article published by Minyanville.com, accusing the study authors of “statistical malpractice.” I also consulted with Ed George, a professor of statistics and department chair at the University of Pennsylvania’s Wharton business school, who examined the study for the purposes of this article.

“When I first read the NTSB report, I thought this is just terrible statistics,” says Brown. “But it goes way beyond that. It’s almost as if someone took some random data and shook it together.”

Read the whole thing over at Reason for an overview of the NTSB’s incredibly sloppy study methodology. Cato’s Randal “The Antiplanner” O’Toole has more.

Epstein didn’t delve into it, but a major issue involved here is the FMCSA’s ability to arbitrarily shut down carriers it views as “imminent hazards,” which is defined as “the existence of a condition that presents a substantial likelihood that death, serious illness, severe personal injury, or a substantial endangerment to health, property, or the environment may occur before a notice of investigation proceeding, or other administrative hearing or formal proceeding, to abate the risk of harm can be completed.”

The enforcement protocol, officially known as the “Imminent Hazard Operations Out-of-Service Order,” is essentially an operating injunction issued by the FMCSA. This is an immense amount of power granted to a federal regulatory agency, a shoot-first-ask-questions-later power that can force a wrongly accused small business in a low-margin industry into bankruptcy nearly overnight. The real story here isn’t that NTSB is churning out garbage studies. Sloppy and biased research published by government agencies are a dime a dozen. It’s that said sloppy and biased research was all that was needed for the FMCSA to arbitrarily wield its huge amount of political power to put entrepreneurs out of business.

Unfortunately, this sort of overzealous enforcement based on bad science is par for the course at FMCSA. For example, this is the agency that recently abused fatigue research in order to justify a new hours-of-service rule for truckers that will likely result in more accidents and cost the economy billions of dollars annually if it comes into force.