Don't trust the CBO, but not for the reasons they're telling you

The Congressional Budget Office is intended to be an objective, non-partisan source of information, telling the American public how much a proposed legislation will cost them. As an “official” source of statistics, most people — at least the ones who pay attention to such things — accept the department’s rulings as fact without a second thought. This is a mistake.

The Hill is reporting that members of the Senate GOP, displeased with the CBO’s score of the latest health care proposal, are putting pressure on the department to revise its numbers.

I’m not here to tell you whether the CBO’s numbers in this case are too low or too high, whether the department is secretly attempting to subvert the agenda of the Trump administration, or whether Republicans are unfairly bullying honest bureaucrats trying their best to tell the truth.

That’s not the point I want to make here. And having worked with several former CBO employees, I believe that they mostly make an honest effort to be impartial. But the problem with the agency, one which it shares with numerous other departments, is a more fundamental one having to do with scope and methodology.

The CBO is essentially a number-crunching system — good at computations but bad at independent thinking. The reason for this is that the employees of the department can only work with the numbers they are given.

They are not allowed to speculate, nor question the validity of the information coming in; they can only do the math. If you feed faulty data into a computer, no matter how sophisticated, you cannot expect to get accurate results. The CBO suffers the same flaw.

A good example of this came from the original Obamacare bill, in which the CBO was forced to assume that the law’s individual mandate would actually compel people to buy insurance, failing to factor in the inevitable levels of deliberate or accidental non-compliance. The result was that the CBO got Obamacare enrollment dramatically wrong and had to repeatedly revise its figures.

This is not surprising. How can anyone be expected to reliably estimate what percentage of people will disobey a new law, before it has even been passed? Obviously human free will makes accurate predictions impossible. Extrapolating based on historical statistics is a poor substitute for the crystal ball the department is expected to have access to.

What the CBO’s black box arrangement means in practice is that legislators exercise a great deal of control over what the CBO produces. If you control what goes in, you can more or less control what comes out, hence the current effort to revise the health care bill — bills can be written with an eye toward the CBO’s analysis.

The more a bill’s drafters know about the scoring process, the better they can game the system to produce a favorable result. The simplest way of doing this is to adjust the time horizon on future expenses.

This is because the CBO only calculates costs within a 10-year budget window. Anything that falls outside those 10 years doesn’t get scored, and is therefore invisible from the point of view of the media, or anyone else who typically relies on CBO scores for analysis. It is therefore trivial for lawmakers, especially on bills with long-term effects, to push major costs out beyond the 10-year budget window, thereby preventing the CBO from reporting them.

Over the last several years, there has been debate over whether the CBO can improve its results by switching from “static scoring” to “dynamic scoring.” Static scoring is the traditional method that assumes no behavioral reaction to policy changes. For example, a 10 percent tax increase will be expected to yield 10 percent more revenues; the actions of people trying to mitigate their tax liability will not be included.

Dynamic scoring seeks to use computer models to account for this behavior, which obviously happens. But, in my view, the shift in methodology makes little difference. It is merely a matter of exchanging one unrealistic set of assumptions (that people do not react to new laws), for another (that we can guess how they will react with any degree of precision).

Economics is the study of human action. The conceit of those government agencies dedicated to predictions and projections is that these actions can be modeled in a computer and tracked, like the behavior of celestial bodies.

But stars and planets have neither minds nor wills, and no government computer will ever be able to account for the various whims and facies of one man, much less the hundreds of millions of people who make up our society. And, for this reason, the CBO and all its projections should always be treated, at best, with suspicion and mistrust.