All over the country, regional planners are working on high-speed rail networks. President Barack Obama has said now is the time to build a modern passenger train system:
BARACK OBAMA: There's no reason why we can't do this. This is America. There's no reason why the future of travel should lie somewhere else beyond our borders. Building a new system of high-speed rail in America will be faster, cheaper and easier than building more freeways or adding to an already overburdened aviation system – and everybody stands to benefit.
And California is first in line to create a high-speed rail that might revolutionize the way we travel. In the past freeways designed for automobiles got the lion's share of public funding and were seen as relatively efficient. But in recent years, people like the President have started to say that freeways have a lot of hidden costs that don't get factored in: like the Gulf oil disaster, or maintaining a military to insure the safety of our overseas sources of oil, or when you begin to think about climate change, pollution, traffic jams, deaths from accidents.
But this is also a time when Americans have little faith in their government’s ability to build big. After Boston’s Big Dig (the tunneling project that went more than $10 billion over budget) and California’s Bay Bridge replacement (which is expected to cost taxpayers $7 billion more than originally estimated) the people of this country are legitimately skeptical.
But California voters decided to take the leap into high-speed rail. The state elected to take out nearly $10 billion in loans in 2008, and in 2009 the Obama administration promised the state another $2.25 billion to get the process started. And that’s just the beginning – the whole system will cost somewhere between $42 and $45 billion…at least that’s the current estimate.
If the train is built, the trip from San Francisco to L.A. is supposed to take around 2 hours and 40 minutes, but today we’re going to get you there in just half-an-hour.
In the next 30 minutes our transportation reporter Nathanael Johnson will be driving the route. In taking a close look at California’s high-speed rail project, he’ll be searching for the forces that cause big infrastructure projects to turn into over-budget boondoggles. What is it exactly that causes this to happen over and over? And how can we make sure that it doesn’t happen this time?
Nathanael picked up his rental car in downtown San Francisco, drove past the TransBay Terminal, which is supposed to be the end of the line, and he’s heading down the 101. The high-speed rail route would follow the Caltrain tracks along the Bay there.
His first stop is in Palo Alto, where he’s meeting the person who caught a flaw in ridership estimates that could increase costs.
* * *
NATHANAEL JOHNSON: So here we are at the house of Elizabeth Alexis. I just have to figure out how to get in here…
ELIZABETH ALEXIS: Hey – good morning. It’s a little chaotic here. This is Sophie.
Elizabeth Alexis doesn’t look like a high-powered transit analyst. At least not today. She’s wearing a grey T-shirt and no shoes. Knock on 50 doors in Palo Alto and you’d probably find a bunch of people a lot like Alexis: smart, working moms with young kids, trying to bring home the bacon and make sure everyone gets to soccer practice on time.
But Alexis is different. She’s one of the few people who really understands what planners did to produce the ridership estimates that were used to determine where the train should go and how it could be financed. There were big mistakes in these estimates. Mistakes missed by planners, peer reviewers and politicians. Mistakes that we only know about because of this mom with two kids and a day job.
ALEXIS: I’m on the civic affairs committee and last January there was this scoping meeting for the H.S.R. project and that was my job, to go attend the meeting and see what it was all about.
The year before, she’d done a little Internet research when the ballot measure to authorize $10 billion in bonds for the project came up. She voted for it. But after the meeting, Alexis started reading through the environmental reports and she began to notice what seemed like mistakes.
ALEXIS: They had some holes in them that were surprising – the kind of thing that just got your attention.
Alexis has a sharp eye for detail. Things that most of us would never notice—like the fact that the existing train corridor between San Jose and Gilroy is just 50 feet wide—she files away, and then is able to recall that when she sees plans assuming a 100-foot corridor on the same section. Before Alexis, nobody had noticed these errors. So she looked closer, and found more problems.
ALEXIS: It meant that they had billions and billions of dollars, gaps in their funding structure, and no one had noticed that.
Alexis decided she had to read the ridership study – something she really, really didn’t want to do. She had studied applied microeconomics in grad school, so she had the skills, but it meant checking the math on over 1,000 pages of numbers, line by line. It took her months just to understand what the planners had done. But once she got a handle on it, she was shocked by what she found.
ALEXIS: I was not expecting to see these just giant mistakes. I was expecting to have obscure arguments over the standard deviations – should it be the cube of this or the square of this? That’s what I was expecting to find and why I didn’t want to read this. I was not expecting to see what I’ll call math mistakes, just enormous flaws like this. That was a shock to me.
This led the state Senate to ask for an independent peer review. The Institute of Transportation Studies at UC Berkeley completed that review in early July. It found, quote, “significant problems that render the key demand forecasting models unreliable for policy analysis,” endquote. Professor Samer Madanat led the peer-review team:
SAMER MADANAT: The best-developed models very often have prediction errors of plus or minus 50%, and very bad models can be wrong by an order of magnitude.
JOHNSON: So these are large, right?
MADANAT: These are large. Which is why I always emphasize that travel demand models are useful but always have to be taken with a grain of salt.
No one can predict the future, and no model is perfect, but these numbers had been bandied-about in the press, politicians had trotted them out as debating points, and they had played an instrumental role in determining the route the train would take for the leap from California’s Central Valley, over the Diablo mountains, to the coast.
Leaving Palo Alto, I drive that route, south through San Jose, then into farmland, until I reach the little town of Gilroy, the garlic capital of the world.
JOHNSON: All right. I’m now in Gilroy, just off Leavesly road, and I’ve walked past the outlet malls through a little windbreak onto a berm, over a canal or slough, and I’m looking out over farmland. There are orchards and greenhouses and above that, these brown hills with black oaks dotting the ridgeline. There are tall grasses all around where I’m standing. You can hear the wind shaking them, birds flying by butterflies pollinating the flowers – there’s an overwhelming scent from the wildflowers. And this is right where the Gilroy station would be.
Environmentalists worry that a station here would turn these fields into housing tracts for commuters. It’s what’s they call "green-field development" – where the train brings new people into a rural area, rather than going to the places where people already live. That’s one reason most environmental groups favored an alternate route – instead of going south, it would run through the more densely populated East Bay, then over the Altamont Pass. On the other hand, it’s hard to build through cities, it’s more expensive, and this route is a straighter shot – it’s about 10 minutes faster.
I turn east off the 101 and wind my way along the narrow road that leads to the Pacheco Pass. It’s rangeland, and steep, craggy wilderness. There would have to be a lot of expensive tunneling through here.
In the end, citing the higher ridership estimation, the regional transportation commission recommended this southern Pacheco route. The nonprofit transit group TransDef is suing over that decision, alleging that planners cooked the books to favor Pacheco. The president of TransDef is David Schonbrunn, and here’s the thing about him: he’s not some small-government crusader who just wants to kill the train. He’s a low-carbon crusader, who desperately wants high-speed rail. When we met, he pressed the end-of-oil book “The Long Emergency” into my hands.
DAVID SCHONBRUNN: We’re environmentalists who are convinced that our future requires great transit, and HSR is absolutely essential to a good future, and these guys are screwing it up every step they take.
The California High Speed Rail Authority, the defendant in this lawsuit, declined to comment for this story. We could get into the details of the lawsuit, but Randy Rentschler, representative for the regional planning authority – the Metropolitan Transportation Commission says that the details are … just details.
RANDY RENTSCHLER: People can have debates, and that’s all fine, but when it comes to the real things that matter, getting distracted by how many angels dance on the head of a pin, I think, obscures what matters more – which is these really big questions: is our county secure economically and from a sense of national security with this level of dependence on liquid fuels? That’s the question.
And at one level it’s impossible to argue with that. The big question is whether or not California musters the political will to build this thing. If it doesn’t, everything else is irrelevant. If the state does build this train, though, all these details will determine how successful it is. But Rentschler says it’s pointless to argue about ridership numbers on Pacheco or Altamont, because the planning projections are so vague. Of course, that didn’t stop him from citing the ridership numbers after I turned off the recorder, to help prove a point.
Samer Madanat, the Berkeley professor whose team criticized the ridership study, says, look, the projections are always vague. The key is that imprecise numbers shouldn’t be used as an authoritative political tool.
MADANAT: Here we are dealing with predicting human behavior – as well of predicting things we have no way of predicting – the economy in 50 years … I think we need to be modest and humble, and frankly truthful, regarding the ability of these models.
As I drive down into the dry heat of the Central Valley, this seems like a central problem. People are neither humble, nor fully truthful in their drive to push projects through. This isn’t corruption per se, it’s just that politicians have learned that if you slow down or go back, the project dies. The infrastructure projects that survive are the ones whose backers are deaf to critiques and just keep the momentum going.
But if inaccuracy in predicting the future were the only problem, we’d be seeing projects completed dramatically under budget, as well as over. From reading the newspaper, it seems like every infrastructure project goes over budget – but do we only hear about the bad ones? To answer this, I spoke to the leading expert in the field.
BENT FLYVBJERG: Bent Flyvbjerg. I’m the BT Professor and Chair of Major Programme Management and Director of Oxford University's BT Centre for Major Programme Management.
JOHNSON: Um. Can you say your name one more time?
FLYVBJERG: Bent Flyvbjerg.
Flyvbjerg is the first person to truly pin down the causes of bias in planning, and his ideas are changing the field. If you want to understand why government building is so problem-filled, he’s probably the best person in the world to explain it. I’ve got the interview I did with him here and it will get us through the long, flat Central Valley portion of the trip.
Flyvbjerg wanted to find out, once and for all, if planners were lying, biased or just inaccurate. So he went back and compared the projected budgets with the final cost of hundreds of projects. OK, here we go.
* * *
FLYVBJERG: Yeah you often hear people say that projects like this go over budget all the time, but that's wrong. It is only 90 percent of the time that the project are over.
JOHNSON: So still a large percentage. Ninety percent is a big number.
FLYVBJERG: It really is, yeah.
JOHNSON: I want to talk specifically about California high-speed rail in just a moment. But I think it would be useful to understand our past mistakes before we look at the future. There are lots of excuses that planners give to explain why these projects go over budget. But you've simplified things and divided these explanations into three tidy categories, the first of which is technical errors. Can you explain, what's a technical error?
FLYVBJERG: This is the most common type of explanation, you know, when budgets or demand forecasts go wrong, people say, yeah, the data wasn't good enough or the models that we used to model the costs or demands were not good enough. But if we get better data and if we get some money to develop better models, then we'll be okay next time. And we've studied this carefully and it just isn't true, you know. The problem is not technical. It's not a question of bad data or bad models.
JOHNSON: If this is simply an honest error, sometimes it should go low as well as high.
FLYVBJERG: Exactly. If this was due to errors, then you'd expect a pretty much normal distribution around zero. So we can say, you know, with a very high level of confidence, that the technical explanations simply do not hold.
JOHNSON: The second category you identify is optimism bias. What is optimism bias?
FLYVBJERG: Well optimism bias is something that we all have. All human beings are subject to optimism, unless - and this has actually been verified through research - unless you are a depressed person. But the average person has optimism. You know, we look at the future through rose-colored glasses. So if you have optimism bias and you're making a budget for, you know, to make a project, a high-speed rail line or whatever, an opera house, what would that look like?
Well, if you would be underestimating the cost that would be an optimistic cost estimate. And if you would be overestimating the revenues or the benefits in more general terms, that would be optimism regarding benefits, and this would actually fit our data. This is what we see in the data that costs are actually underestimated and benefits are actually overestimated for these types of projects.
JOHNSON: So if you could see that in the data, is there a precise way to control for optimism bias?
FLYVBJERG: Yeah, we've actually developed methods for how to do that. We call it de-biasing. We have methods that will de-bias decisions. Like once you've had people make the decision, being subject to the usual optimism biases, we have collected data and developed methods that make us able to go in and then de-bias the decisions.
JOHNSON: So the third category that you mentioned is strategic misrepresentation, which sounds like a charitable way of expressing a less kind, but more direct word. Is that just a euphemism?
FLYVBJERG: Yes it is. I found I was giving lectures on this, which I do quite a lot, that if you talk about lying, which it really is, you know, then people were terribly upset. So I figured I had to find a different term, and the term that I came down on is strategic misrepresentation. It's like when you talk about it in those terms, like people feel that, hey that sounds technical and you need to have some qualifications to do that so they can accept that.
JOHNSON: When you’re talking about strategic misrepresentation, or, to use the word lying, why do planners fudge the facts?
FLYVBJERG: Simply because there’s an incentive structure that encourages this. It’s not irrational lying. It’s actually rational. So, for instance, you go to Washington, D.C. to get money for projects in San Francisco or Chicago or Miami, you have this incentive that the people will try to get as much money out of D.C. as possible with whatever means it takes because they are in competition with other cities. And therefore the cities will compete on their cost/benefit analyses. They will make their projects look good on paper. And how do you do that? Again, you do it by underestimating the costs and overestimating the benefits. Then you get a nice benefit/cost ratio that makes your project look like the best project.
JOHNSON: Indeed, you point out that the way we’ve set things up incentivizes the projects that are least realistic.
FLYVBJERG: Exactly. We actually call this survival of the “unfittest.”
JOHNSON: So what can we do here in California to ensure that we plan for high-speed rail that provides the best service to citizens at the lowest costs?
FLYVBJERG: If it hasn’t been done already, there needs to be a study of whether the decision is biased. Meaning whether it’s optimistic or whether there is strategic misrepresentation.
JOHNSON: You also talk about the importance of taking the outside view, looking at comparable projects around the world rather than building projections from the ground up. So, how does taking the outside view inform the California High-Speed Rail Project?
FLYVBJERG: Well, there are really no high speed rail projects to compare with in the US. The closest thing would be the Boston, New York City, Washington fast rail. I wouldn’t call that real high-speed rail. And urban rail projects in the U.S. generally have a dismal track record. There are a few successes, but most of them don’t look good. So building rail in the US doesn’t have a good track record. Therefore you would have to be extra careful in the planning of a big thing like the California High Speed Rail.
JOHNSON: Let me ask you about a couple other solutions that you’ve proposed to this problem of cost overruns and benefit shortfalls. One that I really like is that you suggest that there should be professional and criminal penalties for planners. As it is, there are no penalties whatsoever in California. How do planners react when you suggest this?
FLYVBJERG: Well, initially they got really upset. But after we’ve had all the corporate scandals and there’s this whole thing about Iraq and all the stuff that went on there, and now with the financial crisis, you actually see people in business and government making decisions that are extremely damaging to the rest of society. If people in the corporate sector are getting punished for doing these kind of things, why shouldn’t civil servants be punished for doing similar things with taxpayer’s money?
JOHNSON: And you also have some ideas for controlling this sort of thing through market mechanisms, how would that work?
FLYVBJERG: Basically, if they make a good decision, they should be rewarded and if they make a bad decision they should have negative consequences. One way or the other, that’s the basic principle. So, for instance, if you are the demand forecaster for the high-speed rail project in California, if your forecast turns out to be completely off, you definitely shouldn’t be paid anything for that forecast. That just doesn’t seem fair that you can make these completely wrong forecasts with no quality whatsoever and get paid for it.
JOHNSON: You also suggest that private money should cover at least one-third of the costs. How would that work?
FLYVBJERG: Here our thesis is, and we’ve tested it against specific projects, is that if you bring in private risk capital, that will induce some discipline into the decision-making process. Because the people who own that capital and bring it in will be very aware of whether it’s spent wisely or what the risks are and so on for bringing in that capital. It would demand more realistic risk assessments, more realistic risk assessments. That’s the idea.
JOHNSON: Well thank you so much, I really appreciate your thoughts on this.
FLYVBJERG: My pleasure, thank you very much and I hope some of this will be useful for you.
* * *
HOST: This is “The Planning Problem” - a Crosscurrents special on high speed rail.
President Obama says we can build high speed rail now:
PRESIDENT BARACK OBAMA: Now, all of you know this is not some fanciful, pie-in-the-sky vision of the future. It is now. It is happening right now. It's been happening for decades. The problem is it's been happening elsewhere, not here.
And the first place it’s supposed to happen, is here, in California.
Our transportation reporter, Nathanael Johnson is driving the length of the proposed route, from San Francisco to L.A. Along the way he’s been talking with experts and people involved to try to understand – what causes these mega-projects to succeed or fail? And what can be done to ensure success?
* * *
JOHNSON: Okay, let’s recap. So far we’ve learned that there are big holes in California’s High Speed Rail plans:
ALEXIS: I was not expecting to see what I’ll call math mistakes, just enormous flaws like this. That was a shock to me.
And that there are incentives that reward politicians who lie, along with planners who intentionally create biased numbers or just don’t stop to check the math when the numbers are going the right way.
FLYBJERG: It's not irrational lying. It's actually rational.
That’s Bent Flyvbjerg, the Oxford professor who has proposed a number of solutions for these problems. Planners all over the world are starting to use some of these solutions. The consultants who prepared the high-speed rail ridership projections have cited Flyvbjerg in their work, but they haven’t fully embraced his techniques for transparent objectivity.
Take one of his most simple suggestions: We have enough data from past projects to know that on average rail projects exceed budgets by 40 percent. So why not just boost projected budgets by these amounts? Doug Kimsey, lead planner for the Metropolitan Transportation Commission, the Bay Area’s regional transit authority, says they are not considering adopting techniques like this:
DOUG KIMSEY: No, I think what we do is what we’ve been doing. We apply industry standards, as we know them today, and then we do sensitivities and contingencies.
JOHNSON: A lot of people are frustrated by standard industry projects – can you see how standard industry practice might seem unacceptable at this point?
KIMSEY: Well, when you look at big projects, yeah – there are all kinds of things that can go wrong that won’t be accounted for even in your contingencies.
And there’s almost always something that goes wrong - something that no one expected. But if you go back and compare past estimates with reality, you’d expect some overestimates and some underestimates, averaging out at zero.
The strange thing is that planners are almost always wrong in the same direction: They overestimate benefits, and underestimate costs. Bent Flyvbjerg found that on average planners overestimated train ridership by 109 percent. There’s probably a simple reason for this: To convince people to invest in your idea – it helps to exaggerate. Call it the James Cameron argument:
JAMES CAMERON: You’re on Pandora ladies and gentleman. Respect that fact every second of every day.
James Cameron makes movies – like Avatar – that go hugely over budget, but they always seem to end up making that money back. The James Cameron argument is that to achieve great things, sometimes you have to give budget estimates that you know are way too low.
But then there’s a counter – call it the Cimino argument:
HEAVEN's GATE TRAILER: A story of a man’s love for a woman, for a people, for a land…
Michael Cimino directed this 1980 movie, Heaven’s Gate. Like James Cameron, he demanded more and more money to finish his film. But if you’ve heard of this movie, it’s probably because it went down in the history books as the worst flop of all time.
So is high-speed rail a Cameron or a Cimino? Hollywood makes these decisions by looking at past performance, so let’s start there.
Elizabeth Alexis, the citizen who caught the mistakes in the high-speed rail plans, has been looking at past projects—like the BART extension to the San Francisco Airport. More than a decade later, that train still doesn’t have the riders it was supposed to get in year one.
ALEXIS: No one has gone back and done the autopsy of what went wrong. No one has gone back to the ridership studies and said, what was really wrong in these studies? So we’re not learning – that actually concerns me more than anything else. I don’t see that we are learning from our mistakes.
As a matter of fact, planner Doug Kimsey does have a good explanation for what went wrong.
KIMSEY: In the case of SFO, I don’t think that anyone predicted the extent of the economic depression as a result of the dot-com bust.
But, a flop is still a flop, no matter what the economy. And there are consequences. These days, James Cameron gets as big a budget as he likes, while Cimino never really worked again. But there don’t seem to be the same consequences in infrastructure. No one was fired for getting the estimates wrong. The consulting firm Parsons-Brinckerhoff led that BART project. They also did the Big Dig in Boston. Then California chose the same company to lead the charge for high-speed rail.
Of course, neither Hollywood nor the public sector can predict the future. But it seems only Hollywood can learn from the past. So what does all this mean for high-speed rail in California?
As I drive over the Grapevine and into L.A., the road turns into a congested parking lot, and all of a sudden high-speed rail seems very attractive.
JOHNSON: I can see my exit, the 134 is right there, but I just can’t get to it because – Oh, now we are moving again. I think we’ve achieved 5 mph. Oh, brake lights.
So this is the kind of thing that might induce people to get out of their cars and jump on the train. It’s frustrating. I can feel the cortisol building up in my blood. It would be really satisfying just to gun it into the back of this car in front of me.
The traffic-jam life is tough, and in general people like the idea of high-speed rail: The California bond for the train passed with 52 percent of voters in favor. Samer Madanat says there is good reason for that. A train system could be transformative. Madanat is the professor who peer reviewed the train’s projected ridership numbers, and he says that the estimates are unreliable, and operations costs might run in the red for years.
MADANAT: But that doesn’t necessarily mean that HSR is a bad idea, right? In many cases, projects like this become game changers. They incentivize other changes in the transportation system that together make the system work better. That’s the "if you build it they will come" approach.
And David Schonbrunn, who is suing the High Speed Rail Authority, still has hope that it could be done right:
SCHONBRUNN: There is something that could be done to save this project, and that is to put the project out to bid. We know that there are various international consortia who want to bid on being able to finance, build, operate and maintain this project.
It has always been the intention of the High-Speed Rail Authority to get the bulk of the money for the project from a private partner. But it remains to be seen if any corporations will actually be interested in investing. Say what you like about corporations, when there is money on the line to be made or lost, businesses actually pay for people like Elizabeth Alexis to catch mistakes.
So I’ve finally reached L.A. – home of Hollywood’s movie mega-projects, land of freeways and hundreds upon thousands of cars. Could this be transformed by high-speed rail? Without good data, it’s a purely speculative question.
Weighing the details of engineering, design and finance are incompatible with a “shovel-ready” age. With many of these projects, numbers, ridership projections and revenues end up serving the politicians, rather than the politicians serving the numbers. Members of the Obama Administration say they are moving toward incorporating Bent Flyvbjerg’s suggestions in infrastructure planning.
But the High Speed Rail Authority is facing deadlines to get its federal funding. And under this pressure it doesn’t feel like Bent Flyvbjerg has an audience in California. The high-speed rail future will either fly on a wing and a prayer, carrying train cars full of unknowns, or die facing what little we can know about it now.
For Crosscurrents I’m Nathanael Johnson.
