Pickrell_(1990)_Urban_Rail_Transit_Projects_Forecast_Versus_Actual_Ridership_and_Cost.pdf

U.S. Department of Transportation

October 1990

I \

Urban Rail Transit Projects:

Forecast Versus Actual Ridership and Cost

Urban Rail Transit Projects:

Forecast Versus Actual Ridership and Costs

Final Report October 1990

Prepared by Dr. Don H. Pickrell Transportation Systems Center Research and Special Programs Administration Kendall Square Cambridge, Massachusetts 02142

Prepared for Office of Grants Management Urban Mass Transportation Administration U.S. Department of Transportation 400 Seventh Street SW Washington, D.C. 20590

Distributed in Cooperation with Technology Sharing Program U .S. Department of Transportation Washington, D.C. 20590

DOT-T -91-04

TABLE OF CONTENTS

Section

SUMMARY V

I.

2.

INTRODUCTION

I. I FEDERAL SUPPORT FOR RAIL TRANSIT INVESTMENTS

1.2 ASSESSING THE PLANNING PROCESS ..

1.3 THE DEFINITION OF "FORECAST" DATA

1.4 WHY REVIEW PAST FORECASTS? .

1.5 TYPES OF FORECASTS EXAMINED

1.6 DATA SOURCES

1.7 PROJECTS CHOSEN FOR STUDY

1.8 COMPARING FORECAST AND ACTUAL DATA .

RIDERSHIP FORECASTS

2. I MEASURES OF RIDERSHIP

2.2 COMPARING FORECAST AND ACTUAL RIDERSHIP

2.2.1 Rail Passengers

2.2.2 Total Transit Ridership

2.2.3 Changes in Transit Ridership . . .

2.3 CAUSES OF FORECASTING ERRORS

2.3.1 Demographic Factors

2.3.2 Transit Service Levels

2. 3. 3 Automobile Costs

2. 3.4 Overall Assessment of Input Errors .

I l

3. CAPITAL OUTLAYS FOR RAIL TRANSIT PROJECTS 31

3.1 FORECAST AND ACTUAL CAPITAL OUTLAYS

3.2 FACTORS CONTRIBUTING TO CAPITAL COST OVERRUNS 34

3.2.1 Changes in Project Scope

3.2.2 Construction Scheduling and Financial Planning 37

3.2.3 Relative Contributions to Cost Overruns 38

3.3 FINANCING OF CAPITAL OUTLAYS . . 40

3.4 ASSESSING CAPITAL COST FORECASTS 42

4. OPERATING EXPENSES

4. I RAIL OPERATING EXPENSES

4.2 SOURCES OF HIGHER UNIT OPERATING EXPENSES 48

4.3 IMPACTS ON SYSTEMWIDE OPERATING EXPENSES 49

5. THE COST-EFFECTIVENESS OF RAIL TRANSIT INVESTMENTS 53

5.1 COST PER RAIL PASSENGER . . 53

5.2 COST PER NEW TRANSIT TRIP

V

TABLE OF CONTENTS (Continued)

Section

6. CONCLUSIONS AND RECOMMENDATIONS

6.1 FORECASTING ERRORS AND THEIR SOURCES 61

6. I. 1 Ridership

6. 1.2 Capital Outlays 63

6. I . 3 Operating Expenses 63

6. 1.4 Cost-Effectiveness

6. 1.5 Sources of Forecast Errors

6.2 FORECASTING ERRORS AND CHOICES AMONG PROJECTS 65

6.3 IMPROVING THE ACCURACY OF FORECASTS 67

6.3.1 Ridership Forecasts

6.3.2 Capital and Operating Cost Forecasts

6.3.3 Expert Review and Oversight of Project Activities 72

6.4 ACKNOWLEDGING UNCERTAINTY IN FORECASTS 73

6.4.1 Uncertainty in Ridership Forecasts

6.4.2 Contingency Allowances to Cover Cost Escalation 74

APPENDIX. DATA SOURCES A-1

VI

LIST OF TABLES

Tablt·

S-1 Forecast and Actual Cost per Passenger for Recent Rail Transit Projects . . . . . . . . . . . . . . . . vii

S-2 Forecast and Actual Cost per New Transit Trip for Recent Rail Transit Projects . . . . . . . . xi

1-1 Characteristics of Recent Rail Transit Projects

2-1 Forecast and Actual Ridership for Recent Rail Transit Projects

2-2 Forecast and Actual Values of Factors Influencing Rail Transit Ridership

3-l Forecast and Actual Capital Outlays for Rail Transit Projects

J-2 Scope Changes and Errors in Financial Planning for Rail

3-3

Transit Projects

Estimated Contributions of Errors in Financial Planning to Capital Cost Overruns for Rail Projects

3-4 Forecast and Actual Financing of Rail Project Capital

24-25

Outlays by Level of Government

4-1 Forecast and Actual Operating Expenses for Rail Transit Projects

4-2 Forecast and Actual Impacts of Rail Projects on Systemwide Transit Operating Expenses

5-l Forecast and Actual Cost per Passenger for Recent Rail Transit Projects

5-2 Forecast and Actual Cost per New Transit Trip for Recent Rail Transit Projects

6-1 Summary of Percentage Forecasting Errors for Recent Rail Transit Projects vii/viii

SUMMARY

Under a variety of grant programs administered by the Urban Mass Transportation Administration (UMTA), the federal government has contributed nearly $12 billion to support U.S. cities' investments in new rail transit facilities. This study evaluates the ridership and cost forecasts that led local officials to select ten rail transit projects that have been constructed with federal financial assistance during the past two decades, by comparing those forecasts to each project's actual costs and ridership. The forecast data employed in making these comparisons were originally reported in published planning documents prepared for each project, while actual data were drawn from a combination of published sources, internal documents provided by local agencies involved in constructing and operating these projects, and direct contacts with employees of those agencies.

Although different forecasts were prepared at varying stages during the planning process for many of the projects examined, this study focuses upon the accuracy of projections that were available to local decision-makers at the time the choice amon alternative transit im rovement ro'ects was actU a y ma e. ecause orecasts o n ers 1p an costs prepare or a spect tc project after it was designated as the locally preferred aJternative cannot have influenced its choice from among competing alternatives, the accuracy of these post-decision forecasts is not a focus of this study.

The study also attempts to identify causes of the divergence between forecast and actual performance of these projects, and makes specific recom mendations intended to improve the accuracy of forecasts prepared for future projects. Its purpose is thus to improve the process currently used to plan and evaluate major transit capital investments, by recommending measures to increase the reliability of information available to local decision-makers when they compare and choose among alternative projects.

The differences between forecast and actual values of transit ridership and costs have important implications for the reliability of the process cur rently used to develop and choose among alternative transit projects:

(I) Actual ridership that differs significantly from its forecast level indi cates that a project's benefits are also likely to vary from the expected level that led to its selection from among the alternatives under study.

IX

(2) Capital costs that differ markedly from their anticipated level can su bstantially increase the financial burden on the government program and agency funding the project, resulting in postponement or cancella tion of other projects competing for its support.

(3) Similarly. operating expenses that exceed their projected level can increase operating deficits or require reductions in the level of other transit services that an agency can operate within its budget.

(4) If the divergence between a project's forecast and actual cost-effective ness in attracting new transit passengers exceeds the margin by which the chosen alternative was preferred to others that were rejected, the planning process may not have led to selection of the most desirable project.

Thus, an important objective is to ensure reasonable accuracy of fore casts prepared to support future choices among alternative transit improve ment projects. To help attain this objective, it is useful to- examine why the actual costs and ridership experienced by past projects diverge so markedly from their forecast values.

RIDERSHIP FORECASTS

Table S-1 compares forecast and actual passenger boardin~s on each of the ten new rail transit projects reviewed as part of this study. As it indi cates, only the extensive rail rapid transit system under construction in Washington, O.C. experiences actual patronage that is more than half of that forecast, and even there ridership remains 28% below that originally antici pated. The number of passengers carried by new rail lines in Baltimore and Portland is somewhat below half of that forecast, while actual ridership on Miami's Metrorail line, as well as on the light rail lines recently completed in Buffalo. Pittsburgh, and Sacramento ranges from 66% to 85% below its forecast levels. Similarly, the two downtown people movers constructed in Miami and Detroit carry 74% and 83% fewer daily passengers than were origi nally anticipated to use them. 2 The consistent over-estimation of future ridership on recent rail transit projects suggests that, with few exceptions, the levels of travel and related benefits they currently provide are far below those originally anticipated by the local decision-makers who selected these projects.

The only forecast of rail ridership available .for the Atlanta rail system applies to a much more extensive system than is presently in operation, and was not expected to be reached until it had been in operation for nearly 15 years. Therefore Table S-1 presents no .forecast ~f rail transit ridership for Atlanta.

The dates to which ridership .forecasts prepared .for light rail lines in Portland. Buffalo, and Sacramenta apply have not yet been reached. Forecasts ~f ridersftip and other variables prepared for these projects applied to the years 1990. /995, and 2000, respectively.

X

Table S-1.

FORECAST AND ACTUAL COST PER PASSENGER

FOR RECENT RAIL TRANSIT PROJECTS

Heavy Rail Transit Projects Light Rail Transit Projects DPM Projects Wash- Bait- · Pitts- Port- Sacra-ington Atlanta imore Miami Buffalo burgh land mento Miami Detroit

Weekday Rail Passengers (thousands)

Forecast 569.6 NF 103.0 239.9 92.0 90.5 42.5 50.0 41.0 67.7 Actual 411.6 184.5 42.6 35.4 29.2 30.6 19.7 14.4 10.8 11.3 % difference -28% -59% -85% -68% -66% -54% -71% -74% -83%

Rail Project Capital Cost (millions of 1988 doUars)

Forecast 4,352 1,723 804 1,008 478 699 172 165 84 144 Actual 7,968 2,720 1,289 1,341 722 622 266 188 175 215 % difference 83% 58% 60% 33% 51% -11% 55% 13% 106% 50%

Annual Rail Operating Expense (millions of 1988 dollars)

Forecast 66.3 13.2 NF 26.5 10.4 NF 3.8 7.7 2.5 7.4 Actual 199.9 40.3 21.7 37.5 11.6 8.1 5.8 6.9 4.6 10.9 % difference 202% 205% 42% 12% 45% -10% 84% 47%

Total Cost per Rail Passenger I (1988 dollars)

Forecast 3.04 NF NF 1.73 2.15 NF 1.68 1.53 0.90 1.14 Actual 8.75 5.93 12.92 16.77 10.57 7.94 5.19 6.53 7. 1 l 10.21 % difference 188% 872% 392% 209% 328% 693% 795%

Annual total cost of rail service divided by annual equivalenf of "Weekday Rail Passengers," computed using numbers of average weekday equivalents per year derived from annual total and average weekday rail ridership reported by project operators. Annual total cost of rail service is the sum of (I) the annualized value of "Rail Project Capital Cost," computed assuming a 40-year project l~fetime and a discount rate of 10% per year, and (2) "Annual Rail Operating Expense."

NF indicates no fat·ecast of a data item was obtainable from published sources.

xi

Calculations using measures of the sensitivity of transit ridership to demographic variables, transit service levels, and automobile costs -- the major inputs into ridership forecasts prepared for these rail projects -- sug gest that errors in projecting their future values typically explain less than half of the observed gap between predicted and actual rail ridership.· Overly optimistic assumptions about the frequency and speed of service that new rail lines would provide, as well as about the quality of bus feeder service on which these lines rely to generate much of their ridership, made the largest contributions to the over-estimation of their future ridership levels.

Even where a significant fraction of the difference between projected and actual rail ridership can be explained by errors in forecasting these inputs, these differences were usually so large that a substantial absolute difference remained unexplained. This suggests that important errors must have arisen from other, less obvious sources, including the structure of the ridership forecasting models, the way in which they were applied, or the misinterpretation of their numerical outputs during the planning process.

CAPITAL COST FORECASTS

Table S-1 also compares these projects' forecast and actual capital outlays, which include costs for acquiring rights-of-way, constructing fixed facilities, and purchasing vehicles. (Both forecast and actual outlays reported in the table are expressed in 1988 dollars). As these comparisons indicate.

capital outlays for Pittsburgh's South Hills light rail reconstruction project were actually J J percent below their forecast value, while cost overruns on other projects ranged from 13% for Sacramento's recently completed light rail line to 106 percent for Miami's downtown "Metromover" project.

Because changes in the physical characteristics of these projects between their planning and construction stages were generally quite minor, and be cause many of the changes that were made should have reduced rather than increased capital outlays. it appears that very little of the substantial cost overruns experienced in building most of these projects can be ascribed to major design changes. Instead, cost escalation must have been the product of many smaller changes in the physical design of facilities or the standards for their performance, no one of which was extremely costly but the cumulative effect of which was often a substantial increase in construction outlays.

Federal assistance under a variety of funding mechanisms ranged from 53 percent to 83 percent of capital outlays for the ten projects reviewed.

Actual federal outlays for these ten projects totaled about $7. 1 billion, al though the equivalent of this figure in 1988 dollars would be considerably higher because much of it was denominated in the higher-valued dollars of previous years. Financing of the remaining share of capital outlays varied widely among these projects, although in nearly every case, either state or local government financed most of the non-federal share of project costs.

(Only one project received substantial support from both state and local government.) Local financial contributions amounted to~ or less of both

XII

planned and actual capital outlays for five of the ten projects studied (those in Baltimore, Buffalo, Pittsburgh, Portland, and Detroit), a surprisingly modest level of local government support considering the highly localized nature of the benefits from transit projects.

Until recently, the federal treasury assumed most of the burden of financing the large cost overruns that have characterized many of these projects. The federal government financed three-quarters or more of the cost overruns experienced by six of the ten projects, including four of the five on which these overruns exceeded the hundred-million dollar mark. In fact, the financing mechanisms originally incorporated in federal capital assistance programs placed on the federal treasury a share of the burden from cost overruns that often substantially exceeded its originally planned share of the project's total cost.

More recently, the declining federal share of cost overruns on rail pro jects appears to indicate that the use of "full-funding" agreements -- which limit the federal government's dollar contribution to a project -- has trans ferred much of the financial risk of cost overruns from the federal govern ment to state and local agencies responsible for selecting these projects. The effectiveness of such arrangements in controlling cost escalation is limited, however, by the fact that they are not entered into until well after the local choice among projects has been made, by which time the estimated cost of constructing the preferred project has often escalated considerably from the forecast on which its selection was based.

OPERATING COST FORECASTS

Table S-1 also compares the forecast and actual values of annual operat ing expenses for rail service provided by each of the ten projects (again, these are expressed in 1988 dollars to remove the effect of errors in fore casting inflation). As the table shows, actual rail operating expenses are above those forecast in all but one case for which this comparison can be made: actual operating expenses for Sacramenta's light rail line are 10% below those forecast, but elsewhere, actual expenses range from 12% to more than 200% above their projected levels.3 Further, except for the two downtown people mover systems, the actual expenses reported in Table S-1 understate the full costs of operating rail service, because they omit the costs of oper ating the networks of feeder bus routes on which these projects rely to generate much of their ridership.

While actual operating expenses would be expected to exceed those forecast if the level of rail service actually provided is higher than that

Operating expenses for Sacramenta's light rail line during 1988 were I 0 percent below the level project to be reached by the year 2000; however, the number of vehicle-miles of service operated during 1988 was 45 percent below the number expected to be operated during the year 2000.

XIII

originally anticipated, this is the case only in Atlanta and Portland. Else where, actual vehicle-miles of service are more typically only one-third to one-half of those originally planned. Instead, the substantial differences between most of these projects' forecast and actual operating .. expenses appear to stem from a combination of lower tabor productivity in rail operations and higher compensation rates than were originally anticipated, together with vehicle operating speeds that are considerably slower than those projected during these projects' planning stages.

COST PER RAIL PASSENGER

Finally, Table S-1 shows that forecasts of total cost per rail rider for seven of the ten rail projects that were studied ranged from slightly under $1.00 to more than $3.00 (when measured in 1988 dollars). However, actual costs per rail passenger carried by these ten projects ranged from somewhat more than $5.00 to nearly $17.00, or from 188% to nearly 900% higher than their corresponding forecast values. The weighted average forecast cost per rail passenger for the seven projects where this forecast could be computed was $2.35, while the weighted average of actual costs per rail rider for all ten projects was $8.56, or 264% percent higher than this forecast average.

Errors in projecting this cost-effectiveness index are a composite of previously documented errors in forecasting each of the three variables that enter into its calculation: rail ridership, project capital outlays, and rail operating expenses. Because no project achieved actual values of ridership, capital outlays, and operating expenses that were each close to those fore cast, every project's actual cost per rail passenger was considerably higher than the value implied by forecasts of these variables.

SYSTEM-WIDE IMPACTS OF RAIL TRANSIT INVESTMENTS

Another important aspect of these projects' performance is their effec tiveness in increasing overall transit ridership and controlling operating costs in the urban areas where they were built -- objectives that were commonly cited by local decision-makers when selecting rail projects over competing alternatives. To assess these projects' performance in doing so, Table S-2 compares forecast and actual changes in total transit ridership and operating expenses accompanying each of the eight heavy and light rail projects that were studied. 4

Because the two downtown people-mover projects are so small that their effects on area wide transit ridership are difficult to isolate, Detroit is excluded from Table S-2, while the combined effects of Miami's Metrorail and Metromover projects on transit ridership and costs are included in the table. The procedures used to estimate these projects' forecast and actual impacts on system-wide transit ridership and operating expenses are discussed in detail in Chapters 2 and 4 of this report.

xiv

Forecast Actual % difference

Forecast Actual % difference

Table S-2.

FORECAST AND ACTUAL COST PER NEW TRANSIT TRIP

FOR RECENT RAIL TRANSIT PROJECTS

Heavy Rail Transit Projects Light Rail Transit Projects Wash- Balt- Pitts- Port- Sacra-ington Atlanta imore Miami Buffalo burgh land mento

New Transit Trips per Average Weekday (thousands)

NF 99.1 NF 262.9 81.0 NF 51.3 13.0

281.3 36.9 37.3 -25.8 -0.4 6.3 11.5 -4.2

-63% -78%

Rail Project Capital Cost (millions of 1988 dollars)

4,352 1,723 804 1,092 478 699 172 165 7,968 2,720 I ,289 1 ,516 722 622 266 188

83% 58% 60% 39% 51% -11% 55% 13%

Annual Operating Expense Impact of Rail Project (millions of 1988 dollars)

Forecast NF NF NF 33.9 4.7 NF 0.6 -2.1 Actual 228.5 53.8 14.7 46.0 13.7 1.4 4.6 5.6 % difference 36% 191% 667%

Total Cost per New Transit Trip

(1988 dollars)

Forecast NF NF NF 1.67 2.20 NF 1.14 3.53 Actual 11.97 29.47 13.56 34.64 9.49 % difference 731%

Annual total cost impact of rail project divided by annual equivalent of "New Transit Trips per Aver age Weekday," computed using numbers of average weekday equivalents per year derived from annual total and average weekday total transit ridership reported by project operators. Annual total cost impact of rail project is the sum of (1) the annualized value of "Rail Project Capital Costs," computed assuming a 40-year project lifetime and a discount rate of I 0% per year, and (2) "Annual Operating Expense Impact of Rail Project."

No actual value can be calculated because ridership declined with the introduction of rail service.

NF indicates no forecast of a data item was obtainable from published sources.

XV

As the table indicates, substantial growth in transit ridership accompa nied the introduction of rail setvice in Washington. Atlanta, and Baltimore, although in no case did the increase in ridership approach that forecast.

Increases in transit ridership accompanying the introductioo of light rail service in Pittsburgh and Portland were more modest, particularly by compar ison to the substantial growth in transit use that was forecast to occur in the latter. In contrast, overall transit ridership in Miami. Buffalo. and Sacra mento -- including travel by both bus and rail -- actually declined rather than increased as rail transit setvice was introduced.

Table S-2 also attempts to identify the impact of introducing rail setvice on these cities' costs for providing transit setvice. This impact consists of the costs associated with investments in new rail facilities, together with any change in the total cost of operating the resulting network of bus and rail transit service. As it shows, the actual capital outlays necessary to construct new rail transit facilities were sharply higher than those initially projected in six of the eight cities that chose to make these major invest ments. The table also reports that while some of these projects' actual im pacts on systemwide operating expenses compare fairly closely to those origi nally forecast. the actual effect of inaugurating rail service on systemwide operating expenses has often been a substantial increase from their "pre-rail" level, even where a reduction or only a slight increase in operating expenses was anticipated. 5 Thus, it appears that savings in total transit operating .

expenses, which were often anticipated to result from substituting rail for bus setvice, may have not have been widely realized. 6

Finally, Table S-2 shows that the projected costs per new transit trip implied by these projects' anticipated additions to areawide transit ridership and costs were quite modest, ranging from somewhat more than $1.00 to slightly over $3.50 in the four cases where a forecast could be computed. In contrast, of the five projects for which actual costs per new transit pas senger could be computed, only that in Portland was accompanied by ridership gains at a cost of less than $10.00 per new trip. While the substantial transit ridership increases in Washington and Baltimore were achieved at costs in the $12.00-13.50 per trip range, Atlanta's similarly impressive ridership gain as well as the more modest gain in Pittsburgh were attained at costs in the neighborhood of $30.00 per new trip.

5 The clear exception is Sacramenta, where only a .small fraction of the major increase in areawide transit service -- and thus in operating expenses -- that was originally projected to occur by the year 2000 has been implemellfed to date.

6 This could occur either because replacing bus service with rail se1vice does not actually reduce operating expenses, or because reductions in bus service originally anticipated to accom pany the introduction of rail service were not actually implemented. It is not possible to distin guish between these two explanations for increaied operating expenses on the basis of the infor mation reported in this study.

xvi

CONCLUSIONS AND RECOMMENDATIONS

Implications of Forecasting Errors

It is difficult to judge whether the subs.tantial errors in forecasting ridership and costs for the rail projects reviewed here led decision-makers to select them when more accurate forecasts might have led them to prefer other alternatives, because the accuracy of forecasts prepared for alternatives that were discarded in favor of the chosen rail projects cannot be evaluated.

Nevertheless. it appears that the divergence between forecast and actual cost per rail rider and per new transit trip for these projects was often larger than the entire range of values of these measures over all of the alternatives from which these pfojects were selected. Thus, it is certainly possible that decision-makers acting on more accurate forecasts of costs and future rider ship for the projects reviewed here would have selected projects other than those reviewed here, at least in some cases.

Even if cost and ridership forecasts for each alternative considered were subject to the same sources and degree of inaccuracy as those examined here, the systematic tendency to over-estimate ridership and to under-estimate capital and operating costs introduces a distinct bias toward the selection of capital-intensive transit improvements such as rail lines. This bias arises because. as a variety of studies has shown, rail becomes the economically preferred transit·mode only when its substantial capital costs and fixed operating expenses can be spread over large passenger volumes. 7 Thus, even if cost and ridership forecasts prepared for transit improvement projects entailing investments in different modes or technologies prove to be equally optimistic (that is, to represent equal proportional over-estimates of future ridership and under-estimates of costs), the planning process will still be biased toward selection of the most capital-intensive alternatives under con sideration.

Recognizing the sensitivity of local transportation officials' choices among alternative transit improvement projects to the reliability of their projected future ridership and anticipated costs, it is important that steps be undertaken to improve the accuracy of forecasts prepared to support future transit investment decisions. These steps should include specific technical improvements in the procedures used to develop and check cost and ridership forecasts, some of which have already been at least partially incorporated into UMTA project planning guidelines. They should also include subjecting forecasts to review by outside experts, as well as acknowledging to local officials and the public that the resulting forecasts are still accompanied by considerable uncertainty. ·

The earliest such study is reported in John R. Meyer, John F. Kain, and Martin Wohl, The Urban· Transportation Problem, Cambridge, Massachusetts, Harvard University Press, 1965, Chapters 8-11. Sev_eral subsequent studies arrive at the same conclusion, although for a variety of reasons they di;tfer regarding the exact ridership threshold at which rail becomes the most cost-e,ffective transit mode.

XVII

Technical Improvements in the Accuracy of Forecasts

The errors in ridership and cost forecasts documented in this report are so large that it seems unlikely they would have been eliminated by simple technical changes in the way forecasts are generated. Nonetheless, it should be possible to reduce substantially the magnitude of future errors by combin ing technical improvements in the preparation of forecasts with stronger incentives for local agencies planning these projects to develop more realistic projections of their costs and future ridership. Among the potentially valu able procedural improvements, most of which have already been incorporated into the current UMTA process for planning major transit projects, are the following:

(l) Using a nearer "horizon" year for ridership and operating expense fore casts, as is now strongly encouraged under the UMT A planning process.

Although this study found that errors in forecasting variables that serve as inputs to travel demand forecasting models (demographic and economic variables, transit performance levels, and automobile travel conditions and costs) were not solely responsible for overestimation of future ridership, they did contribute significantly in a few cases. Input as sumptions could be made more accurate by shortening the period between their preparation and the future year to which they apply. This should reduce the number of major developments during the intervening period (often as long as twenty years in current practice) that can cause projections of these input variables to be inaccurate, such as changes in the performance of the local economy, or reorientat\on of travel pat terns in response to changing geographic distributions of jobs and pop ulation.

(2) Developing procedures that allow the effects of individual factors on costs and transit ridership to be isolated and examined separately. One approach would be to prepare "forecasts" using existing or "base year" population, employment, and transportation system characteristics, as if each proposed transit project could be implemented instantaneously.

This could provide a more realistic estimate of the costs and increased ridership that would result from the service characteristics associated with each of the proposed transit improvements, since it would allow the effects on ridership of these changes to be distinguished from those of growth in overall travel demand due simply to ~emographic growth or changes. in other exogenous factors (such as gasoline prices) that affect each transit alternative under consideration.

(3) Conducting s~nsitivity analyses to examine the effects of alternative assumptions affecting cost and ridership projections. This type of analysis can be very helpful both for refining forecasting procedures themselves, as well as for examining the likely effects on costs and ridership of uncertainty regarding such factors as demographic growth, transit service levels, energy prices. future escalation of construction costs, or design and performance characteristics of transit facilities.

xviii

(4) Checking the reasonableness of construction and operating cost fore casts, of ridership forecasts, and of inputs to these forecasts, by com paring them to the record established by previous projects. Significant discrepancies with empirical data for simi1ar projects and urban areas should require either that these discrepancies be convincingly justified, or that revised forecasts be prepared using adjusted procedures or input assumptions.

Subjecting Forecasts to Expert Review

Another potentially effective strategy for establishing the reliability of cost and ridership forecasts, as well as of the assumptions and models used to generate them, is to subject them to review and verification by independent experts. Although such a review of each urban area's forecasting efforts is presently conducted by UMTA staff members, the growing number of projects for which planning is actively underway, together with a reduction in resour ces available for this effort, has reduced the scope and depth of review activities that can be performed.

For example, local agencies responsible for conducting the UMTA plann ing process could be required to designate a peer review panel with respon sibility for assessing the credibility of input assumptions, technical proced ures, and forecast results when they are still subject to review and revision.

The responsibilities and powers of individual members comprising such a panel would need to be clarified prior to their selection, but wider use of such groups offers the potential for bringing valuable judgement and experience to bear in generating reliable information to support local decision-makers c hoices among alternative transit improvement projects.

Once projects have been selected and arrangements for financing their implementation have been completed, independent expertise in activities such as construction management, testing of completed systems, and actual initia tion of service could also be employed by local agencies serving as project sponsors. Such expertise has recently been provided to sponsors of some recent federally-financed rail transit construction projects by Project Manage ment Oversight (PMO) contractors designated by UMTA, who have been retained to provide specific assistance and oversight activities agreed to by UMTA and the local project sponsor. More widespread use of such contract ors, incJuding their participation before problems have arisen with project construction timetables or financing, is thus a potentially useful strategy for bringing independent expertise to bear on project implementation activities.

Acknowledging Uncertainty in Ridership Forecasts

The errors in forecasting ridership and costs identified in this study are so large that they appear unlikely to be eliminated completely by techni cal changes in the procedures for developing and reviewing forecasts. It thus

XIX

seems prudent that both ridership and cost forecasts prepared to support future choices among alternative projects be prepared and presented in a manner that explicitly recognizes the existence of uncertainty about whether their exact values will be achieved. Perhaps most important. this recognition also needs to be conveyed to the local political officials that will ultimately rely on these forecasts to choose among alternative projects, as well as to the more general public.

One obvious way to acknowledge that such uncertainty surrounds even the most carefully prepared and assiduously reviewed projections of ridership would be to report a ran~e of patronage levels that could reasonably be ex pected to result from tmp ementing each project under consideration. While this procedure may slightly complicate the calculation and interpretation of the cost-effectiveness measures local project sponsors are required to prepare.

it will simply represent a formal acknowledgement that the performance of each alternative under consideration cannot be predicted with certainty, and that local officials' selection of a preferred alternative must recognize a variety of other criteria in addition to cost-effectiveness measures.

Increasing Contingency Allowances to Cover Cost Escalation

Recognizing that capital cost estimation and financial planning for major public works projects such as the construction of rail transit lines is an inherently difficult and risky activity, it seems prudent in project budgeting to provide contingency allowances that are adequate to cover capital cost escalation of the magnitude typically experienced by such projects. On the basis of the results reported in this study, it appears that such contingency allowances have been consistently inadequate to allow local project sponsors to absorb unforeseen developments without incurring major increases in their projects' budgets, and should be increased substantially for future projects.

Although it is difficult to specify the exact size of allowance that should be provided in capital budgeting for future transit projects. it does appear that some increase in those historically provided is warranted. The most prudent course would probably be for UMTA to draw upon the experi ence of other major public works projects, in combination with the record established by past major transit capital projects (including those reviewed here), to establish guidelines for the size of contingency allowances in rela tion to foreseeable project expenditures. Even within the scope of major capital grant programs administered by the various other branches of the U .S.

Department of Transportation, there probably exists considerable project budgeting and oversight experience that could be called upon to develop guidelines for more realistic estimation of adequate contingency provisions in budgeting for future federally-supported transit investments.

XX

I. INTRODUCTION

1.1 FEDERAL SUPPORT FOR RAIL TRANSIT INVESTMENTS

Over the past two decades. the federal government has contributed nearly $12 billion in support of U .S. cities' investments in new rail transit facilities and vehicles. through a variety of grant programs administered by the Urban Mass Transpot1ation Administration (UMTA). Grants for new rail transit construction thus have represented nearly a third of the federal government's $37 billion in total capital assistance to the nation's urban transit industry. 1 As this substantial financial commitment illustrates. support for major new investments in rail transit service has served as one of the cornerstones of federal urban transportation policy.

During this period. UMTA has developed an increasingly formalized and rigorous planning process to be used by localities in designing, evaluating, and selecting among alternative transit capital projects. The intent of this pro cess is to ensure cost-effective decisions at each stage of project develop ment, and its use -- illustrated schematically in Figure 1-1 -- is required as a condition of locally chosen projects' subsequent eligibility for federal financial assistance. Federal financial support is also available for conducting the detailed studies required at each stage; detailed review of these studies and formal consent by U MTA is required for a local agency to progress to each subsequent stage.

1.2 ASSESSING THE PLANNING PROCESS

Although not a11 rail transit projects constructed with federal financial assistance are products of a planning process as formalized as that currently in place, most have emerged from a selection procedure with important simi larities to the present U MTA process. This procedure entails the design of a variety of alternative possible transit improvement projects. and evaluation of these alternatives on the basis of their forecast costs and performance in meeting state and local transportation objectives. The preferred project is

77tis estimmc is compiled fi"om information reported in Urban Mass Transportation Ad ministration, "Grants A~~istancc Programs: Statistical Summaries." 1988, Tables B, 15, 17, and 20.

Cl Oenotes local a.:tivitiel funded by UMTA

Figure 1-l.

THE UMTA PROJECT DEVELOPMENT PROCESS

1. System Planning

2. Alternatives Analysis/ Draft EIS

3. Preliminary Engtneering Final EIS

5. Const1uction

Oeno• UMTA decision then selected by local decision-makers in a process that weighs each alterna tive's projected benefits against its projected costs. While the formality of the evaluation procedures leading to the selection of recent rail construction projects has varied considerably, the underlying logic of different local pro ject selection processes was remarkably consistent even before UMTA devel oped specific, formal project selection procedures and required their use as a condition for receiving federal financial assistance.

This study examines one critical element of the process by which recent federally funded transit capital projects have been evaluated and select~d. It assesses the accuracy of certain forecasts that contributed to the designation of a specific alternative as the locally preferred project. Specifically, for each selected alternative, it compares projected and subsequent actual values of three critical variables: ridership, capital costs, and operating and maint enance costs. Unfortunately. it is impossible to perform similar comparisons for the rejected alternatives -- a step that would be necessary to fully evaluate the overall accuracy of the entire forecasting process -- for the obvious reason that no actual information is available for projects that were never built.

1.3 THE DEFINITION OF "FORECAST" DATA

Although different forecasts were prepared at varying stages during the planning process for many of the projects examined, this study focuses upon the accuracy of projections that were available to local decision-makers at the time the choice among alternative transit improvement pro~ects was actually made.~ In the context of the present planning process, t is refers to forecasts prepared as part of the Alternatives Analysis stage. The major published product of this stage typically is a Draft Environmental Impact Statement (DEIS) that compares forecast cost and ridership, as well as en vironmental, community, and other projected impacts, for a variety of alterna tive transit improvement projects.3

Some local transit planners have expressed the view that the absolute accuracy of fore casts prepared for any single alternative (even the subsequently chosen one) at this stage of the planning process is irrelevalll, since they are intended only to facilitate local decision-makers' comparison and choice among alternatives. Hence, they contend that these forecasts are only required to be of roughly equal accuracy for each altemative considered in order for an unbiased choice among alternatives to be made. However, the systematic tendency of the planning process to overestimate ridership and underestimate costs for proposed rail transit projects, documented in detail in subsequent sections of this report, biases the decision process toward high-capital alternatives such as rail transit. Thus, even if forecasts of cost and ridership incorporate rough ly equal errors for each alternative under study, a11 unbiased choice among alternative projects of different capital intensity and with varying ridership levels cannot result.

One exception is the forecasts of nominal-dollar capital outlays analyzed in Chapter 3, which are typically prepared as part of the Preliminwy Engineering stage of the planning process (see Figure 1-1). 11lese forecasts are generally not reported until publication of the Final En-

Forecasts of ridership and costs for a specific project that were pre pared after it was designated as the locally preferred alternative are fr equentTYClted in public discussions and press accounts. Not surprisingly.

these post-decision forecasts often have proven to be more accurate than those available to decision-makers when multiple alternatives were still under consideration. This is due both to the shorter time span between preparation of the forecasts and the future date to which they applied (sometimes as short as a few months). and to the greater scrutiny afforded a single alterna tive after it was designated as the locally preferred one. However. because such forecasts cannot have influenced the choice of a preferred project from among competing alternatives. their accuracy is not a focus of this study.

As an illustration. the Draft Environmental Impact Statement prepared for Portland's Banfield corridor was released in March. 1978. providing a detailed comparison of ridership. capital and operating costs. and other pro jected impacts for eleven alternative transit improvements in the corridor.

By November of that year. each of the four responsible local jurisdictions had voted unanimously to select light rail transit in the Banfield/Burnside Street alignment as its preferred alternative. 4 As indicated in the subsequent Final Environmental lm~act Statement (FEIS) prepared for the project. "Data con tained in the DEI ... provided the basis for selection of the preferred alterna tive by the jurisdictions." [emphasis added f

By the August, 1980 release of the FEIS, however. the estimated con struction cost for the transit elements of the joint transit-highway project had increased from the original $172 million to $210 million (both estimates are expressed in 1988 dollars to eliminate the effect of inflation between their publication dates), or by 22% from the forecast on which the responsible local jurisdictions had based their selections. At the same time. the forecast of annual operating expenses for the project had been raised from $3.8 million to $7.0 million (both figures are again expressed in 1988 dollars), or by 84%.

while projected ridership had been revised downward from 42.500 to 30.800 daily passengers, a reduction of 28% from the level on which decision-makers representing each of the four local jurisdictions had previously acted. Fur-vironmental Impact Statement (FEIS) or submission. or an application for federal .fzmding.

These were Tri-Met. the agency responsible .{(1r building and subsequently for operaTing the project, Multnomah County. and the cities or l'ortland and Grcsham: sec Federal Highway Administration and Urban Mass Transportation Administration. Banficld Transitway Project: Final Environmental Impact Statement. August 1980. pp. 2-51 and 2-52.

Banfic!d Transitway Project: Final Environmental Impact Statement. p. 2-53.

ther. the forecast of ridership was subsequently revised downward to 18,100 daily passengers. 6

Although each of these revised forecasts ultimately proved to be more accurate than that on the basis of which the light rail project was selected, their publication occurred too late in time to cause local decision-makers to seriously reconsider their choices. The only recourse available to local offi cials would have been to require the preparation of similarly revised forecasts for the ten other alternatives that had previously been discarded, in effect returning the planning process to the Alternatives Analysis phase (see Figure 1-1). This is obviously an unattractive option for local planners and political officials. particularly in view of the substantial time and other resources expended in moving the process forward from that stage. Recognizing its undesirability. this study emphasizes the accuracy of forecasts that were available to local decision-makers at the time they chose among alternative projects, and recommends measures to improve the reliability of cost and ridership forecasts developed to support future choices among projects.

1.4 WHY REVIEW PAST FORECASTS?

Examining the divergence between forecast and actual values of transit ridership and costs provides an important assessment of the project develop ment process. First. actual ridership that differs significantly from its fore cast level indicates that the project's benefits vary from the expected level that led to its selection from among the alternatives under study. Second, project costs that differ markedly from their anticipated level can substan tially increase the financial burden on the government program or agency funding the project. resulting in the postponement or exclusion of other projects competing for that program or agency's support. If as a result of such errors the divergence between a project's forecast and actual cost effectiveness in attracting new transit passengers exceeds the margin by which the chosen alternative was preferred to others that were rejected, the planning process may not have led to selection of the most desirable project.

A detailed examination of cost and ridership forecasts for past transit lhese figures are reported in l1arsons Rrinckerho.ff Quade & Douglas blC. and Louis T.

Klauder & Associates. Capital Cost Estimates/Operations & Maintenance Costs, Technical Memo randum No. I 0, July 1980. Figure 2, p. 30. Figure 3. p. 31 (original and revised construction cost forecasts). and p. 44 (revised opaating cost and ridership forecasts); Tri-Met, East Side Transit Opera/ions. December 1977, Table 5. p. 3 7 (original ridcrship forecast), and Table 6, p. 40 (original operating cost .forecast); and "Ban.field LRT Patronage projections," Tri-Met lntemal Memorandum, undated, pp. I and 4. . All dollar .figures were converted to their 1988 dollar equivalents using the change in the Implicit Price De.flator .for Gross National Product between the year of their original publication and 1988, reported in U.S. Department o.f Commerce, Survey of Current Business, various issues.

projects should also help to improve the accuracy of forecasts prepared to support future choices among alternative projects. thus fostering better urban transportation investment decisions. Recurring errors in projecting specific variables may signal the need to develop improved procedure·S for forecasting their future levels, to improve the accuracy of inputs that are relied upon to generate those forecasts (such as assumptions about future demographic trends), or to examine the sensitivity of planning decisions to a range of possible future values of variables that remain chronically difficult to forecast accurately. Because each of these potential means for improving the accuracy of cost and ridership forecasts is both complex and costly. the division of effort among them should be guided by an awareness of where those efforts are likely to prove most productive. A review of past forecasting errors should provide an important contribution to developing such an awareness.

1.5 TYPES OF FORECASTS EXAMINED

This study reports comparisons between forecast and actual values for four types of measures.

(l) Ridership --Recognizing the critical role of utilization in determining the transportation and other benefits resulting from the substantial capital investments represented by these projects. the study focuses first on how their forecast and actual ridership levels compare, and investi gates why they differ.

(2) Capital costs and financing -- Next. the study compares actual capital outlays for each project to forecasts of their magnitude developed during the planning process.