♦
By far, perhaps the most controversial and eyebrow-raising element of Project Connect’s self-styled “study” to select a route for “high-capacity transit” has been the heavy use of projections. Crucial data inputs involved projections of population, employment, economic activity, travel, and congestion, mostly for the year 2030, 17 years into the future.
All of these, plus some current data, appear in a large Excel-based “Sub-Corridor Comparison Matrix” which forms the basic data foundation for justifying Project Connect’s route (“sub-corridor”) decision — the “Highland” and “ERC” sectors — which, amazingly, happen to be almost exactly the very route profile local officials wanted in the first place. (See: Surprise! Mayor and Project Connect select same routes they wanted in the first place.)
As Dan Keshet, a local software developer and research analyst, has shown, Project Connect’s own evaluation calculations have relied overwhelmingly on these questionable projections of key mobility-related factors rather than current factual data. (See, e.g., A first glance at Project Connect data.)
Similarly, Julio Gonzalez Altamirano, an Austin management consultant, has conducted his own analysis of Project Connect’s selection of the “Highland” sector (“sub-corridor”):
Highland Score
On the basis of his quite comprehensive analysis, Julio concludes that “it is clear that Highland’s score is the result of speculative assumptions about the nature of development in the sub-corridor.” His explanation seems to imply that, because planners would like to see certain types of development in certain areas, they just go ahead, assume it will happen, and voila! these become the projected inputs in the “evaluation” matrix.
Julio summarizes some of the key problems with Project Connect’s “projections” this way:
The bottom-line is that the methodology awards points to larger parcels that are relatively undeveloped and (seemingly?) car-dependent based on an optimistic outlook on future land-use variances being granted. It is not a prediction of how parcels will behave in the future given what similarly-zoned parcels have done in the past. It is not a sophisticated statistical model of what is likely to be that uses historical data to forecast. Instead, it is scoring based on where the methodology-designers would like development to go that uses a proven statistical technique to classify the parcels into different bins based on parcel attributes. The methodology has not been validated, so we don’t know if it actually predicts anything.
The public reasoning presented to CCAG by the PC team for Highland’s score is ‘significant future development’. Obviously, projects such as the ACC Highland Mall redevelopment and Austin’s evolving medical scene will have an impact on employment and residential density in the sub-corridor. But it also appears that the methodology selected assigns significant points to the type of parcels available in the sub-corridor. Needless, to say, the presence of certain types of parcels we’d like to see developed in a sub-corridor does not mean development will happen. Worse, it doesn’t mean they exist in a contiguous stretch that make sense for a transit route.
Another way of saying this is that Project Connect’s planners have converted their own wishful thinking into actual data inputs, that are then deployed to make their evaluation. Wishes are used to try to make the wishes come true.
In an effort to deflect criticism, Project Connect and some of its backers have mounted a “straw man” defense, trying to portray critics as absurdly renouncing the role of projections in planning major projects. But there’s a vast distinction between developing conservative, reliable, and plausible forecasts, and producing exaggerated, fantasy-like projections, as Project Connect has done, heavily skewed toward certain geographical areas.
The question of projections has been an extremely contentious issue in Project Connect’s urban rail “study”. For many critics, the agency’s “projections” have represented de facto fantasies about what they would like to see, rather than the solidly reliable output of competent predictive analytics.
So, where do the “2.9 million daily trips in East Riverside” come in? This is the result of plugging Project Connect’s demographic and economic projections for 2030 into their own transit ridership prediction model.
Apparently in an attempt at a gesture toward some kind of prediction of future transit ridership, one of the metrics Project Connect decided to use in their Comparison Matrix is a “Transit Orientation Index” (TOI), a ridership demand assessment model developed in 1997 by consultants for Portland, Oregon’s TriMet transit agency. To explain their use of the Transit Orientation Index (TOI), Project Connect has posted some documentary material:
The TOI model evidently was developed in conjunction with travel demand planning in Portland for the target year of 2015. The documentation describes its intent as “an effort to quantify the dependence of transit demand on land use variables …. The purpose is to develop a means of assessing transit demand for transportation zones in 2015, based on land use projections.”
The documentation also makes clear that the ultimate objective is to render a prediction of daily transit trips: “For the purposes of this assessment, the transit demand in a Regional Transportation Zone (RTZ) is represented by weekday transit trip productions per acre….”
According to the documentation, the TOI metric is envisioned to assess transit ridership demand at the level of a small analysis zone (Regional Transportation Zone), apparently a travel analysis zone (TAZ, described in the ARN posting Project Connect’s “corridor” study — without corridors!) Project Connect planners, however, have applied the model to considerably larger sectors covering several square miles with hundreds and even thousands of acres.
Whether it’s a small zone or a much larger area such as one of Project Connect’s study sectors (“sub-corridors”), the ultimate output of the model is (1) a “Transit Orientation” index (TOI) which, applied to the acreage in the zone for which it’s calculated, renders (2) both a low-end and high-end ridership prediction. Simply put, the TOI is calculated from three basic inputs:
• Household density (households per acre)
• Retail employment density (retail employment/acre)
• Total employment density (total employment/acre)
These are related in a formula, with various coefficients, analytical relationships, etc., developed by the Portland researchers, to calculate the TOI, which is then intended to predict daily transit trips for zones or sectors (based on trips per acre). Incidentally, it’s worth noting that the TOI was designed to predict ridership potential for an entire urban network, not to evaluate discrete corridors or disparate areas of a city in a competition for a major transit investment.
For Austin’s near-current (2010) data, the TOI model yields ridership levels that seem fairly plausible and reasonably close to actual experience. But when Project Connect’s own long-term future projections (i.e., fantasies) are plugged in — into its own model — the resulting daily transit ridership predictions (for both the “low” and “high” ends of the prediction range) are nothing short of jaw-dropping.
Much of the controversy has focused on the ridership predictions for the two sectors chosen by Project Connect (and embraced by both its “Central Corridor Advisory Group”, CCAG, and, most recently, the Austin City Council) — “ERC” (associated loosely with the East Riverside Drive travel corridor) and “Highland” (a heavily gerrymandered sector nestled next to I-35 north of the UT campus, then ballooning at its north end to encompass the Highland ACC site).
The “ERC” ridership predictions for 2030 have been targets of especial ridicule. These are shown in the snippet below from the Project Connect “evaluation” matrix.
Snippet from Project Connect’s “evaluation” matrix shows implausibly high year-2030 daily ridership projections, both low and high, for the “ERC” sector.
The high-end daily ridership prediction of 2.9 million, it’s been pointed out, roughly equals the combined daily transit system ridership of the cities of Chicago and Philadelphia.
However, Project Connect’s planners called Foul on this figure. For the top TOI index achieved by this sector, they point out that the TOI model yields an “infinite” multiplier to calculate the high-end ridership prediction. So Project Connect, with an index result predicting “infinitely” high ridership, decided to bring it down somewhat closer to Planet Earth’s reality by substituting an upper-bound value of 999 as the acreage multiplier to calculate ridership … and produced the “2.9 million” figure.
However, the low-end figure — daily ridership of 492,682 (493K) — is equally preposterous, exceeding the total system daily ridership of entire large cities. For example:
• Portland, 332K
• Seattle, 476K
• Denver, 324K
• Atlanta, 453K
Remember — 493K is the low-end predicted ridership for one single small segment of the Austin area. Currently, Capital Metro’s daily ridership for the entire service area is about 120K.
For the “Highland” sector, the TOI model results for 2030 are similarly off the scale. Whereas current 2013 ridership is about 5K (5100/day), the “low” TOI prediction for 2030 is about 127K — an increase of 2,440%. The “high” prediction (no need for upper-bound substitution in this case) is 279K — a predicted increase of 5,480%.
Put another way, to meet the lower-end ridership suggested by the demographic and economic projections, average daily ridership in the “Highland” sector would have to exhibit sustained average daily ridership growth of about 7,200 each year for 17 years.
To most reasonable analysts concerned with the accuracy of their forecasts, such wildly implausible predictions would surely prompt a re-evaluation of the basic data inputs — i.e., the projections. But not Project Connect’s planners, who have simply hunkered down and circled the wagons around the inviolability of their projection methodology.
Project Connect personnel, and at least one City Councilman, have attempted to disparage this criticism by noting that the actual ridership predictions have not been used as a metric in Project Connect’s “evaluation” matrix.
But that sidesteps the main issue! First, Project Connect’s matrix does use the TOI, itself based on the same dubious projection inputs, to render a metric score to bolster their preferred sectors (“sub-corridors”) in the competition they’ve set up. For the 2030 metric it gives a score of 1.00 to the “ERC” sector, and 0.65 to “Lamar”, in a points system where every hundredth of a point does count.
But, even more importantly, the TOI for 2030, dependent as it is upon Project Connect’s “projections” (de facto fantasies), exposes their absurdity. No wonder Project Connect and its entourage don’t want these used … no wonder they attempt to distance themselves from them!
It’s very simple — plug Project Connect’s own projections into this otherwise fairly realistic model, and you get bizarrely, unbelievably exaggerated results. Maybe a hint that the original projections are bizarrely unbelievable?
In effect, the TOI is performing here somewhat like a “canary in the coalmine” — telling Project Connect, and all of us, that something is terribly wrong with their demographic and economic projections for 2030.
Obviously, it’s a message that Project Connect, and a certain segment of Austin’s civic leadership, don’t want to hear.
_______________Austin Rail Now 