Operational Analytics Keeps Bay Area Trains on Track

Whether you're heading in from San Francisco International Airport or out to downtown Berkeley, if you're taking rapid transit in the Bay Area then you're in for an analytics ride -- one that starts the moment you pass through the fare collection gate.

That's because on-time train service is the most important issue to Bay Area Rapid Transit's 370,000 daily riders, as one customer satisfaction survey after another shows, according to Roy Henrichs, manager of reliability engineering for the San Francisco Bay Area Rapid Transit District. And the transit agency can't possibly know if its trains are running on time and patrons arriving at their destinations as expected if it doesn't engage in some rather sophisticated operational analytics.

"You can do management by walking around -- but it doesn't work. You obviously need analytics," Henrichs shared with me during a recent phone interview.

His team conducts numerous types of operational analytics, including system performance analysis, passenger flow modeling (PFM), and delay analysis, as well as a variety of other modeling used for forecasting and what-not, he told me. And the data required for the analytics is complex and voluminous.

BART's data infrastructure comprises an IBM Maximo asset management system on an Oracle database/Linux platform. BART relies on code from SAS, this site's sponsor, to move the data around and do the analytics, Henrichs says, noting that the code was easily ported from the agency's previous mainframe-based environment .

With on-time service top priority, PFM is one of the most critical applications running in its environment. Using time series analysis, combined with econometric data, the team runs models aimed at forecasting ridership. The goal is making sure train schedules are optimized to keep customer satisfaction high while curbing BART's costs. "We don't want to run trains that are under- or overloaded… and the PFM, among other things, captures or estimates train loadings for use in generating the train schedules."

Henrichs's team can find out if a train that's supposed to arrive at Embarcadero Station at 10:17 a.m. actually showed up early, on time, or late. And because it also has data coming from its fare gates, it can monitor the motion of passengers through the system, too, using patron ID numbers. "What this department wants to know is whether Joe Citizen, who got on at Berkeley Station at 9:15 a.m. and off at Embarcadero Station at 10:45 a.m., arrived on time or not... This is not a Big Brotherish kind of thing."

Rather, he said, it allows BART to address the three stages a passenger goes through when arriving at a station. "You arrive at the station, and the first question you ask is, 'Where's my train?' Then you ask, 'Where's my seat?' And finally, 'Will I be on time?' "

With its operational analytics, BART is able to get to a "Yes" answer more often than not and keep satisfaction high.

Beth Schultz, Editor in Chief

Beth Schultz has more than two decades of experience as an IT writer and editor.  Most recently, she brought her expertise to bear writing thought-provoking editorial and marketing materials on a variety of technology topics for leading IT publications and industry players.  Previously, she oversaw multimedia content development, writing and editing for special feature packages at Network World. In particular, she focused on advanced IT technology and its impact on business users and in so doing became a thought leader on the revolutionary changes remaking the corporate datacenter and enterprise IT architecture. Beth has a keen ability to identify business and technology trends, developing expertise through in-depth analysis and early adopter case studies. Over the years, she has earned more than a dozen national and regional editorial excellence awards for special issues from American Business Media, American Society of Business Press Editors, Folio.net, and others.

Midmarket Companies: Bring on the Big Data

The "big" in big data is no reflection of the size of the organization embracing its potential.

Push Yourself to New Analytical Discoveries

Take inspiration from Christopher Columbus as you pursue your analytical journeys.

Re: Reliable BART
  • 5/18/2012 8:08:34 PM

This is a really interesting and practical use of operational analytics.  Planes, Trains, Buses...all types of schedule dependent transportation can benefit from this type of analytics. I really like how BART put themselves into the shoes of the customer and this helps them to narrow scope and address real needs.

Very impressive.

Re: Reliable BART
  • 5/17/2012 5:46:05 PM

Lyndon, funny, I hung up with Roy Henrichs just a bit ago and among some points of clarification one of the things he wanted to stress is the criticality of beating the pavement, as it were. All the automation is fine, but managers, etc., still have to walk the stations, assess situations, and check out trains & other equipment as well. One shouldn't exist without the other.

Re: Reliable BART
  • 5/17/2012 4:48:58 PM


Beth writes

 ...thanks for all this great insight into transit systems and how/why some are more capable of enabling operational analytics and automated operations than others.

I'm not a great fan of automation of transit service (I'd like to see a lot more real human interface) ... but I do see the potential for a greatly expanded role for analytics in all areas (and modes) of service. 

However, as I was suggesting in today's E-chat with Bill Franks on Big Data, I see a major obstacle to be the cost of the data-gathering technology that is needed, such as APCs (automatic passenger counters) and NextBus-PID (passenger information display) systems. Closely related here is GIS-based AVL (automatic vehicle location) systems...

Ching! Ching! Ching! That's the sound of the cash register ringing up the pricetag for all that APC, NextBus, PID, GIS, AVL, etc. hardware and software that's needed to make all this data-gathering and passenger benefit happen. Transit agencies are particularly cost-strapped right now, so this is a huge hill to climb.

But I'm a believer, I do think this hill can be climbed, and I'm professionally turning more of my own focus on the issues involved.


Re: Analytics Enhancements would increase ridership
  • 5/17/2012 4:16:31 PM


Maryam writes

For me I can't truly type on virtual keyboards I use one finger and make lots of mistakes that need correction, I wish all devices had the option of the Bluetooth keyboard especially my kindle, my efficiency level would multiply many fold. None of my mobile devices can offer the speed of typing I enjoy on a latop or keyboard.

This is off-topic for the transit issue, but I just had to comment on this because it struck a chord.

From what I've seen, tablets and similar devices offering "virtual" keyboards are designed for receiving, not outputting. I strongly agree with Maryam on this. 

I'm sure this will vary somewhat by individual user, but I'd be driven nuts trying to write and create and manipulate graphics without a keyboard. I note that the friends and colleagues I know who do use these devices respond to me with extremely short, rather cryptic messages. One will write longer messages when he's on his PC or laptop.

I'd really wonder whether any truly creative, highly productive users rely on tablets.

The current techno-fad among device makers is to churn out shiploads of tablets. Netbooks (my own preferred mobile device) seem to be becoming extinct. I'm hoping that the market will prompt a re-thinking of this tablet craze and resuscitate the fortunes of netbooks (or an equivalent small, lightweight, low-cost device with a real keyboard).


Analytics Enhancements would increase ridership
  • 5/17/2012 12:25:34 PM

For me I can't truly type on virtual keyboards I use one finger and make lots of mistakes that need correction, I wish all devices had the option of the Bluetooth keyboard especially my kindle, my efficiency level would multiply many fold. None of my mobile devices can offer the speed of typing I enjoy on a latop or keyboard.

Re: Operational Analytics Keeps Bay Area Trains on Track
  • 5/17/2012 10:51:16 AM

This use of analytics is great.  I agree that most people are willing to give up a bit of their privacy to get the trains to run effectively and to be able to predict schedules etc.  It would also be good for ferries..then even more elements could be included like weather conditions, ship traffic etc.

Re: Reliable BART
  • 5/17/2012 10:10:53 AM

Hi Lyndon, I'm glad you enjoyed this post as I definitely had  you in mind as I  wrote it! And thanks for all this great insight into transit systems and how/why some are more capable of enabling operational analytics and automated operations than others. In talking to Roy for my blog, I also learned that BART has hundreds of thousands of assets from which it gathers data. Understanding what's happening with those assets is critical as BART is looking at a fleet overhaul. It has 669 "revenue vehicles," or train cars, and wants a few pilot prototypes in place in 3 years or so, "to be followed by replacement of the entire lot," Roy told me. With so much at stake, the agency needs a firm handle on reliability, etc., he says.


Re: Reliable BART
  • 5/17/2012 9:50:42 AM


Vincent writes

On track maybe, but on time, not really. Swiss trains are doing a much better job being on schedule...


Good grief — the Swiss are in ... SWITZERLAND! You know, Clockland! No wonder they're more on time!

Seriously, you make a very valid point. European systems in general (as well as, say, Japanese, Chinese, and those in various other major countries) run far more reliable rail public transport systems (and bus systems for that matter). They just sorta have the knack...

There are other problems on BART and other U.S. systems that contribute to delays, of course. Inadequate infrastructure maintenance is very likely at the top of the list, along with car maintenance as Vincent mentions. 

BART has a history of problems with its automated control system (caused a train to run off the end of track on an elevated structure back in 1973), but I think that with upgrades and improved technology a lot of this has been solved. However, automation does introduce a lot of extra quirks and fussiness and reliability problems.

One other disruption I'd throw in the mix — terrorism alerts. An unattended bag on a train can stop an entire transit line, maybe an entire system.




Re: Reliable BART
  • 5/17/2012 2:50:06 AM

On track maybe, but on time, not really. Swiss trains are doing a much better job being on schedule, mainly because they are not subject to


  • 5 minutes pause every time there is a micro earthquake that nobody can feel
  • Old cars that stopped working properly and not well maintained, getting stuck in a tunnel, causing delays
  • Police activity (happens on a daily basis - police chasing small criminals in a train or a station, causing delays)
  • Medical emergency (some homeless having a cocaine overdose?)


A lot of the delays are caused by excessive regulations, and the fact that these trains have are frequented by a lot of homeless and criminals (the cars stink permanently), unless Swiss trains.

Also, financial management is terrible - they could benefit from superior, even basic analytics.

Re: Reliable BART
  • 5/17/2012 1:00:51 AM


Beth, very nice article.

BART was one of the first 2 new-generation U.S. rail transit systems to use basically automatic control of its trains. The very first (beating BART by about half a dozen years) was the Port Authority Transit Corporation (PATCO) High-Speed Line that originally connected center-city Philadelphia with Lindenwold, New Jersey. My then-wife and I rode this frequently on visits to her parents in South Jersey from our base in NYC (Amtrak to Philly, local subway to PATCO, PATCO to Lindenwold).

I've also had the opportunity to sit in the cab (motorman's seat) of a BART train while it was speeding through the Transbay tunnel. Other than watch out the front window, I didn't do a thing — the train ran itself. Still, it was kinda exciting...

Total automation is now the norm for new rail rapid transit lines (totally grade-separated from all other traffic, etc.). And of course just about everybody is familiar with those automated peoplemovers in many airports, shuttling passengers between terminals.

Older systems (including Chicago's El) still have a way to go to reach the level of automation (and analytics) of the newest lines, but they're slowly getting there...

BART is able to track passenger movements (boarding and deboarding points) because you must use a ticket to both enter and exit the system, vua turnstiles. Most other new rail systems however don't do this, and "tracking" passenger movements is more difficult.

The other feature of interest is train arrival technology, also pioneered by new systems like BART. This has basically been adapted for various surface operations, especially bus and light rail (including streetcars), using GIS to track the different transit vehicles and then digesting this information, calculating arrival times, and displaying to passengers in stations via PIDs (Passenger Information Displays). This is usually called NextBus technology, although that's actually a brand name.

I'm trying to pull together through research some of the main applications of analytics in public transportation, for both planning and operations, and maybe an article will issue forth...


Page 1 / 2   >   >>