Performance Monitoring to Improve Service Quality and Efficiency Through Better Management of Bus Operators

Summary: TriMet, the regional transit provider in Portland, OR metropolitan area has been an industry leader in the collection and analysis of transit operations data targeted towards improving service quality and reducing the costs of service provision. The agency has recently embarked upon a innovative program that seeks to improve service quality and efficiency by addressing inconsistencies in operator behavior through use of performance monitoring. The primary focus of the program is to provide detailed, comprehensive information related to operator performance to garage managers and field supervisors so that corrective actions can be taken, if warranted.

Description: The innovation stems from an upper level management directive called the Productivity Improvement Process (PIP) which is a agency-wide framework that seeks to continually improve efficiency and effectiveness and save money through use of innovative processes, procedures, and new technologies. Introduced in 1999, PIP is based upon the idea that the best ideas for improving productivity come from frontline employees who are actively engaged in the work. In 2002, the Service Performance PIP team was formed to improve service quality and utilize agency resources more efficiently. Two goals of the team are to increase system-wide on-time performance (OTP) to 90% and to increase the percentage of on-time pull outs to 95%. The team is also working towards increasing ridership, improving operator participation in the scheduling process, and changing the managerial and operating culture related to OTP. The innovation process is shown in Figure 1. The three key departments involved with the initiative to address inconsistencies in operator behavior through performance monitoring are management, operations, and scheduling. The process started with a directive from senior management to improve productivity The impetus for the innovation came from operations personnel who saw opportunities to improve service quality and address operational inefficiencies through ad hoc discussions on operations analysis. Performance

monitoring plays a central role in the process as it not only helps with problem identification but also with measuring and monitoring changes in service characteristics following implementation. Once operational problems are identified, it must be ascertained whether these problems are due to operator behavior or simply because of incorrect or outdated schedules. In the case of poorly performing operators, additional analysis can determine whether problems are related to a particular class of operators such as those with minimal experience. If it is shown that inexperienced operators are leaving the beginning of the line excessively early or late, then training staff need to become more involved. For operators that truly are poor or inconsistent performers, closer supervision is necessary. The process is iterative (shown by orange colored arrows in the figure) in that once operator-related problems are addressed, service performance will have to be reevaluated and schedules further adjusted.

[Figure 1] The innovation has two primary goals: 1) provide reliable service to customers and 2) reduce service inefficiencies through reductions in run time variability resulting from differences in operator behavior, thereby allowing excess run time and recover/layover times to be removed from schedules. Previous analyses of operations data at TriMet has yielded important information concerning the role of operator experience and behavior in relation to service reliability and service efficiency. One study found that the impact of the effects of individual operators on bus running times varied considerably and that the effects were normally distributed. The same study also found that nearly 70% of the amount of recovery/layover time in schedules was needed to address differences in operator behavior rather than actual operating conditions1. Another study found that the amount of headway delay at peak loads point was largely attributable to headway delays at trip origins2. A more recent internal effort looked at TriMet’s 15 frequent service bus routes where it was found that 19% of all buses started trips more than three minutes late. In looking at operator length of service and terminal departure behavior, 19% of operators with greater that three years of experience were found to be departing trip origins more than three minutes late with operators with three years or less departing late 25% of the time. This last finding suggests more adequate training is needed. Overall, these studies suggest that efforts to reduce departure delay variability at trip origins will yield important service quality and schedule efficiency benefits. Historically, schedulers and operations management personnel have made decisions with limited information about various states of the transit system at any point in space and time. Like most agencies that have AVL systems in

1 Strathman, J. G., Kimpel, T. J., Dueker, K. J., Gerhart, R., & Callas, S. (2002). Evaluation of Transit Operations: Data Applications of Tri-Met's Automated Bus Dispatch System. Transportation, 29, 321-345. 2 Strathman, J. G., Kimpel, T. J., Callas, S. (2002). Headway Deviation Effects On Bus Passenger Loads: Analysis of Tri-Met’s Archived AVL-APC Data (Report No. PR126). Portland, OR, Center for Urban Studies.

place, TriMet polls bus locations at regular time intervals and transmits this information to dispatch centers in-real time for operations control purposes. TriMet is somewhat unique among North American transit properties in that its AVL system was also designed to collect stop-level information on bus operations which is linked with automatic passenger count data and used for off-line analysis of transit operations. At TriMet, 100% of the bus fleet is AVL-equipped. This fact, coupled with stop-based AVL, provides the agency with a complete picture of the system thereby making it possible to 1) directly monitor the behavior of each operator in relation to service quality on a continual basis and 2) indirectly address service inefficiencies resulting from poorly performing operators through the scheduling process. The thrust of TriMet’s innovation is that by providing detailed information on operator behavior to supervisory personnel, they become empowered and are thus more likely to take corrective actions. Operator-specific information is disseminated through performance reporting. The reports are readily accessible by agency staff through customized query interfaces connected to an enterprise-level database. Figure 2 is a screenshot of the Service Performance PIP reports interface. These specific reports largely focus on the performance of individual operators and routes and are available for the current signup period (booking), as well as the previous week.

[Figure 2 ] Operator OTP at time points for the current signup is presented in Figure 3. In addition to the percentage of trips on-time, early, and late, peer on-time is also presented along with the on-time difference in relation the operator’s peer group. Peer on-time is the weighted average on-time percentage of all operators driving the same route and direction during similar time periods throughout the day. Peer on-time is an important variable because it provides managers with a more accurate basis for comparison of OTP among operators. It would not be realistic to compare an operator who drives a radial trunk during the peak hour to someone who operates a low ridership suburban feeder during the midday time period. Since the peer on-time measure is weighted average, it also works for extra-board operators who operate different runs from day to day. Begin of line late represents the percentage of departures more than three minutes late. The report also shows information related to average lifts per 8 hours of work since some lateness can be explained by lift operation activity. Figure 4 shows operator OTP by month of service for an individual operator. The variables are similar to the ones presented in the previous figure except that the percentage of trips departing late from the beginning of the line is broken out by three and five minute intervals. The importance of this report is to track the performance of an individual operator to see if a recently identified problem is a continuing trend, a new pattern, or an isolated incident.

[Figure 3]

[Figure 4]

One of the first elements of the Service Performance PIP process was to investigate the pattern of operators leaving their respective garages consistently late. Figure 5 presents information on late operator pull outs. The data presented in the pull out late report includes general operator information including badge, name, and type of duty, percent pull out late, average minutes late, and the number of pull out observations used in the analysis. Figure 6 presents information on operator pull outs for a single operator. More detailed information is presented in this report including scheduled pull out time and actual pull out time, minutes late, pull out notes, and pull out to location. Information related to scheduled departure time, actual arrival time, and minutes late at the first time point are presented as well. The report also includes information related to minutes late at the second time point. Situations where operators pulled out late and were more than three minutes late at the first two time points are denoted by a double asterisk. Since buses do not operate independently from one another, the scheduling implications resulting from highly variable pull out delays are immense. Designing schedules to accommodate extreme operator variability is a second best solution compared to better management of operators at the beginning of the line.

[Figure 5]

[Figure 6]

User Assessment: The innovation is primarily targeted towards management personnel, particularly the station (garage) managers, trainers, and field supervisors. A large amount of a station manager’s time is spent responding to customer complaints. Operator performance information can be readily queried from the Service Performance PIP database as well as other databases to see if the complaints are valid and whether disciplinary actions are warranted. Armed with various performance reports, field supervisors can now take a more proactive approach in dealing with problems, allowing them to allocate their time more efficiently to other areas. Schedulers can also use this information to ascertain whether certain problems are due to operator inconsistencies or actual changes in operating conditions. The attitude of the users towards change is open given that the innovation provides comprehensive information about the status of each individual operator along multiple performance criteria. The Service Performance PIP has resulted in increased levels of communication between supervisors, operations, and

scheduling personnel. Prior to the PIP initiative, the information was not that well known to supervisors.

Technology Assessment: The relative advantage of the Service Performance PIP initiative over current techniques is that supervisory personnel now have the information that they need to make more informed decisions regarding variability in operator behavior. Furthermore, the development of a new performance measure (peer OTP) in relation to operator peer group provides managers with a useful point of reference. The present innovation was relatively easy to implement given the strength of performance reporting at TriMet to begin with. Since the innovation makes use of existing data, the costs are negligible outside of staff time required to build the database queries and design the reporting interface. The innovation simply makes use of existing data by summarizing it in a novel manner. Transit agencies that do not have AVL systems in place would have considerable difficulty implementing a similar program given that data would need to be collected manually. A study focusing on operator performance in the absence of AVL technology would likely have to be a one-off study rather than a continual one due to the high costs of manual data collection. Since the Service Performance PIP initiative is fairly new, it would be premature to say that the benefits outweigh the costs. The measurement of benefits is partially confounded by the fact that operators can choose different pieces of work each quarter based upon seniority. Thus, it would prove difficult to measure changes in service reliability or schedule efficiency at the route or trip level resulting from improved management of bus operators. It would be possible, however, to measure changes in the performance of an individual operator over time. The consequences of failure are small since the worst outcome is that the status quo simply continues. As such, there is very little risk in attempting to reduce variability in operator behavior given the potentially large benefits of improved service quality and more efficient service provision.

Project Status: Continuous

Disclaimer Notice: This document is disseminated under the sponsorship of the United States Department of Transportation, Federal Transit Administration, in the interest of information exchange. The United States Government assumes no liability for the contents or use thereof.

The United States Government does not endorse products or manufacturers. Trade or manufacturers' names appear herein solely because they are considered essential to the contents of the report.

Contacts: For more information contact: Steve Callas, Service & Performance Analysis Coordinator Tri-Met 4012 SE 17th Ave. Portland, OR 97202 (503) 239-3000 Thomas J. Kimpel, Research Associate Center for Urban Studies College of Urban and Public Affairs Portland State University Portland, OR 97207 (503) 725-4020

Figure 1: Innovation Process: Performance Monitoring of Operator Behavior

Operations

Management

Performance Monitoring

Data Analysis and

Reporting

Garage/Field Management

Data Collection

Scheduling

TrainingSenior Management

Schedule Adjustment

PIP Implementation

Initiative Proposal

Figure 2: Service Performance Productivity Improvement Process Reports Query Interface

Figure 3: Operator OTP by Badge

Figure 4: Individual Operator OTP by Month

Figure 5: Pull Out Late by Operator

Figure 6: Pull Out Late by Operator by Stop and Date