1

Ryan LanyonManager, Smart CommuteACT Canada Sustainable Mobility Summit 2012November 5, 2012

Communicating SuccessRevisiting Key Performance Indicators

2

Agenda

Introduction to Smart Commute and Metrolinx Brief History Start-up Indicators Foundation Indicators Growth Indicators Future Indicators Next Steps

Thanks to:

Mark SinghKyle KellamRebekah McGurranMeaghan MendonçaGeoff Noxon

3

Introduction – Metrolinx

Established in 2006 Serving the GTHA 6.5 m people 30 municipalities Three operating divisions:

GO Transit Presto Airport Rail Link

Two programs: Transit Procurement Initiative (TPI) Smart Commute

4

Introduction – Smart Commute

Established 2001, 2005 Serving the GTHA Two-tiered structure

Metrolinx 11 TMAs (soon to be 13)

Metrolinx and municipal funding Some sponsors and private

sector fees Focus on work trips 250+ workplaces 600,000+ commuters

55

1999 – Roundtable Breakfast and TMA Steering Committee 2000 2001 – Black Creek Regional TMA 2002 2003 – “Smart Commute” Black Creek 2004 – 404-7 2005 – Smart Commute Assocation (SCA), Carpool Zone

Mississauga, NE Toronto, North Toronto & Vaughan 2006 – TMA Toolkit, Brampton-Caledon, Halton, Hamilton 2007 – Central York, Durham, SC Awards 2008 – Metrolinx, Toronto-Central 2009 – Discount Transit Pass Programs 2010 – Pearson Airport Area 2011 – Workplace Program 2012 – Scarborough 2013 – Triple program impacts over 2009

Brief History – Decade of Development

Exploration

Experimentation

Start-Up

Foundation

Growth

6

Start-up Indicators

Contribution Agreement with Transport Canada Awareness Participation Satisfaction Project Benefits

7

Start-up Indicators

Awareness Measures Percentage of commuting population aware of Smart Commute services Percentage of commuting population aware of how to reach Smart Commute (e.g. 800

number, website) Number of commuters directly exposed to Smart Commute information by direct

outreach efforts. Percentage of information requests received through various referral sources Percentage of regional and local commuters who currently use alternative modes for

commuting Percentage of regional and local commuters who would be willing to try alternative

modes for commuting Percentage of regional employers that are aware of Smart Commute Association

services; percentage of local employers that are aware of TMA services

8

Start-up Indicators

Participation Measures Commuters

• Number of commuter requests for various services offered/supported by Smart Commute Association or the individual TMA

• Number of participants in Smart Commute Association’s Project or the TMAs services.• Number of ridematch applicants and number per 1,000 commuters• Percentage of applicants who use ridematch information sent to them.

Employers• Number of employer requests for information and assistance• Number of employer clients participating in Smart Commute Association’s services (e.g.,

transportation fairs)• Number of regional employers implementing worksite TDM measures (parking spaces saved,

bike storage and shower/change facilities added, site design)

9

Start-up Indicators

Satisfaction Measures Commuters

• Percentage of users who rate various Project services as “excellent” or “very good” overall• Percentage of users who request improvements in Project services• Percentage of ridematch applicants who receive ridematches• Commuter ratings on service quality features (e.g. time to obtain assistance, convenience of

service access/availability, accuracy/quality of information provided) Employers

• Percentage of employers that rate various Smart Commute Association or TMA services as “excellent” or “very good” overall

• Employer ratings on service quality features (e.g. time to obtain assistance, usefulness of information and products provided, knowledge and expertise of outreach staff)

10

Start-up Indicators

Project Benefits Measures Percentage and number of applicants placed in rideshare modes after

receiving Smart Commute Association or TMA services or information (continued and temporary/trial placements)

Average vehicle trips reduced per placement Number of daily vehicle trips reduced by commuters who received Smart

Commute Association or local TMA services Number of VKT reduced by commuters who received Smart Commute

Association or local TMA services Number of tonnes of emissions reduced by commuters who received Smart

Commute Association or local TMA services (also reported by mode)

11

Start-up Indicators

Challenges Indicators negotiated and created with little experience Too many indicators

• Competing priorities• Did not paint a clear picture

Usefulness of some indicators Measurement feasibility and accuracy What defined success?

12

Foundation Indicators

Opportunity to Reflect on Experiences Collaborative approach Refinement of Start-up Indicators

• Built on foundation provided• Elimination of unnecessary indicators• Simplification of indicators• Alignment with existing data collection

Used only for monitoring, not evaluation Short-term usage

13

Growth Indicators

Triple Productivity in Three Years (2010-11 to 2012-13) Number of employers “active” in the program Number of commuters at active employers Number of registered online users

14

Growth Indicators

15

Growth Indicators

Challenges Indicators mandated by Metrolinx senior management

• Partner and stakeholder reaction

Isolating productivity to what is within our influence• Regional and TMA-level accountability

Focus on recruitment and retention• KPI too narrow

Drive to have users register and stay registered• Concerns around how to attract users, rather than incent behaviour

change

16

Future Indicators

Return to a collaborative approach; pro-active Sharper refinement based on greater experience Balance of needs for Metrolinx, Smart Commute,

stakeholders Distinguishing between regional, TMA, employer

What funders need to know What delivery agents need to know What employers need to know

17

Future Indicators

Challenges Regionalize the KPI Measure depth of engagement at workplaces Measure the efficiency of program delivery Isolate performance from the broader context Inputs vs. outputs Balancing measurement accuracy with resources Indicators vs. collected data

18

Future Indicators

Reach Breadth and depth of engagement

Satisfaction Client satisfaction with programs & services

Impact Change in commuter behaviour

Cost-effectiveness Efficiency in investment

19

Future Indicators

Workplace Program – Measuring Depth Programming Promotions Tracking SOV reductions Investments and incentives Infrastructure and trip-end facilities

Designation:-Gold Workplace-Silver Workplace-Workplace

20

Future Indicators – Reach

Employer Reach # of SC Workplaces

• Gold Workplaces• Silver Workplaces• Workplaces

Proportion of all GTHA employers engaged (%)

Commuter Reach # of Commuters at Active SC employers

• Gold Workplaces• Silver Workplaces• Workplaces

Proportion of all GTHA commuters engaged (%)

21

Future Indicators – Reach (con’t)

Carpool Zone Participation # of active registrants # of new registrants (per year)

Program Awareness % of Commuters in GTHA aware of Smart Commute

22

Future Indicators – Satisfaction

Employer Satisfaction Employers rating services as very good or excellent (%)

• Value for money• Understanding of needs• Staff knowledge and expertise• Usefulness of information and products• Time to obtain assistance

Employer Retention Retention of employers from preceding year (%)

Carpool Zone Satisfaction Online registrants rating the service as very good or excellent

23

Future Indicators – Impact

Change in vkt Set a baseline vkt through census survey

• Mandatory minimum response rate for extrapolation• Alternatively assume non-responders to be SOVs• Assumptions:

– Average commute distance from a more rigourous data source– Average number of days spent commuting

Conduct a follow-up• Calculate vkt in a similar manner• Assume most vkt is reduced in first year and then held steady until survey• Tally all surveys for a regional vkt reduction

24

Future Indicators – Cost-effectiveness

Service Efficiency Service efficiency index

• Inputs:– # of Employers– Depth of engagement by workplace designation– # of Employees– Cost of Services

Leveraged Funding• TMA funding leveraged from non-MX sources (%)

– From non-MX sources– From non-government sources

25

Future Indicators – Cost-effectiveness

Service Efficiency Methodology Employer Size Factors:

• ≥ 50 and ≤ 499 employees = 1.0• ≥ 500 and ≤ 4,999 employees = 2.0• ≥ 5,000 and ≤ 49,999 employees = 3.0• ≥ 50,000 and ≤ 499,000 employees = 4.0

Designation weightings:• Gold Workplace = 6• Silver Workplace = 4• Workplace = 1

Employer Efficiency Formula: • [(1 + log10(Employer “AA” size/50)) * Employer “AA” designation weight]

TMA Service Efficiency Formula: ∑ (Employer AA Efficiency + Employer BB Efficiency + Employer CC Efficiency)

Cost of Services

26

Next Steps

Finalize future KPI Determine exact methodology for calculating vkt Ensure buy-in from management and municipalities

Restrategize around revised KPI Five-year strategic and operational plan Realign efforts to focus on new KPI

27

Ryan LanyonManager, Smart Commute416-874-5933ryan.lanyon@metrolinx.com