c o n f i d e n t i a l upa 2007 what reporting has shown us jenni mckienzie august 22, 2007
TRANSCRIPT
C O N F I D E N T I A LUPA 2007
What Reporting Has Shown Us
Jenni McKienzie
August 22, 2007
C O N F I D E N T I A L 2 SpeechTEK 2007
Reporting Background
• Some data goes directly from the IVR into a database where we use Business Objects to report on it
• Other data is tracked through Genesys
• Reports are all custom-built in-house
• All reports are done by DNIS (which we often refer to as a campaign)
• Our corporate travel campaigns are all about routing to the right agent, so most of what we look at is our leisure travel campaigns
C O N F I D E N T I A L 3 SpeechTEK 2007
Report Inventory
• DNIS report– Available from the beginning– Call volumes
• Menupick report – Available from the beginning– Menupick is a variable that is set at the point of transfer to tell us where to
route to and what was happening in the call
• Utterance report– Still in the works, but close– Tracks what people are saying at each menu, both the ultimate choice and
the synonym recognized
• Where callers end– Still in the works, but close– Will track where a caller is in the call flow when the call ends, trying to
determine for hangups if they got what they wanted or hung up unhappy
C O N F I D E N T I A L 4 SpeechTEK 2007
DNIS Report Sample
CallType DNIS Month 7/9/2007 7/8/2007 7/7/2007 7/6/2007 7/5/2007 7/4/2007Corporate Blue 3315 922 407 68 23 198 205 21Corporate Green 4433 3,706 1363 197 157 892 971 126Corporate Red 4466 17 8 2 0 3 4 0Corporate Yellow 1583 129 39 7 4 38 32 9Leisure Blue local 2799 268 52 52 37 43 57 27Leisure Blue toll free 2118 18,950 4586 2205 2386 3499 4134 2140Leisure Green 2117 4,832 1266 418 509 999 1161 479Leisure Yellow local 1089 1,112 184 226 196 183 231 92Leisure Yellow toll free 1099 8,492 1879 1133 1115 1636 1721 1008TVLY Main 1809 72,846 16078 9562 9312 13732 15221 8941TVLY Service 1093 8,721 1689 1509 1160 1730 1705 928TVLY VIP 1143 4,364 1178 580 479 792 884 451SCC Total 119,585 26,912 15,685 15,194 22,614 25,114 14,066TBiz Total 4,774 1,817 274 184 1,131 1,212 156TOTAL 124,359 28,729 15,959 15,378 23,745 26,326 14,222
C O N F I D E N T I A L 5 SpeechTEK 2007
Menupick Report Sample – Raw Data
Menu Pick Month 7/9/2007 7/8/2007 7/7/2007 7/6/2007 7/5/2007 7/4/2007AgentRequested 22932 3376 1762 1748 2799 3271 1742Calls Ended in IVR 37759 4608 3513 3200 4659 4917 2764CancelAir 2835 326 177 188 378 408 209CancelCar 899 125 72 64 87 133 86CancelHotel 2306 342 183 145 278 337 206ChangeAir 17397 1964 1226 1377 2263 2564 1336ChangeCar 1696 249 155 112 219 230 148ChangeHotel 3132 462 243 260 362 468 225CustChampAir 4625 485 385 370 572 624 343CustChampHotel 4557 505 376 550 749 563 389FailInitial 4749 686 386 391 613 642 388FailPhone 519 70 48 41 68 73 47FailTripID 2529 329 218 246 317 302 206Flifo 3015 471 246 242 376 428 240NewAir 14815 2133 1163 1239 1835 1943 1191NewCar 2022 310 158 163 263 266 172NewCaribbeanVacation 749 113 58 60 95 101 64NewCruise 349 54 20 37 44 57 27NewDisneyVacation 1085 174 115 113 120 129 93NewHawaiiVacation 716 104 66 69 85 101 45NewHotel 12440 1710 1062 1066 1568 1740 1109NewLastMinute 202 35 15 13 28 30 18NewMexicoVacation 706 75 71 45 83 130 65NewOtherVacation 4007 562 355 344 484 541 332NewReservation 6023 898 440 482 763 889 479NewSkiVacation 215 30 18 19 21 25 25NewVegasVacation 855 120 85 63 119 127 62PhoneBSFail 389 216 62 29 13 19 7PhoneCityDateProb 1899 240 149 145 256 256 166PhoneNothingFound 3366 467 232 251 388 470 272
C O N F I D E N T I A L 6 SpeechTEK 2007
Menupick Report Sample – Percentages
Menu Pick Month 7/9/2007 7/8/2007 7/7/2007 7/6/2007 7/5/2007 7/4/2007AgentRequested 11.97% 12.15% 10.87% 11.18% 11.98% 12.63% 11.86%Calls Ended in IVR 19.71% 16.58% 21.67% 20.47% 19.93% 18.99% 18.81%CancelAir 1.48% 1.17% 1.09% 1.20% 1.62% 1.58% 1.42%CancelCar 0.47% 0.45% 0.44% 0.41% 0.37% 0.51% 0.59%CancelHotel 1.20% 1.23% 1.13% 0.93% 1.19% 1.30% 1.40%CancelTotalTrip 0.45% 0.32% 0.28% 0.46% 0.55% 0.52% 0.42%ChangeAir 9.08% 7.07% 7.56% 8.81% 9.68% 9.90% 9.09%ChangeCar 0.89% 0.90% 0.96% 0.72% 0.94% 0.89% 1.01%ChangeHotel 1.63% 1.66% 1.50% 1.66% 1.55% 1.81% 1.53%ChangeTotalTrip 1.47% 1.06% 1.04% 1.36% 1.54% 1.81% 1.37%CreditCard 0.30% 0.25% 0.38% 0.33% 0.32% 0.29% 0.31%CustChampAir 2.41% 1.74% 2.37% 2.37% 2.45% 2.41% 2.33%CustChampHotel 2.38% 1.82% 2.32% 3.52% 3.20% 2.17% 2.65%CustChampSomethingElse 2.42% 3.63% 2.78% 2.61% 2.08% 2.16% 2.00%FailInitial 2.48% 2.47% 2.38% 2.50% 2.62% 2.48% 2.64%FailPhone 0.27% 0.25% 0.30% 0.26% 0.29% 0.28% 0.32%FailTripID 1.32% 1.18% 1.34% 1.57% 1.36% 1.17% 1.40%Flifo 1.57% 1.69% 1.52% 1.55% 1.61% 1.65% 1.63%NewAir 7.73% 7.67% 7.17% 7.92% 7.85% 7.50% 8.11%NewCar 1.06% 1.12% 0.97% 1.04% 1.13% 1.03% 1.17%NewCaribbeanVacation 0.39% 0.41% 0.36% 0.38% 0.41% 0.39% 0.44%NewCruise 0.18% 0.19% 0.12% 0.24% 0.19% 0.22% 0.18%NewDisneyVacation 0.57% 0.63% 0.71% 0.72% 0.51% 0.50% 0.63%NewHawaiiVacation 0.37% 0.37% 0.41% 0.44% 0.36% 0.39% 0.31%NewHotel 6.49% 6.15% 6.55% 6.82% 6.71% 6.72% 7.55%NewLastMinute 0.11% 0.13% 0.09% 0.08% 0.12% 0.12% 0.12%NewMexicoVacation 0.37% 0.27% 0.44% 0.29% 0.36% 0.50% 0.44%NewOtherVacation 2.09% 2.02% 2.19% 2.20% 2.07% 2.09% 2.26%NewReservation 3.14% 3.23% 2.71% 3.08% 3.26% 3.43% 3.26%NewSkiVacation 0.11% 0.11% 0.11% 0.12% 0.09% 0.10% 0.17%
C O N F I D E N T I A L 7 SpeechTEK 2007
Initial Menu Rewrite
• Moved away from leading with phone number prompt to offering the caller phone number or trip ID, but highly recommending trip ID.
• Callers who gave a phone number then had to say where they were going and when for security reasons, which was proving problematic.• Before: To see what I have on file for you now, say or enter
your phone number. (pause) You can also say one of the following: use my trip ID, shop for new reservations, make a new reservation, flight information, Travelocity Master Card, or gnome joke of the day.
• After: The fastest way to look up current reservations is with your trip ID. So, say use my trip ID, use my phone number, or I don’t have a reservation yet.
• Implemented on 1 of 4 IVRs on Travelocity branded campaigns for one week to do an A/B comparison.
C O N F I D E N T I A L 8 SpeechTEK 2007
Initial Menu Rewrite Findings
• Transfers from problems stemming from giving the city and date after the telephone number dropped from 9.7% to 3.2%.
• Customer championship transfers (which means we found their trip and they’re in the middle of it) went up from 7.9% to 9.9%.
• Calls completed in the IVR actually went down by about 0.8%. Our best guess is that fewer people are getting upset to the point of hanging up.
• All in all, if you take all the changes directly related to the change, we’ve seen a shift from problem related transfer reasons to correctly identified trip related ones of 6.2%.
C O N F I D E N T I A L 9 SpeechTEK 2007
Broader Findings
• We then implemented the change across all leisure campaigns.• Before the change, one partner (who we’ll call Blue) had 33% of all
calls for sales. After the change, it has been about 46%. Another campaign (Green) went from 30% to 41%. These were the only two campaigns to see this kind of shift.
• Travelocity VIP callers were saying “agent” at the initial menu or during the sign-in process 30% of the time. It dropped to 13%. This is the biggest shift of any campaign. They are now getting their tasks identified much better, allowing them to get to the right agent.
• Our Green partner callers were the worst about obediently giving us their phone number even when they didn’t have a trip booked. Their transfers due to problems trying to look up a trip by phone number have dropped from 17% of calls to 3%.
C O N F I D E N T I A L 10 SpeechTEK 2007
Reward Points Upgrade
• Our blue partner uses a reward points system. Many of their calls are for help in redeeming these points. They recently had a scheduled upgrade to this system that prevented customers from using that feature for a couple of days. We added messaging to the IVR and website.
• We saw a definite dip in sales calls and increase in calls ended in the IVR during the upgrade. Afterwards, there was higher than usual call volume, seeming to indicate people calling back to do what they couldn’t do over the weekend.
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
5/1
5/8
5/1
5
5/2
2
5/2
9
6/5
6/1
2
6/1
9
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Sales
End in IVR
Volume
C O N F I D E N T I A L 11 SpeechTEK 2007
Sales Call Analysis – June 2007
NewAir
NewCar
NewCruise
NewHotel
Unknown
Destination
Package
• 23.8% of calls are for new reservations• It varies wildly by campaign• Have used this data to analyze IVR offerings and staffing
0%5%
10%15%20%25%30%35%40%45%50%
Blue Green Yellow TVLYMain
TVLYService
TVLYVIP
C O N F I D E N T I A L 12 SpeechTEK 2007
Existing Air Reservations
• Recently added staff that is only trained in air reservations• Expanded Cancel, Change, and Service menupicks to include the
product• For June 2007, 17.0% of calls resulted in menupicks of CancelAir,
ChangeAir, or ServiceAir• Menupick report helps us forecast staffing for that agent group• It also shows us the potential value in adding functionality to the
IVR around these types of requests
C O N F I D E N T I A L 13 SpeechTEK 2007
Adding CTI
• Looking to add CTI to one of our call centers. But how many calls would benefit from this?
• Looked at the menupick report to see how many calls were in the Service, Cancel, or Change categories since these are all calls where the trip was identified and could be passed to the agent.
• We are using this data to help determine how many licenses are needed.
C O N F I D E N T I A L 14 SpeechTEK 2007
Happy/Sad Report Sample
Dnis CallType DATE Result Menu Name H/S Calls % of Type % of All
3237 leisure 07/31/2007 callerHangup AfterFlifo H 2 9.52% 6.90%3237 leisure 07/31/2007 callerHangup RepeatMenu H 3 14.29% 10.34%3237 leisure 07/31/2007 callerHangup Flifo_DepOrArr S 1 4.76% 3.45%3237 leisure 07/31/2007 callerHangup Flifo_FlightOrCity S 2 9.52% 6.90%3237 leisure 07/31/2007 callerHangup FlifoRightYN S 1 4.76% 3.45%3237 leisure 07/31/2007 callerHangup GetCity_FromCityPair_Arr S 1 4.76% 3.45%3237 leisure 07/31/2007 callerHangup InitialMenu S 9 42.86% 31.03%3237 leisure 07/31/2007 callerHangup OnTrip S 1 4.76% 3.45%3237 leisure 07/31/2007 callerHangup Time_Departure S 1 4.76% 3.45%3237 leisure 07/31/2007 transfer Destination H 2 25.00% 6.90%3237 leisure 07/31/2007 transfer NewResProduct H 2 25.00% 6.90%3237 leisure 07/31/2007 transfer InitialMenu S 4 50.00% 13.79% 3237 leisure 07/31/2007 Total Calls 29 3237 leisure 07/31/2007 Total Hangups 21 72.41%3237 leisure 07/31/2007 Total Transfers 8 27.59%3237 leisure 07/31/2007 Total Happy 9 31.03%3237 leisure 07/31/2007 Total Sad 20 68.97%3237 leisure 07/31/2007 Happy Hangups 5 23.81% 17.24%3237 leisure 07/31/2007 Sad Hangups 16 76.19% 55.17%3237 leisure 07/31/2007 Happy Transfers 4 50.00% 13.79%3237 leisure 07/31/2007 Sad Transfers 4 50.00% 13.79%
C O N F I D E N T I A L 15 SpeechTEK 2007
Utterance Report
• Initial menu has “shop for” and “make” new reservations, based on usability testing. What’s the usage for each? Is it warranted to keep them both?
• Error prompt in trip lookup process is as follows:– OK then, to try again, say try a different phone number or use my
trip ID. (small pause) You can also say shop for new reservations, make a new reservation, flight information, or agent if none of these are what you need.
– Impression from call monitoring is that people tune out all these choices and jump on agent. Anecdotal evidence is dangerous. I need the data to back it up before I contemplate a change here.
