![Page 1: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/1.jpg)
Orly Amrany
General Manager, Wix
When the question ‘why?’ is not relevant
The Unexpected Facts of Life
![Page 2: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/2.jpg)
Hi, I’m Orly Amrany General Manager, Head of Premium Services @ Wix
![Page 3: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/3.jpg)
Question #01
![Page 4: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/4.jpg)
• Assume you have 3 Million Subscription (Monthly/Yearly)
• Each recurring charge is done on it’s purchase date
• Re-try upon failure is done every 7 days
• Success rate of recurring charge is ~90%
• Which day do we have the best changes to charge ?
24th or 25th?
![Page 5: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/5.jpg)
85.25%
81.94%
84.14%
79.80%
82.79%
85.67%
85.13%
83.24%
81.59%
86.80%
83.55%
85.48%
86.49%86.96%
79.00%
80.00%
81.00%
82.00%
83.00%
84.00%
85.00%
86.00%
87.00%
AU BR CA ES FR GB US
24th Vs. 25th, initial recurring
![Page 6: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/6.jpg)
85.25%
81.94%
84.14%
79.80%
82.79%
85.67%
85.13%
83.24%
81.59%
86.80%
83.55%
85.48%
86.49%86.96%
79.00%
80.00%
81.00%
82.00%
83.00%
84.00%
85.00%
86.00%
87.00%
AU BR CA ES FR GB US
24th Vs. 25th, initial recurring
+0.5% increase in approval rate
![Page 7: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/7.jpg)
85.25%
81.94%
84.14%
79.80%
82.79%
85.67%
85.13%
83.24%
81.59%
86.80%
83.55%
85.48%
86.49%86.96%
79.00%
80.00%
81.00%
82.00%
83.00%
84.00%
85.00%
86.00%
87.00%
AU BR CA ES FR GB US
24th Vs. 25th, initial recurring
+0.5% increase in approval rate
+15k saved subscriptions
![Page 8: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/8.jpg)
85.25%
81.94%
84.14%
79.80%
82.79%
85.67%
85.13%
83.24%
81.59%
86.80%
83.55%
85.48%
86.49%86.96%
79.00%
80.00%
81.00%
82.00%
83.00%
84.00%
85.00%
86.00%
87.00%
AU BR CA ES FR GB US
24th Vs. 25th, initial recurring
+0.5% increase in approval rate
+15k saved subscriptions
+1.5M
![Page 9: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/9.jpg)
And the winner is…
![Page 10: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/10.jpg)
The 24th!
![Page 11: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/11.jpg)
You’ve won 1,500,000million dollars
![Page 12: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/12.jpg)
Question: Why checking the 24th vs. 25th?
![Page 13: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/13.jpg)
Question: Why checking the 24th vs. 25th?
Answer: Because we could
• We had data of million of transactions
• We actually checked all days of the month
• A lot of other assumptions failed..(15th Vs 16th)
• Should we AB test? (NO)
![Page 14: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/14.jpg)
On the 24th there is more changes for charges to succeed
We gained +1.5M USD with a simple change!
When you have tones of data, you can find out a lot of interesting and unexpected facts
You should do it as soon as possible: Time is money
Make all recurring charges on the 24th ? (NO) Improve re-try logic (Which Tool)
So what did we learn?
![Page 15: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/15.jpg)
So what did we learn?
Don’t ask yourself
‘why?’
Ask yourself
‘why didn’t I check this before?’
![Page 16: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/16.jpg)
Some more fun examples
![Page 17: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/17.jpg)
Question #02
![Page 18: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/18.jpg)
• Same data
• Which day do we have the best changes to charge?
Weekdays or weekends?
![Page 19: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/19.jpg)
And the winner is…
![Page 20: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/20.jpg)
Weekday!
![Page 21: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/21.jpg)
4.41%
-1.85%
6.36%
2.42%
5.38%
8.00%
-0.59%
1.18%
5.67%
-4.84%
6.39%
0.45%
2.97%
5.94%
-6.67%
0.10%
-0.62%
5.21%
7.62%
1.50%
2.59%3.22%
-5.88%
3.41%
0.66%
11.80%
6.81%
-0.35%
8.05%
7.64%
2.39%
-10.00%
-5.00%
0.00%
5.00%
10.00%
15.00%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Approval rate difference - weekdays vs. weekends
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
![Page 22: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/22.jpg)
4.41%
-1.85%
6.36%
2.42%
5.38%
8.00%
-0.59%
1.18%
5.67%
-4.84%
6.39%
0.45%
2.97%
5.94%
-6.67%
0.10%
-0.62%
5.21%
7.62%
1.50%
2.59%3.22%
-5.88%
3.41%
0.66%
11.80%
6.81%
-0.35%
8.05%
7.64%
2.39%
-10.00%
-5.00%
0.00%
5.00%
10.00%
15.00%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Approval rate difference - weekdays vs. weekends
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
+1.3% increase in approval rate
![Page 23: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/23.jpg)
• Same data
• Which hour of the day would be the best? (in Brazil for example)
Time or Day?
![Page 24: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/24.jpg)
And the answer is…
![Page 25: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/25.jpg)
8am – 10pm
![Page 26: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/26.jpg)
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
55.00%
60.00%
65.00%
70.00%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Approval Time, Recurring/UTC time
all
![Page 27: Unexpected facts of life by Orly Amrany last - Nov 2015](https://reader033.vdocuments.mx/reader033/viewer/2022042611/58ecbafc1a28ab7a628b4713/html5/thumbnails/27.jpg)
Thanks