unexpected facts of life by orly amrany last - nov 2015

Orly Amrany

General Manager, Wix

When the question ‘why?’ is not relevant

The Unexpected Facts of Life

Hi, I’m Orly Amrany General Manager, Head of Premium Services @ Wix

Question #01

• Assume you have 3 Million Subscription (Monthly/Yearly)

• Each recurring charge is done on it’s purchase date

• Re-try upon failure is done every 7 days

• Success rate of recurring charge is ~90%

• Which day do we have the best changes to charge ?

24th or 25th?

85.25%

81.94%

84.14%

79.80%

82.79%

85.67%

85.13%

83.24%

81.59%

86.80%

83.55%

85.48%

86.49%86.96%

79.00%

80.00%

81.00%

82.00%

83.00%

84.00%

85.00%

86.00%

87.00%

AU BR CA ES FR GB US

24th Vs. 25th, initial recurring

85.25%

81.94%

84.14%

79.80%

82.79%

85.67%

85.13%

83.24%

81.59%

86.80%

83.55%

85.48%

86.49%86.96%

79.00%

80.00%

81.00%

82.00%

83.00%

84.00%

85.00%

86.00%

87.00%



+0.5% increase in approval rate

85.25%

81.94%

84.14%

79.80%

82.79%

85.67%

85.13%

83.24%

81.59%

86.80%

83.55%

85.48%

86.49%86.96%

79.00%

80.00%

81.00%

82.00%

83.00%

84.00%

85.00%

86.00%

87.00%




+15k saved subscriptions

85.25%

81.94%

84.14%

79.80%

82.79%

85.67%

85.13%

83.24%

81.59%

86.80%

83.55%

85.48%

86.49%86.96%

79.00%

80.00%

81.00%

82.00%

83.00%

84.00%

85.00%

86.00%

87.00%




+15k saved subscriptions

+1.5M

And the winner is…

The 24th!

You’ve won 1,500,000million dollars

Question: Why checking the 24th vs. 25th?

Question: Why checking the 24th vs. 25th?

Answer: Because we could

• We had data of million of transactions

• We actually checked all days of the month

• A lot of other assumptions failed..(15th Vs 16th)

• Should we AB test? (NO)

On the 24th there is more changes for charges to succeed

We gained +1.5M USD with a simple change!

When you have tones of data, you can find out a lot of interesting and unexpected facts

You should do it as soon as possible: Time is money

Make all recurring charges on the 24th ? (NO) Improve re-try logic (Which Tool)

So what did we learn?

So what did we learn?

Don’t ask yourself

‘why?’

Ask yourself

‘why didn’t I check this before?’

Some more fun examples

Question #02

• Same data

• Which day do we have the best changes to charge?

Weekdays or weekends?

And the winner is…

Weekday!

4.41%

-1.85%

6.36%

2.42%

5.38%

8.00%

-0.59%

1.18%

5.67%

-4.84%

6.39%

0.45%

2.97%

5.94%

-6.67%

0.10%

-0.62%

5.21%

7.62%

1.50%

2.59%3.22%

-5.88%

3.41%

0.66%

11.80%

6.81%

-0.35%

8.05%

7.64%

2.39%

-10.00%

-5.00%

0.00%

5.00%

10.00%

15.00%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Approval rate difference - weekdays vs. weekends

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

• Same data

• Which hour of the day would be the best? (in Brazil for example)

Time or Day?

And the answer is…

8am – 10pm

20.00%

25.00%

30.00%

35.00%

40.00%

45.00%

50.00%

55.00%

60.00%

65.00%

70.00%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Approval Time, Recurring/UTC time

all

Thanks

unexpected facts of life by orly amrany last - nov 2015

Business