predictive modeling with major donors the 2002 cara summer workshop peter wylie, margolis wylie...
TRANSCRIPT
PREDICTIVE MODELING WITH MAJOR DONORS
The 2002 CARA Summer WorkshopPeter Wylie, Margolis Wylie Associates
PREDICTIVE MODELING: AN OVERVIEW
WHAT IS IT?
WHY DO IT?
HOW DO YOU DO IT?
DOES IT REALLY WORK?
SHOULD YOU DO IT YOURSELF OR HAVE IT DONE FOR YOU?
WHAT IS IT?
A WAY TO USE THE RICHNESS OF YOUR DONOR DATABASE TO IDENTIFY GOOD PROSPECTS
CAN GET TECHNICALLY COMPLICATED BUT CONCEPTUALLY SIMPLE
WHY DO IT?
1. You can learn huge amounts about who your donors are
2. You can save big money on appeals
3. You can generate lots more money for your mission
How Do You Do It?1. DECIDE WHAT YOU WANT TO PREDICT2. PICK A LIMITED NUMBER OF POSSIBLE
PREDICTORS3. BUILD A FILE (RANDOM SAMPLE FROM YOUR
DATABASE)4. IMPORT THE FILE INTO A STAT SOFTWARE
APPLICATION5. SPLIT THE FILE IN HALF AT RANDOM6. SEARCH FOR PREDICTORS ON ONE HALF OF THE
FILE AND BUILD A MODEL7. CHECK THE MODEL OUT ON THE OTHER SAMPLE8. TEST THE MODEL9. IMPLEMENT THE MODEL
Let’s Walk Through An Example
from The U of Minnesota Annual Fund
Step 1: Decide What You Want To Predict
Randy Bunney & Pete Wylie decide on:
– Life to date giving– Total number of gifts
Step 2: Pick a Limited Number of Possible
Predictors
These are some of the ones we chose:– Job Title– Gender– Birth Date– Marital Status– Grad Year– Degree Count– Bus Phone– Email
Step 3: Build A Random Sample
IS folks built an Excel file of 10,000 random records from a database with over 700,000 living alumni and friends
Steps 4 & 5:Importing And Splitting
Working over the phone, we imported the excel file into the stat application (Datadesk) and randomly divided the file into two halves of 5,000 records each
Step 6: (on 1/2 of the file )Find predictors. Build a model.
Some of promising predictors we found:– Job title listed (Yes/No)– Marital status listed as “married” (Yes/No)– Born before 1948 (Yes/No)
Job Title Status
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North1.7
6.2
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
NUMBER OF GIFTS
NOT LISTED JOB TITLE LISTED
MEAN (AVERAGE) NUMBER OF GIFTS GIVEN BY WHETHER OR NOT JOB TITLE WAS LISTED IN
DATABASE
Marital Status
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North1.2
4.5
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
NUMBER OF GIFTS
NOT LISTED LISTED AS MARRIED
MEAN (AVERAGE) NUMBER OF GIFTS GIVEN BY WHETHER OR NOT LISTED AS "MARRIED" IN DATABASE
Age as a Factor
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North1.8
6.2
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
NUMBER OF GIFTS
OTHER BORN BEFOR '48
MEAN (AVERAGE) NUMBER OF GIFTS GIVEN BY WHETHER OR NOT BORN BEFORE 1948
The Model We Came Up With
Score = (Bus Phone Good) + (Home Phone Good) + (Job Title Listed) + (Married) + (Born Before 1948)
Step 7: Check Model Against the Other Sample
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North
0.41.4
3.5
9.7 10.1
14.3
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
# GIFTS
S0 S1 S2 S3 S4 S5
SCORE LEVEL
MEAN (AVERAGE) NUMBER OF GIFTS GIVEN BY SCORE LEVEL ON UMINN CROSS VALIDATION SAMPLE
Step 8:Test the Model
0
10
20
30
40
50
60
70
80
90
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
East
West
North
0.2
1.7
3.5 4.2
6.2
8.6
0.0
2.0
4.0
6.0
8.0
10.0
%
S0 S1 S2 S3 S4 S5
SCORE LEVEL
UNIVERSITY OF MINNESOTA MAIL CAMPAIGN: PERCENTAGE OF
GIVING BY SCORE LEVEL
More Testing
0.14 0.832.96
5.568.02
35.42
0.00
10.00
20.00
30.00
40.00
$
S0 S1 S2 S3 S4 S5
SCORE LEVEL
UNIVERSITY OF MINNESOTA MAIL CAMPAIGN: MEAN (AVERAGE)
DOLLARS RECEIVED BY SCORE LEVEL
Step 9: Implement The Model
UM decided to only re-appeal to records scored 3 or above.
An Other Experiment
Oklahoma State University
SCORE = (Bus Phone Yes) + (Oc-Tit Listed) + (Emplr Listed) + (Bus City Listed) + (Stud Org Listed) + (Alum Member) + (Mrtl Code Listed) + (Child Fir Nam Listed) + (Child Birth Date Listed)
Oklahoma State UniversityPledges By Score
5
9
12
15 16 17
19
23
13
0
5
10
15
20
25
%
S0 S1 S2 S3 S4 S5 S6 S7 S8
SCORE LEVEL
OKLAHOMA STATE UNIVERSITY PHONE CAMPAIGN: PERCENTAGES OF ALUMS MAKING PLEDGES BY
SCORE LEVEL
Oklahoma State UniversityDollars Pledged by Score
1.15
2.89
6.16 6.66 6.28
7.74
15.2915.97 16.41
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
$
S0 S1 S2 S3 S4 S5 S6 S7 S8
SCORE LEVEL
OKLAHOMA STATE UNIVERSITY PHONE CAMPAIGN: MEAN (AVERAGE) DOLLARS PLEDGED BY SCORE
LEVEL
What About Major Giving?
Will modeling work as well
as it does for the annual fund?
Data From 5 Other Schools
Large samples of records noting if a person:– Had given a total of $1,000 or more or not to
the school– Had a business phone listed or not– Had an e-mail address listed or not– Had an age of 52 or older listed or not
Business Phone Status
19
6
16
3
12
6
9
54
1
0
2
4
6
8
10
12
14
16
18
20
%
SCHOOL A SCHOOL B SCHOOL C SCHOOL D SCHOOL E
PERCENTAGES OF DONORS GIVING $1000 OR MORE BY WHETHER OR NOT A BUSINESS PHONE IS LISTED
BUS PHONE LISTED
NOT LISTED
E-mail Status
12
7
24
4
14
6
9
65
1
0
5
10
15
20
25
%
SCHOOL A SCHOOL B SCHOOL C SCHOOL D SCHOOL E
PERCENTAGES OF DONORS GIVING $1000 OR MORE BY WHETHER OR NOT AN E-MAIL ADDRESS IS LISTED
E-MAIL LISTED
NOT LISTED
Giving and Age
14
4
26
3
23
4
15
4 4
1
0
5
10
15
20
25
30
%
SCHOOL A SCHOOL B SCHOOL C SCHOOL D SCHOOL E
PERCENTAGES OF DONORS GIVING $1000 OR MORE BY WHETHER OR NOT THEY ARE LISTED AS 52 OR OLDER
52 OR OLDER
OTHER
LET’S LOOK AT AGE AT ONE OF THESE SCHOOLS
33
9
4
1
17
0
5
10
15
20
25
30
35
%
1953 OREARLIER
1954-1962 1963-1969 1970 OR LATER NOT LISTED
PERCENTAGE OF RECORDS GIVING $50,000 OR MORE BY BIRTH YEAR
Multiple Factors
3
11
22
27
2
14
33
44
4
11
16
38
3
9
14
33
13
5
15
0
5
10
15
20
25
30
35
40
45
%
SCHOOL A SCHOOL B SCHOOL C SCHOOL D SCHOOL E
PERCENTAGES OF DONORS GIVING $1000 OR MORE BY WHETHER NONE, ONE, TWO, OR THREE ATTRIBUTES (BUSPHONE, E-MAIL, AND
52 OR OLDER) ARE LISTED
NONE
ONE
TWO
THREE
Modeling: In-house Or Have It Done For You?
• Doing it all by yourself isn’t feasible.
• Besides, there are excellent products and services out there that shouldn’t be ignored.
A NEW KIND OF RESEARCHER IN ADVANCEMENT?
• Without an inside specialist, the data enhancement products and services you purchase are less likely to be used effectively. A blunt question. In the past five years, have you spent more than $25,000 on enhancing your database with estimates of wealth, capacity to give, and so on, only to have the information untapped and unused by your development officers? Why buy the stuff if you’re not going to use it? An inside data analyst can not only help you use data effectively, he or she also can be a persistent thorn in your side until you do use it.
• A good data analyst is worth the effort and cost of creating a new position.
In-house Modeling & Analysis
Worth the consideration because of:• The richness of info in your database
• Speed
• Continuity
• The “Big Picture”
• Vendor screening
Questions & Comments