Detecting Changes in Social Networks Using Statistical

Process Control

Cadet Matthew R. Webb

Advisor: Major Ian McCulloh, D/Math USMA

20 May 2007

NetSci 2007 Contributed Talk

Agenda• Social Network Change Detection

– Components• Social Network Analysis• Statistical Process Control

– Feasibility Study• Application: Al-Qaeda

– Method– Results– Lessons Learned

• Application: TOEP– Models

• Future Researchand Applications

What is Social Network Change Detection?• Social Network Analysis

– the mathematical methodology of quantifying connections between individuals and groups.

• Statistical Process Control (SPC)– Calculates a statistic from sequential

measurements of a random process and compares it to a control limit.

– Used extensively in quality engineering

Network Measures• Density

• Betweenness Centrality

• Closeness Centrality

• Many More

Graph courtesy of Steve Borgatti

Statistical Process Control• Cumulative-Sum Control Chart

– Good at detecting small changes in mean over time– Built-in change point detection– Requires normally distributed data

• Calculate Zt transform value for each time-period, t.

• Two Charts (To Detect An Increase or Decrease)

• Chart Signals when C+ or C- statistic exceeds control limit

• Reference value and control limit are arbitrarily set for each process

/0 tt xZ

},0max{ 1

ttt CkZC

},0max{ 1

ttt CkZC

Feasibility Study• Cadet Summer Training

– How does communication effect an organization’s performance?

– Can changes in communication patterns be detected in larger, well-established organizations?

TimeTime

ExperienceExperience

Al-Qaeda Application

• The Office of Naval Research maintains network data on Al-Qaeda.

• Ran CUSUM Control Chart for three Network Measures (Betweenness, Closeness, and Density)

Al-Qaeda Network Measures

0

0.002

0.004

0.006

0.008

0.01

0.012

1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004

Time

Mea

sure

Val

ue

Betweeness

Density

Closeness

Closeness CUSUM Statistic for Al-Qaeda (1994-2004)

0

1

2

3

4

5

6

7

8

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004

C+

Results

Change Point

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004Closeness 0.0027 0.003 0.0028 0.0028 0.0031 0.003 0.0032 0.0034 0.0024 0.0015 0.0004Z -0.8729 1.0911 -0.2182 -0.2182 1.7457 1.0911 2.4004 3.7097 -2.8368 -8.7287 -15.9299C+ 0 0.5911 0 0 1.2457 1.8368 3.7372 6.9469 3.6101 0 0C- 0.3729 0 0 0 0 0 0 0 2.3368 10.5655 25.9955

Signal 0 0 0 0 0 0 0 1

The control chart signals a change in the network in 2001.

Most Likely Estimate of the change point is 1997: - Re-establish base in Afghanistan - Terrorist Attack in Luxor, Egypt - Feb ’98 Islamic Front - Embassy bombings in ’98

1997 was a critical planning year for Al-Qaeda

Control Limit = 4

Signal

Lessons Learned

• SPC Change Detection Method Can Detect Changes in a Social Network.

• Further Testing Requirements– Need Dataset with Higher Resolution– More Complete Network Information

TOEP Application• Tactical Officer Education Program is a master’s

degree program for 24 Army officers.• Patch on their OUTLOOK allows collection of all

sent emails and reconstruction of their email network.

Results• Monitored Email Traffic for 24 Weeks

– Collected viable data from 9 TOEPs– Network measures normally distributed

• Developed Model of TOEP Closeness

TOEP Email Network Measures

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Week

Val

ue

Betweenness

Closeness

Density

Regression Model• Used data from first semester to develop model

– TOEP planning calendar and interviews to establish possible predictors (Group Projects, Administrative Changes, Academic Requirements, and Planned Social Outings).

• Closeness Model

• Conducted SPC on closeness predictions to detect network changes during second semester

Closeness = 0.18 - 0.11 Group Projects + 0.11 Social + 0.0074 # of Emails

Predictor Coef SE Coef T P VIFConstant 0.18 0.034 5.40 0.000Group Projects -0.11 0.050 -2.13 0.046 1.3Social 0.11 0.040 2.89 0.009 1.3# of Emails 0.0074 0.00084 8.77 0.000 1.0

S = 0.096 R2 = 82.4% R2(adj) = 79.8%PRESS = 0.30 R2(pred) = 70.9%

Most Likely Estimate of the change point is Week 21: - The week prior to Spring Break - Two weeks prior to Comprehensive Exam - Interviews with TOEPs cite this as the last week for study before the exam

Week 21 represents a change in previous group dynamics from first

semester

Closeness Model CUSUM Statistic for TOEPs (21 JAN - 31 MAR)

0

0.5

1

1.5

2

2.5

3

3.5

4

15 16 17 18 19 20 21 22 23 24

Week Number

C-

SPC on ClosenessWeek 15 16 17 18 19 20 21 22 23 24

20070121 20070128 20070204 20070211 20070218 20070225 20070304 20070311 20070318 20070325Closeness 0.3332 0.5134 0.2760 0.3332 0.5406 0.6536 0.4977 0.1258 0.2646 0.5226Model 0.4712 0.3798 0.3798 0.3562 0.5243 0.5745 0.3916 0.2913 0.4215 0.4152Z -1.971 1.909 -1.483 -0.329 0.233 1.130 1.516 -2.364 -2.241 1.534C+ 0 1.409 0 0 0 0.630 1.646 0 0 1.034C- 1.471 0 0.983 0.811 0.079 0.000 0 1.864 3.606 1.571

Signal 0 0 0 0 0 0 0 0 1

Change Point

Control Limit = 3

Signal

Future Research and Applications• Further Research

– Continue to Monitor TOEP Email Network• Compare Data across multiple years• Optimize k value and control limit to detect desired type of

changes

– Future Datasets• Use networks with less variance• Monitor a “more routine network” to establish better

baselines

– Examine Possible Relationships between Network Measures and an Organization’s Performance

• Applications for this Method– Evaluating Friendly Command and Control Networks– Monitoring Terrorist Communication Networks

USMA Network Science Workshops

• USMA/ARI Network Science Workshop 18-20 April 2007

http://www.netscience.usma.edu/nsw/

• USMA/ASA-ALT Network Science Workshop 18-20 April 2007

Questions?