behaviometrics: behavior modeling from heterogeneous sensory time-series
TRANSCRIPT
1
Thesis Defense for the degree of Doctor of Philosophy Electrical and Computer Engineering Carnegie Mellon University
Jiang Zhu [email protected]
Thesis Committee
Prof. Joy Zhang, Chair Prof. Jason Hong
Prof. Patrick Tague Dr. Fabio Maino, Cisco Research
2
• Data collected from physical and soft sensors • Identify various behavioral factors from sensor streams • Build behavioral models to generate quantifiable metrics • Apply these models to a mobile security application: SenSec
Study the fundamental scientific problem
modeling a mobile user’s behavior
heterogeneous sensory time-series
of from
3
4
North America
Global
0
50
100
150
200
250
300
2011 2011-2016
15.5 58.3
33.6
256.7 Tablets
North America
Global
0
500
1000
1500
2000
2500
2011
2011-2016
138 253
698
2027 Smartphones
• Cisco Visual Networking Index VNI 2012, Cisco Systems Inc. 2012
Smartphone Tablets 3-82-4 2016
5
0%
10%
20%
30%
40%
50%
60%
Mobile Device Loss or theft
Strategy One Survey conducted among a U.S. sample of 3017 adults age 18 years older in September 21-28, 2010, with an oversample in the top 20 cities (based on population).
• “The 329 organizations polled had collectively lost more than 86,000 devices … with average cost of lost data at $49,246 per device, worth $2.1 billion or $6.4 million per organization.
"The Billion Dollar Lost-Laptop Study," conducted by Intel Corporation and the Ponemon Institute, analyzed the scope and circumstances of missing laptop PCs.
6
Reliable Non-intrusive
Password
Application
Usability
A major source of security vulnerabilities. Easy to guess, reuse, forgotten, shared
Different applications may
have different sensitivities
Authentication too-often or sometimes too loose
7
Passwords Normal passwords are not strong enough: usually meaningful words that can be remembered Stringent strong password can be annoying
Most users do not use the password-aid tools (Hong et al. 2009)
Fingerprint? Iris recognition? Face recognition? Voice recognition?
Password for the DHS E-file: Contain from 8 to 16 characters Contain at least 2 of the following 3 characters: uppercase alphabetic, lowercase alphabetic, numeric Contain at least 1 special character (e.g., @, #, $, %, & *, +, =) Begin and end with an alphabetic character Not contain spaces Not contain all or part of your UserID Not use 2 identical characters consecutively Not be a recently used password
8
• Provides a way to passively authenticate while using common, sensitive applications and services.
• Allows for rapid detection of unauthorized users Block their access as quickly as possible.
• Uses a variety of sensors available on common smartphones
8
9
• Derived from
• Behavioral: the way a human subject behaves
• Biometrics: technologies and methods that measure and analyzes biological characteristics of the human body
• Finger prints, eye retina, voice patterns
• Behaviometrics: Measurable behavior to Recognize or to Verify • Identity of a human subject, or
• Subject’s certain behaviors
Behavioral Biometrics Behaviometrics
10
• Mobile devices come with embedded sensors • Accelerometers, gyroscope, magnetometer • GPS receiver • WiFi, Bluetooth, NFC • Microphone, camera, • Temperature, light sensor • “Clock” and “Calendar”
• Connect with other sensors • EEG, EMG, GSR
• Mobile devices are connected with the Internet • Upload sensor data to the cloud • Viewing information computing on the server side
• Users carry the device almost at all time. • My phone “knows” where I am, what I am doing and my future
activities.
11
• Motion Metrics
• Location Metrics
• Interaction Metrics
• System Metrics
• Accelerometer • activity, motion, hand trembling, driving
style • sleeping pattern • inferred activity level, steps made per
day, estimated calorie burned • Motion sensors, WiFi, Bluetooth
• accurate indoor position and trace. • GPS
• outdoor location, geo-trace, commuting pattern
• Microphone, camera: • From background noise: activity, type
of location. • From voice: stress level, emotion • Video/audio: additional contexts
• Keyboard, touches, slides • Specific tasks, user interactions, …
12
• Monitor and track user behavior on smartphones using various on-device sensors
• Convert sensory traces and other context information to Behaviometrics
• Build statistical models with these features and use them for calculation of Certainty Scores as security measure
• Trigger various secondary Authentication Schemes when certain application is launched or certain system function is invoked.
13
Raw Data Preprocessing
Modeling
Metrics
Evaluation
Metrics
Metrics Ground Truth
Metrics
Metrics
+
14
Heterogonous Sensor Data
Feature Extraction Behavioral Text. Frequency Rep.
n-gram Skipped n-gram Helix, NN,LR,
DT, RF, SVM…
Motion Metrics
Accuracy TPR & FPR DR & FRR
Location Metrics
Interaction Metrics
Sim. Attacks Ctrl. Exp.
Auth. Records
MobiSens Framework
+
15
16
• Human behavior/activities share some common properties with natural languages
• Meanings are composed from meanings of building blocks • Exists an underlying structure (grammar) • Expressed as a sequence (time-series)
• Apply rich sets of Statistical NLPs to mobile sensory data
3
3.5
4
4.5
5
5.5
6
0 20 40 60 80 100 120 140 160 180 200
log
(fre
q)
Rank of words by frequency
Zipf’s Law
17
• Is this play Shakespeare’s work?
• Comparing the play to Shakespeare’s known library of works
• Track words and phases patterns in the data
• Calculate the probability the unknown U given all the known Shakespeare’s work {S}
• Compare with a threshold θ • Authentic work (a=1) • Fake, Forgery or Plagiarism (a=0)
a = sign[P (U |{S}) > �]
18
Quantization Clustering
19
• Convert feature vector series to label streams – dimension reduction
• Step window with assigned length
A1 A2 A1 A4
G2 G5 G2 G2
W2 W1 W2
P1 P3 P6 P1
A2 G2G5 W1 P1P3 A1A4 G2 W1W2 P1
20
Applications
Modeling
PreprocessingSensing
Feature Construction
Behavior Text Generation
N-gramModel
Classification
Threshold
Skipped N-gramModel
Helix
DT
EvaluationROC Accuracy
PrecisionRecall UX
Prediction RecognitionSVMDT SVM
Ground Truth
Anom
aly D
etection
...
21
Quantization
Risk Analysis Tree
Clustering
Activity Recognition
<
Application Sensitivity
Application Access Control
Certainty of Risk
Sensor Fusion and Segmentation
Application Access Control
22
Inference
ModelingPreprocessingSensing
FeatureConstruction
Behavior TextGeneration
N-gramModel
Cla
ssifie
rB
inary
Cla
ss
ifierThreshold
UserAuthentication
UserClassification
• SenSec collects sensor data • Motion sensors • GPS and WiFi Scanning • In-use applications and their traffic patterns
• SenSec modulebuild user behavior models • Unsupervised Activity Segmentation and model the sequence using Language model • Building Risk Analysis Tree (DT) to detect anomaly • Combine above to estimate risk (online): certainty score
• Application Access Control Module activate authentication based on the score and a customizable threshold.
23
• Accelerometer • Used to summarize acceleration stream • Calculated separately for each dimension [x,y,z,m] • Meta features: Total Time, Window Size
• GPS: location string from Google Map API and mobility path
• WiFi: SSIDs, RSSIs and path
• Applications: Bitmap of well-known applications
• Application Traffic Pattern: TCP UDP traffic pattern vectors: [ remote host, port, rate ]
24
• Offline data collection (for training and testing) Pick up the device from a desk Unlock the device using the right slide pattern Invoke Email app from the "Home Screen” Some typing on the soft keyboard Lock the device by pressing the "Power" button Put the device back on the desk
25 25
26
• 71.3% True-Positive Rate with 13.1% False Positive with only 2-3 days of training data
27
• Various Behaviometric applications be framed as a classification problem
• Prerequisite: a learned Behaviometric model • Input: a given observation of user’s behavior, • Output: some decisions based on the observation and the model
• Behavioral text representation may lead to huge features space • Algorithms to handle large feature space • Smart feature set construction to limit size of the feature space
• Identification and anomaly detection work better if the users are performing the same activity
• Activity recognition before identification/detection
• High FPR • Incorporate other factors and metrics • UX improvements
28
29
• The language model in general builds a single model for all types of activities.
• Often the way people perform a certain activity is enough to distinguish them(as in PoC).
• Models to identify between the diff. of how people perform the same, or same class of activities.
0
5
10
15
20
0 20 40 60 80 100 120 140
acceleration
time
Accelerometer X-axis
standingwalkingrunning
Activity Class
Extraction
AC-1 Model
AC-2 Model
AC-n Model
…
30
4.3 MODELING PERSONAL FACTORS FROM MOTION SENSOR DATA 50
0
5
10
15
20
0 20 40 60 80 100 120 140
acceleration
time
Accelerometer X-axis
standingwalkingrunning
0
100
200
300
400
500
600
700
0 1 2 3 4 5 6 7
amplitude
frequency
Accelerometer X-axis
standingwalkingrunning
0
5
10
15
20
0 20 40 60 80 100 120 140
acceleration
time
Accelerometer Y-axis
standingwalkingrunning
0
200
400
600
800
1000
1200
1400
0 1 2 3 4 5 6 7
amplitude
frequency
Accelerometer Y-axis
standingwalkingrunning
0
5
10
15
20
0 20 40 60 80 100 120 140
acceleration
time
Accelerometer Z-axis
standingwalkingrunning
0
100
200
300
400
500
600
0 1 2 3 4 5 6 7
amplitude
frequency
Accelerometer Z-axis
standingwalkingrunning
Figure 4.7: Accelerometer readings for standing, walking and running along with the discretefourier transformation
31
• Extract the activity level of motion time series data by examining magnitude of the data and frequency composition via DFT
• Activity Level =
• Activity Class can be arbitrary segments of this function which has range [0, 1)
0
5
10
15
20
0 20 40 60 80 100 120 140
acce
lera
tion
time
Accelerometer Y-axis
standingwalkingrunning
0
200
400
600
800
1000
1200
1400
0 1 2 3 4 5 6 7
ampl
itud
e
frequency
Accelerometer Y-axis
standingwalkingrunning
32
• t-SNE to map feature space into 2-D plane
• First tested activity recognition method on ambulatory behavior with 381 samples of standing, walking, and running. Correctly classifies 359/381 samples, giving an accuracy of 94.23%
• For user identification we ranked the features and used the top features to create models using classical ML algorithms
33
• We approach the authentication problem by building a Multivariate Gaussian distribution for each activity class
• We fit the parameters, mean and standard deviation, for each feature for a training set for which we define to contain only non-anomalous data
• The calculate the probability of some unknown data being generated by the model with
34
• Each range is an activity class with lower ranges representing activities with low magnitude or low frequency and higher ranges representing activities with high magnitude and high frequency.
• 110GB dataset of accelerometer data from 25 users.
35
• Using only motion data can lead to a very high false positive rate
• Combine motion with location factor to mitigate
• Verify most users spend the majority time in a small number of places. Incorporate location in experiments lead to reduced FPRs
36
37
• Hypothesis: the micro-behavior a user interacts with the soft keyboard reflects his/her cognitive and physical characteristics.
Cognitive fingerprints: typing rhythms, correction rate, delay between keys, duration at each key…. Physical characteristics: area of pressure, amount of pressure, position of contact, shift …
38
• Keystroke Dynamics are a popular subject Many papers—focusing primarily on desktops
Great success for passwords, good success for arbitrary text
Typing rate, key-to-key latencies are the primary features
Once people are skilled at typing, they develop natural rhythms (on desktops)
• Detecting keystroke patterns on mobile phones is challenging Focus on Desktop-like attributes
Typing rate, timing, di-graphs, tri-graphs, etc.
• Use background applications to “sniff” keystrokes Without direct access to keyboard
Successful demonstrations using accelerometers
38
Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 39
• Frequent use • Typically single user
• Context awareness • Protected applications vs. Non-
protected • Current and historical patterns
• Touchscreens provide wealth of data • Touch location, pressure, finger
size, finger drift • Wide variety of other sensors
• Accelerometers, gyroscopes
• Limited computing power • Need to use efficient algorithms
• Finite battery life • Users are sensitive to battery life
impact • Highly mobile
• Typical usage: lying down, sitting, walking, passenger in car/train/subway system
• Need to behave gracefully
40
• Location pressed on keys • Length of press
key down to key up • Force of press
Varies across device types • Change of force over key press • Size of finger
Varies across device types • Drift of finger during press • Recent accelerometer history
41
42
43
44
• From finger down to finger up
45
46
• 13 initial users after short recruiting drive
• 2 week long collection period
• 86,000 keystrokes
• 430,000 data points @ ~5/keystroke
• Data split into training and testing:
• 99% accuracy with just 1 key stroke
Training Data for Model 50%
CV 15%
Training for Keys 15%
Final Testing 15%
CV for Keys 10%
47
48
Anomaly Detection Rate: 67.7%
FRR(FPR):4.6%
49
Anomaly Detection Rate:84.8%, increased from 67.7% for 5 strokes
FRR(FPR): 2.2%, reduced from
4.6% for 5 strokes
50
51
• Alpha test started in 6/2012, 1st Google Play Store release in 10/2012
• False Positive: 13% FPR still annoying users sometimes
• 75% alpha users mentioned feeling annoyed when SenSec prompted for passcode, at least once. Some of them are confused by the UI.
“I couldn’t get the passcode right multiple times when trying to answer an important phone call because I was single-handed”
“I just entered my passcode a min ago and it asked me again” “I was so lost about this UI. What’s this training and testing knob?”
52
• Use adaptive model • Adding the trace data shortly before a false positive to the training data and
update the model
• Add sliding pattern in addition to passcode validation
• A confirmed false positive will grant a “free ride” for a configurable duration
• Assumption: just authenticated user should control the device for a given period of time
• “free ride” period will end immediately if abrupt context change is detected.
• Interaction Metrics added as part of the sensory fusion • Motion, location and interaction metrics work together • Different user flow: Reactive triggering vs. Proactive triggering
53
6.1 USER INTERFACE AND USER EXPERIENCE CHALLENGES 118
Figure 6.3: SenSec Protection Options: Secured, Protected via either Passcode or Sliding Pattern
Figure 6.4: SenSec Home Screen
54
6.1 USER INTERFACE AND USER EXPERIENCE CHALLENGES 117
Figure 6.2: SenSec Tutorial
6.1 USER INTERFACE AND USER EXPERIENCE CHALLENGES 118
Figure 6.3: SenSec Protection Options: Secured, Protected via either Passcode or Sliding Pattern
Figure 6.4: SenSec Home Screen
55
• Data collected from physical and soft sensors • Identify various behavioral factors from sensor streams • Build behavioral models to generate quantifiable metrics • Apply these models to a mobile security application: SenSec
Study the fundamental scientific problem
modeling a mobile user’s behavior
heterogeneous sensory time-series
of from
56
• Propose a concept Behaviometric to study the fundamental scientific problem of modeling a mobile user’s behaviors from heterogeneous sensor time series
• Adopt a Language approach to solve Behaviometric problems via various NLP techniques
• Unsupervised algorithm Helix to discover the hierarchical structure, i.e. grammar, in activity recognition
• Investigate existing statistical learning algorithms being adopted and applied to Behaviometric context when NLP approach is less sufficient
• Derive an effective yet simple “activity level” metric from time series to be used in identification and detection. Use location metrics to augment motion metrics to reduce FPR.
• Develop and deploy versions of SenSec app and adapt through user feedbacks.
57
• Extended data set for System Metrics TCP, UDP traffic; sound; ambient lighting; battery status, etc.
• Data and Modeling Gain more insights into the data, features and factorized relationships among various sensors
• Enhanced security of SenSec components prepared for commercial release
Integration with Android security framework and other applications
• Privacy as expectation (Liu et al., 2012) Users need to know where the data resides, how the data is going to be used and shared. Whom to trust the data with?
• Energy efficiency
58
• CyLab at Carnegie Mellon
• Northrop Grumman Cybersecurity Research Consortium
• Cisco Research aware for “A Language Approach in Behavioral Modeling” Research award for “Privacy Preserved Personal Big Data Analytics through Fog Computing’’
• Special thanks: Dr. Hao Hu, Jiatong Zhou, Dr. Flavio Bonomi, Cisco Systems; Sky Hu, Twitter Inc.; Yuan Tian, Google Inc.; Pang Wu, Samsung Research North America.
58
Cybersecurity Research Consortium
59
“Mobile behaviometrics: Models and applications” In Proceedings of the Second IEEE/CIC Inter- national Conference on Communications in China (ICCC), Xi’An, China, August 12-14 2013., [with H.Hu, S.Hu, P.Wu, J.Zhang]
“MobiSens: A Versatile Mobile Sensing Platform for Real-world Applications”, MONE, 2013, [with P.Wu, J.Zhang]
"SenSec: Mobile Application Security through Passive Sensing," to appear in the Proceedings of International Conference on Computing, Networking and Communications. (ICNC 2013). San Diego, USA. January 28-31, 2013 [with P.Wu, X.Wang, J.Zhang]
“Towards Accountable Mobility Model: A Language Approach on User Behavior Modeling in Office WiFi Networks”, accepted to ICCCN 2011, Maui, HI, Aug 1-5, 2011 [with Y.Zhang]
"Retweet Modeling Using Conditional Random Fields," in the Proceedings of DMCCI 2011: ICDM 2011 Workshop on Data Mining Technologies for Computational Collective Intelligence, December 11, 2011.[ with H.Peng, D.Piao, R.Yan and Y.Zhang]
“Mobile Lifelogger - recording, indexing, and understanding a mobile user's life", in the Proceedings of The Second International Conference on Mobile Computing, Applications, and Services, Santa Clara, CA, Oct 25-28, 2010 [With S.Chennuru, P.Cheng, Y.Zhang]
"SensCare: Semi-Automatic Activity Summarization System for Elderly Care", MobiCase 2011, Los Angeles, CA, October 24-27, 2011. [with Pang Wu, Huan-kai Peng,Joy Ying Zhang] "Helix: Unsupervised Grammar Induction for Structured Human Activity Recognition," to appear in the Proceedings of The IEEE International Conference on Data Mining series (ICDM), Vancouver, Canada, Dec 11-14, 2011.[with Huan-Kai Peng, Pang Wu, and Ying Zhang] "Statistically Modeling the Effectiveness of Disaster Information in Social Media," to appear in the Proceedings of IEEE Global Humanitarian Technology Conference (GHTC), Seattle, Washington, Oct. 30 - Nov. 1st, 2011.[with Fei Xiong, Dongzhen Piao, Yun Liu, and Ying Zhang] "A dissipative network model with neighboring activation," to appear in THE EUROPEAN PHYSICAL JOURNAL B.[with F. Xiong, Y. Liu, J. Zhu, Z. J. Zhang, Y. C. Zhang, and J. Zhang]
"Opinion Formation with the Evolution of Network," to appear in the Proceedings of 2011 Cross-Strait Conference on Information Science and Technology and iCube, TaiBei, China, Dec 8-9, 2011.[with F.Xiong, Y.Liu, Y.Zhang]
Thank you.
61
Location
Time of the Day
Day of the week
R =f(handholding,
indoor loc, app)Alert!
Activity
R =f(WiFi trace,
app, time)
Traveling Speed
Gait R =f(geo trace,
app, time)
At home At work Between home and work
8am-6pm Other
Mon.-Fri. Sat. & Sun idle
Walking Driving
1