extracting places and activities from gps traces using hierarchical conditional random fields...
TRANSCRIPT
![Page 1: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/1.jpg)
Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields
2012311529Yong-Joong Kim
Dept. of Computer ScienceYonsei University
Lin Liao, Dieter Fox, and Henry Kautz, In International Journal of Robotics Research (IJRR), 26(1), 2007
![Page 2: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/2.jpg)
Contents• Motivation
• Hierarchical Activity Model
• Preliminaries : Conditional Random Fields– Overview– Inference– Parameter Learning
• Conditional Random Fields for Activity Recognition– GPS to street map association– Inferring activities and types of significant places– Place detection and labeling algorithm
• Experimental Results– Experimental environment– Example analysis– Extracting significant places– Labeling places and activities using models learned form others
• Conclusions
![Page 3: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/3.jpg)
Motivation (cont’)
• Application areas of learning patterns of human behavior from sensor data– Intelligent environments– Surveillance– Human robot interaction
• Using GPS location data to learn to recognize the high-level activities
• Difficulties in previous approaches– Restricted activity models– Inaccurate place detection
![Page 4: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/4.jpg)
Motivation
• A novel, unified approach to automated activity and place labeling– High accuracy in detecting significant places by taking a user’s context into
account– By simultaneously using CRF (Conditional Random Field)
• Estimating a person’s activities• Identifying places • Labeling places by their type
• Research goal– To segment a user’s day into everyday activities– To recognize and label significant places
![Page 5: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/5.jpg)
Hierarchical activity model (cont’)
• GPS readings– Input to proposing model– Segmenting a GPS trace spatially in order to generate a discrete sequence of
activity nodes
• Activities– Being estimated for each node in the spatially segmented GPS trace– Distinguishing between navigation activities and significant activities
• Significant places– Playing a significant role in the activities of a person
![Page 6: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/6.jpg)
Hierarchical activity model
• Two key problems for probabilistic inference– Complexity of model
• Solved by approximating inference algorithm
– Not clear how to construct the model deterministically from a GPS trace• Solved by constructing the model as part of this inference
![Page 7: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/7.jpg)
Preliminaries : Conditional Random Fields
![Page 8: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/8.jpg)
Overview (cont’)
• Definition of CRFs– Undirected graphical models developed for labeling sequence data– Properties
• Directly represent the conditional distribution over hidden states • No assumptions about the dependency structure between observations
• Nodes in CRFs– Observation :– Hidden states : – Defining conditional distribution over hidden states y
• Cliques– Fully connected sub-graphs of a CRF– Playing a key role in the definition of conditional distribution
Preliminaries: Conditional random fields
![Page 9: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/9.jpg)
Overview
• Conditional distribution over hidden state :
where
Preliminaries: Conditional random fields
![Page 10: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/10.jpg)
Inference (cont’)
• Inference in CRF can have two tasks :– To estimate the marginal distribution of each hidden variable – To estimate the most likely configuration of the hidden variables (i.e. the maximum a posteriori, or MAP, estimation)– Using Belief propagation to solve these tasks
• Two types of BP algorithms : – Sum-product for marginal estimation– Max-product for MAP estimation
Preliminaries: Conditional random fields
![Page 11: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/11.jpg)
Inference (cont’)
• Sum-product for marginal estimation– Message initialization :
• Initializing all messages as uniform distr. over
– Message update rule :
– Message update order :• Iterating the message update rule until it (possibly) converges
– Convergence conditions :
– After convergence, calculation of marginals
Preliminaries: Conditional random fields
![Page 12: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/12.jpg)
Inference
• Max-product for MAP estimation– Very similar to the sum-product– Replaced summation with maximization in the message update rule
– After convergence, calculating the MAP belief
– Then, each component of
Preliminaries: Conditional random fields
![Page 13: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/13.jpg)
Parameter learning (cont’)
• Goal of parameter learning– To determine the weights of the feature functions– Learn the weights discriminatively
• Two method– Maximum likelihood (ML) estimation– Maximum pseudo-likelihood (MPL) estimation
• Parameter sharing– Learning algorithm to learn the same parameter values (weights) for different
cliques in the CRF
Preliminaries: Conditional random fields
![Page 14: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/14.jpg)
Parameter learning (cont’)
• Maximum likelihood (ML) estimation
– Object function
– The gradient of object function
Preliminaries: Conditional random fields
![Page 15: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/15.jpg)
Parameter learning (cont’)
• Maximum pseudo-likelihood (MPL) estimation
• : local feature counts involving variable •
– Object function
– The gradient of object function
Preliminaries: Conditional random fields
![Page 16: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/16.jpg)
Parameter learning
• Parameter sharing– Learn a generic model that can take any GPS trace and classify the locations in
that trace
– Achieved by making sure that all the weights belonging to a certain type of feature are identical
– Calculating gradient for a shared weight by the sum of all the gradients computed for the individual cliques
Preliminaries: Conditional random fields
![Page 17: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/17.jpg)
Conditional Random Fields for Activity Recognition
![Page 18: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/18.jpg)
GPS to street map association (cont’)
• Desirable to associate GPS traces to a street map– (e.g.) to relate locations to addresses in the map
• Constructing a CRF – Taking into account the spatial relationship between GPS readings– Generating a consistent association
Conditional Random Fields for Activity Recognition
![Page 19: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/19.jpg)
GPS to street map association (cont’)
• Distinguishing tree types of cliques– Measurement cliques (dark grey)
– Consistency cliques (light grey)
– Smoothness cliques (medium grey)
Conditional Random Fields for Activity Recognition
![Page 20: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/20.jpg)
GPS to street map association
• Using these feature function, conditional distribution can be written as :
– : measurement feature function weight– : consistency feature function weight– : smoothness feature function weight
Conditional Random Fields for Activity Recognition
![Page 21: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/21.jpg)
Inferring activities and types of significant places (cont’)
• Generating a new CRF, to estimate– Activity performed at each segment– A person’s significant places
Conditional Random Fields for Activity Recognition
![Page 22: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/22.jpg)
Inferring activities and types of significant places
• Activity node’s features– Temporal information such as time of day, day of week, duration of the stay– Average speed through a segment– Information extracted from geographic databases– Connected to its neighbors
• Place node’s feature– Activities that occur at a place strongly (consider weekly frequency)– A limited number of different homes or work places
• Possibility of generating very large cliques– Resolve this problem by converting to tree-structured CRFs
Conditional Random Fields for Activity Recognition
![Page 23: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/23.jpg)
Place detection and labeling algorithmConditional Random Fields for Activity Recognition
![Page 24: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/24.jpg)
Experimental Results
![Page 25: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/25.jpg)
Experimental environment
• Collected GPS data from four different persons– Seven days of data– Roughly 40,000 GPS measurements (10,000 segments)– Manually labeled all activities and significant places
• Using leave-one-out cross-validation for evaluation– Training data : 3 persons (MPL estimation for learning)– Testing data : 4 persons
Experimental Results
![Page 26: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/26.jpg)
Example analysisExperimental Results
![Page 27: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/27.jpg)
Extracting significant places
• Comparing experiment– Proposing system– A widely-used approach (time threshold)
Experimental Results
![Page 28: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/28.jpg)
Labeling places and activities using models learned form others (cont’)
Experimental Results
![Page 29: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/29.jpg)
Labeling places and activities using models learned form others
Experimental Results
![Page 30: Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields 2012311529 Yong-Joong Kim Dept. of Computer Science Yonsei](https://reader035.vdocuments.mx/reader035/viewer/2022070412/56649e175503460f94b03340/html5/thumbnails/30.jpg)
Conclusions
• A novel approach to performing location-based activity recognition– One consistent framework– Iteratively constructing a hierarchical CRF– Discriminative learning using pseudo-likelihood– Being performed the Inference efficiently using loopy BP
• Achieving virtually identical accuracy both with and without a street map