dissertation new2

7/31/2019 Dissertation New2

1/44

Ricky Jones

Movement classification using Android sensor data

Computer Science Tripos

Jesus College

2012


2/44


3/44

Proforma

Name Ricky Jones

College Jesus College

Project Title Movement classification using Android sensor data

Examination Computer Science Tripos Part II July 2012

Approximate Word Count 9,200

Project Originator Kiran Rachuri

Project Supervisor Kiran Rachuri

Original Aims

The aim of the project was to investigate, implement and evaluate various algorithms for movement

classification using Android accelerometer data. Movement will be classed as stationary, walking,

cycling etc. This would be done by developing an application that would allow multiple algorithms to

be used and tested. The algorithms will be evaluated by accuracy and energy usage. The project

aimed to investigate efficient methods such as adaptive sampling, where a low power classifier

triggers a more energy intensive classifier.

Work Completed

I have achieved the original aims and have met the success criteria. Four main classification

algorithms are investigated, and their accuracy and energy usage evaluated.

Special Difficulties

None


4/44


5/44

Declaration of Originality

I, Ricky Jones of Jesus College, being a candidate for Part II of the Computer Science Tripos, hereby

declare that this dissertation and the work described in it are my own work, unaided except as may

be specified below, and that the dissertation does not contain material that has already been used

to any substantial extent for a comparable purpose.

Signed:

Date:


6/44


7/44


8/44

Table of contents

1 Introduction ......................................................................................................................................... 1

1.1 Related Work ................................................................................................................................ 1

1.1 Motivation ..................................................................................................................................... 1

1.1.1 Exercise monitoring ............................................................................................................... 1

1.2.2 General context awareness ................................................................................................... 2

1.2 Results ........................................................................................................................................... 2

2 Preparation .......................................................................................................................................... 4

2.1 Overview ....................................................................................................................................... 4

2.2 Background research .................................................................................................................... 4

2.3 Project management..................................................................................................................... 5

2.3.1 Requirements analysis ........................................................................................................... 5

2.3.2 Planning and management .................................................................................................... 5

2.4 Setting up the development environment ................................................................................... 5

2.4.1 Android development ............................................................................................................ 5

2.4.2 Version control and backup ................................................................................................... 6

2.5 Android development ................................................................................................................... 6

2.5.1 Activity lifecycle ..................................................................................................................... 6

2.5.2 Android manifest ................................................................................................................... 6

2.5.3 Data storage ........................................................................................................................... 7

2.6 MATLAB ......................................................................................................................................... 7

3 Implementation ................................................................................................................................... 8

3.1 Overview ....................................................................................................................................... 8

3.2 Ground Truth Data Application ..................................................................................................... 8

3.2.1 Requirements ......................................................................................................................... 83.2.2 Implementation ..................................................................................................................... 9

3.3 Classification ............................................................................................................................... 10

3.3.1 Classification methodology .................................................................................................. 10

3.3.2 Feature extraction ................................................................................................................ 11

3.3 MATLAB scripts ........................................................................................................................... 13

3.3.1 Loading the data .................................................................................................................. 13

3.3.2 Cleaning the data ................................................................................................................. 14

3.3.3 Feature extraction ................................................................................................................ 15


9/44

3.3.4 Appling the functions to the collected data......................................................................... 15

3.4 Classification Application ............................................................................................................ 16

3.4.1 Requirements analysis ......................................................................................................... 16

3.4.2 Overview .............................................................................................................................. 16

3.4.2 Application structure ........................................................................................................... 16

3.4.3 Simple classifier .................................................................................................................... 17

3.4.5 Nave Bayes classifier ........................................................................................................... 18

3.4.6 k-NN classifier ...................................................................................................................... 19

3.4.7 Voting classifier .................................................................................................................... 20

3.4.8 Adaptive classifier ................................................................................................................ 20

3.4.9 Additional features .............................................................................................................. 20

4 Evaluation .......................................................................................................................................... 22

4.1 Overview ..................................................................................................................................... 22

4.2 Offline evaluation of classification algorithms............................................................................ 22

4.3 On device testing ........................................................................................................................ 23

4.4 User testing ................................................................................................................................. 24

4.5 Energy usage ............................................................................................................................... 26

5 Conclusions ........................................................................................................................................ 29

5.1 Project success ............................................................................................................................ 29

5.2 Further work ............................................................................................................................... 29

Bibliography .......................................................................................................................................... 31

Appendices ............................................................................................................................................ 32


10/44


11/44


12/44

1

1 Introduction

This project aims to develop a mobile phone application that can determine what type of movement

the user (who is carrying the device) is doing. Most smartphones today contain a variety of sensors

that collect useful information, such as location (GPS), the orientation of the phone and the amount

of light falling on the phone. This dissertation uses one of these sensors, the accelerometer.

All Android phones have a triaxial accelerometer embedded within them. The sensor returns a real

valued estimate of the acceleration along the x, y and z-axis. The goal is to convert this data into

semantic information about the uses activity.

The activities focused on are Walking, Cycling and Running, although any could be added with extra

time. This process of converting input data into a semantic result is called classification. An overview

of classification is presented in 2.2. Multiple classification algorithms are implemented, described in

3.4 and evaluated in 4.

1.1 Related Work

There are many examples of accelerometer data being used for useful applications. Last year a

student did a Part II project on inferring a users transportation mode (Train, Bus, Car) using this data

(Piggott, 2011). Apples Fall Detection sensor, which uses an embedded accelerometer in a laptop,

can detect falls or shocks.

Making an application able to detect the users movement fits in to the general problem of context

awareness. Context awareness determining the users current activity and location is a key

functionality in many pervasive computing applications. The motivation for context awareness ishighlighted in many papers; Dey A K (2001) presents a brief summary at the start of his book.

Movement inference is a major component of context awareness. While there has been a lot of

research in determining a users location (Balakrishnan et al, 2000), interest in motion classifcation

has been growing. Bao & Intille (2004) used five wearable acceleromters on different locations on

the body, getting subjects to perform a variety of everyday tasks.

More specifically work has been done testing various classification algorithms (Ravi, 2005), using

similar classification algorithms to this dissertation, but with the accelerometer in a fixed postion on

the user.

Rather than using multiple sensors, this project uses an accelerometer built into Android phones. No

special equipment is required. However this means the the acclerometer is not in a fixed position, it

is free to be anywhere on the user, just as they would carry their phone. For this reason, analysis

techniques not effected by the orientation of the accelerometer are used.

1.2 Motivation

There are many ubiquitous computing applications that could benefit from knowledge of the users

movement. I have listed a couple of applications below.

1.2.1 Exercise monitoring


13/44

2

Perhaps the most obvious application would be to monitor and record all the movement of a user at

all times. This data could be used to provide the user with feedback on the type and duration of their

activities over the day or week, encouraging the user to do more exercise. This data could be

coupled with information such as the calorific impact of various exercises to present the user with an

estimate of calorie loss over the day. A social aspect could be introduced, where users can share or

compare exercise activity, further incentivising the user.

1.2.2 General context awareness

As described in the previous section context awareness has numerous benefits for application

developers. For instance if a user is cycling, and receives a call, their phone could automatically pass

on a preprogramed message to the caller. The music player could be adapted to change choose

tracks depending on what the user is doing, changing to some upbeat lively music if the user is going

for a jog.

1.4 Summary of results

The finished application is evaluated extensively in 4, this subsection presents a summary of the

findings. The aim of this project is to compare different algorithms, in terms of accuracy and energy

usage. Accuracy testing is performed in three separate ways. Firstly the accuracy of the classifiers

are tested offline (on the desktop), using labelled data reserved for testing (4.2).

Table 1 - % Accuracy of classification algorithms in offline testing

Next the algorithms are tested online (on the device) by myself (4.3). As I was involved in collecting

training data the accuracy was similar to the offline testing above. Finally the accuracy was tested in

the most interesting case, with people who had not been involved in the training (4.4).

70

75

80

85

90

95

100

Nave Bayes k-NN k=3 k-NN k=5 Voting

Stationary

Walking

Cycling

Running

Overall


14/44

3

Table 2 - % Accuracy of classification algorithms in online testing with new users

Overall the accuracy was very good. As is expected the algorithms performed worse when tested on

new users. This could be improved by having more participants involved in the training, to create a

larger training set. This is explained in more detail later.

The energy usage of the various classifiers was also measured (4.5), it was found that the more

accurate classifiers were, the more energy they required. Choosing between higher accuracy or

energy efficiency would depend on the application domain. A number of techniques are presented

and tested to reduce energy usage.

Within this dissertation some explanation may be duplicated. I have tried to keep this to a minimum

and in the cases where it does occur; it is intended to help the reader maintain understanding of the

algorithms and methods used.

70

75

80

85

90

95

100

Nave Bayes k-NN k=3 k-NN k=5 Voting

Stationary

Walking

CyclingRunning

Overall


15/44

4

2 Preparation

2.1 Overview

This chapter describes the work that was done before any implementation of the project began.

Initially time was spent learning about classification, which I describe briefly, along with background

reading that was done. I describe the foundations for a professional implementation, such as

planning, setting up a robust development environment, and progress monitoring. Finally I touch on

some key technologies that I learnt about, namely Android and MATLAB.

2.2 Background research

This chapter describes the work that was done before any implementation of the project began.

Initially time was spent learning about classification, which I describe briefly, along with background

reading that was done. I touch on some technologies that I learnt about, such as Android

development, and MATLAB. The chapter then describes how I set about monitoring my progress,

and preparing a professional development environment.

After discussing with my supervisor he recommended I read the Wikipedia pages about

classification, in particular Nave Bayes as this was the first classification algorithm I was going to

implement. I also read a couple of papers in the field (DeVaul & Dunn 2001), and (Ravi et al. 2005),

the latter of which being particularly on topic.

In the next subsection Im going to briefly describe how classification works as this knowledge will

help general understanding of this dissertation.

2.2.3 Classification

Like many problems in AI, the problem of classification can be seen as the challenge of creating a

signal-to-symbol converter. We have some data as input, and we want to infer some semantic

information about the source of the data, or what it represents. An example is optical character

recognition (OCR), where the input data is an image of a character, possibly handwritten. The

classifier attempts to discern what underlying letter actually is. To convert the messy input signal,

into a symbol readily understood.

In this project, the input data is a series of accelerometer readings. A single reading is a float array of

length 4, the beginning contains a timestamp, followed by the acceleration in the x, y and z axis.

These readings are clumped together in a 2 dimensional float array, representing a sample of

acceleration over time. This data will be recorded from the mobile phone, and the classification task

here is to discern what movement the user was doing during that time. One expects that if they

were walking, the acceleration should be somewhat different to them standing around.

However initially, a classification algorithm is just a piece of code, it has no knowledge of how

Walking, or Running, or Cycling or any movement is likely to be represented by acceleration. This is

where the first main task of this project comes in, training. In order to train a classifier you need to

have correctly labelled input data. It is vital that the training data is labelled accurately as if it is not


16/44

5

the classifiers will learn incorrectly, reducing the end accuracy. How exactly the classifiers learn is

implementation dependent and will be discussed more in the following chapter.

2.3 Project management

This section covers the requirements analysis, the planning and the overall administration that was

done in order to ensure the project would be completely correctly and on time.

2.3.1 Requirements analysis

In the project proposal a number of aims are presented, using those follow success criteria were

identified:

Create and test an application for gathering labelled training data. Use the application to gather a large amount of accurate training data.

Implement a classification application for simple motion detection Implement a more advanced classification application for more classes, such as Walking,

Cycling and Running

As an extension I aimed to look at the energy use of the application and try to minimise it using

adaptive sampling techniques.

2.3.2 Planning and management

For the project proposal a detailed schedule for completed work was produced, this outlined what

should be completed on a fortnightly basis. It also included succinct milestones, and when they

needed to be achieved. The benefits of such planning are enormous, something I have learnt frompast experience.

Regular meetings with other people active in the project were also extremely important. My

supervisor and I organised twice weekly meetings in the computer laboratory. The purpose was to

ensure I was making progress and not having any problems.

Throughout the project emphasis was placed on planning for evaluation. The requirements for

specific pieces of code include the need to build in methods for testing.

2.4 Setting up the development environment2.4.1 Android development

In order to develop for Android a number of technologies have to be installed. Android applications

are written in java so one has to install the JDK. As I use Java regularly I had this and Eclipse running

on my computer.

Android offers a custom plugin for development in Eclipse, Android Development Tools. The tool is

designed to give you an environment where it is easy to deploy and debug Android applications

much like you would standard java applications. In general, developing with the ADT plugin in Eclipse

is a highly recommended approach. It extends the abilities of Eclipse to let the developer quickly set


17/44

6

up new Android projects, develop an UI, debug applications and export signed APKs (Application

package files) to distribute your application.

Another handy tool in the SDK is the Dalvik Debug Monitor Server (DDMS). This provides lots of real-

time information about the device, such as thread and heap information, along with the logs. It also

provides port-forwarding services, screen capture, incoming call and SMS spoofing.

Lastly as I was using a real device rather than the emulator, the USB drivers needed to be installed.

2.4.2 Version control and backup

I used Subversion (SVN) for version control, this allowed me to track changes and undo changes if

they caused a problem on the device. I used Unfuddle as a host, which provides free SVN and GIT

hosting. This set up also serves the purpose of a conventional backup, if something went wrong with

my development machine.

2.5 Android development

In the next two sections I describe the two key technologies I learnt about whilst preparing for the

project.

The first piece of preparation required was getting up to speed with how applications are developed

on Android. Although I used Android during an internship in 2010, I had not used it since so I spent

the first couple of weeks reading the developer website developer.android.com. In the following

sections I touch briefly on the main Android specific things I learnt about.

2.5.1 Activity lifecycle

All applications that a user can interact with have an Activity component. Every Activity is given a

window in which to draw content that is displayed on the screen. Activities need to be managed

with respect to what happens at different stages of their lifecycle. This is done by implementing call-

back methods.

An activity can be in three different sates: Stopped: this is when the activity is completed obscured

by another activity, when it is in the background; Paused: this is when another activity is in the

foreground and has focus but is not completely covering the main activity; Resumed: the activity is

running in the foreground and has user focus.

What happens when an activity transitions between these states needs to be controlled by the

developer, otherwise data can be lost, and the UI can crash. For example when an Activity is paused,

its entire stack is popped, destroying all of the variables. When the application is resumed, if these

have not been stored correctly then it is very likely the program will crash due to a null pointer. The

solution to this is to use the saved instance state paradigm. An android construct that allows the

developer to easily save variable data when a pause event is triggered, and then recover it on then

the activity resumes.

2.5.2 Android manifest


18/44

7

Every application has a manifest file: AndroidManifest.xml. This contains meta-data about the

application, mainly a list of components that make up the application. Quite often if some piece of

code that is obviously correct isnt functioning correctly; it is probably due to an incorrect

declaration in the manifest file.

An activity is declared with information such as its name, and an icon. Within an application thereare components such as Activities and Services. The most important component of the application

declaration are permissions:

Nearly any interaction with the rest of the phone by an application requires a uses-permission. The

first example above is required for an application to use the internet, for http, ftp, or socket

programming. The second is used to prevent an application from losing CPU resources when the

phone is locked, or the application is in the background.

2.5.3 Data storage

There are a variety of ways to store persistent information in an application, which is best depends

on exactly what the developer is aiming to achieve. Apart from using the internet to store data on a

server, the key methods of data storage are SharedPreferences, and using the internal file system.

The SharedPreferences class provides a framework that allows the programmer to store key-value

pairs in any primitive data type. This data is persistent across user sessions, even when the

application is killed.

For larger amounts of data, such as accelerometer traces, using the internal storage is required. Eachsubclass of Activity has access to some simple methods for using the file system. The

openFileOutput() is used to open a file for writing, and openFileInput() for reading. The other key

methods are deleteFile(), and fileList(), the latter of which returns an array of files that the

application has created in the past.

2.6 MATLAB

MATLAB is a fantastic tool for prototyping ideas and algorithms quickly. I spent an afternoon with

one of the researches from the Systems group who uses MATLAB in his work. He gave me a quick

overview of how the software works, and I was able to get up to speed quite fast. In theimplementation chapter there is more on MATLAB and how it was used in this particular project.


19/44

8

3 Implementation

3.1 Overview

The project is made up of three separate pieces of work, but which are all needed to create a

successful application. The first is an Android application designed to record accelerometer readings.

It records the data, along with a label submitted by the user. This is the basis for the training data

described earlier in 2.3. The second are a series of MATLAB functions and scripts that are used to

process the data collected by the first application into a form suitable for the classification

algorithms. The final, and most significant piece implemented is the classification application. This is

the application that will run on the phone and infer in real time what activity a user is performing.

Although this is the most interesting part of the project, without the first two pieces, it wouldnt be

able to function.

Figure 1 - Classification application, user is stationary then cycles for a period in the middle

To make the reading more interesting, it may be preferable to skip the section describing

classification.

3.2 Ground Truth Data Application

3.2.1 Requirements

The motivation for this application was to collect a large amount of sampled training data. It was

important to make the application simple and reliable. One of the worst things would be if someone

had volunteered to collect some data and the data had been lost. The key requirements of the

application are listed below:


20/44

9

Allow the user to label a sample of accelerometer data with the movement they areperforming

Automatic uploading of data over Wi-Fi (if there is no Wi-Fi the application will store thesample for later upload).

Ability to cancel a sample if it has been ruined (for example if the user went for a cycle thenhave to stop because of a phone call).

Initially the application had other features such as the ability to tag the data with extra information,

but after some use I decided it was best to make the application as simple as possible.

Figure 2 - GTD Application

3.2.2 Implementation

Android allows an application to record accelerometer readings at various sample frequencies,

FASTEST, GAME, UI and NORMAL. At the beginning of the project neither myself or my supervisor

were sure of what was best, so it was decided to allow the GTD Application to record all the

frequencies concurrently.

The main class is the GTDActivity class which contains all the UI Event handlers. If the user starts

collecting this class creates instances of the AccelerometerListener class to capture the

accelerometer readings at various sample rates. The class also starts a separate data upload thread

that continually monitors the applications internal file space for new sample files. When new files

are discovered they are then compressed before being uploaded over Wi-Fi. I decided to use FTP to

upload the data to a webserver. The key benefit was having a guaranteed server 24/7, also the

Apache FTP libraries are very easy to use.


21/44

10

Figure 3 - General structure of the GTD Application

3.2.3 Initial analysis of the collected data

The first version of the application was distributed, so that my supervisor and some of his colleagues

could collect information. One of the concerns my supervisor had was that having four concurrent

threads reading values from the accelerometer might reduce the frequency each was receiving. To

test for this I ran the GTD application with all four sample frequencies being recorded, and then just

with the FASTEST.

On my device there was little difference, when collecting FASTEST in parallel with the others the

frequency was 94 Hz, and on its own it was 96 Hz. However for my supervisor and his colleagues the

difference was much greater, with the frequency dropping to 16 Hz when collected concurrently.

This turns out to be a difference in how Android 2.1 and 2.2 operate. This was the first major

problem in the project, should I just collected FASTEST and forget about the other frequencies.

The motivation behind wanting to get all sample rates was that it would be potentially useful

information to help save energy later in the classification application. For instance if one can classify

simply MOVING/STATIONARY using a low frequency classifier, then switch to a higher frequency formore accurate classification, energy would be saved as opposed to always using the latter.

After some experimentation and analysis of the data it was apparent that one could generate all the

sampling frequencies from the FASTEST rate, by offline resampling. For this reason I decided to

further simplify the GTD application such that it would only collect at the fastest rate, removing the

problem with Android 2.2, and reducing the amount of data to needed to be sent over the network.

3.3 Classification

3.3.1 Classification methodology

Before diving into the exact implementation of the classification algorithms, this section will describe

classification in general and how it is applied in this project. In order to improve the understanding

of the reader some points will be made again even though they were included in the Preparation

section.

Classification is the problem of identifying which of a set of classes a new observation belongs, based

on information from a training set. The previous section, regarding the GTD Application explains the

source of this training set. In the following sections some terms will be used frequently and it a good

idea to define them now.


22/44

11

Class

A class is a category that an observation can belong to. In this project each class is a type of

movement, such as walking, cycling or running.

Accelerometer reading

An accelerometer reading is a single measurement of the accelerometer at a point in time, stored as

a float array [timestamp, x-acceleration, y-acceleration, z-acceleration]

Figure 4 - Acceleration axis

Observation / Sample

An observation or sample is what is being classified. In this project is it a set of 256 consecutive

accelerometer readings, representing the acceleration of the phone over some period of time.

Before the sample is classified, it is processed into what is called a feature vector. This is designed to

represent the sample more informatively. The feature vector contains feature values. In general

features can be anything about the data, but in this project I used the following: Mean, Standard

Deviation, Energy1, Energy2, Energy3 and Energy4 to create feature values of length 6. The meanand standard deviation are obvious choices, the idea of energy values was used by (Ravi et al. 2005)

to good effect.

3.3.2 Feature extraction

This section describes in detail how the features are extracted from an observation. These equations

are used in both the following chapters, for the MATLAB scripts and for the classification application.


23/44

12

Figure 5 - Feature extraction

Step 1: Removing gravity

Recall an observation is an array of accelerometer readings. As the phone is always experiencing the

force of gravity, that needs to accounted for. For this reason when the device is resting on a table,

the accelerometer reads a magnitude of g = 9.81 m/s. In order to measure the real acceleration of

the device, the contribution by the gravity must be eliminated. The recommended method on the

Android developer site is to apply a high-pass filter.

Let represent gravity in the x-axis at time t, and be the reading from the accelerometer inthe x-axis. The actual acceleration (without gravity) in the x-axis is then given by:

where is approximated by:

1

Step 2: Calculating the magnitude

Once the gravity has been removed, the magnitude of the acceleration is calculated for eachpoint in time. This generates a 256 length magnitude vector, representing the sample.

Calculating the features on the magnitude vector rather than on individual axis means the features

are independent of the devices orientation. That is vital for this project as people can have their

phones on them in any position.

Step 3: Calculating the features

The mean and standard deviaton are calculated as you would expect. The energy values are

calculated by taking the fourier transform of the sample, to produce an array representing the

frequency components of the data. Each energy value is the energy from a frequency band. The

frequency spectrum is devided into four, the lower quartile for energy1, up to the upper quartile for

energy4.

1 is the value used in the Android documentation (Google, 2012)


24/44

13

Let be the fourier tranform of the magnitude vector. Let be energy feature of band .

The energy of a given frequency is simply the square of that frequency component; the sum and the

denominator give the average energy over a band of frequencies. The mean, standard deviation and

the energy values are put together to form a feature vector for the sample.

Figure 6 - Example feature vectors taken from labelled walking data

3.3 describes a concrete implementation of this process in MATLAB, for processing the labelled

training data. The same process is also used on the device when classifying a new observation.

3.3 MATLAB scripts

MATLAB was used to process the collected labelled data into a form suitable for the on-device

classification algorithms to use as training parameters. The best way to explain what was

implemented using MATLAB is to describe each function in turn.

3.3.1 Loading the data

function [a_m, a_x, a_y, a_z, freq] = loadAccel( filename )

This function loads the raw accelerometer values from a .csv file generated by the ground truth data

application. Below is the layout of one of those .csv files. The columns are: timestamp, x-axis

acceleration, y-axis acceleration and z-axis acceleration.

Figure 7 - Raw accelerometer readings

The function loads the values from the disk into a matrix within MATLAB, a high-pass filter is applied

as described in the previous section to remove the gravity component. There is a built in function for

this, filter.

a_x_temp = filter([alpha -alpha], [1 -alpha], accel_x);

The next part generates a vector representing the magnitude of acceleration at reach point:

a_m_temp = sqrt(a_x_temp.^2 + accel_y_temp.^2 + accel_z_temp.^2);

4.8915860 3.4314873 55.478584 0.86720960 0.68431590 0.07791250

4.7732573 3.3957403 49.822770 0.86712134 0.45566043 0.14041372

10839906912000, -2.030283, 6.129156, 10.9175610839938332000, -0.9576807, 5.3247046, 9.423578

10840365885000, -2.528277, 7.3166804, 7.469909


25/44

14

The data in the .csv file is not split up into separate 256 element observations, so this has to be done

before features can be calculated. The vector is converted into a two dimensional array of

observations.

a_m[i][j] = a_m_temp[i*256 + j]

Figure 8 A flattened version of a_m from a[0] to a[10000].

The figure above shows the result of the processing so far, this is the magnitude vector of all the

windows flattened out. This data was labelled as walking, however you can see that there is a

section where the acceleration drops down to almost zero. The next function is designed to remove

such areas.

3.3.2 Cleaning the data

function [a_m, a_x, a_y, a_z] = cleanData(a_m, a_x, a_y, a_z)

One of the problems identified in the data are anomalous sections, for example in the trace above

from 0.4 0.5 the acceleration drops to zero. As this is a trace of someone walking there should

always be some acceleration. This problem occurs when the user collecting stops to do something

forgetting that the application is building a trace.

The purpose of this function, cleanData, is to remove portions of the trace that are too far from the

median. After some experimentation, an accepting range of median +/- 1*standard deviation was

applied.

% Accepted if (mean_F1 - std_F1) < f1 < (mean_F1 + std_F1)for i = (w_count:-1:1)

if(F(i, 1) < (median_F1 - std_F1) ||F(i, 1) > (median_F1 + std_F1) )% delete window ia_m(i, :) = [];

endend


26/44

15

After this function has been applied the section from 0.4-0.5 is removed, along with some trimming

from the start and end of the trace. This is expected as when the user is just starting the data

collection he will be moving the device into a pocket, or removing at the end.

Figure 9 The result of the cleanData function

3.3.3 Feature extraction

function [ F, N ] = calculateFeatures( a_m, a_x, a_y, a_z )

Once the data has been loaded and filtered, the next process is to calculate features. At this point it

will be helpful to the reader if the read 3.3.2 to understand the motivation for feature extraction.

The main body of this function is a for loop that runs through each window calculating the feature

values, mean, std, energy1, energy2, energy3, and energy4. These are returned in a feature matrix F,

where F[i][j] is the result of feature j calculation on window i.

for i = (1:w_count)S = real(fft(a_m(i,:)));F(i, 1) = mean(abs(a_m(i,:)));F(i, 2) = std(abs(a_m(i,:)));F(i, 3) = energy1(a_m(i,:), S);F(i, 4) = energy2(a_m(i,:), S);F(i, 5) = energy3(a_m(i,:), S);F(i, 6) = energy4(a_m(i,:), S);

end

This function also contains the procedures for each feature calculation.

3.3.4 Appling the functions to the collected data


27/44

16

All the functions are used together to convert a raw accelerometer data file to a list of feature

vectors, one for each observation in the file. These functions are then applied to multiple input files,

generating all the feature vectors for a given class.

3.4 Classification Application

At this stage in the project training data had been collected by the Ground Truth Data application,

and processed into feature vectors using MATLAB. This section of the dissertation describes the final

component of the project, the Classification Application. The phone application classifies

accelerometer data in real time using one of the selected algorithms. This chapter will describe the

requirements of the application, the general implementation, the various classification algorithms

and finally any additional features that the application had.

In total there were four main classification algorithms, Simple, Nave Bayes, k-NN and Voting, these

will be described individually later in this chapter. In the evaluation chapter the accuracy of the

various classifiers are presented.

3.4.1 Requirements analysis

The key requirements of the application are listed below:

Record and classify accelerometer data in real time, presenting the results on screen Allow the user to choose between the various classification algorithms Update the parameters for the algorithms dynamically Code for easy monitoring and debugging, to aid evaluation

3.4.2 Overview

The classifier application is an android application that shows the real time results of classification on

accelerometer data. The application records readings from the accelerometer, bundling them into

samples of 256 readings for classification. The sample is passed into one of the algorithms and then

the result is displayed to the user.

The application updates the classifiers dynamically by checking a webserver for the most up to date

parameters. This means once the application has been distributed it can be updated without

requiring the user to install a new application.

Another feature implemented is automatic log uploading. All debug messages produced by the

application pass through a logger class that uploads them to a database running on a webserver.

This feature was implemented because it greatly improves testing; one can go out with the phone

for a cycle, then come back and look at the webserver for an exact trace of what was happening.

3.4.2 Application structure


28/44

17

Figure 10 - Movement classifier class structure. *in the Android SDK

As is common with Android applications that interact with a user, there is a subclass of Activity, the

MovementClassifierActivity, abbreviated MC Activity in the class diagram above. This class is

responsible for setting up the layout and controlling user interaction. It is also the entry point for the

application.

The DownloadParamatersTask will be explained in more detail in 3.4.8, but it is used to keep the

algorithm parameters up to date.

The AccelerometerListener class the same as in the GTD Application. It simply records accelerometer

readings into a 2 dimensional float array, when 256 readings are reached; the array is wrapped in a

Sample class, then passed to the current classifier instance for classification.

The Logger class extends the normal functionality of the Android Log class, it allows for automatic

uploading of log messages to a server, to aid in the debugging process.

The abstract class Classifier is instantiated as one of the concrete classification classes depending on

the user selection. These classification classes will be explained individually in the following

sections.

3.4.3 Simple classifier

The key method that all the instances of the classification algorithm have to implement is the

following:

publicabstractint classify(Sample sample);

What is passed to the classify function is an instance of Sample, representing the 256 accelerometer

readings recently collected. The sample class wraps the float array with functionality and state. The

functionality is used to calculate the features as described in 3.3.2. Firstly gravity is removed, and

then the features are calculated. The state is used to store these feature values. This is very

important as it prevents the same features from being calculated repeatedly. In some cases a sample

may be classified by multiple algorithms. This class reduces the marginal cost significantly.


29/44

18

The simple classifier classifies between STATIONARY and MOVING. It does this using only the mean,

comparing it with a decision value. The result is returned as an int. The abstract classifier class

contains a method to convert this into a string for the UI.

3.4.5 Nave Bayes classifier

3.4.5.1 Theory

This subsection describes the mathematics behind Nave Bayes. The probability model for the

classifier is a conditional model | over a random variable C, which can take class values,conditional on a number of feature variables to . Using Bayes theorem, we write:

| |

We are aiming to find the maximal probability, i.e. given the feature values to what the class is most likely. The denominator of the above fraction is irrelevant to the classifier as itsconstant for all possible class values.The numerator can be split up using the chain rule:

| | |

| | |

| | | |

We assume that each feature is conditionally independent of each other meaning:( ) |

This allows one to form the standard Nave Bayes result:

| |

To calculate

|we assume that for a given class C, each feature

is normally distributed, with

a mean and standard deviation . For each feature the mean and standard deviation iscalculated from the training data.

Once we have the mean and standard deviation for each feature for each class, calculating the

conditional probability is simple:

|

( )


30/44

19

So finally to classify, one calculates | for each class, taking the maximum to be theclassification.

[] |

3.4.5.2 Implementation

The first part of the constructor loads the parameters (mean and standard deviation values for each

feature and class) from a stored file. The values are parsed and converted in Features. A Feature

class contains the featureID, its mean, and its standard deviation. For each movement type, a

FeatureSet is created, storing the six Features that make up that type. The classifier maintains a list

of FeatureSets, one for each class of movement.

Figure 11 - Naive Bayes class structure

When a sample is passed to the classifier, the probability of it being from each class is computed in

the method described in the previous section. The source code for this class can be found in

Appendix A.

3.4.6 k-NN classifier

3.4.6.1 Theory

The k-Nearest Neighbours algorithm (k-NN) uses more memory than Nave Bayes, but unlike that

algorithm, it does not require any prior probability knowledge. The algorithm is simple. The training

data consists of a series of feature vectors, labelled with their source classification. All of these

vectors are used as is in the k-NN algorithm, rather than just the mean and standard deviation of

their values as in Nave Bayes.

When a new sample is recorded, the k-NN algorithm searches for the closest matches from the

training data. If you imagine the vectors in a 6-dimensional vector space, and the sample vector a

point in that space, the algorithm searches for the k nearest neighbours to that point. The final result

is simply a vote between the k neighbours.

3.4.6.2 Implementation

In the concrete implementation I used 800 labelled feature vectors for each class. As there are 4

classes (Walking, Stationary, Driving and Cycling), a total of 3200 vectors are stored on the device.

When a new observation is collected, it is processed into a feature vector. The k-k-NN algorithm

classifies this vector by searching the vectors stored for the k nearest neighbours. The new

observation is classified by adding up the labels, and returning the most common.


31/44

20

The implementation is quite simple, when a new feature vector is being classified, the Euclidian

distances from that point to all the other points in the vector space are calculated, they are then

sorted in reverse, and the top k-elements are the nearest neighbours.

For instance if k = 5, and the top five vectors are labelled W, W, R, W, C, then Walking is returned. In

the instance where there is a tie the result with the closest vector is returned.

In order for one feature not to have too much influence, the vectors are normalised prior to

searching.

3.4.7 Voting classifier

A voting classifier was recommended in Ravi et al. This classifier uses a combination of Nave Bayes

and k-NN with various k values and then performs a vote on the result. As described earlier, the float

array of the sample is wrapped in the Sample class, which maintains values for calculated features.

The k-NN classifier maintains a record of the previous sample it classified, if it is the same, the

nearest neighbours are not recalculated meaning classifying the same sample for another k value is

free. These design choices mean the processing required for the Voting classifier is less than the sum

of the three components run separately.

3.4.8 Adaptive classifier

There are actually three versions of the adaptive classifier, Adaptive Nave Bayes, Adaptive k-NN and

Adaptive Voting. The adaptive classifier is designed to reduce processing power and thus save

energy. It classifies a given sample using the Simple classifier, if that returns Stationary it returns

Stationary, however if the sample is classified as Moving, the sample is then classified with a morecomplex classifier. This saves processing power in the instances when the phone is stationary.

3.4.9 Additional features

The application contains a number of additional features touched on earlier. These are described

below.

3.4.9.1 Dynamic parameter updating

Every time the application is started it checks for an internet connection, if this is available it

downloads a file from a webserver containing all the available parameter files and their associatedhash values. It then compares the filename-hash pairs with the files currently stored on the device. If

any differ it downloads the new file from the webserver.

The dynamic updating is done by the DownloadParamatersTask class, an extension of AyncTask.

AsyncTask is an Android SDK class which enables background work to be completed while facilitating

proper communication with the UI Thread.


32/44

21

Figure 12 - Server file containing filename - hash pairs, used for dynamic updating

The filename-hash pairs are stored using shared preferences, described in 2.5.3.

The SharedPreferences class provides a general framework that allows you to save and retrieve

persistent key-value pairs of primitive data types. The parameter hash file is downloaded from the

server using an URL connection.

Once any modified parameter files have been downloaded the key-hash value pairs are updated in

the applications shared preferences. This is done using an instance of shared preferences Editor

class.

3.4.9.2Log uploading

Uploading log messages is done using the message uploader thread. When a log message is written

using logger.write(tag, msg) the message is added to a queue ready for upload. Its also passed

on to the standard Log.d(tag, msg) android logging function. The thread starts by creating a

FTPClient to connect to the webserver. The thread then polls the message queue for a new message.

The standard JDK Linked Blocking Queue was used to allow blocking.

naivebayes.csv 0c13791ee21654f67c8357c24b159592d59e91f7

KNNParameters_walking.csv 576b754df3a82f3364313e4a2fb6916f5ee81da4

...


33/44

22

4 Evaluation

4.1 Overview

Evaluation of the project is done in two ways. Firstly I evaluate how the algorithms perform offline

by testing them against some labelled data that was reserved for this use rather than for training.

Secondly the performance of the program running on the device is evaluated. The first approach can

tell us how the algorithms are performing for data gathered using the Ground Truth Data

application. If the algorithms are accurate in the offline testing then that gives a baseline of

confidence in their efficacy.

However there are a number of factors that may lead to a lower rate of accuracy for on device

testing, compared with offline testing. There could be bugs in the android application that lead to

inaccurate classification. The device, having to now collecting and analysing samples simultaneously,

may collect samples at a lower frequency than the training data, affecting the feature values. The

algorithms may be suffering from over fitting, where they are accurate with the offline test sample

(most of which was collected by the same couple of people), but not with new data from new test

subjects. These factors are the motivation behind doing online testing on the device.

4.2 Offline evaluation of classification algorithms

Initially I planned to do this stage in MATLAB. However it was possible there could be subtle

variations in how the calculations were done between Java and MATLAB, in particular with theFourier Transform libraries. In order to check that the algorithms that the device application uses are

working, it seemed appropriate to test hose algorithms directly. I developed a small application in

java that took unused labelled training data (around 30% of the total), imported the exact algorithms

the device was using, and computed their accuracy on the labelled test data. This also allowed me to

experiment with the k number for the k-NN algorithm.

Throughout the next section a common representation for results will be used, this representation is

a confusion matrix. The leftmost column represents the label of the test samples in that row. The

next four columns, S, W, C, R, indicate the classification results for those samples. The final column is

the accuracy for that class.

For the offline testing 1044 samples were used. Below are the results for each classifier.

Naive Bayes

S W C R Accuracy

Stationary 122 0 3 0 97.6

Walking 0 275 24 2 91.4

Cycling 0 10 247 1 95.7

Running 0 22 4 334 92.8

94.8


34/44

23

Table 3 Nave Bayes, offline testing confusion matrix

k-NN (K = 3)

S W C R Accuracy

Stationary 124 0 1 0 99.2Walking 0 292 5 4 97.0

Cycling 4 10 243 1 94.2

Running 0 25 0 335 93.0

95.8

Table 4 k-NN k = 3, offline testing confusion matrix

k-NN (K = 5)

S W C R Accuracy


Walking 0 295 5 1 98.0Cycling 3 8 246 1 95.3

Running 0 26 0 334 92.7

96.3

Table 5 k-NN k = 5, offline testing confusion matrix

Voting (Nave Bayes, K = 3, K = 5)

S W C R Accuracy


Walking 0 295 5 1 98.0

Cycling 3 7 247 1 95.7

Running 0 26 0 334 92.8

96.4

Table 6 Voting classifier, offline testing confusion matrix

The results are very good. All the classifiers are accurate to at least 92% for any class. This is

expected as the test data used here was collected by the same users as the training data on the

same device so in the data is very similar. Although it does give confidence that there are no major

bugs or errors in the classification algorithms.

Also the other key point is all the algorithms are performing to a very similar level. The Voting

classifier is only marginally better than the others for twice the processing power. However the

difference increases in the user testing.

4.3 On device testing

The next stage of the evaluation procedure was to evaluate the performance of the classifiers

running on the device. This was done by recording the results of the real-time classification

application, while maintaining knowledge of the real movement type. As in the previous section, a

confusion matrix is used to represent the results. However the matrices for Nave Bayes and K-NN

have been omitted for brevity.


35/44

24

Voting (Nave Bayes, K = 3, K = 5)

S W C R Accuracy

Stationary 34 0 0 0 100

Walking 0 94 0 2 97.9

Cycling 0 3 86 0 96.7

Running 0 5 0 72 93.5

97.0

Table 7 Voting classifier, on device testing confusion matrix

Below is a table showing the accuracy of all the classifiers:

NB K = 32

K = 5 Voting

Stationary 100 100 100 100

Walking 93.8 91.7 97.9 97.9Cycling 96.7 96.7 96.7 96.7

Running 93.0 93.0 96.1 97.4

Overall 97.0 95.4 97.7 98.0

Table 7 Overall on device accuracy of the classifiers, using personal data

This testing was done by me, and as I made up a large component of the training data, is it

unsurprising that the classifiers show a high level of accuracy. Again there is little to no improvement

using the voting classifier as opposed to the simpler methods. However as in the previous section

this shows that the classification application is working as intended. In the next section, theclassifiers are evaluated using new users that were not involved in the training or testing so far.

4.4 User testing

In this part of the evaluation I organised a number of users (my housemates) to use the application

while I recorded what activity they were doing. The purpose of this was to test the accuracy of the

classifiers with users who had not been involved in the training data. Overall the results were lower

than in the offline testing, as to be expected.

NB K = 3 K = 5 Voting

User1

Stationary 93.4 94.0 94.0 94.6

Walking 84.4 84.4 87.5 84.4

Cycling 82.0 92.0 90.0 90.0

Running 82.2 86.6 88.4 90.0

Overall 85.7 89.2 90.0 90.0

User2

Stationary 100.0 100.0 100.0 100.0

2K=3, and K=5, represent a k-NN classifier as described in the previous section, with the given k value


36/44

25

Walking 48.6 74.3 71.4 74.3

Cycling3 - - - -

Running - - - -

Overall 74.3 78.2 87.5 87.2

User3

Stationary 88.6 88.6 89.2 89.2

Walking 83.3 86.7 86.7 86.7

Cycling 85.2 88.5 88.5 90.1

Running 85.7 86.5 86.5 88.7

Overall 85.7 87.6 87.7 88.7

User4

Stationary 92.2 93.4 92.2 94.0

Walking 82.1 85.7 85.7 85.7

Cycling 100.0 100.0 100.0 100.0Running 90.4 92.3 92.3 96.0

Overall 91.2 92.8 92.5 93.9

Table 8 - User testing

There are a couple of notable points with regards to the results. Firstly, they are generally very high

which leads me to conclude that the classification application is successful in what it was designed

for. All the classification algorithms performed badly on user2. This I suspect is because User2 was

female and all the training data had been gathered by males. More investigation is needed. If this

was the case, in order to improve accuracy you could have the user answer a few questions about

their sex, possibly age, and use these factors along with the accelerometer data for classification.

To conclude this section I have collated the accuracies above, to give an idea of the total accuracy of

each classifier.


Accuracy 84.2 87.0 89.4 90.0

Table 7 Overall on device accuracy of the classifiers, with housemates


Accuracy 87.5 89.9 90.1 90.9

Table 8 Overall on device accuracy of the classifiers, with housemates excluding user2

Overall the voting classifier is no better than using k-NN with k set to 3. I had collected the

processing time of each of the classifiers, as I was going to investigate the trade-off between

3Running and Cycling are omitted as this user was unavailable to test the application


37/44

26

accuracy and power for each of the classifiers, but as the Voting classifier isnt any better than the

simple k-NN algorithm it wouldnt be a valuable addition to this evaluation.

4.5 Energy usage

To analyse the energy usage, I used an application called PowerTutor. It allows you to profile the

energy used by the system, and filter by individual applications. The testing was performed with the

dynamic updating, and log uploading turned off, to remove the energy required to use the internet.

The screen was also turned off, leaving the CPU to be the only component of energy usage.

The energy used when applying different algorithms varies greatly as would be expected. When

using the simple classifier the energy is minimised. In the simple case, the application is performing

three main tasks: collecting a sample, removing gravity and calculating the mean.

When Nave Bayes is used, the algorithm is doing all the above, but also calculating more complex

features, and then performing some calculations to get a maximal probability. For k-NN the features

are calculated, then the distance from that feature vector to all the others is calculated, and finally

these distances are sorted.

Classifier Time(S) Energy (mJ) Power (mW)

Simple 328 990 3.02

NB 565 2250 3.98

k-NN 365 2100 5.75

Voting 372 2200 5.92

Table 9 - Rate of energy usage (final column) for each classification algorithm

Using the k-NN classifier is significantly more energy intensive than NB, a 45% increase. However the

voting classifier is not much more. This is because if you are calculating k-NN, an extra k-NN with a

different k value is free. Plus NB is not very expensive because all the features will have been

calculated, leaving just the probability calculation, which is likely to be what is causing the extra

0.17mW above k-NN alone.

The next chart shows the classifiers when plotted against accuracy and energy usage.


38/44

27

Table 10 - Accuracy / Energy trade off4

From this one can see that in order to have higher accuracy classification, more energy is needed. If

one was developing a commercial application the need for accuracy would have to be balanced with

the need for low power.

The adaptive classifier, described in 3.4.8, is another technique for reducing power. In the casewhen the user is stationary, the adaptive classifier would only use the power required for the simple

classifier, however if motion is detected it would trigger a more complex classifier. If the application

were running constantly to provide context awareness, it can be assumed that a large amount of

time will be spent stationary. The simple classifier uses 49% less power than voting. Assuming the

device is stationary for just half of the time the application is running, the energy usage overall

would drop by 25%.

4k value of 5 used for k-NN. Accuracy taken as overall accuracy for user testing, as in table 7

83

84

85

86

87

88

89

90

91

3 3.5 4 4.5 5 5.5 6 6.5

Accuracy%

Energy usage (mW)

Nave Bayes

k-NN

Voting


39/44

28


40/44

29

5 Conclusions

5.1 Project success

Looking back at the requirements analysis, 2.3.1, the core aims of the project were to:

Create and test an application for gathering labelled training data. Use the application to gather a large amount of accurate training data. Implement a classification application for simple motion detection Implement a more advanced classification application for more classes, such as Walking,

Cycling and Running

All of this was completed. Also the energy usage of the application was evaluated. I had planned to

go further and investigate varying the accelerometer sample rates to reduce power but I wasnt able

to get that finished. In hindsight I didnt give enough time for the collection of accurate ground truthdata. This part of the project took up almost the whole of first term. There were a lot of problems

with people uploading inaccurate data by mistake. I think overall not enough people were used for

the training data which lead to problems such as a low accuracy when tested with woman rather

than men.

That said all of the pieces are there to make the application more accurate. The software for

collecting and processing new training data is complete, and the algorithms on the device

automatically update without having to be reinstalled.

5.2 Further work

There are many ways the project could be extended, something I hope to do over the summer. More

activity classes could be added, such as driving, (Ravi, 2005) even used vacuum cleaning as an

activity type. Adding an activity would not involve any development because one could use the

software developed in this project. However it would require management, time and a lot of

volunteers.

Other methods for reducing the energy usage of classification could be investigated. One idea would

be to collect sensor readings at one of the slower rates, such as NORMAL, when the user is not

moving, and then have user movement trigger a faster sampling frequency.


41/44

30


42/44

31

Bibliography

DeVaul R, D. S. (2001). Real-Time Motion Classification for Wearable Computing Applications.

Technical report, MIT Media Laboratory.

Dey, A. K. (2001). Understand and Using Context. Personal and Ubiquitous Computing. In A. K. Dey,

Understand and Using Context. Personal and Ubiquitous Computing (pp. 4 - 7).

Google. (2012). Retrieved from developer.android.com: http://developer.android.com

Intille, L. B. (2004). Activity Recognition from User-Annotated.

Lee S, M. K. (2002).Activity and Location Recognition Using Wearable Sensors. IEE.

Nissanka B. Priyantha, A. C. (2000). The Cricket Location-Support System.

Piggott, D. (2011). Inferring Transportation Mode using Smartphone Sensor Data. Fitzwilliam College.

Ravi, D. M. (2005). Activity Recognition from Accelerometer Data.


43/44

32

Appendices


44/44

dissertation new2

Documents