Relaxed Context-Aware Machine Learning

Midddleware (RCAMM) for Android Jitesh Punjabi

Student, Computer Engineering

V.E.S. Institute of Technology

Mumbai, India

jitesh.punjabi@gmail.com

Gaurav Taneja

Student, Computer Engineering

V.E.S. Institute of Technology

Mumbai, India

gauravrtaneja@gmail.com

Shekhar Parkhi

Student, Computer Engineering

V.E.S. Institute of Technology

Mumbai, India

sheks.paar@gmail.com

Dr. Nupur Giri

Head of Department, Computer Engineering

V.E.S. Institute of Technology

Mumbai, India

nupur.giri@vesit.edu

Abstract— Context Aware Computing is a promising

approach of developing mobile applications that provide

experiences and services in a manner that is fine-tuned based on

the user's preferences. Applications such as Google Now, Apple

Siri learn the User's activities from context related information

and subsequently provide suggestions to the users in real-time.

However, in almost all cases, application developers have to

develop the same set of mechanisms to consume the context

information and storing it in an appropriate form rather than

focusing on the parts of the application that consume the context

information. This approach results in the repetition of the same

task and multiple copies of data.

This paper presents our work detailing the development of a

middleware that handles context information collection and its

storage. The work provides a framework that allows the

developers to easily implement context aware applications that

consume the services provided by the middleware. Applications

will only have to react to context data (past and present) while

the middleware takes care of everything else such as the

background service for context information collection and

storage, thus reducing the redundancy, increasing adaptability

and flexibility, and simultaneously supporting developers in

rapid prototyping of context-aware applications. Thus the paper

presents our work towards building sustainable Android

Framework which follows the principle of Reformat, Reduce,

Regenerate, Reuse and Repurpose.

Keywords— Context; context-awareness; Android;

middleware; framework; machine-learning

I. INTRODUCTION

A system is said to be context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task and where context is any information that can be used to characterize the situation of an entity. An entity is a person, place or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves [1].

Context Aware systems can greatly improve the user experience by presenting only that information to the user

which he/she needs "NOW" [2]. The user experience can be improved further by the use of machine learning. Machine learning has many application areas for mobile context-aware applications such as location prediction [3], adaptable user interfaces [4], human behavior recognition [5], facial recognition [6], and smart music players that adapt to user preferences [7]. In the 21

st century, Smartphones are giving a

new dimension to context-aware application development. Smartphones are equipped with sensors such as such as accelerometers, GPS, light sensors, distance sensors, video/photo cameras, microphones, etc. Smartphones are being accepted as personal companions.

The Android Operating System developed for Smartphones uses a middleware infrastructure approach to separate context sensing and context usage, supports a range of physical components, and provides software components to offer the middleware for the physical components and context preprocessing capabilities. However, the Android platform lacks a generalized interface for context management and a discovery component for adaptation. In addition, Android does not include any machine learning capabilities [14]. The work presented in this paper aims at bridging this gap. However, the work relaxes the strict semantics imposed by the layered architecture to propose a soft layered architecture, the Relaxed Context-Aware Machine Learning Middleware (RCAMM). RCAMM adheres to the middleware infrastructure style that Android follows. RCAMM is a two component solution which includes:

1. Middleware Runtime - manages the middleware components and the context-aware applications that are installed on the Smartphone.

2. Application Development Library - an Architecture Implementation Framework that provides the necessary abstract classes and reduces the job to reacting to the context information coming in from the runtime.

The rest of the paper is organized as follows: Section 2 describes the related work, Section 3 outlines the scope of RCAMM, Section 4 discusses the context model, Section 5 discusses the implementation of the RCAMM, Section 6

2013 IEEE Recent Advances in Intelligent Computational Systems (RAICS)

978-1-4799-2178-2/13/$31.00 ©2013 IEEE 92

describes the use of the RCAMM to develop a context-aware power manager "BatriCon", Section 7 discusses the short-comings and finally, Section 8 concludes the paper with a check to ensure that all the requirements outlined were met.

II. RELATED WORK

A. Context Toolkit

In [9], Dey et al. describe their Context Toolkit, which is a framework for rapid prototyping of context-aware applications. The framework is a loosely-held architecture built around the following components: Context widgets that provide applications with access to context information, Interpreters that transforms context information to a higher abstraction level, Aggregators that collect multiple logically related pieces of context information and are store them in a common repository, Services that change the state of activators and Discoverers that maintain a registry of existing capabilities in the framework.

B. Hydrogen

Hydrogen [10] uses a hard layered architecture approach with its context acquisition scheme specialized for mobile devices. The architecture consists of three layers which are all located on the same device. The Adaptor layer retrieves raw context data by querying sensors and supports concurrent consumption from different applications. The Management layer consumes sensor data from adapters and provides and retrieves contexts. The Context server sub-layer offers the stored information via synchronous and asynchronous methods to the client applications. The Application layer consists of the appliance code implemented to react on specific context changes. On account of the Limited resources on Mobile platforms back then, the Hydrogen Project did not store any historic context information. Balduf et al [11] have done a survey of Context aware systems including the above two as well as CASS [12], CoBra [13], and SOCAM [14]. An important observation reported by them is that even though many of these systems store contextual information, none of them use learning techniques to provide the context-aware service proactively.

C. CAMF - Context Aware Machine Learning Framework for

Android

CAMF [14], is inspired from the Context Toolkit and Balduf & Miroui's Layered Architecture Approaches. It has been developed keeping in mind Android in particular and also validates Android as a platform capable of supporting context-aware applications. It uses the Object-Oriented style of handling Context Information as done in Hydrogen. This data is collected and stored in local Context Stores and is later used for Machine Learning Purposes. It exposes discovery and adaptation layers however there is no specification on how these activities are to be performed. The layered architecture is rigid leading to an overhead of querying via the pre-processing components in the case when the application needs to consume raw sensor data.

The focus of RCAMM is different than the ones discussed above. It uses a hybrid context model and overcomes the rigidity imposed by the layered architectures, introduces the concept of a central entity for context management, and on account of its component based nature, supports addition of support for new contexts even after deployment. Development of context consumers and context providers is made swift and easy with the help of the provided API.

III. REQUIREMENTS AND SCOPE

The scope of RCAMM is defined in the following points:

It should be developed using the Middleware

Approach that allows the components to be re-usable.

Data collected by components should be made

accessible to other applications as well as

components. Thus to facilitate re-purpose.

The components should be loosely held so that there

are no strong dependencies among the components.

Different versions of the components should be

usable interchangeably and help in Re-generation.

All communication between the processes should be

non-blocking and should be possible in a thread-safe

manner, thus be faster.

The context model is expected to be simple, flexible,

extensible, generic and expressive. Thus facilitating

Re-format.

It should allow more components to be installed as

and when required.

IV. CONTEXT MODEL

A context model is needed to define and store context data that can be processed by a machine. Various models such as key-value based models, ontology based models, object oriented models and mark-up based models exist[11].

A “Scope” is defined as an identifier for a type of context information, e.g. a plugin with "location.coarse" as a provided scope multicasts changes in coarse location and shares historic data about the same. The use of a hierarchical namespace is recommended to ensure readability.

Instead of using the standard context models, RCAMM uses a hybrid model which uses JSON to encode context values and a Relational Database table to store context values. Following is the generic structure of a database table storing context values:

id timestamp value

Use of JSON ensures that the data is encoded in a platform-independent format. The structure of context values remains flexible enough to accommodate varying levels of details. Adaptability is achieved as changes in the context model will not lead to changes in the Relational Database Tables.

A ContextValue class needs to implement the

iContextValue interface and hence the methods that decode the JSON to values of native data types such as strings, doubles etc.

93

The tags attribute allows the developers to add custom tags such as confidence, support and other domain-specific labels and quantities based on their requirements.

The context model meets all the requirements as discussed in the scope

1) Simplicity: the model designed is fairly simple to use

as it is a combination of two widely used approaches.

It does not require special training for development.

2) Flexibility and Extensibility: The use of JSON within

tuples ensures that developers have the flexibility to

modify the context model according to their wishes.

3) Genericity: The JSON based context values and tags

ensure that this context model is not limited to any

specific context.

4) Expressiveness: Structured expressiveness is supported

on account of JSON. Support for the expression of

Machine Learning Data is ensured by the appropriate

use of the tags field.

V. . RCAMM – RELAXED CONTEXT AWARE MACHINE-

LEARNING MIDDLEWARE

RCAMM comprises of the Runtime and the different context plugins, as seen in Fig. 1.

Fig. 1. RCAMM Components.

The RCAMM uses the following core components:

A. Plugins

The Plugins are the components that:

Collect context information.

Inform the runtime about context-value changes.

Keep a time-stamped record of the changes in the

context-values in a content provider accessible to

applications with the required permissions. Plugins may service one or more scopes. Each scope may

be serviced by more than one plugin. They are classified on the basis of their functionality into 3 types:

1) Sensor Plugin: Listens to Context changes, stores

them in a database (optional), brought into the

memory by the runtime only when an application is

consuming its services.

2) Background Context Listener Service: begins at boot

and runs in the background at all times collecting

different context inputs. Upon change in context

information values, they multicast the new values in a

message to the different applications that listen to the

scopes provided by it.

3) Reasoner Plugin: can consume context of one or more

scopes and generate context value change messages

thus making them ideal for machine learning

applications that need to react proactively by

consuming the changing and past context values.

Each plugin is characterized by the scopes it provides and the permissions it requires.

Fig. 2 is a component diagram that captures the interactions between the various RCAMM components used in developing the sample application “BatriCon” which is discussed in detail in the subsequent secion.

Fig. 2. Component Diagram.

The Plugin Lifecycle confirms to the OSGi component Life cycle specifications as shown in Fig. 3.

Fig. 3. Component Lifecycle

1) Starting: Bind to the runtime. (Overriding the

onBind method)

2) Active: Respond to context change events using the

notifyListener method.

3) Running: unBind from the runtime and die

(Overriding the onUnbind method).

Resolution and Activation are handled by the runtime (discussed in the following section).

94

B. Runtime:

There is need for proper passing of information from the plugins to the applications that need it. To a context-aware application, this is the only visible part. The runtime sends the context updates and historic data requests from the applications to the appropriate plugins and the replies from the plugins to the applications thus acting as a proxy.

There are 3 tasks performed by the Runtime:

1) Resolving Plugins:

A resolution mechanism is needed to ensure that the

runtime adapts to the available plugins and that there is no

hard dependency imposed. This phase ensures that all the

dependencies of the various plugins are met before allowing

them to be activated.

The resolution algorithm uses two sets viz. Resolved(R)

and Unresolved(U) and a map of all plugins with their

ProvidedScopes(A,S) used, comprising of two stages:

Stage 1 - Identifying unresolved dependencies:-

i. Look out for changes in A.

ii. Iterate over R to determine if any dependency is

broken.

iii. If broken link found, move that plugin to U.

iv. Repeat iii for all broken dependencies.

Stage 2 - Identifying resolved dependencies:-

i. Iterate over U to determine if any dependency is

now met.

ii. If dependencies being met, move that plugin to R.

iii. Repeat ii for all broken dependencies.

2) Activating and De-activating plugins:

The resolution mechanism was run in reaction to

change in the plugins. However, the Activation mechanism

is used to react to the binding, unbinding, registering and

de-registering by the different context aware applications.

This mechanism regulates the lifecycle of the plugins thus

effectively ensuring that plugins are running only when

their services are being consumed by one or more

applications. Working:

a) Pre-processing:

Remove all unresolved plugins from

activePlugins. Compute the set of the needed plugins (Derived from the keys of the

scopesToListeners map maintained as a part of Context Management). A check is made to ensure that the permissions possessed by the listener match the permissions possessed. This check ensures that the Android Permissions Model is not violated.

b) Handling communication:

scopesToListeners maps the scopes to the

applications that are consuming those scopes. In the event of a context value change, the plugin sends a message containing the new context value. The Runtime basically does a multicast to all the applications listening to that specific scope. The prime benefit of the scheme is that all the

communication used is asynchronous and is non-blocking. The benefit obtained as such is that there is no blocking of threads at both the context- aware application as well as the runtime. However, the asynchronous communication model comes with an associated cost, which is further discussed in the limitations. An Application Development Framework is provided as a part of the project to ensure that the application developers are not needed to learn about the OSGi based component lifecycle. The Framework contains all the abstract classes necessary for the development of Context-Aware

Applications (ContextListenerActivity) as well as the different plugins.

3) Context Management: The runtime achieves context management by the use

of RPC calls made from the plugins and the applications. These calls are made in the various methods involved during

binding and unbinding. The iContextManagement and

iContextManagementListener interface are implemented to provide the server side code for context management.

The interaction between the various components is explained with the help of Fig. 4

Fig. 4. System Sequence Diagram

1. requestContextUpdates: The Context Aware

application sends a request to the runtime for its

desired scopes.

2. updateScopesToListeners: The Runtime verifies that

the application has all the necessary permissions for

the required scopes.

95

3. Resolve Plugins: Resolution of plugins as discussed.

4. connectContextServicePlugin: Activate and Connect

to the plugin that provides the requested scope(s).

5. onContextChange: send the updated context value to

the runtime.

6. onContextValueChange: use the scopesToListeners

map to identify the recipients of the new context value

and send the update message to them.

7. OnContextValueChanged: The application processes

the new context value and reacts accordingly.

8. unRegisterContextTypes: Request discontinuation of

context updates for the specified scopes.

9. disconnectContextServicePlugin: deactivates the

plugin and subsequently stops the process.

VI. VALIDATION OF FRAMEWORK

RCMM was implemented on the android platform using the standard Android SDK (Software Development Kit). This ensures that the middleware and the components can be installed on existing mobile phones running android without having to flash them with customized versions of the operating system.

The Context Aware application built was "BatriCon", which basically senses the geo-fence that the user is currently in and depending on the distance from the geo-fence in which the phone is usually charged and the current battery level, appropriate recommendations are given to the user.

The component diagram in Fig. 4 shows a higher view of the relation between the components. The layered architecture approach shows:

A. Context Aware Application - "BatriCon":

Fig. 5. (L)Geo-fences using different color markers.

(R) Final Application with recommendations

It is implemented by extending the

ContextListenerActivity class provided by the framework. It has a GUI (Graphical User Interface) as shown in Fig 5-b and displays the current geo-fence on a map along with the recommendations. It consumes the following scopes:

TABLE I. CONSUMED SCOPES

Sr. no. Scope Provided by

1. battery.level Battery Sensor Plugin

2. power.connected Battery Sensor Plugin

3. location.coarse Location Aggregator Plugin

4. location.fine Location Aggregator Plugin

5. location.latest Location Aggregator Plugin

6. geofencing.latest Geo-fencing Plugin

The application subscribes to these scopes and thus delegates the task of context collection to the plugins.

B. Battery Sensor Plugin:

It is implemented by extending the SensorService class and provides the information about events such as change in battery charge, battery voltage and battery temperature. This plugin does not store this information for future processing and is functional only when the provided scopes are being consumed by some application or plugin. Sample Context Value: scope: battery.level : { value: 74.8 }

C. Location Aggregator Plugin:

It provides the current coarse or fine location on location

change events and also the more latest of the two locations by

using the standard location selection algorithm prescribed by

the android SDK guide. These location values are time

stamped and stored in a database as "historic data". The

frequency of data collection and its accuracy can be adjusted

by changing appropriate parameter values. Sample Context

Value: {value: {latitude: xyz, longitude: mno}}

D. Geo-fencing Plugin:

A geo-fence is a virtual perimeter for a real-world geographic area. The Geo-fencing plugin provides two functions:

1. Provides the set of geo-fences to the applications that

need it

2. Broadcasts the change of geo-fence

The Geo-fencing plugin is a Reasoner Plugin that takes care of the creation of geo-fences and tagging the geo-fences appropriately (e.g. Charging geo-fence, etc.). It consumes the Battery Sensor Plugin services to add appropriate support to a geo-fence each time the charging activity is detected in the geo-fence. It consumes the "location.latest" to know the latest location and update listening applications about the current geo-fence.

It uses a version of the DBScan density based clustering approach to cluster the locations based on the distance between coordinates. The Location Aggregator Plugin provides the data set using which the algorithm generates the seed coordinates of the geo-fences.

96

As seen in Fig. 5-a, the red and blue markers are indicative of the different geo-fences detected. Every time the user connects the phone to a charger, the support for the same is updated in the tags section of the geo-fence database.

TABLE II. SAMPLE GEO-FENCE DATABASE ENTRY

id Time value tags

1 123 { value: [{ latitude: ,

longitude: } , { latitude: ,

longitude: }]}

{ charging_support:

12, label: "home;" }

VII. LIMITATIONS

RCAMM though is very promising in terms of the benefits it proposes, has certain limitations, as follows:

1. Developers eventually have to learn about the OSGi object

lifecycle to be able to develop applications efficiently.

This induces a steep learning curve.

2. Debugging of the applications becomes difficult as the

points of error increase on account of the distribution of

the work amongst different plugins. Existing debuggers do

not support the debugging of processes that are

dynamically forked.

3. Even though "historic data" is de-centralized, the runtime

is a central entity and thus, a "single point of failure".

4. Even though RCAMM promises scalability and

sustainability an appropriate scalability, sustainability

evaluation needs to be done to quantify it.

5. The process life-cycle in Android is not OSGi compliant.

Hence, appropriate changes may be necessary for the

optimization of the framework for android.

6. Android is an open-source project and is frequently

updated. The framework has been developed in

accordance with the process models employed by Android

4.0. Other versions of the OS may need suitable

modifications.

VIII. CONCLUSION

RCAMM provides the Application developers with an infrastructure that provides them with context data updates and historic data, thus reducing the development time and code significantly. The delegation of aggregation tasks to different plugins, reduces the redundant activities of context pre-processing and labeling, to one plugin, thus ensuring efficient processing and higher battery life. It can be easily extended to develop more applications and plugins like location based alarms, vision based plugins. Thus, RCAMM is definitely a sustainable, efficient, flexible and extensible developer friendly framework for the development of Context Aware Applications on the Android Platform.

REFERENCES

[1] G. D. Abowd, A. K. Dey, P. J. Brown, N. Davies, M. Smith, and P. Steggles, "Towards a Better Understanding of Context and Context-Awareness," in Proceedings of the 1st international symposium on Handheld and Ubiquitous Computing Karlsruhe, Germany: Springer-Verlag, 1999.

[2] http://www.google.com/landing/now/

[3] T. Anagnostopoulos, C. Anagnostopoulos, S. Hadjiefthymiades, M. Kyriakakos, and A. Kalousis, "Predicting the location of mobile users: a machine learning approach", in Proceedings of the 2009 international conference on Pervasive services London, United Kingdom: ACM, 2009.

[4] A. Shankar, S. J. Louis, S. Dascalu, L. J. Hayes, and R. Houmanfar, "User-context for adaptive user interfaces," in Proceedings of the 12th international conference on “Intelligent user interfaces”, Honolulu, Hawaii, USA: ACM, 2007.

[5] D. H. Wilson, A. C. Long, and C. Atkeson, "A context-aware recognition survey for data collection using ubiquitous sensors in the home," in CHI '05 extended abstracts on Human factors in computing systems Portland, OR, USA: ACM, 2005.

[6] M. Davis, M. Smith, J. Canny, N. Good, S. King, and R. Janakiraman, "Towards context-aware face recognition," in Proceedings of the 13th annual ACM international conference on Multimedia Hilton, Singapore: ACM, 2005.

[7] S. Dornbush, A. Joshi, Z. Segall, and T. Oates, "A Human Activity Aware Learning Mobile Music Player," in Proceeding of the 2007 conference on Advances in Ambient Intelligence: IOS Press, 2007.

[8] Dey, A.K. (2001) The Context Toolkit – A Toolkit for Context-aware Applications, http://www.cs.berkeley.edu/~dey/context.html.

[9] Dey, A.K. and Abowd, G.D. (2000a) ‘The context toolkit: aiding the development of context-aware applications’, Workshop on Software Engineering for Wearable and Pervasive Computing, Limerick, Ireland.

[10] Hofer, T., Schwinger, W., Pichler, M., Leonhartsberger, G. and Altmann, J. (2002) ‘Context-awareness on mobile devices – the hydrogen approach’, Proceedings of the 36

th Annual Hawaii International

Conference on System Sciences, pp. 292–302.

[11] Baldauf, M., Dustdar, S. and Rosenberg, F.(2007) ‘A survey on context-aware systems’, Int. J. Ad Hoc and Ubiquitous Computing, Vol. 2, No. 4, pp.263–277.

[12] Fahy, P. and Clarke, S. (2004) ‘CASS – a middleware for mobile context-aware applications’, Workshop on Context Awareness, MobiSys 2004

[13] Chen, H. (2004) An Intelligent Broker Architecture for Pervasive Context-Aware Systems, PhD Thesis, University of Maryland, Baltimore County

[14] Gu, T., Pung, H.K. and Zhang, D.Q. (2004a) ‘A middleware for building context-aware mobile services’, Proceedings of IEEE Vehicular Technology Conference (VTC), Milan, Italy

[15] Ahmad, . Q. (2010). Context-Aware Machine Learning in Android. (Student Paper). Norges teknisk-naturvitenskapelige universitet Institutt for datateknikk og informasjonsviten

97