real-time integrated platform for services and analytics ...cs620/ripsac_overview.pdf · ripsac...
TRANSCRIPT
TCS Confidential i
Real-time Integrated Platform for Services and Analytics (RIPSAC)
An Overview
RIPSAC Overview
TCS Confidential i
Contents
1. Background ...................................................................................................... 2
1.1 Motivation ....................................................................................................................................... 2
1.2 Problems addressed by RIPSAC ...................................................................................................... 3
2. RIPSAC Components ......................................................................................... 5
2.1 Core Platform Components ............................................................................................................. 5
2.2 Multi-tenancy and isolation .............................................................................................................. 5
2.3 RIPSAC Architectural layers ........................................................................................................... 5
2.4 Extended Platform Services ............................................................................................................ 8
2.5 Analytics Services............................................................................................................................ 9
3. Vertical Domain Focus ..................................................................................... 10
3.1 Transportation .............................................................................................................................. 10
3.2 Electrical Energy and Smart Grid ................................................................................................... 10
3.3 Wind Forecasting Services............................................................................................................. 10
3.4 Remote Healthcare and Wellness .................................................................................................. 11
RIPSAC Overview
2 30 October 2012
1. Background
RIPSAC (Real-Time Integrated Platform for Services & Analytics) is Platform-as-a-Service (PaaS)
cloud computing platform that allows quick and easy development, deployment and administration of
sensor driven applications. RIPSAC provides sensor device management, data acquisition, data
storage and analytics services. These services are made available to application developers in form of
APIs and SDKs. RIPSAC aims to provide a highly scalable platform for sensor integration, sensor
data storage, analytics (including real-time and Big Data processing), rich query capabilities
(including geo-spatial queries and continuous queries) and visualization. RIPSAC is currently under
development in TCS Innovation Labs.
At the core of RIPSAC is Sensor Web Enablement (SWE) services – a set of services related to sensor
description, discovery, integration, sensor observation and measurement capture, storage and query.
These services are built as per standard interface and schema specifications published by the Open
Geospatial Consortium. RIPSAC provides these services in form of APIs and libraries.
App developers will develop, test, deploy and manage Java applications in RIPSAC. RIPSAC
supports multi-tenancy and provides secure sandboxes for testing and deployment of applications by
each tenant. End users will download Apps, subscribe & unsubscribe to them, control their privacy
settings, and view usage history and billing information.
1.1 Motivation
Open, sharable sensor data repository
There is a need to create a platform where sensor configuration data and sensor observations data is
available as a service. This platform must support virtually any kind of sensor and sensor
observations. The schema and the semantics must be based on open standards and ontologies.
Depending on data sharing policies set by users, sensor data may be shared either publicly or to
targeted users or application developers. Application developers can query the repository for sensor
data. Multiple different types of sensor observations may be combined to create different kinds of
intelligent apps. Sensor data is no longer locked up in separate application specific silos.
Crowd sourcing of sensor based apps
When a shareable, open repository is available, the sensor data services may be exposed via open
APIs. These APIs will allow third party developers to develop their apps. Developers would be
allowed to export anonymized sensor data from the repository so that they can develop and test their
data mining and machine learning algorithms. Such a facility will allow newer and more interesting
apps to evolve over time.
Gaps in existing PaaS Platforms
There are various PaaS offerings like Google App Engine, Heroku, Microsoft Azure etc; however
they do not provide specialized services required in IOT/ Cyber Physical Systems domain. In this
domain, there is a need for specialized services to cater to applications that want to leverage web
RIPSAC Overview
3 30 October 2012
connected sensors and sensors available as part of smart mobile devices. Sensor discovery,
description, interfacing, query and tasking are some of the key requirements.
Additionally, the following capabilities essential for sensor driven applications are not available on
general purpose PaaS platforms –
Sensor based applications need to be event driven and therefore require capabilities such as
event processing or stream processing
Supports for various types of databases such as RDBMS, NOSQL and Object Stores
Requirement for specialized analytics such as data mining, machine learning, statistical
processing, etc.
Specialized visualization
None of the known PaaS platforms provide support for all these features in a single platform
On the other hand, there are sensor platforms notably from Pachube, Sun Microsystem Sensor
Networks; however they mainly focus on sensor data storage services with very rudimentary support
for application development. Additionally, there is very little support in these platforms for location
based processing, spatial and spatio-temporal processing. Additionally, these sensor platforms provide
no support for crowd sourced applications to be developed and deployed on these platforms.
Then there are sensor and gateway device vendors like Digi, Mobile Devices who has their sensor
network platform along with data cloud; however they cater for sensor and device from that
particular vendor only and not suitable for multi-vendor generic sensor data & observation processing.
Additionally, these platforms have very limit support for sensor data storage and analytics.
TCS RIPSAC platform addresses the blind spots which are not covered by these other categories of
offerings.
1.2 Problems addressed by RIPSAC
RIPSAC platform greatly simplifies development of IOT applications involving sensor data and
observations. Platform users can expedite application development in different verticals like Energy,
Utility, Government sector, Transportation, Healthcare, Education by using RIPSAC platform and its
application services
Some of the key problems addressed by RIPSAC is as follows –
RIPSAC provides an integrated platform for sensor data capture, storage, analytics,
visualization. Existing systems do not provide such comprehensive support in a single
platform.
RIPSAC enables easy development and deployment of applications developed by many
different third party developers. RIPSAC services are made available in form of APIs (
Application Programming Interfaces) and Software Development Kits.
Ability to support many different application developers, application end users and sensor
providers in a single platform is one of the key challenges addressed by RIPSAC. RIPSAC is
RIPSAC Overview
4 30 October 2012
a multi-tenant platform. It allows multiple sensor data providers, multiple application
developers and application end users to use the platform in a secure and mutually isolated
way.
RIPSAC also allows sensor data to be shared across applications and users. RIPSAC allows
policy driven data privacy and policy driven sharing of sensor data across users and
applications.
RIPSAC supports virtually any kind of sensor to be interfaced with RIPSAC platform. The
schema used by RIPSAC to capture and store sensor data is generic and allows virtually any
kind of sensor observations to be captured, stored and queried. Additionally, the interface use
by applications to access captured sensor data is standard and independent of the type of
sensor or sensor observation.
RIPSAC provides scalable sensor data storage for a wide variety of sensors and sensor
observations. Different types of data storage mechanisms such as relational databases, schema
free document databases and object storage systems are provided.
RIPSAC provides scalable analytics services. Depending on the amount of computational
load for analytics, the computation resources dedicated to analytics can be dynamically
adjusted.
Current Status
Version 1 of RIPSAC Beta is available and hosted in TCS R&D datacentre in Chennai. It is available
at https://ripsac.web2labs.net. Two applications are already hosted on RIPSAC. The first one is a
Bus aTracking System for TCS Siruseri campus. The second one is a Wind Forecasting Solution.
RIPSAC uses an infrastructure cloud called OpenStack for its infrastructure layer.
RIPSAC Overview
5 30 October 2012
2. RIPSAC Components
RIPSAC has three main compenents in its architecture - Core Platform Components, Extended
Platform Services and Analytics Servcies.
2.1 Core Platform Components
These are basic features made available on the current RIPSAC platform.
1. Application hosting & deployment services
a. Support for Java and Play web applications and batch applications
2. Sensor Services
a. Sensor Web Enablement
b. Remote Device Monitoring & Management
c. Sensor Data Storage & Query
3. Platform Services
a. Scalable database services – including SQL, NOSQL and Object Storage services
b. Data Exports
c. Event Distribution & Messaging
2.2 Multi-tenancy and isolation
d. Service Integrations, Orchestration and Choreography
4. Basic Security Services
a. Policy driven access and usage controls
b. Identity management
5. Basic Analytics Services
a. Scalable and load balanced R analytics services
6. Basic Visualization Services
7. Integration with Infrastructure-as-a-Service cloud platform – OpenStack.
2.3 RIPSAC Architectural layers
The architectural layers and description is given below.
RIPSAC Overview
6 30 October 2012
Data Aggregation Device Monitoring & Management
Device Integration & Management Services
Visualization Components
Statistical Analysis
Machine Learning
Rule Engines
Event Processing
Reasoning & Ontlology
Optimization
Analytics Services
Application Components
Application Services
Data Storage Data Access /
Query Continuous
Queries
Storage
Data & Event Distribution
Messaging
Messaging & Event Distribution Services
Core Services
Application Services
Presentation Services
Portal User Interfaces
Application Support Services
Integration & Orchestration
Planners Platform API & SDKs
Secu
rity
& P
riva
cy
RIPSAC Overview
7 30 October 2012
Category Components Description, Features / Functions
Portal Application Developers Portal This component is used to application developers to register themselves, their applications, create databases, upload and test analytics programs etc.
Administrators Portal This component is used by the RIPSAC administrator to manage tenants, monitor and manage the underlying software and hardware infrastructure, monitor, manage and control usage of platform services by tenants.
Services Device Integration and Management Services
Services for interfacing gateway devices, sensors, mobile devices and their network addresses in RIPSAC
Support for various network protocols for data communication between these devices and RIPSAC
Ability monitor the health and status of these devices
Ability to deploy software on these devices from RIPSAC
Messaging & Event Distribution Services
Providing an infrastructure for passing of messages and events across RIPSAC services and applications
Data Storage & Query Services Large scale, distributed sensor data storage and query facility, including support for geo-spatial queries
Continuous Query processors
Analytics Services Consisting of following types of libraries and servers –
Machine Learning packages
Statistical Processing packages
Rule Engines
Complex Event Processing and Stream Processing
Knowledge driven processing
Application Support Services Support for service integration and orchestration
Core Sensor Services Sensor and Senor Observations Description Services
Sensor Discovery
Describing features and phenomena
Inserting Observations
Querying observations
User Interface & Visualization Services Libraries and tools for creating rich visualizations and reports from sensor data
Security & Privacy Services Identity management Tenant and End User Authentication Policy driven Authorization Policy driven data privacy controls Policy driven data masking
Software Development Tools
API Web Services and language specific bindings to RIPSAC services
RIPSAC Overview
8 30 October 2012
Data Exports Ability to create anonymized and masked data exports from sensor database
Applications Producer Applications Applications that publish sensor data to RIPSAC platform
Consumer Applications Applications that query and use sensor data from RIPSAC platform
Producer cum Consumer Applications Applications that are both producers and consumers
Software Infrastructure
Application Servers Containers / Virtual Machines / hosts on which user applications are run
Relational Databases Relational Database services provided in RIPSAC.
Document Databases Document database services provided in RIPSAC
Data Center Infrastructure Services
Compute, Network & Disk Storage Services
Underlying hardware / virtual hardware infrastructure on which RIPSAC components finally run. Consists of Servers, disks and network resources
File Services File storage services provided to servers
Firewall Services Services to create secure zones based on policies to separate different tenants from each other
2.4 Extended Platform Services
These are enhanced platform components on RIPSAC which are needed for specialized use case
scenarios. In addtion to performance and scalability, these services also look into features like privacy
and visualization which are end-user experience level services.
1. Middleware Services
a. Lightweight yet reliable communication protocols between edge gateway and
backend over internet
2. Privacy and Security
a. Data Masking/Obfuscation and Anonymization
b. Privacy vs. Utility
c. Identity based Encryption
3. Data Visualization
a. Framework for generic Visualization support
4. Parallel and Distributed Computing
a. Framework for distributed execution of analytics
b. Big Data Support
5. Simulation
RIPSAC Overview
9 30 October 2012
a. Sensor model driven synthesis
b. Cyber-physical system level simulation and modeling
2.5 Analytics Services
These are collection of algorithms and toolboxes to help application developers on RIPSAC to
analyze the data collected from sensors and create intelligent applications. These services tend to be
sensor-specifc and in some cases, use case specific. Typically the kind of analytics services provided
can be classified as the following context understanding systems –
a) Physical Objects
Recognition (which), Localization (where), State and Event Recognition (what),
Diagnosis and Prognosis, Interoperation, Actuation
b) Human Beings
Recogniton (who), Localization (where), Physical Activity Detection (what),
Physiological Activity Detection (inside human body), Network Discovery
The following modules can be envisaged under RIPSAC for providing the above services -
1. Sensor Signal Processing and Informatics
a. Sensor Data Compression and Disaggregation
2. Specialized Sensing
a. Mobile phone based sensing
b. 3D camera based sensing
c. Web-as-a-soft-sensor
d. Physiological sensing
3. Real-time processing
a. Event Processing
b. Stream Processing and Stream Reasoning
c. Statistical Data Processing
4. Model-driven Analytics
a. Semantic Interoperability of Sensors
b. Sensor physical and environmental model driven system identification and prognosis
c. Cyber-physical system model driven prediction and analysis
RIPSAC Overview
10 30 October 2012
3. Vertical Domain Focus
3.1 Transportation
Transportation for TCS campus at Siruseri in Chennai is a test bed for RIPSAC. A set of two pilot
projects are being executed for TCS Siruseri campus and these projects are being implemented using
RIPSAC. RIPSC itself will be hosted out of CTO Private Cloud. These projects are as follows –
Bus Tracking System – This is a real time bus tracking and passenger authentication system. Buses
will be tracked using on board GPS devices provided by Digi. Buses will also have a TCS ID card
reader. The system once implemented will provide a real time visibility of buses and their operation,
reduce unauthorized travel, improve utilization level of buses and reduce paperwork for TCS
administration.
Cab Tracking System – This system comprises of hand held device based cab identification and
passenger data collection system. Handheld devices are used to automatically capture the registration
number of cabs using image processing, identify cab passengers by scanning their ID cards and
automatically fill in and upload “trip sheets”.
3.2 Electrical Energy and Smart Grid
The need to conserve energy and climate change issues is forcing utilities to transcend into homes of
consumers and provide greater consumer empowerment and better energy services. Smart home
devices, advanced analytical algorithm and convergence in communication provide a fascinating
potential to achieve green targets without compromising the convenience of consumer. The Utility of
the Future will need to exploit these technologies and social enablement in the market place to
integrate the consumer more dynamically, thus achieving operational and regulatory goals along with
increased customer satisfaction. This would pose new challenges in customer data privacy and
security along with this fine grained in-home integration. TCS has come up with Home Energy
Management Solution (HEM), a platform which provides necessary components that can be
integrated to backend systems in the Utility to realize powerful use cases for the customer.
Though the current solution is done on a reference hardware board, in next phase it will be using
ConnectPort X2e Digi gateway – a low cost IP to ZigBee SE (Smart Energy) gateway targeted at
residential and light commercial use.
The whole work is being done in collaboration with ERU ISU and there is already one pilot in
progress with Greeniant, Netherlands.
3.3 Wind Forecasting Services
The Wind Forecasting system is currently being developed on top of RIPSAC in collaboration with
Infrastructure Lab. The application provides improved wind speed forecasts at wind turbine locations.
The Wind Forecasting Service is being planned as a SaaS offering.
The heart of the wind forecasting system consists of an innovative multi-forecast fusion algorithm
using Ada Boost technique. Data from multiple wind prediction systems are combined to create a
RIPSAC Overview
11 30 October 2012
more accurate forecast. The algorithm is trained using past forecasts and actual wind turbine wind
speed data as captured by SCADA systems.
RIPSAC platform will be used in the following way –
RIPSAC will provide the basis application hosting environment
RIPSAC sensor services will be used to capture and report both actual wind speed data,
individual predictor values and also to store values of output from the combined predictor.
The algorithm is run on the scalable R platform that is available as part of the RIPSAC
platform. The algorithm needs to be run for each individual turbine once every half hour.
RIPSAC visualization services are used to create visualizations of the wind turbine and
dynamic status of the sensor combination algorithm.
3.4 Remote Healthcare and Wellness
Remote Medical Consultation
The aim of this project is to prepare an end to end successful demonstration of smart healthcare
service. It is a Remote Medical Consultation Solution where the expert doctor can view patient’s
physiological data collected by a technician at a rural health center or at patient’s home using portable
wireless medical instruments.
The solution is already built as a demonstrator on top of RIPSAC platform. However the current
solution is very much device vendor-specific that only works with the medical instruments and
gateways from a company called Etcomm in China. In the next phase, we want to make it medical
instrument vendor independent, through using a generic gateway device from Digi, who is a COIN
partner in M2M devices space.
For the above purpose, following things are planned -
Making Digi Platform ready for accepting medical records
Developing 433 MHz communication capability in Digi platform
Integrating all Etcomm medical devices into RIPSAC via Digi Cloud
Developing some intelligent analytics and rich visual tools for medical records in RIPSAC
cloud
TCS Fit4Life
The aim of the project is to use mobile phone based sensing and its associated processing to do
automatic detection of exact distance run, mode of running (brisk walking, jogging, running),
estimates of calories burnt, and instant pulse rate measurement. The existing Fit4Life mobile app can
only measure distances based on GPS and do not have any of the above features. It also does not work
indoors. The proposed solution uses sensors available on smartphones like accelerometer, compass,
gyroscope and camera. The solution is being developed jointly with Mobility and Big Data Unit.
RIPSAC Overview
12 30 October 2012