real-time integrated platform for services and analytics ...cs620/ripsac_overview.pdf · ripsac...

14
TCS Confidential i Real-time Integrated Platform for Services and Analytics (RIPSAC) An Overview

Upload: others

Post on 17-Jun-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

TCS Confidential i

Real-time Integrated Platform for Services and Analytics (RIPSAC)

An Overview

Page 2: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

TCS Confidential i

Contents

1. Background ...................................................................................................... 2

1.1 Motivation ....................................................................................................................................... 2

1.2 Problems addressed by RIPSAC ...................................................................................................... 3

2. RIPSAC Components ......................................................................................... 5

2.1 Core Platform Components ............................................................................................................. 5

2.2 Multi-tenancy and isolation .............................................................................................................. 5

2.3 RIPSAC Architectural layers ........................................................................................................... 5

2.4 Extended Platform Services ............................................................................................................ 8

2.5 Analytics Services............................................................................................................................ 9

3. Vertical Domain Focus ..................................................................................... 10

3.1 Transportation .............................................................................................................................. 10

3.2 Electrical Energy and Smart Grid ................................................................................................... 10

3.3 Wind Forecasting Services............................................................................................................. 10

3.4 Remote Healthcare and Wellness .................................................................................................. 11

Page 3: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

2 30 October 2012

1. Background

RIPSAC (Real-Time Integrated Platform for Services & Analytics) is Platform-as-a-Service (PaaS)

cloud computing platform that allows quick and easy development, deployment and administration of

sensor driven applications. RIPSAC provides sensor device management, data acquisition, data

storage and analytics services. These services are made available to application developers in form of

APIs and SDKs. RIPSAC aims to provide a highly scalable platform for sensor integration, sensor

data storage, analytics (including real-time and Big Data processing), rich query capabilities

(including geo-spatial queries and continuous queries) and visualization. RIPSAC is currently under

development in TCS Innovation Labs.

At the core of RIPSAC is Sensor Web Enablement (SWE) services – a set of services related to sensor

description, discovery, integration, sensor observation and measurement capture, storage and query.

These services are built as per standard interface and schema specifications published by the Open

Geospatial Consortium. RIPSAC provides these services in form of APIs and libraries.

App developers will develop, test, deploy and manage Java applications in RIPSAC. RIPSAC

supports multi-tenancy and provides secure sandboxes for testing and deployment of applications by

each tenant. End users will download Apps, subscribe & unsubscribe to them, control their privacy

settings, and view usage history and billing information.

1.1 Motivation

Open, sharable sensor data repository

There is a need to create a platform where sensor configuration data and sensor observations data is

available as a service. This platform must support virtually any kind of sensor and sensor

observations. The schema and the semantics must be based on open standards and ontologies.

Depending on data sharing policies set by users, sensor data may be shared either publicly or to

targeted users or application developers. Application developers can query the repository for sensor

data. Multiple different types of sensor observations may be combined to create different kinds of

intelligent apps. Sensor data is no longer locked up in separate application specific silos.

Crowd sourcing of sensor based apps

When a shareable, open repository is available, the sensor data services may be exposed via open

APIs. These APIs will allow third party developers to develop their apps. Developers would be

allowed to export anonymized sensor data from the repository so that they can develop and test their

data mining and machine learning algorithms. Such a facility will allow newer and more interesting

apps to evolve over time.

Gaps in existing PaaS Platforms

There are various PaaS offerings like Google App Engine, Heroku, Microsoft Azure etc; however

they do not provide specialized services required in IOT/ Cyber Physical Systems domain. In this

domain, there is a need for specialized services to cater to applications that want to leverage web

Page 4: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

3 30 October 2012

connected sensors and sensors available as part of smart mobile devices. Sensor discovery,

description, interfacing, query and tasking are some of the key requirements.

Additionally, the following capabilities essential for sensor driven applications are not available on

general purpose PaaS platforms –

Sensor based applications need to be event driven and therefore require capabilities such as

event processing or stream processing

Supports for various types of databases such as RDBMS, NOSQL and Object Stores

Requirement for specialized analytics such as data mining, machine learning, statistical

processing, etc.

Specialized visualization

None of the known PaaS platforms provide support for all these features in a single platform

On the other hand, there are sensor platforms notably from Pachube, Sun Microsystem Sensor

Networks; however they mainly focus on sensor data storage services with very rudimentary support

for application development. Additionally, there is very little support in these platforms for location

based processing, spatial and spatio-temporal processing. Additionally, these sensor platforms provide

no support for crowd sourced applications to be developed and deployed on these platforms.

Then there are sensor and gateway device vendors like Digi, Mobile Devices who has their sensor

network platform along with data cloud; however they cater for sensor and device from that

particular vendor only and not suitable for multi-vendor generic sensor data & observation processing.

Additionally, these platforms have very limit support for sensor data storage and analytics.

TCS RIPSAC platform addresses the blind spots which are not covered by these other categories of

offerings.

1.2 Problems addressed by RIPSAC

RIPSAC platform greatly simplifies development of IOT applications involving sensor data and

observations. Platform users can expedite application development in different verticals like Energy,

Utility, Government sector, Transportation, Healthcare, Education by using RIPSAC platform and its

application services

Some of the key problems addressed by RIPSAC is as follows –

RIPSAC provides an integrated platform for sensor data capture, storage, analytics,

visualization. Existing systems do not provide such comprehensive support in a single

platform.

RIPSAC enables easy development and deployment of applications developed by many

different third party developers. RIPSAC services are made available in form of APIs (

Application Programming Interfaces) and Software Development Kits.

Ability to support many different application developers, application end users and sensor

providers in a single platform is one of the key challenges addressed by RIPSAC. RIPSAC is

Page 5: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

4 30 October 2012

a multi-tenant platform. It allows multiple sensor data providers, multiple application

developers and application end users to use the platform in a secure and mutually isolated

way.

RIPSAC also allows sensor data to be shared across applications and users. RIPSAC allows

policy driven data privacy and policy driven sharing of sensor data across users and

applications.

RIPSAC supports virtually any kind of sensor to be interfaced with RIPSAC platform. The

schema used by RIPSAC to capture and store sensor data is generic and allows virtually any

kind of sensor observations to be captured, stored and queried. Additionally, the interface use

by applications to access captured sensor data is standard and independent of the type of

sensor or sensor observation.

RIPSAC provides scalable sensor data storage for a wide variety of sensors and sensor

observations. Different types of data storage mechanisms such as relational databases, schema

free document databases and object storage systems are provided.

RIPSAC provides scalable analytics services. Depending on the amount of computational

load for analytics, the computation resources dedicated to analytics can be dynamically

adjusted.

Current Status

Version 1 of RIPSAC Beta is available and hosted in TCS R&D datacentre in Chennai. It is available

at https://ripsac.web2labs.net. Two applications are already hosted on RIPSAC. The first one is a

Bus aTracking System for TCS Siruseri campus. The second one is a Wind Forecasting Solution.

RIPSAC uses an infrastructure cloud called OpenStack for its infrastructure layer.

Page 6: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

5 30 October 2012

2. RIPSAC Components

RIPSAC has three main compenents in its architecture - Core Platform Components, Extended

Platform Services and Analytics Servcies.

2.1 Core Platform Components

These are basic features made available on the current RIPSAC platform.

1. Application hosting & deployment services

a. Support for Java and Play web applications and batch applications

2. Sensor Services

a. Sensor Web Enablement

b. Remote Device Monitoring & Management

c. Sensor Data Storage & Query

3. Platform Services

a. Scalable database services – including SQL, NOSQL and Object Storage services

b. Data Exports

c. Event Distribution & Messaging

2.2 Multi-tenancy and isolation

d. Service Integrations, Orchestration and Choreography

4. Basic Security Services

a. Policy driven access and usage controls

b. Identity management

5. Basic Analytics Services

a. Scalable and load balanced R analytics services

6. Basic Visualization Services

7. Integration with Infrastructure-as-a-Service cloud platform – OpenStack.

2.3 RIPSAC Architectural layers

The architectural layers and description is given below.

Page 7: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

6 30 October 2012

Data Aggregation Device Monitoring & Management

Device Integration & Management Services

Visualization Components

Statistical Analysis

Machine Learning

Rule Engines

Event Processing

Reasoning & Ontlology

Optimization

Analytics Services

Application Components

Application Services

Data Storage Data Access /

Query Continuous

Queries

Storage

Data & Event Distribution

Messaging

Messaging & Event Distribution Services

Core Services

Application Services

Presentation Services

Portal User Interfaces

Application Support Services

Integration & Orchestration

Planners Platform API & SDKs

Secu

rity

& P

riva

cy

Page 8: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

7 30 October 2012

Category Components Description, Features / Functions

Portal Application Developers Portal This component is used to application developers to register themselves, their applications, create databases, upload and test analytics programs etc.

Administrators Portal This component is used by the RIPSAC administrator to manage tenants, monitor and manage the underlying software and hardware infrastructure, monitor, manage and control usage of platform services by tenants.

Services Device Integration and Management Services

Services for interfacing gateway devices, sensors, mobile devices and their network addresses in RIPSAC

Support for various network protocols for data communication between these devices and RIPSAC

Ability monitor the health and status of these devices

Ability to deploy software on these devices from RIPSAC

Messaging & Event Distribution Services

Providing an infrastructure for passing of messages and events across RIPSAC services and applications

Data Storage & Query Services Large scale, distributed sensor data storage and query facility, including support for geo-spatial queries

Continuous Query processors

Analytics Services Consisting of following types of libraries and servers –

Machine Learning packages

Statistical Processing packages

Rule Engines

Complex Event Processing and Stream Processing

Knowledge driven processing

Application Support Services Support for service integration and orchestration

Core Sensor Services Sensor and Senor Observations Description Services

Sensor Discovery

Describing features and phenomena

Inserting Observations

Querying observations

User Interface & Visualization Services Libraries and tools for creating rich visualizations and reports from sensor data

Security & Privacy Services Identity management Tenant and End User Authentication Policy driven Authorization Policy driven data privacy controls Policy driven data masking

Software Development Tools

API Web Services and language specific bindings to RIPSAC services

Page 9: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

8 30 October 2012

Data Exports Ability to create anonymized and masked data exports from sensor database

Applications Producer Applications Applications that publish sensor data to RIPSAC platform

Consumer Applications Applications that query and use sensor data from RIPSAC platform

Producer cum Consumer Applications Applications that are both producers and consumers

Software Infrastructure

Application Servers Containers / Virtual Machines / hosts on which user applications are run

Relational Databases Relational Database services provided in RIPSAC.

Document Databases Document database services provided in RIPSAC

Data Center Infrastructure Services

Compute, Network & Disk Storage Services

Underlying hardware / virtual hardware infrastructure on which RIPSAC components finally run. Consists of Servers, disks and network resources

File Services File storage services provided to servers

Firewall Services Services to create secure zones based on policies to separate different tenants from each other

2.4 Extended Platform Services

These are enhanced platform components on RIPSAC which are needed for specialized use case

scenarios. In addtion to performance and scalability, these services also look into features like privacy

and visualization which are end-user experience level services.

1. Middleware Services

a. Lightweight yet reliable communication protocols between edge gateway and

backend over internet

2. Privacy and Security

a. Data Masking/Obfuscation and Anonymization

b. Privacy vs. Utility

c. Identity based Encryption

3. Data Visualization

a. Framework for generic Visualization support

4. Parallel and Distributed Computing

a. Framework for distributed execution of analytics

b. Big Data Support

5. Simulation

Page 10: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

9 30 October 2012

a. Sensor model driven synthesis

b. Cyber-physical system level simulation and modeling

2.5 Analytics Services

These are collection of algorithms and toolboxes to help application developers on RIPSAC to

analyze the data collected from sensors and create intelligent applications. These services tend to be

sensor-specifc and in some cases, use case specific. Typically the kind of analytics services provided

can be classified as the following context understanding systems –

a) Physical Objects

Recognition (which), Localization (where), State and Event Recognition (what),

Diagnosis and Prognosis, Interoperation, Actuation

b) Human Beings

Recogniton (who), Localization (where), Physical Activity Detection (what),

Physiological Activity Detection (inside human body), Network Discovery

The following modules can be envisaged under RIPSAC for providing the above services -

1. Sensor Signal Processing and Informatics

a. Sensor Data Compression and Disaggregation

2. Specialized Sensing

a. Mobile phone based sensing

b. 3D camera based sensing

c. Web-as-a-soft-sensor

d. Physiological sensing

3. Real-time processing

a. Event Processing

b. Stream Processing and Stream Reasoning

c. Statistical Data Processing

4. Model-driven Analytics

a. Semantic Interoperability of Sensors

b. Sensor physical and environmental model driven system identification and prognosis

c. Cyber-physical system model driven prediction and analysis

Page 11: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

10 30 October 2012

3. Vertical Domain Focus

3.1 Transportation

Transportation for TCS campus at Siruseri in Chennai is a test bed for RIPSAC. A set of two pilot

projects are being executed for TCS Siruseri campus and these projects are being implemented using

RIPSAC. RIPSC itself will be hosted out of CTO Private Cloud. These projects are as follows –

Bus Tracking System – This is a real time bus tracking and passenger authentication system. Buses

will be tracked using on board GPS devices provided by Digi. Buses will also have a TCS ID card

reader. The system once implemented will provide a real time visibility of buses and their operation,

reduce unauthorized travel, improve utilization level of buses and reduce paperwork for TCS

administration.

Cab Tracking System – This system comprises of hand held device based cab identification and

passenger data collection system. Handheld devices are used to automatically capture the registration

number of cabs using image processing, identify cab passengers by scanning their ID cards and

automatically fill in and upload “trip sheets”.

3.2 Electrical Energy and Smart Grid

The need to conserve energy and climate change issues is forcing utilities to transcend into homes of

consumers and provide greater consumer empowerment and better energy services. Smart home

devices, advanced analytical algorithm and convergence in communication provide a fascinating

potential to achieve green targets without compromising the convenience of consumer. The Utility of

the Future will need to exploit these technologies and social enablement in the market place to

integrate the consumer more dynamically, thus achieving operational and regulatory goals along with

increased customer satisfaction. This would pose new challenges in customer data privacy and

security along with this fine grained in-home integration. TCS has come up with Home Energy

Management Solution (HEM), a platform which provides necessary components that can be

integrated to backend systems in the Utility to realize powerful use cases for the customer.

Though the current solution is done on a reference hardware board, in next phase it will be using

ConnectPort X2e Digi gateway – a low cost IP to ZigBee SE (Smart Energy) gateway targeted at

residential and light commercial use.

The whole work is being done in collaboration with ERU ISU and there is already one pilot in

progress with Greeniant, Netherlands.

3.3 Wind Forecasting Services

The Wind Forecasting system is currently being developed on top of RIPSAC in collaboration with

Infrastructure Lab. The application provides improved wind speed forecasts at wind turbine locations.

The Wind Forecasting Service is being planned as a SaaS offering.

The heart of the wind forecasting system consists of an innovative multi-forecast fusion algorithm

using Ada Boost technique. Data from multiple wind prediction systems are combined to create a

Page 12: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

11 30 October 2012

more accurate forecast. The algorithm is trained using past forecasts and actual wind turbine wind

speed data as captured by SCADA systems.

RIPSAC platform will be used in the following way –

RIPSAC will provide the basis application hosting environment

RIPSAC sensor services will be used to capture and report both actual wind speed data,

individual predictor values and also to store values of output from the combined predictor.

The algorithm is run on the scalable R platform that is available as part of the RIPSAC

platform. The algorithm needs to be run for each individual turbine once every half hour.

RIPSAC visualization services are used to create visualizations of the wind turbine and

dynamic status of the sensor combination algorithm.

3.4 Remote Healthcare and Wellness

Remote Medical Consultation

The aim of this project is to prepare an end to end successful demonstration of smart healthcare

service. It is a Remote Medical Consultation Solution where the expert doctor can view patient’s

physiological data collected by a technician at a rural health center or at patient’s home using portable

wireless medical instruments.

The solution is already built as a demonstrator on top of RIPSAC platform. However the current

solution is very much device vendor-specific that only works with the medical instruments and

gateways from a company called Etcomm in China. In the next phase, we want to make it medical

instrument vendor independent, through using a generic gateway device from Digi, who is a COIN

partner in M2M devices space.

For the above purpose, following things are planned -

Making Digi Platform ready for accepting medical records

Developing 433 MHz communication capability in Digi platform

Integrating all Etcomm medical devices into RIPSAC via Digi Cloud

Developing some intelligent analytics and rich visual tools for medical records in RIPSAC

cloud

TCS Fit4Life

The aim of the project is to use mobile phone based sensing and its associated processing to do

automatic detection of exact distance run, mode of running (brisk walking, jogging, running),

estimates of calories burnt, and instant pulse rate measurement. The existing Fit4Life mobile app can

only measure distances based on GPS and do not have any of the above features. It also does not work

indoors. The proposed solution uses sensors available on smartphones like accelerometer, compass,

gyroscope and camera. The solution is being developed jointly with Mobility and Big Data Unit.

Page 13: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for

RIPSAC Overview

12 30 October 2012

Page 14: Real-time Integrated Platform for Services and Analytics ...cs620/RIPSAC_Overview.pdf · RIPSAC Overview 2 30 October 2012 1. Background RIPSAC (Real-Time Integrated Platform for