Cps innovation lab kolkata iiest

Download Cps innovation lab kolkata iiest

Post on 09-Aug-2015




0 download

Embed Size (px)


<ol><li> 1. 1 Copyright 2014 Tata Consultancy Services Limited Internet-of-Things and Cyber-physical Systems - Exploratory Research in Signal Processing, Communication and Computing Dr. Arpan Pal Principal Scientist and Research Head Innovation Lab, Kolkata TCS 17-May-15 </li><li> 2. 2 Cyber-physical Systems Internet-of-Everything Humans Physical Objects and Infrastructure Computing Infrastructure Physical Context Discovery INTERNET OF EVERYTHING Physical Context Discovery What is happening, where and when People Context Discovery Who is doing what, where and when, who is thinking what Internet of Digital Internet of Things Internet of Humans ABI Research. May 7, 2014 "In the next century, planet earth will don an electronic skin. It will use the Internet as a scaffold to support and transmit its sensations. - Neil Gross 1999 </li><li> 3. 3 Its a Connected Universe NEVER FORGET YOUR PILLS MONITOR YOUR ACTIVITYMONITOR THE AGED source: http://postscapes.com/internet-of-things-examples/ HEAT YOUR HOME EFFICIENTLY MAKE SURE THE OVEN IS OFF TRACK DOWN THOSE LOST KEYS KEEP THE CITY CLEAN RECEIVE POLLUTION WARNINGS USE ELECTRICITY MORE EFFICIENTLY Connected Individual Connected Home Connected City </li><li> 4. 4 Research Programs Outline Mobile Phone Sensing Camera Sensing Other SensorsBio-Sensing Signal and Image Processing Protocols and Networking Parallel and Distributed Computing Data Analytics (Computational and Semantic) and Modeling E D G E C L O U D Personal Context Discovery (Location, Activity, Psychology) Mobiles Cameras Bio-sensing Manage Scale Reduce Network Load Increase Compute Capacity Reduce Storage Requirement Handle Interoperability Easy-to-use Analytics Physiological Sensing using Mobile Phones Mobile phone and Robot based sensing Human Activity Detection and Behavior ModelingIoT Platform Solutions Affordable Wellness &amp; Healthcare Mobile Interactive Remote Sensing PROGRAMS </li><li> 5. 5 Click to edit Master title styleProgram: IoT Platform Solutions </li><li> 6. 6 Integrated Platform for Intelligent Enterprise People Feedback &amp; Emotions Social Media Integrated Services Sensors &amp; IoT Platform Legacy Monitoring &amp; Control Systems Enterprise Data Smart Integration Platform Transportation Human Resources Energy OperationsSafety Asset Tracking Smart Integrated Services Sense Analyze Extract Respond Intelligence Smart Domain Services Supply Chain Security and Surveillance Sense: People Context, Appliances, Building, Plant, Utility Infrastructure Sync Transportation with Remote Operations Link Asset Tracking and Safety with Surveillance Employee Wellness and Energy Preservation as Community Initiatives Intelligent Integration Platform Integrated Intelligent Services </li><li> 7. 7 Requirements and Challenges for IoT Need for a Platform Applications need support for Visibility Capture &amp; store data from sensors Insights Patterns, relationships and models Control Optimize and actuate TCUP TCS Connected Universe Platform A horizontal platform for addressing the IoT Software and Services market Model-driven Development Model the Domain Knowledge Model the Infrastructure Network, Storage, Compute Model the Analytics map to Domain Requirements Model the Architecture Device and Cloud TCUP Platform Model the Sensor Semantics, Phenomenon </li><li> 8. 8 TCUP Design and Architectural Highlights 18 patents filed, Standard Body Contribution - IETF and Singapore ITSC Fog Computing Utilize unused compute power of edge devices Distributed Computing on Edge Devices To reduce network congestion Adequate Security and Reliability Adaptive, Lightweight yet Secure Communication Protocols For economical scaling of sensor data store Efficient Compression Statistical and Information-theoretic measure to find out potential privacy- breaching content Sensitivity Measurement and Privacy Preservation Semantic annotation of sensors Sensor Search Engine Semantic Enabled Sensor Explorer Algopedia Algorithm Repository, Search and Recommendation Semantic Sensor Web Model-driven Development Manage Scale, Reduce Cost Handle Privacy Manage Diversity and Inter-op Ease of Development </li><li> 9. 9 Horizontal operators (semantic integration) operates on data from heterogeneous sources to created integrated data streams. Semantic Sensor Web - From Data to Wisdom temperature humidity odor image high temperature gaseous odor light concentrated light high temperature indicates fire gaseous odor indicates gas discharge Fire from Gas Leak, evacuate immediately, send fire fighting team equipped with gas leakage data information knowledge wisdom Vertical operators (semantic abstraction) operates on artifacts at each level and transcends them to the next level F PCS(Data, KB*) Information F PCS(Knowledge, KB) Wisdom F PCS(Information, KB) Knowledge KB: Knowledge base Adopted from: Physical-Cyber-Social Computing: An early 21st Century Approach, Amit Sheth et. al. </li><li> 10. 10 Research Outcomes Some of the Results Publications in ACM Sensys, Ubicomp, Infocomm, Middleware </li><li> 11. 11 Program: Human Behavior Modeling and Data Collection </li><li> 12. 12 Human Data Collection and Behaviour Modelling - Program Overview Research Goals - Given a context, predict behaviour. Given behaviour, find out context. Focus Domains Organization Behavior, Consumer Behavior Current Available Models Statistical, Need of the Day Models based on physical data Meetings Group or One-on-one Digital Communication Individual @Work or Leisure Individual Day in the Life Of Strength and Polarity of Relationships Meeting Flow Emotive State Outcome Formal / Informal, Business / Social Strength and Polarity of Relationships Tenor of Communication Nature of Communication Location and Time spent Mood and Physical State Engagement Level Cognitive Load Location and Time spent Activity Behavioral Routine Social Interaction Moment of decision making Microphone Email / Knome Mobile Phone, Desktop Mounted Cameras, EEG/GSR Mobile Phone, Kinect, EEG/GSR, Smart Meter Surveillance Camera </li><li> 13. 13 Human Identification and Activity Detection using Kinect Human Identification Skeleton Model Based / Depth based 20 joints of skeleton data 2D Camera with IR depth sensor Excitation by IR light pattern Human Identification Gait cycle detection Feature extraction from skeleton joints Training Recognition Papers in IEEE Fuzz, CEC, IEEE SMC, UbiComp and ECCV Activity Sitting Standing Walking Human activity recognition using RGB-D Accuracy is above 90% </li><li> 14. 14 Activity Detection using Mobile Phone Inertial Sensors Activity Detection Uses Accelerometer Data Gyroscope and Magnetometer for orientation correction Step Count, Stride Length Estimation Walking, Brisk Walking, Running, Sitting, Falling Classification Continuous Data Stream Windowed Data Zero Normalization Linear Interpolation Low Pass Filtration Frequency Spectrum Identifying non-activity window using frequency spectrum Peak Detection and Step Validation using IPA; calculating step cycle lengths for all valid steps in the window Classification of window activity using step frequencies derived from step cycle lengths Continuous Data Stream Windowed Data Zero Normalization Linear Interpolation Low Pass Filtration Frequency Spectrum Identifying non-activity window using frequency spectrum Peak Detection and Step Validation using IPA; calculating step cycle lengths for all valid steps in the window Classification of window activity using step frequencies derived from step cycle lengths Peak Detection and Step Validation using IPA; Calculating Step cycle lengths for all valid steps in the window Classification of window activity using step frequencies derived from step cycle lengths Noise Cancellation and pre-processing Calorie Count from Step Count and Type of Activity Papers in UbiComp ~90% Accuracy ~80% Accuracy </li><li> 15. 15 Geo-fencing Using Magentometer Proximity Detection Using Bluetooth RSSI Inertial Navigation Step Count + Stride Length (personalized model) Gyroscope and Magnetometer-corrected Inertial Navigation Wi-Fi based Zoning and Triangulation Based on RSSI of known location of 3 or more access points Attenuation modeling of the building Unsupervised Learning through physical modeling Fusion, Tracking and Correction Kalman Filter based Tracking Particle Filter based Correction Mobile Phone based Indoor Localization Inertial and Wi-Fi Colleague Finder in Large Offices Shopper Localization in Retail Stores Emergency Evacuation in Large Buildings Papers in Mobiquitous, UbiComp Selected for Indoor Localization Competition in IPSN 2014 </li><li> 16. 16 Cognitive Load on Human Brain Cognitive Load 23+45=? 1846890129 + 2374609823=? Use Cases Personalized education User interface design EEG GSR Papers in IEEE SMC, IEEE Fuzz, ACM BIBE </li><li> 17. 17 Emotion and Engagement level using Camera Why Camera Unobtrusive Sensing @Work Purpose Identifying the mood a person at work How a person is engaged at work Scope Facial emotion analysis on unconstrained environment using camera on desktop/laptop Mood and human engagement at work place using desktop camera Partial occlusion for facial expression recognition Engagement analysis Micro emotions / expressions http://www.ecse.rpi.edu/~cvrl/tongy/aurecognition.html Irfan Essa (1994), Analysis, interpretation and synthesis of facial expressions, PhD Thesis, MIT, Cambridge, MA, USA. (Advisor: Alex (Sandy) Pentland) </li><li> 18. 18 Program: Affordable Healthcare and Wellness </li><li> 19. 19 Affordable Healthcare using Mobile Phones Sensing various physiological parameters using smartphone sensors with minimal attachments Low cost solution for initial screening in preventive healthcare / wellness Solutions need to be stable, repeatable and robust New disease diagnostics and treatment protocols Requires long observation over large set of patients Geriatric care, monitoring chronic patients Go into Wearable in future Heart condition Heart rate, Heart rate variability, Blood Pressure, ECG Foetal Heart rate Lung condition Spirometry, Respiratory Rate Pupil condition Pupillary dilation response Pulse Diagnostics Purpose Scope Robust Solutions using camera, microphone and accelerometer </li><li> 20. 20 PoC Approach and Expected Novelty HR, HRV, RRPPG extractionRealtime Video Audio from Mic Accelerometer Cardiovascular Model SpO2 BP, ECG Spirometry Breathing Rate Image of eye Pupillary Reflex </li><li> 21. 21 Photo-plethysmography (PPG) using Mobile Phone Camera Subject1 Subject2 Subject3 Actual Detected Actual Detected Actual Detected 68 66 66 63 85 84 2.9% 4.5% 1.1% Papers at Mobihoc, IEEE BIBE, SenSys, ICASSP Data set Pd Ps PP-diff &lt; 15 Standard dataset (14 features) 92.9% 74.7% 77.9% TCS dataset - add height, weight, age 99.3% 82.7% 85.5% </li><li> 22. 22 Program: Mobile Interactive Remote Sensing </li><li> 23. 23 Sensing the Physical World Mobile phone based crowd sensing Robot assisted sensing www.popularmechanics.com www.engadget.com www.allthingssd.com apollo2.cs.illinois.ed u Camera based sensing </li><li> 24. 24 Intelligent Transportation Vehicle Model Driven Sensor Data Analysis KNOWN PARAMETERS EFFECTS TARGET INDUSTRY Vehicle Type &amp; Driving Behavior Road Condition Monitoring City Municipality Road Condition &amp; Driving Behavior Car Prognosis Automotive Road Condition &amp; Vehicle Type Driving Behavior Analysis Insurance Acceleration a(t) = f (H(t), v(t), R(t), D(t)) H(t) Papers in ICST, Percom </li><li> 25. 25 Phone Microphone based Sound Scaping Solution Overview Event driven with participatory sensing aided audio surveillance system Classification of Traffic Noise (Honk Detection) and Crowd Noise Papers in CODIS, ISSNIP, ISDA </li><li> 26. 26 Phone Camera based 3D Reconstruction from 2D images Input Images Dense Reconstruction without using mobile inertial sensors - 20 images, compute time (4 core, 1GPU) ~ 20 min - 120 images, compute time (16 core, 1GPU) ~ 30 min - Bandwidth saving ~ 8 times, if done on mobile Sparse Reconstruction using Mobile Inertial Sensors for Camera Position Estimation 20 images, compute time (4 core, 1GPU) ~ 3 min (without using inertial sensors) 20 images compute time (4 core, 1GPU) ~10 sec. (with inertial sensors) Bandwidth saving ~ 200 times, if done on mobile Sparse good enough for many applications Mobile Sensing and ACCV (submitted) Dense Reconstruction with mobile inertial sensors under progress with more number of images target &lt; 1min Dense Reconstruction -120 images Dense Reconstruction - 20 images Low cost solution for 3D reconstruction from multiple 2D images captured from mobile device. Motion information from the inbuilt inertial sensors for camera position estimation Applications in Agro-advisory service, Remote Diagnostics, Remote Healthcare </li><li> 27. 27 Multi-sensor Fusion for Robot-assisted Sensing Application in remote sensing in hazard-prone areas Robot carries 2D camera and heat / chemical sensors on a rotating arm 3D reconstruction from the 2D vision Estimation of Heat / gas leak / sound Source (direction and range) through passive directional signal processing Fusion of heat / gas / sound map on reconstructed 3D vision map www.ese.wustl.edu Ongoing Work Possible reuse from 2D-3D reconstruction and sound classification Cloud point from 3D vision Possible gas / heat source (ROI) Source direction and intensity </li><li> 28. 28 Innovation @TCS </li><li> 29. 29 TCS - Pioneering IT Innovation in India 1970s Offshore Model Alliances with Major IT Players University Alliances, Systems Engineering 1970s -1980s New Labs, Bio Suite Dhruvam Silicon Valley Ecosystem Mastercraft, Revine, Quartz Program, Tools Foundry, Rice Husk Ash 2000 - 2005 2007- 2011 IPR Focus and Policy Talent Management TCS COIN TM Software Tools - Casepac Migration Re-engineering Industry &amp; Services Practices Set Up Foundation of TRDDC- Indias first Industrial R&amp;D Center in IT, S/W Engg CTO/R&amp;D structure Innovation Framework More Domain Labs 2006 1981 1987 1991 1998 2012 4E Model Scaled Invention, Co-Innovation The journey continues </li><li> 30. 30 Innovation@TCS - Innovation Labs Bangalore, India1 TCS Innovation Labs - Bangalore Chennai, India2 TCS Innovation Labs - Chennai TCS Innovation Labs - Retail TCS Innovation Labs - Travel &amp; Hospitality TCS Innovation Labs - Insurance TCS Innovation Labs - Web 2.0 TCS Innovation Labs - Telecom Cincinnati, USA3 TCS Innovation Labs - Cincinnati Delhi, India4 TCS Innovation Labs - Delhi Hyderabad, India5 TCS Innovation Labs - Hyderabad TCS Innovation Labs - CMC Kolkata, India6 TCS Innovation Labs - Kolkata Mumbai, India7 TCS Innovation Labs - Mumbai TCS Innovation Labs - Performance Engineering Peterborough, UK8 TCS Innovation Labs - Peterborough Pune, India9 TCS Innovation Labs - TRDDC - Process Engineering TCS Innovation Labs - TRDDC - Software Engineering TCS Innovation Labs - TRDDC - Systems Research TCS Innovation Labs - Engineering &amp; Industrial Services 1 2 3 4 5 97 6 8 Associates in R&amp;D2000+ Innovation Labs19 Papers Published in last two years...</li></ol>