Conquering the Edge with Data Virtualization

Lakshmi Randall

Director of Product Marketing, Denodo

Twitter: @LakshmiLJ

2

Source: Gartner Blog

(http://blogs.gartner.com/thomas_bittman/2017/03/06/the-edge-will-eat-the-

cloud/)

The edge will eat the cloud. And this is

perhaps as important as the cloud

computing trend ever was.”

Which workloads will need an edge presence?The need for an edge presence will be defined & driven

by data requirements …

SpeedAvailability& reliabilityof access

Rate of generation or security

…or any combination

Source: 451 Research

3

Which deployment locations do you plan to use to store and analyze IoT data in 2017Source: 451 Research, Voice of the Enterprise: IoT Budgets and Outlook, 2017. n = 288

4

5

IoT Network Architecture

• IoT networks will contain multiple ‘layers’ of compute systems

• Each layer will persist data

• Some storage (e.g. at the sensor) might be temporary

• e.g. last hour of readings

• Not all data can be transmitted to next layer

• Regulatory, privacy, security reasons leaving data where it is

• Data moving to higher levels is typically aggregated for analytics

and monitoring

6

Layer 1 Layer 2 Layer 3 Layer 4

Layered Architecture

7

Data Virtualization: Five Essential Capabilities

4. Self-service data services

5. Centralized metadata, security & governance

1. Data abstraction

2. Zero replication, zero

relocation

3. Real-time information

8

• Data Virtualization provides data access at the edge layer and data center/cloud layer

• Connected DV platforms optimize data access by pushing processing to the data

• Data Virtualization can be deployed at any intermediary layers between ‘edge’ and data center/cloud layer

• e.g. Organizational, Regional, or Country based layers

Data Virtualization at the Edge

9

Connect rather than Collect ParadigmCase Study Example from Gartner

Single Logical View

10

DV Layer @ EdgeDV Layer @ Data

Center/Cloud

Layer 1 Layer 2 Layer 3 Layer 4

Layered Architecture

Business Need Solution Benefits

11

Leading Heavy Equipment Manufacturer Improves Service Delivery and Revenue

• Competitive pressure from low-cost Chinese manufacturers

• Needed a proactive approach to customer service to differentiate

• Sought to improve equipment and services delivery through predictive maintenance

• Phased rollout systematically improved asset performance and proactive maintenance

• Increased revenue from sale of services and parts

• Reduced warranty costs of parts failure

• Future – optimize pricing for services and parts among global service providers

• Telemetry (IoT) data from sensors embedded in the equipment is stored in Hadoop to perform predictive analytics

• Denodo integrates analytics data with parts, maintenance, and dealer information stored in traditional systems

• It then feeds the predictive maintenance information to a customer dashboard

In business for over 90 years and is the world’s leading manufacturer of construction and mining equipment, diesel and natural gas engines, industrial gas turbines and diesel-electric locomotives with 2014 net revenue of $54.6 billion.

Denodo @ Layer 4 (Data Center/Cloud)

12

Denodo @ Layer 4 (Data Center/Cloud)Telemetry Analytics

Business Need Solution Benefits

13

World’s Largest Independent Biotechnology Firm Successfully Increases Production Yield and Plant Efficiency

• Manufacturing data had to be aligned in real time to identify “weak signals” that could hinder production and cost millions

• Streaming data from manufacturing, laboratory facilities, etc. had to be analyzed for optimal plant efficiency

• “Golden Batch” had to be defined for increased production yield after combining current/historical data

• Improving yield by 1–3% has saved $10 –30 million per plant

• Concerns with operating a new plant at low capacity are eliminated

• ROI of Denodo easily justifies all work –will be used for all future applications

• Data Virtualization through Denodo is cheaper and more efficient than ETL

• Denodo Platform combines current manufacturing process data with historical data and feeds it to analytical systems

• The systems analyze virtualized data, determine “golden batch,” and take corrective steps to ensure ideal yield

• Denodo combines data from 46 source systems, of which 12 are key systems

Headquartered in Thousand Oaks, California, the company is the world’s largest independent biotechnology firm, and sells a number of products treating autoimmune diseases, rheumatoid arthritis, etc. In 2015, the company reported a revenue of $20 billion with its 18,000 employees.

Denodo @ Layer 3 (at the Edge)

14

Denodo Beyond the Edge

• Data Virtualization can also be leveraged beyond the edge – at

local data collection level

• e.g. factory or production line, retail store, ICU in hospital, etc.

• Collection point for real-time data streams from sensors, devices

• Stored in local databases, files, etc.

• For sensor management and control

• DV provides data access for analytics and reporting

• Lightweight deployment for lower capability systems

• e.g. Denodo in Docker Container

15

DV Layer @ EdgeDV Layer @ Data

Center/CloudLightweight DV Layer

Beyond Edge

Layer 1 Layer 2 Layer 3 Layer 4

Layered Architecture

Business Need Solution Benefits

16

Large US Telecom Company Improves Inventory Alignment with Virtualized Master Data Views

• Business users needed single view of networking device, customer, and site information from systems

• The single views were needed for ticket and event enrichment, compliance and audit, and inventory alignment

• The information was dispersed across multiple systems with varying levels of completeness and data quality

• Customer satisfaction: High quality MDM data helped end customer receive better quality information

• Inventory alignment: Improved information availability to internal and external applications

• Improved compliance: Better inventory management and analytics helps with better compliance and audit standards

• Denodo is used as a virtualized master data management (MDM) system

• Denodo connects to multiple systems and Hadoop

• Denodo aggregates and exposes the master data views via an integrated API layer

• Data is consumed by ticket and event enrichment, dashboard, and compliance / audit applications

The company is the second largest cable company in the U.S. by revenue, operating in 29 states, with corporate headquarters located in Midtown Manhattan, New York City. The company recorded revenue of US$23.69 billion in 2015. It’s Network Operations group is a thriving business providing the entire range of telecommunications services to both individual and commercial consumers.

Denodo Beyond the Edge

17

Denodo and Streaming Data

• Integrate with streaming data

systems e.g. Spark, Kafka, IBM

Streams, etc.

• Read data from queues

or streams (e.g. Spark

data frames) during

ingestion process

• Also enrich streaming

data through Denodo

Platform

• Access temporal data in specific

time window

• e.g. last 2 hours of

readings/data

Read temp window buffers

SQL basedenrichment

Secure +

Combine +

Enrich

18

Complete enterprise information, combining Web, cloud, streaming, and structured data

ROI realization within 6 months, with the flexibility to adjust to unforeseen changes

An 80% reduction in integration costs, in terms of resources and technology

Real-time integration and data access, enabling faster business decisions

“Get it Real-time and Get it Fast!”

The Benefits of Data Virtualization

Thank you!

© Copyright Denodo Technologies. All rights reservedUnless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.

#DenodoDataFest