ieg 201602 share_sachin mittal_big data lake

1 Copyright © 2015 HCL Technologies Limited | www.hcltech.com

Big Data Lake

Proposition OverviewSachin Mittal

DirectorHCL

Copyright © 2015 HCL Technologies Limited | www.hcltech.com

HCL Big Data Lake Proposition Overview

BIG DATA LAKE PROPOSITION OVERVIEW


BUSINESS CONTEXT

Business are seeking greater access to data and flexibility to drive innovations

Acquire raw data ID business use-cases Promote agile adoption Prove ROI / Lower TCO Improve performance Explore any data Establish Foundation

services Business use-case

driven Correlated data Integrate analytics ID new services

No historical trendOperational reporting No business intelligence

or Analytics

Limited historical trend Integrated operational

reporting No business intelligence

or Analytics

Complex & Expensive Less Agile Performance challenges Limited historical trendOperational reporting Business intelligence Analytics silos

Complex & Expensive Less Agile High TCO Performance challenges No data exploration Insights based on EDW Limited historical trendOperational reporting Business intelligence Analytics silos

LEVEL

INVESTMENTS

CHALLENGE

EDW +Big Data Lake

+ Analytics

Data repository,a copy of

production data

Operational Data Store

ODS + EDW on conventional

database + Analytics

ODS+ EDW Appliances +

Data marts + Analytics

DataRepository ODS

ODS+EDW+

Analytics

ODS+EDW+

DM Analytics

Big DataLake

Top-down: Business use-case drivenBottom-up: TCO drivenOptimize performance and reduce cost


BIG DATA LAKE

Use a Big Data Lake, businesses have access timely access to any data enabling deeper and more holistic analysis of all the enterprise data assets. The Big Data Lake co-exist with traditional EDW and ETL platforms expanding business case around increasing functionality

WHAT IS IT?

Enable business to innovate and operate at the speed of digital commerce with rapid access to the any data and right analytics AGILITY – acquire, process and analyze any data, any time, and complexity internal and external to the enterprise FLEXIBILITY - correlate, combine, and consolidate all the data for rapid delivery ACCESS – publish data to end-user reporting and analytics applications Time-to-market – provide the insights to necessary to grow market share, improve operations, and customer satisfaction

WHAT IS THE BUSINESS NEED?

Agile Big Data & Analytics methodology, DIF, meta-M, Data Processing Framework, Data Distribution Framework, SEAR, ROAR,

HCL ASSETS

Organization today must have tools and technique to gather any data and reduce the time-to-insights for decisions makers, customers, suppliers, and employees. HCL’s approach is to enable a unified platform integrating new digitally focused business processes to facilitate innovation providing our customer with competitive advantages: Business use cases include: price optimization, digital data enablement, predictive analytics, compliance reporting, process offloading and platform modernization.

HCL POINT OF VIEW (POV)

Big DataLake


BIG DATA OPPORTUNITIES: MARKET DRIVERS

Migrate from EDW to Big data Lakes by off-loading: Old data being stored in

expensive EDW platforms

ETL processes

Some mainframe processes

Storage of cold data

Provide business access to any data, any time Reduce time-to-insights

Publish data sets specific to analytical needs

Increase value of data delivered for advanced analytical analysis

Unified architectures Co-exist with existing enterprise

data warehouse and ETL platforms

Connect to existing BI and analytics reporting environments

Business use-case driven implementation delivers value quickly Use-case drive building foundational

services

Real-time offer generation

Recommendation engines

Analytics sandbox for ad hoc analysis

Data services

Identify risk across divers data sources

Enable predictive analytics

Top-down: Business use-case drivenBottom-up: TCO drivenOptimize performance and reduce cost


HCL’S BIG DATA REFERENCE ARCHITECTURE

DATA LANDING AND HOSTING

DATA PROCESSING

Data quality routine processing

Rules processing

Function processing

Master/reference data referencing

DATA PUBLISH

Search data Data access APIs

Advanced analytics

Data visualization

Data security

EnterpriseStructured

data

Unstructured Data

LogsFiles

Web data

DATA SOURCES

Error Handling Logging Monitoring

OPERATIONAL INFORMATION

Metadata

Master data

Reference data

Rules

Indexes

Workflows

DATA CAPTURE METAINFORMATION

Data storage Data compression

Data security Data versioning

Subject area

Op. Dashboard

Fast data processing

Streaming data

ingestion

Batch data ingestion

Machine Data

Data quality routines

Functions

Security

HCL IP DaaS

HCL IP Meta-m

HCL IP UDF

HCL IP DIF


HCL POINT OF VIEWBIG DATA LAKE TRANSITIONS WITHIN AN ORGANIZATION

Business Use-case driven migration strategyTraditional BI & Analytics dashboard & reportsPersonalized / role-based & self-service BI and analyticsReal-time analytics feeds to any device “As-a-Service” delivery: BI, Analytics, Data

Business DriversProvide insights not currently available within the EDWConnect with customers across all channelsTime-to-Market with innovation and insightsApply analytics to improve top line & bottom line

EDW OptimizationInfrastructure & SchemasOracleTeradataDB2MS SQLVertica

Traditional EDW

NoSQL DB

Graph DB

Business value realization

Service OrientedArchitecture

BIG Data LakeClients are using Hadoop Distribution such as:HortonWorks, ClouderaBig Insights, MapR, Pivotal

Publish data sets for analytics and reporting

Foundational ServicesAcquire Process Analyze Publish

BI Pathways Business & IT alignmentStrategic objectivesTactical execution

Incremental & Agile Deployment

Spatial

IT DriversReduce cost of traditional EDWAcquire new data sources Improve access to dataProvide new data and analytics services

Business Driven Imperatives


DRIVERS: ACROSS INDUSTRY VERTICALS

COST RISK REGULATORYCOMPLIANCE

INNOVATION CUSTOMEREXPERIENCE

ADVANCEDANALYTICS

Increasing transaction volume from multiple sources leading to high storage and maintenance costs

Long-lead-time-to-insightsLack of insights to address the risk -market risk, credit risk, supply-chain, attrition, and customer risk

Complex Global RegulationsFATCA/GATCA reporting for regulatory compliance

Creating competitive advantagesDigital banking initiatives like Communication surveillance (identify and raise for review, trading anomalies), fraud and log analysis

Improving the customer experience Know the voice of the customer and identify social networks to incorporate sentiment, preferences, and opinion into ideation and product research

Provide superior data for analyticsReduce downtime by predicting failure, optimize maintenance plans, placements of sparsEnable business users to use data science insight analysis


OUR IMPLEMENTATIONS - DRIVING BUSINESS BENEFITS

DATA TRANSFORMATION & PROCESS OFFLOADING

LEADING AUSTRALIAN BASED FINANCIAL SERVICES ORGANISATION

Offload processes dependent on legacy systems for cost savings & better scalability.

Simplify operations to reduce cost of Mainframe application (including cost of mainframe MIPS and mainframe storage)

Improved time to market – business was able to launch products sooner

TCO reduction & improved processes

BUSINESS BENEFITS

DRIVE OPERATIONAL EFFICIENCIES ACROSS VALUE CHAIN

GLOBAL INTEGRATED LOGISTICS PROVIDER

Single view of cargo has been challenge given the organization has undergone various acquisitions and there are disparate data sources that monitor siloed view of cargo.

Optimization of value chain leading to better view of cargo and ability to predict capacity and visibility.

RISK OFFLOADING

LARGEST INVESTMENT BANK BASED OUT OF GERMANY:

Changes to Risk Rules was taking close to 4-6 months Simulation was limited to a sample data set Very manual intensive process leading to human errors

Improved time to market of implementing Risk rule changes Ability to test with large data set and get more accurate results

DATA LAKE IMPLEMENTATION FOR CUSTOMER INSIGHTS

US BASED NATIONAL PROVIDER OF WIRELESS & DATA SERVICES

Customer insights from social media, web analytics, unstructured and semi structured data.

Correlating the data coming from multiple sources (silos) and building NexGen IDW

Analytics and intelligence on the go by providing real time data for customer intelligence

Improved efficiencies by 36% in Retail Operation Center for Services, a system that reports on customer transactions from multiple dimensions

BUSINESS BENEFITS

BUSINESS BENEFITS

BUSINESS BENEFITS


THE HCL ADVANTAGE – LEAD, INNOVATE, DEPLOY & MANAGE

Global experience• Multiple Big Data implementations and management initiatives• Technology and business driven approaches• Trusted advisor helping clients establish scalable foundation services• Comprehensive view of the Big Data life cycle and team requirements

Scalable resources• HCL has a global team experience in Big Data implementation• Competencies supporting the full lifecycle;

o Program/Project Definition & Managemento Business Analyst, Data Scientist, Big Data Analysto Hadoop specialists, Visualization specialists

Big Data assets• Agile Big Data and Analytics methodology• Innovation labs around the globe• Technical capabilities to extend Hadoop core capabilities• Accelerators and implementation frameworks

Data ingestion and management Metadata, Security, Data Quality, Data Publication

Strong Partner ecosystem• IBM, Oracle, HP, SAP• HortonWorks, Cloudera, MapR, Pivotal• Talend, Tableau, Qlik View

Top-down: Business use-case drivenBusiness Value Realization

Bottom-up: TCO drivenOptimize performance and reduce cost

Flexible approach for your organization

Rapid-Start – Agile delivery producing results quickly

Ideate & PanBig Data Lake

Apply Analytics / Visualizations

Deliver results

Plan next increment – scale & enhance


Thank You.


Community Focused

Volunteer Driven

Knowledge Share

Accelerated Learning

Collective Excellence

Distilled Knowledge

Shared, Non Conflicting Goals

Validation / Brainstorm platform

Mentor, Guide, Coach

Satisfied, Empowered Professional

Richer Industry and Academia

ABOUT INFORMATION EXCELLENCE GROUP

Progress Information Excellence

Towards an Enriched Profession, Business and Society


ABOUT INFORMATION EXCELLENCE GROUP

Reach us at:

blog: http://informationexcellence.wordpress.com/

linked in: http://www.linkedin.com/groups/Information-Excellence-3893869

Facebook: http://www.facebook.com/pages/Information-excellence-group/171892096247159

presentations: http://www.slideshare.net/informationexcellence

twitter: #infoexcel

email: [email protected]@gmail.com

Have you enriched yourself by contributing to the community Knowledge Share..

Copyright © 2015 HCL Technologies Limited | www.hcltech.com

Progress Information Excellence Towards an Enriched Profession, Business and Society

Progress Information Excellence Towards an Enriched Profession, Business and Society