Five In-depth Technology and Architecture Sessions on Data Virtualization

Session 1: Denodo Platform – Architecture

Today’s Speaker

■ Paul Moxon

Senior Director, Product Management

Architect-to-Architect Series

■ Series of five webinars over next 2 months

■ Deeper look into Denodo Platform

■ Architectural Overview (today’s sessions)

■ Performance

■ Scalability

■ Data Discovery and Governance

■ Security

Denodo Express

■ Denodo Express

■ Free to Download

■ Fully functioning Data Virtualization Platform

■ Single user, supports common data sources

■ Many of the same capabilities of Denodo Platform

■ Performance, Data Discovery, Governance, internal Security, Publishing, …

• Intuitive graphical design tool

• Drag-and-drop, point-and-click modeling and integration

• Connectivity to all types of data sources

• Native support for hierarchical data structures via ERM

• Extensive on-the-fly data transformation & data cleansing and matching functions

• Multiple ‘publishing’ protocols

• Metadata exploration

• Data lineage, dependency analysis, change impact analysis, etc.

• Global Search

• Search data and metadata across all data sources

• Team/distributed development models

• Version Control System integration …

Denodo Platform – Design time

• Proven scalability & performance

• Scales from departmental/project to enterprise-wide with geographically distributed nodes

• Handles largest transactional (users, transactions, or sources) and analytical loads (data volumes)

• Advanced query optimizer

• Cost-based & source constraint-based query optimization incl. query delegation, query rewrite, join optimization, data movement, …

• Intelligent caching

• Full, partial, time-based caching

• Fine-grained security model

• Integrate with external systems

• Enterprise–grade monitoring and auditing

• Integration with System Management products e.g. HP BTO, Unicenter, Tivoli, etc.

• Full development & deployment lifecycle support

Denodo Platform - Runtime

Denodo Platform – Detailed Architecture

Performance – Architecture Modules

Performance Optimization

■ Advanced Query Optimization

■ Cost and Source Constraint Based Query Plans

■ Query Delegation

■ Automatic Query Rewriting

■ Join Optimizations

■ Data Movement

■ Asynchronous Multi-threaded Processing

■ Server Throttling Mechanisms

■ Scalability

■ Caching

■ Multiple configuration modes – full or partial

Performance – Optimizer, etc.

■ Optimizer

■ The Optimizer applies state-of-the-art optimization techniques to relational and non-relational sources.

■ Query Plan Generator

■ The Plan Generator is in charge of generating possible execution plans for the query and selecting the optimum one.

■ Execution Engine

■ Responsible for executing the selected query plan, executing the necessary sub-queries on the sources (or collecting data from cache as appropriate) and integrating the results to generate the global response.

Scalability – Architecture Modules

Scalability – HA and Clustering

■ High Availability Architecture

■ Active-Passive

■ Active-Active

■ Vertical Scalability

■ VM Support

■ Metadata Sharing

■ Horizontal Scalability

■ Clustering

■ Shared Cache or Individual Cache

Scalability – HA and Clustering

■ Denodo Cache

■ The Denodo Platform incorporates an advanced cache system to improve performance

■ For data materialization it makes use of an external JDBC database (physical or in-memory)

■ Denodo High-Availability

■ Set of features provided by the Denodo Platform to support High Availability and Load Balancing

■ Denodo Cluster Management

■ Tool for monitoring and managing a cluster of Denodo servers

Data Discovery and Governance – Architecture Modules

Data Discovery and Governance

■ Metadata Repository

■ Apache Derby database

■ Contains view and data source configuration data

■ Relationships between views – dependencies, etc.

■ Searchable – Catalog searches, etc.

■ Data Lineage

■ Trace how data changes between source and consumer

■ Change Impact Analysis

■ What is the impact of a change in a data source schema?

Data Discovery and Governance

■ RESTful Web Services

■ Supports Global Search functionality

■ Index and search data sources

■ ‘Google’ like search

Security – Architecture Modules

Security

■ Authentication & Authorization

■ Built-in User/Role Management Module

■ Integration with external entitlement servers (LDAP/AD)

■ Multi-level access controls

■ Database, View, Row, Column, and Cell

■ Policy-based Security and Workload Management

■ Enforcement of custom policies for query execution according to security / workload considerations

Extensibility – Architecture Modules

Extensibility

■ Stored Procedures

■ Extension point for custom behavior (based on a Java SDK)

■ Eclipse Plugin for ease-of-development

■ Custom Functions

■ Extension point for custom transformation and data quality functions

■ Custom Wrappers

■ Extension point for custom data sources

■ Denodo API

■ Integration point for external tools and utilities

Q&A

Data Virtualization – Next Steps

Move forward at your own pace

Download Denodo Express –

The fastest way to Data Virtualization

Denodo Community: Documents, Videos, Tutorials, more.

Attend Architect-to-Architect Series

Performance

Scalability

Move forward with one of our Data Virtualization experts

Phone: (+1) 877-556-2531 (NA)

Phone: (+44) (0)20 7869 8053 (EMEA)

Email: info@denodo.com | www.denodo.com

Data Discovery and Governance

Security

Five In-depth Technology and Architecture Sessions

on Data Virtualization

Thank You!

Next Session

Session 2 Denodo Platform: Performance