netezza pure data

IBM PureData System for Analytics

Powered by Netezza

Hossein Sarshar

Agenda• What is PureData and Netezza

o History

o Characteristics

o Product chain

• PureData Hardware Architectureo Introduction

o Hardware architecture

o Paralleled structures

• Analytics with PureDatao Introduction

o In-database analytics tools

• Demo

IBM® PureData™ for Analytics 2

What is PureData and Netezza

PureSystems

PureFlex PureApplication


In 2010, IBM bought a new analytics platform called

Netezza. It was founded in 2000 at Marlborough, CA.

IBM later rebranded it to PureData.

What is PureData and Netezza

PureSystems

PureFlex PureApplication PureData


PureSystems Product Family

PureFlex: o Combines and optimizes compute, storage, networking and virtualization

capabilities under a single, unified management console into an

infrastructure system.

PureApplication:o Is a platform system designed and tuned specifically for transactional web

and database applications.

PureData:o Based on Netezza technology, PureData is all data experts need in a

single well tuned appliance.


PureData

Operational Analytics

Transactions Analytics

PureSystemsCharacteristics

• Built-in Expertso No indexing/tuning/partitioning

o Fully parallel, optimized in-Database Analytics.

o No storage administration.

o No software installation.

• Integration by Design:o Server, Storage, Database in one easy to use package.

o Automatic parallelization and resource optimization to scale economically

o Enterprise-class security and platform management

• Simplified Experience:o Up and running in hours.

o Minimal ongoing administration.

o Standard interfaces to best of breed Analytics, BI, and data integration tools.

o Built-in analytics capabilities allow users to derive insight from data quickly.

o Easy connectivity to other Big Data Platform components


Each of these come as an appliance equal to simplified yet strong private clouds with

minimal administration

PureData Introduction• It is a datawarehousing and data analytics

appliance that is fast enough to process terabytes of data in seconds. It is a fully parallel machine.

• Netezza’s main technology is using FPGA (Field Programmable Gateway Array) to filter unnecessary files in parallel manner.

• PureData uses Netezza technology to perform deep analytics on huge amount of data in a reasonable time.

• It is purpose-built for high performance analytics.

• It supports all DB structures (3NF, Star, De-Normalized table)


PureData Architecture


Disk storage

RAID 1 disksHigh speed data

streams

SMP Host

Redhat linuxservers

OptimizerCompiler

A gateway to the system

Snippet-Blades

Query accelerator using FPGAs

S-Blades (SPU)


S-Blades


Intel Quad-Core

Dual-Core FPGADRAM

IBM BladeCenter Server Netezza DB Accelerator

SAS Expander

Module

SAS Expander

Module

S-Blades Overview• There are 8 intel core on IBM Blade-Center Server

and 8 FPGA on Netezza DB accelerator.o FPGA has similar dimensions a CPU has, consumes 5 times less power and

clock speed is about 5 times less

o More caching capability

o Low latency and high throughput

• Each of these S-Blades takes ownership of 6-8 disks.

• The queries are divided into subqueries that are

processed by S-Blades.


PureData AMPP (Shared-Nothing) Architecture

12

Advanced Analytics

Loader

ETL

BI

Applications

FPGA

Memory

CPU

FPGA

Memory

CPU

FPGA

Memory

CPU

Hosts

SMP

Host

Disk

EnclosuresS-Blades™

Network

Fabric

Netezza Appliance

FPGA Secret Sauce


FPGA Core CPU Core

Uncompress Project Restrict,

Visibility

Complex ∑

Group by, …

select DISTRICT,

PRODUCTGRP,

sum(NRX)

from MTHLY_RX_TERR_DATA

where MONTH = '20091201'

and MARKET = 509123

and SPECIALTY = 'GASTRO'

Slice of table

MTHLY_RX_TERR_DATA

(compressed)

where MONTH = '20091201'

and MARKET = 509123

and SPECIALTY = 'GASTRO'

sum(NRX)

select DISTRICT,

PRODUCTGRP,

sum(NRX)

Using FPGA reduces a tremendous among of

unnecessary data movement

PureData System Configuration

14IBM® PureData™ for Analytics



Single Rack System Multi Rack System

Specs N3001-

002

N3001-

005

N3001-

010

N3001-

020

N3001-

040

N3001-

080

Racks 1 1 1 2 4 8

Active S-Blades 2 4 7 14 28 56

CPU Cores 40 80 140 280 560 1120

FPGA Cores 32 64 112 224 448 896

User Data in TB 32 98 192 384 768 1536

N3001 is the newest IBM PureData

What is Achievable• Having agile analytics platform.

• No administration effort to install/manage

• Scalability in petabyte level

• Linear speedup scalability by adding additional

racks.

• Big Data Meets Deep Analytics => No need to

sample


High Performance Analytics Architecture


PureData Analytics Modules


Netezza In-Database Analytics Options

Classification Time Series Clustering

Associate Rules

Simulation and Monte

Carlo AnalysisGeospatial


Demo• Installation

• Client Tool Exploration

• Command Execution


Summary• A system for analytics

• Out-of-the-box solution

• It uses FPGA technology to boost query execution

• It uses nothing-shared approach.

• PureData uses open standards to communicate to

outside world

• It has many NZ in-database and 3rd party in-

database options to enrich our analytics


References• http://www-01.ibm.com/software/data/netezza/

• http://www.ibm.com/ibm/puresystems/ca/en/


http://www-01.ibm.com/software/data/netezza/

http://www.ibm.com/ibm/puresystems/ca/en/

netezza pure data

Data & Analytics

netezzaibm puredata

puredata introductionit

data analytics appliance

new analytics platform

data experts

platform system

deep analytics

terabytes of data