doag 2014 konrad-haefeli_in-memory-option-on-exadata

2014 © Trivadis

BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA

2014 © Trivadis

Exadata with In-Memory Option – the

best of all ?!?

Konrad HÄFELI

Senior Solution Manager

Infrastructure Engineering

DOAG-Konferenz 2014

In Memory Option on Exadata - the best of all?!?

1

2014 © Trivadis

Trivadis ist führend bei der IT-Beratung, der Systemintegration,

dem Solution-Engineering und der Erbringung von IT-Services

mit Fokussierung auf und Technologien

im D-A-CH-Raum.

Unsere Leistungen erbringen wir aus den strategischen Geschäftsfeldern:

Trivadis Services übernimmt den korrespondierenden Betrieb

Ihrer IT Systeme.

Unser Unternehmen

DOAG-Konferenz 2014


B E T R I E B

2

2014 © Trivadis

Mit über 600 IT- und Fachexperten bei Ihnen vor Ort

3

12 Trivadis Niederlassungen mit

über 600 Mitarbeitenden

200 Service Level Agreements

Mehr als 4'000 Trainingsteilnehmer

Forschungs- und Entwicklungs-

budget: CHF 5.0 Mio. / EUR 4.0

Mio.

Finanziell unabhängig und

nachhaltig profitabel

Erfahrung aus mehr als 1'900

Projekten pro Jahr bei über 800

Kunden

Stand 12/2013

Hamburg

Düsseldorf

Frankfurt

FreiburgMünchen

Wien

Basel

ZürichBern

Lausanne

3

Stuttgart

DOAG-Konferenz 2014


3

Brugg

2014 © Trivadis


Trivadis an der DOAG

Ebene 3 - gleich neben der Rolltreppe

Wir freuen uns auf Ihren Besuch.

Denn mit Trivadis gewinnen Sie immer.

DOAG-Konferenz 2014

4

2014 © Trivadis

A year ago DOAG 2013…

DOAG-Konferenz 2014


5

2014 © Trivadis

Chose another topic… Flash Cache

DOAG-Konferenz 2014


6

2014 © Trivadis

… finaly in July 2014

Previews…

Betas Versions

Announcments: 10th of June

Releasing 22nd of July

12.1.0.2 full release

DOAG-Konferenz 2014


7

2014 © Trivadis

It’s an option, is it worth it?

On a normal Database Server?

Hopefully, if requirements accordingly…

But what about Exadata Database Machine?

Are the «expensive» Storage Server Feature

Smart Scan

Flash Cache

a benefit for IMCS (In Memory Column Store) ?

Resp. vice versa?

DOAG-Konferenz 2014


8

2014 © Trivadis

AGENDA

1. Exadata Feature

2. In-Memory Overview

3. In-Memory Configuration

4. In-Memory Management

5. In-Memory with Exadata Tests

6. Conclusion

DOAG-Konferenz 2014


9

2014 © Trivadis

DOAG-Konferenz 2014


Exadata Feature

10

2014 © Trivadis

Architecture Features Overview

11

Source: Oracle Corporation

DOAG-Konferenz 2014


Smart Scan Capabilities

Predicate Filtering

Column Filtering

Join Processing

Scans on encrypted Data

Scans on compressed Data

Create Tablespace –

Storage Server formats

blocks

Backup and Recovery – I/O

for incremental Backup is

much more efficient, only

changed Blocks are

returned to Server

2014 © Trivadis

Exadata Hybrid Columnar Compression (HCC)

12

A compression unit is a logical structure spanning multiple DB blocks

Each row is self-contained within a compression unit

Data is organized by column during data load

Each column is compressed separately

Smart Scan is supported

DOAG-Konferenz 2014


2014 © Trivadis

DOAG-Konferenz 2014


In-Memory Overview

13

2014 © Trivadis

DOAG-Konferenz 2014


14

Columnar Storage: Example for Columnorientation

2014 © Trivadis

In-Memory Option – What is it ?

Oracle Databases traditionally stores Data in Row Format

Oracle Database In-Memory Option uses an In-Memory Column Store

A new memory component in the SGA called In-Memory Area

Data are stored in column format in the In-Memory Column Store (IMCS)

Both Memory Structure (Buffer Cache and IMCS) will be populated

Dual Format Architecture does not double the Memory requirement

The Optimizer is fully aware about the In-Memory Column Store

DOAG-Konferenz 2014


15

Source: Oracle

2014 © Trivadis

In-Memory Option – Database Tiering

With the In-Memory Area not all data must fit in Memory

Less performance sensitive can reside on lower cost Disk

IM Option acts transparently against the applications

On Exadata acts on the top of all existing Storage tiers

It is also fully compatible with the new Multitenant architecture

DOAG-Konferenz 2014


16

Source: Oracle

2014 © Trivadis

In-Memory Option – SIMD

In-Memory Column Store are scanned with SIMD Vector Processing

SIMD = Single Instruction Multiple Data

SIMD Processing are evaluated in a single CPU Instruction

IM Column Store are specifically designed to maximize the number of

column entries that can be loaded into Vector Registers

Theoretically Scan of billions of rows per

second are possible

DOAG-Konferenz 2014


17

2014 © Trivadis

Summary: Oracle Database In-Memory

Extreme Performance: Analytics & OLTP

Extreme Scale-Out & Scale-Up

Extreme Availability

Extreme Simplicity

Powering the Real-Time Enterprise

All In-Memory Benefits With No Application Changes


DOAG-Konferenz 2014

18

2014 © Trivadis

DOAG-Konferenz 2014


In-Memory Configuration

19

2014 © Trivadis

In-Memory Option – The way to use it…

DOAG-Konferenz 2014


20

On Exadata there are some prerequisites…

Storage Server

Min. 12.1.1.1.0*

RDBMS

12.1.0.2

Database Server

Min. 12.1.1.1.0

Grid Infrastructure

12.1.0.2

* Full Offload Support for 12.1.x RDBMS

2014 © Trivadis

In-Memory Option – Configuration

Configuration is very simple

The Initializations Parameters INMEMORY_SIZE controls the In-

Memory Area

DOAG-Konferenz 2014


21

SQL> alter system set inmemory_size=16G scope=spfile sid=‘*’;

SQL> select * from v$sga;

NAME VALUE CON_ID

-------------------- --------------- ----------

Fixed Size 5304248 0

Variable Size 11274289224 0

Database Buffers 108716359680 0

Redo Buffers 263131136 0

In-Memory Area 17179869184 0

2014 © Trivadis


DOAG-Konferenz 2014


22

Initialization Parameter Comment

inmemory_clause_defaultenables you to specify a default In-Memory

Column Store (IM column store) clause for new

tables and materialized views

inmemory_force

allows you to specify whether tables and

materialized view that are specified as

INMEMORY are populated into the In-Memory

Column Store or not

inmemory_max_populate_serversspecifies the maximum number of background

populaserverste

inmemory_queryINMEMORY_QUERY is used to enable or

disable in-memory queries for the entire

database at the session or system level

inmemory_sizeINMEMORY_SIZE sets the size of the In-

Memory Column Store (IM column store) on a

database instance

inmemory_trickle_repopulate_ser

vers_percent

limits the maximum number of background

populate servers used for In-Memory Column

Store (IM column store) repopulation

optimizer_inmemory_awareenables or disables all of the optimizer cost

model enhancements for in-memory

2014 © Trivadis


Tables or Tablespace must be enabled to use IM Columnn Store

Depending on configuration the IMCS will be populated

Activation at Tablespace and Table Level

At Table Level configuration for compression and priority are possible

DOAG-Konferenz 2014


23

2014 © Trivadis


Compression Types

Priority Levels

DOAG-Konferenz 2014


24

Priority Level Description

Critical Objects are populated immediately

High Objects are populated after all critical objects are populated, if remains space

MediumObjects are populated after all critical and high objects are populated, if remains

space

LowObjects are populated after all critical, high and medium objects are populated, if

remains space

None Objects are populated only after they are scanned

Compression Type Description

No Compress No Compression

Memcompress for DML Compression optimized for DML

Memcompress for Query Low Compression optimized for Query (default)

Memcompress for Query High Compression optimized for Query and Space saving

Memcompress for Capacity Low Compression optimized with orientation towards Space saving

Memcompress for Capacity High Compression optimized for Space saving

2014 © Trivadis


Activation at Tablespace Level

At Table Level configuration for compression and priority are possible

memcompress for query low and priority none are the default

settings

Table activation with compression and priority settings

DOAG-Konferenz 2014


25

SQL> alter tablespace SOE inmemory;

SQL> alter table orders inmemory;

SQL> alter table orders inmemory memcompress for query low priority

critical;

2014 © Trivadis

DOAG-Konferenz 2014


In-Memory Management

26

2014 © Trivadis

In-Memory Option – Licensing

License is need for using the In-Memory Option

~ 18000 € / per Processor

The Option is installed with the RDBMS Installation, however not

activated

Bug 19308780 – Wrong Feature Tracking, fixed in PSU October 2014

Very “lovely” discussion between Maria Colgan (Oracle IM Option Product

Manager) and Kevin Closson (“independent” Oracle/Exadata specialist)

DOAG-Konferenz 2014


27

2014 © Trivadis

In-Memory Option – Management

Few v$ Views are in place for In-Memory Column Store management

Not all v$ Views are documented (only red ones)

New IM Memory Option PL/SQL Package

DOAG-Konferenz 2014


28

SQL> desc dbms_inmemory

..

PROCEDURE DISABLEFASTSTART

PROCEDURE ENABLEFASTSTART

PROCEDURE POPULATE

PROCEDURE REPOPULATE

SQL> select table_name from dict where table_name like 'V$IM_%';

TABLE_NAME

------------------------------

V$IM_COLUMN_LEVEL

V$IM_COL_CU

V$IM_HEADER

V$IM_SEGMENTS

V$IM_SEGMENTS_DETAIL

V$IM_SEG_EXT_MAP

V$IM_SMU_CHUNK

V$IM_SMU_HEAD

V$IM_TBS_EXT_MAP

V$IM_USER_SEGMENTS Only V$ Views in red are documented

Only Procedure in red are documented

2014 © Trivadis

In-Memory Option – Management

New dba_/all_/user_ views attribute are been implemented

Monitoring of In-Memory Segment

DOAG-Konferenz 2014


29

SQL> desc user_tables

Name Null? Type

----------------------------------------- -------- ----------------------------

TABLE_NAME NOT NULL VARCHAR2(128)

.

INMEMORY VARCHAR2(8)

INMEMORY_PRIORITY VARCHAR2(8)

INMEMORY_DISTRIBUTE VARCHAR2(15)

INMEMORY_COMPRESSION VARCHAR2(17)

INMEMORY_DUPLICATE VARCHAR2(13

SQL> select owner, segment_name, segment_type, inmemory_size, bytes_not_populated,

populate_status from v$im_segments;

OWNER SEGMENT_NAME SEGMENT_TYPE INMEMORY_SIZE BYTES_NOT_POPULATED POPULATE_

---------- -------------- --------------- ------------- ------------------- ---------

EXA IM2 TABLE 1179648 0 COMPLETED

EXA IM4 TABLE 8519680 0 COMPLETED

2014 © Trivadis

DOAG-Konferenz 2014


In-Memory with Exadata Tests

30

2014 © Trivadis

147 Mio Rows, 10GB

IM is factor 10 faster then jus Buffer cache

First population of IM data factor 4 faster with offloading

80% offload efficiency

Dedicated Query

DOAG-Konferenz 2014


31

SQL> SELECT SUM(amount_sold) FROM sales;

Cell

offload_processing

FALSE TRUE CACHE FALSE TRUE

INMEMORY_QUERY DISABLE DISABLE DISABLE ENABLE ENABLE

Elapsed time 56.82 13.51 1.17 0.13 013

Factor 1 4 48 437 437

2014 © Trivadis

Dedicated Query

DOAG-Konferenz 2014


32

SQL> SELECT t.time_id,

TO_CHAR(SUM(amount_sold), '9,999,999,999') AS sales,

TO_CHAR(AVG(SUM(amount_sold))

over(ORDER BY t.time_id range BETWEEN INTERVAL '2' DAY

preceding AND INTERVAL '2' DAY following), '9,999,999,999')

AS centered_5_day_avg

FROM sales s, times t

WHERE s.time_id = t.time_id

GROUP BY t.time_id

ORDER BY t.time_id;

Cell

offload_processing

FALSE TRUE CACHE FALSE TRUE

INMEMORY_QUERY DISABLE DISABLE DISABLE ENABLE ENABLE

Elapsed time 01:30.14 29.49 05.95 05.49 05.48

Factor 1 3 15 16 16

2014 © Trivadis

In-Memory Option – Exadata Test Enviroment

Eighth Rack (2 DB Server – 256 Gb RAM, 3 Storage Server)

Swingbench (charbench) used as Benchmarking Tool (2 Min)

OLTP Profile with 100 concurrent Users

- 60/40% Read/Write ratio

DSS Profile with 16 concurrent Users

- 100/0% Read/Write ratio (Rollup and Cube Function)

In-Memory Area with 16G Memory (SGA 128GB)

All Columns of involved Tables (full Schema) are stored in the IM

Column Store (~ 10GB) and populated

Duplicate Feature intended for RAC Environment was not used

This Feature should be considered in HA or Scalability Environments

Our first steps in this way, was not so encouraging in terms of scalability

DOAG-Konferenz 2014


33

2014 © Trivadis

In-Memory Option – Test Cases

1. Smart Scan off , no In-Memory

2. Smart Scan on, no In-Memory

3. Smart Scan off, In-Memory compression “for DML”

4. Smart Scan on, In-Memory compression “for DML”

5. Smart Scan on, In-Memory compression “for query low”

6. Smart Scan on, In-Memory compression “for query high”

7. Smart Scan on, In-Memory compression “for capacity low”

8. Smart Scan on, In-Memory compression “for capacity high”

DOAG-Konferenz 2014


34

2014 © Trivadis

In-Memory Option – Test Results OLTP

DOAG-Konferenz 2014


35

Total completed Transaction

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

Total Completed Transaction

2014 © Trivadis


DOAG-Konferenz 2014


36

Average CPU Utilization

0

10

20

30

40

50

60

70

80

90

Avg User CPU

Avg Sys CPU

CPU Usage

2014 © Trivadis

Total completed Transaction

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

0

10

20

30

40

50

60

70

80

90


DOAG-Konferenz 2014


37

2014 © Trivadis

In-Memory Option – Test Results DSS

DOAG-Konferenz 2014


38

Total completed Transaction (Reports)

0

20

40

60

80

100

120

Total Completed Transactions

2014 © Trivadis


DOAG-Konferenz 2014


39

Average CPU Utilization

0

10

20

30

40

50

60

Avg User CPU

Avg Sys CPU

CPU Usage

2014 © Trivadis

0

20

40

60

80

100

120

0

10

20

30

40

50

60


DOAG-Konferenz 2014


40

Total completed Transaction (Reports)

2014 © Trivadis

In-Memory Option on Exadata – Result discussion

OLTP

factor 12 :-o

No offloading due to no direct path functionality

All in cache with high CPU load

40% read profit of IMCS

OLTP less CPU with IMCS due to SIMD benefit

DSS

No Offloading/No IMCS as fast but a lot of CPU Buffer Cache

Offloading, as fast but less CPU work done in Storage Server

IMCS 25% plus work, less CPU

The dataset and ressources are not in a good balance to test

Much more data needed

DOAG-Konferenz 2014


41

2014 © Trivadis

Exadata Database Machine works fine

Oracle IMCS Option works fine

Testing and interpretation is not trivial

The option costs 50% of EE core price

A serious evaluation is needed

PoC to find the best of all, unless

you need a:

In-Memory Option on Exadata – Conclusion

DOAG-Konferenz 2014


43

2014 © Trivadis

Questions and answers ...

2014 © Trivadis

BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA

Konrad HÄFELI

Tel: +41 58 459 59 69

[email protected]

DOAG-Konferenz 2014


44

doag 2014 konrad-haefeli_in-memory-option-on-exadata

Technology