partner webcast – oracle database in-memory: accelerate business

http://twitter.com/oracleimc

https://www.facebook.com/oracleimc

https://plus.google.com/108612216492389783792

https://www.youtube.com/OracleIMCTeam

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Oracle Database In-Memory Powering the Real-Time Enterprise

Rhine Singh Migration consultant Database & Options Oracle AxizWorkgroup Partner Hub South Africa


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3


Why did Oracle introduce Database In-Memory ?


Oracle Database In-Memory option Goals

5

Real-Time Analytics Trivial to Implement

No Application Changes Not Limited by Memory

100X

Risk-Free

Proven Scale-Out, Availability, Security

Accelerate Mixed Workload

Analytics Transactions

Run analytics on Operational

Systems

Enable Real-Time Business Decisions

Real-Time Analytics

100X


What is an analytic query?

Which products give us our highest

margins?

Who are the top 10 sales reps in the north

west region this month?

If I get a 20% discount on widget A, how

much will our margins improve?


Row Format Databases vs. Column Format Databases

Rows Stored Contiguously

Transactions run faster on row format

– Example: Query or Insert a sales order – Fast processing few rows, many columns

Columns Stored

Contiguously

Analytics run faster on column format

– Example : Report on sales totals by region – Fast accessing few columns, many rows

SALES

SALES

7

Until Now Must Choose One Format and Suffer Tradeoffs

Query

Qu

ery


How has Oracle dealt with this ?


Breakthrough: Dual Format Database

• BOTH row and column formats for same table

• Simultaneously active and transactionally consistent

• Analytics & reporting use new in-memory Column format

• OLTP uses proven row format

9

Normal Buffer Cache

New In-Memory Format

SALES SALES

Row Format

Column Format

SALES


How does it work ?


Oracle In-Memory Columnar Technology

• Pure in-memory column format • Not persistent, and no logging

• Quick to change data: fast OLTP

• Enabled at table or partition • Only active data in-memory

• 2x to 20x compression typical

• Available on all hardware platforms

11

SALES

Pure In-Memory Columnar


Dual-Format Database

ID PROD STORE QTY AMOUNT

983 MILK SF 5 20

111 BREAD LA 20 45

695 APPLE LA 10 5

12

SALES

ID 983 111 695

PROD MILK BREAD APPLE

STORE SF LA LA

QTY 5 20 10

AMOUNT 20 45 5

Column Store

Rotate 90°

SALES

Row Store


Dual-Format Database Logical Representation

13

1:983,MILK,SF,5,20-2:111,BREAD,LA,20,45- 3:695,APPLE,LA,10,5

983:1,111:2,695:3-MILK:1,BREAD:2, APPLE:3-SF:1,LA:2:3-20:1,45:2,5:3

ROW Representation

Column Representation


Populating : In-Memory Column Store

• New INMEMORY ATTRIBUTE

• Following segment types are eligible

• Tables

• Partitions

• Subpartition

• Materialized views

• Following types not eligible

• IOTs

• Hash clusters

• Out of line LOBs

CREATE TABLE customers ……

PARTITION BY LIST

(PARTITION p1 …… INMEMORY,

(PARTITION p2 …… NO INMEMORY);

ALTER TABLE sales INMEMORY;

ALTER TABLE sales NO INMEMORY;

Pure OLTP Features


Users And Schemas Used In This Demo

RTL schema is a stripped-down and

modified version of Oracle Retail

Data Model.

LABUSER schema is a copy of the

Sales History (SH) schema, which

is a part of the example schemas supplied by the Oracle Database.


Verify IN-Memory Area Configuration

Connect to Database • CONNECT / AS SYSDBA

Check if the In-Memory parameter is enabled • SHOW PARAMETER INMEMORY

Current value for SGA – Not set for In-Memory yet • SELECT NAME, VALUE FROM V$SGA;


Initialization Parameters Related To IM Column Store

• INMEMORY_SIZE

• INMEMORY_FORCE

• INMEMORY_QUERY

• INMEMORY_CLAUSE_DEFAULT

• INMEMORY_MAX_POPULATE_SERVERS

• NMEMORY_OPTIMIZER_AWARE

• INMEMORY_TRICKLE_POPULATE_SERVERS_PERCENT


The In-Memory Clause

• MEMCOMPRESS – To Enable the object for the IM column store

• DUPLICATE – In a RAC environment each database node has its own IM Column Store.

• DISTRIBUTE - This attributes also pertains to a RAC environment. When an objectis too big to fit into the IM Column


In-Memory Column Store Compression

• IMCUs compressed during population

• Controlled by MEMCOMPRESS sub-clause

• Ascending order of compression levels: • FOR DML (for heavily updated tables, slower for queries)

• FOR QUERY LOW (default: light compression and fastest)

• FOR QUERY HIGH (slightly more compression)

• FOR CAPACITY LOW (balances capacity and performance)

• FOR CAPACITY HIGH (maximizes capacity)

• Allows in-memory storage tiering

19

DML

QUERY LOW

QUERY HIGH

CAPACITY LOW

CAPACITY HIGH H

ott

er

Par

titi

on

s C

old

er

Par

titi

on

s


Enable The In-Memory Column Store

Need to Alter the system in order to set the In-Memory size • ALTER SYSTEM SET INMEMORY_SIZE=1G SCOPE=SPFILE;

View the details of the new In-Memory parameter • SHOW PARAMETER SGA_TARGET

Need to Shut down & start-up the DB after the In-Memory area has been set • SHUTDOWN IMMEDIATE • STARTUP

Once restarted In-Memory parameter new value becomes visible • SHOW PARAMETER INMEMORY_SIZE

View the change in the SGA area • SELECT NAME, VALUE FROM V$SGA;

Query the amount of available memory allocated to each pool • SELECT POOL, ALLOC_BYTES, USED_BYTES FROM V$INMEMORY_AREA;


Enabling A New Object CONNECT LABUSER/labuser

Creating a new table and enabling it for In-Memory CREATE TABLE PARTS ( PART_ID NUMBER, NAME VARCHAR2(100), MFR VARCHAR2(50), CATEGORY VARCHAR2(30), BRAND VARCHAR2(50)) INMEMORY; Checking the In-Memory attributes for table ‘PARTS’ SELECT INMEMORY, INMEMORY_PRIORITY, INMEMORY_COMPRESSION, INMEMORY_DISTRIBUTE, INMEMORY_DUPLICATE FROM USER_TABLES WHERE TABLE_NAME = 'PARTS';

Disable In-Memory on an object ALTER TABLE PARTS NO INMEMORY ;


Oracle Database In-Memory Advisor

• Analytic processing performance improvement factors based on – Elimination of user I/O waits, cluster transfer waits, buffer cache latch waits, etc.

– Cost: Estimated memory size with various compression options • Certain query processing advantages related to specific compression types.

• Decompression cost heuristics per specific compression types.

– Benefit: Estimated DB time reduction of processing analytic workload

• Report & SQL plan selectivity, number of columns in the result set, etc

– Vary IM Column Store size to receive specific loading plan

– Generate DDL script with all the tables/partitions/sub-partitions recommended

– Top SQL benefits from given configuration


– A name for the SPA Task

– The STS you’ve created containing the SQL to be tested with Database In-Memory

– The parameter you wish to test – ‘inmemory_query’

– The two values of the parameter – ‘DISABLE’ and ‘ENABLE’.

– The metric to be used for comparison between the two trials.

For testing Database In-Memory


Oracle In-Memory Advisor Reports


Why is an In-Memory scan faster than the buffer cache?

SELECT COL4 FROM MYTABLE;

26

X X X X X

RESULT

Row Format

Buffer Cache


Why is an In-Memory scan faster than the buffer cache?

SELECT COL4 FROM MYTABLE;

27

RESULT

Column Format

IM Column Store

RESULT

X X X X X


Oracle In-Memory Column Store Storage Index

• Each column is made up of multiple column units

• Min / max value is recorded for each column unit in a storage index

• Storage index provides partition pruning like performance for ALL queries

28

Memory

SALES Column Format

Min 1 Max 3

Min 4 Max 7

Min 8 Max 12

Min 7 Max 15

Example: Find all sales from stores with a store_id of 8

?


Orders of Magnitude Faster Analytic Data Scans

• Each CPU core scans local in-memory columns

• Scans use super fast SIMD vector instructions

• Originally designed for graphics & science

• Billions of rows/sec scan rate per CPU core

• Row format is millions/sec

29

Vec

tor

Reg

iste

r Load multiple region values

Vector Compare all values an 1 cycle

CPU

Memory

REG

ION

CA

CA CA

CA

Example: Find sales in region of CAlifornia

> 100x Faster


Joining and Combining Data Also Dramatically Faster

• Converts joins of data in multiple tables into fast column scans

• Joins tables 10x faster

30

Example: Find total sales in outlet stores

Sales Stores

Sto

re ID

StoreID in 15, 38, 64

Type=‘Outlet’

Typ

e

Sum

Sto

re ID

Am

ou

nt


Generates Reports Instantly

• Dynamically creates in-memory report outline

• Then report outline filled-in during fast fact scan

• Reports run much faster

• Without predefined cubes

• Also offloads report filtering to Exadata Storage servers

31

Example: Report sales of footwear in outlet stores

Sales

Stores

Products

In-Memory Report Outline

Footwear

Ou

tlet

s

$ $$

$ $$$

Footwear

Sales Outlets


How does it impact OLTP environments ?


Complex OLTP is Slowed by Analytic Indexes

• Most Indexes in complex OLTP (e.g. ERP) databases are only used for analytic queries

• Inserting one row into a table requires updating 10-20 analytic indexes: Slow!

• Indexes only speed up predictable queries & reports

Table

1 – 3 OLTP

Indexes

10 – 20 Analytic Indexes

33


OLTP is Slowed Down by Analytic Indexes

Insert rate decreases as number of indexes

increases

# of Fully Cached Indexes (Disk Indexes are much slower)

34


Column Store Replaces Analytic Indexes

• Fast analytics on any columns

• Better for unpredictable analytics

• Less tuning & administration

• Column Store not persistent so update cost is much lower

• OLTP & batch run faster

Table

1 – 3 OLTP

Indexes In-Memory

Column Store

35


How much of a difference does it make when running a simple aggregation query

Step1: We will connect as the RTL user Enable In-Memory on all tables in the RTL schema • ALTER TABLE DWB_RTL_SLS_RETRN_LINE_ITEM INMEMORY; • ALTER TABLE DWB_RTL_TNDR_LI INMEMORY; • ALTER TABLE DWB_RTL_TRX INMEMORY; • ALTER TABLE DWR_ORG_BSNS_UNIT INMEMORY; • ALTER TABLE DWR_SKU_ITEM INMEMORY;

Step 2: Populate the RTL tables. Execute a simple COUNT(*) query on each of the tables in RTL schema to trigger their population, as shown below. • SELECT COUNT(*) FROM DWB_RTL_SLS_RETRN_LINE_ITEM; • SELECT COUNT(*) FROM DWB_RTL_TNDR_LI; • SELECT COUNT(*) FROM DWB_RTL_TRX; • SELECT COUNT(*) FROM DWR_ORG_BSNS_UNIT; • SELECT COUNT(*) FROM DWR_SKU_ITEM;

Step 3: Running A Simple Aggregation Query SET TIMING ON; SELECT MAX(EXTENDED_AMT) FROM DWB_RTL_SLS_RETRN_LINE_ITEM;

Step 4: View the time laps difference. Execute the query against the buffer cache ALTER SESSION SET INMEMORY_QUERY=DISABLE; SELECT MAX(EXTENDED_AMT) FROM DWB_RTL_SLS_RETRN_LINE_ITEM;


How much of a difference does it make when using a filter

Step 1: SET TIMING ON SELECT SUM(EXTENDED_AMT) FROM RTL.DWB_RTL_SLS_RETRN_LINE_ITEM WHERE ACTN_CD = 'Return'; SET TIMING OFF

Step 2: Now, execute the same query against the buffer cache ALTER SESSION SET INMEMORY_QUERY=DISABLE; SET TIMING ON SELECT SUM(EXTENDED_AMT) FROM RTL.DWB_RTL_SLS_RETRN_LINE_ITEM WHERE ACTN_CD = 'Return'; SET TIMING OFF ALTER SESSION SET INMEMORY_QUERY=ENABLE;


How can I scale this solution ?


Scale-Out In-Memory Database to Any Size

• Scale-Out across servers to grow memory and CPUs

• In-Memory queries parallelized across servers to access local column data

• Direct-to-wire InfiniBand protocol speeds messaging on Engineered Systems

39


In-Memory Speed + Capacity of Low Cost Disk

DISK

PCI FLASH

DRAM

Cold Data

Hottest Data

Active Data

• Size not limited by memory

• Data transparently accessed across tiers

• Each tier has specialized algorithms & compression

• Simultaneously Achieve:

• Speed of DRAM

• I/Os of Flash

• Cost of Disk

40


Scale-Up for Maximum In-Memory Performance

M6-32

Big Memory Machine

32 TB DRAM

32 Socket

3 Terabyte/sec Bandwidth

• Scale-Up on large SMPs

• Algorithms NUMA optimized

• SMP scaling removes overhead of distributing queries across servers

• Memory interconnect far faster than any network

41


Oracle In-Memory: Industrial Strength Availability

RAC

ASM

RMAN

Data Guard &

GoldenGate • Pure In-Memory format does not

change Oracle’s storage format, logging, backup, recovery, etc.

• All Oracle’s proven availability technologies work transparently

• Protection from all failures

• Node, site, corruption, human error, etc.

42


Oracle Database In-Memory: Unique Fault Tolerance

• Similar to storage mirroring

• Duplicate in-memory columns on another node

• Enabled per table/partition

• E.g. only recent data

• Application transparent

• Downtime eliminated by using duplicate after failure

43

Only Available on Engineered Systems


Example: Duplicate Strategy For a Star Schema

• Fact tables are distributed by partition

• Dimension tables are duplicated (DUPLICATE ALL)

• Co-locates joins between the distributed fact table partitions and the dimension tables

44

D

D

D

D

F D

D

D

D

F D

D

D

D

F


How easy is it to get started ?


Oracle In-Memory: Simple to Implement

1. Configure Memory Capacity

• inmemory_size = XXX GB

2. Configure tables or partitions to be in memory

• alter table | partition … inmemory;

3. Later drop analytic indexes to speed up OLTP

46


Oracle In-Memory Requires Zero Application Changes

Full Functionality - ZERO restrictions on SQL

Easy to Implement - No migration of data

Fully Compatible - All existing applications run unchanged

Fully Multitenant - Oracle In-Memory is Cloud Ready

Uniquely Achieves All In-Memory Benefits With No Application Changes

47


How are customers using Oracle Database In-Memory ?


In-Memory Use Cases OLTP

• Real-time reporting directly on OLTP source data

• Removes need for separate ODS

• Speeds data extraction

Data Warehouse

• Staging/ETL/Temp not a candidate

• Write once, read once

• All or a subset of Foundation Layer

• For time sensitive analytics

• Potential to replace Access Layer

49


Getting The Most From In-Memory

• Fast cars speed up travel, not meetings

• In-Memory speeds up analytic data access, not: – Network round trips, logon/logoff

– Parsing, PL/SQL, complex functions

– Data processing (as opposed to access) • Complex joins or aggregations where not much data is filtered before processing

– Load and select once – Staging tables, ETL, temp tables

Understand Where it Helps

50

Know your bottleneck!


Oracle In-Memory Applications Examples

CHANGE

Oracle Application Module Improvement Elapsed Time

Demantra - Promotion Planning 102x Faster 18.3 min to 11 second

Oracle In-Memory Transportation Mgmt 1031x Faster 17 min to < 1 seconds

Oracle E-business Suite - In-Memory Cost Management

628x Faster

58 hours to 6 minutes

Demantra - Consumption Driven Planning 13.5x Faster 12.7 hours to 56 min

JD Edwards - Supply Chain Planning and Analytics

76x Faster 28 minutes to 3 min

Siebel Marketing - List Import 141x Faster 115 mins to 49 seconds

51


Getting The Most From In-Memory

• Avoid stop and go traffic – Process data in sets of rows in the Database

– Not one row at a time in the application

• Plan ahead, take shortest route – Help the optimizer help you: Gather representative statistics using DBMS_STATS

• Use all your cylinders – Enable parallel execution

– In-Memory removes storage bottlenecks allowing parallelism to increase

The Driver Matters

52


Oracle Database In-Memory Key Differentiators

#1 OLTP Scales with Unique Dual Format

#2 Both Scale Up & Scale Out supported

#3 Maximum Availability is Built In

#4 Database Size Isn’t Limited by Memory

#5 All Applications are Transparently Compatible

#6 No new Personnel Needed


Where can I get more information ?


Join the Conversation

55

https://twitter.com/db_inmemory

https://blogs.oracle.com/in-memory/

Related White Papers • Oracle Database In-Memory White Paper • Oracle Database In-Memory Aggregation Paper • When to use Oracle Database In-Memory • Oracle Database In-Memory Advisor

Related Database In-Memory Free Webcasts • Oracle Database In-Memory meets Data Warehousing

Related Videos • In-Memory YouTube Channel • Database Industry Experts Discuss Oracle Database In-Memory (11:10) • Software on Silicon

Any Additional Questions • Oracle Database In-Memory Blog

https://www.facebook.com/OracleDatabase

http://www.oracle.com/goto/dbim.html

Additional Resources

Oracle Partners Sales guided learning path is intended for sales representatives at partner organizations who specialize in selling and positioning Oracle Database In-Memory to their own customer base.

http://www.oracle.com/technetwork/database/in-memory/overview/twp-oracle-database-in-memory-2245633.html?ssSourceSiteId=ocomen



http://www.oracle.com/technetwork/database/bi-datawarehousing/inmemory-aggregation-twp-01282015-2412192.pdf



http://www.oracle.com/technetwork/database/in-memory/overview/twp-dbim-usage-2441076.html



http://www.oracle.com/technetwork/database/manageability/inmemory-advisor-2412222.html



http://www.bitpipe.com/detail/RES/1420858423_861.html



http://www.oracle.com/ocom/groups/systemobject/@mktg_admin/documents/webcontent/videoplayer-ocom.html?bctid=3717442023001&playerType=single-social&size=c23&shareUrl=http://medianetwork.oracle.com/video/player/3717442023001

https://www.youtube.com/channel/UCSYHgTG68nrHa5aTGfFH4pA








https://www.youtube.com/watch?v=anT99OvEpNg

https://blogs.oracle.com/In-Memory/



https://competencycenter.oracle.com/opncc/full_glp.cc?group_id=29751


Q&A

56

Rhine Singh Oracle AxizWorkgroup ISV Migration Center Consultant [email protected] ISV Migration Center blog: http://blogs.oracle.com/imc ISV Migration Center email: [email protected]

http://blogs.oracle.com/imc

mailto:[email protected]


• Oracle.com Partner Hub oracle.com/partners/goto/hub-ecemea

• Migration Center Team Blog blogs.oracle.com/imc

feeds.feedburner.com/oracleIMC

• Partner Webcast Recordings youtube.com/OracleIMCteam

• Partner Webcast Presentations slideshare.net:/Oracle_IMC_team

• [email protected]

Oracle Partner Hub ISV Migration Center • twitter.com/OracleIMC

• plus.google.com/+OracleIMC

• facebook.com/OracleIMC

• linkedin.com/groups/Oracle-Partner-Hub-Migration-Center-4535240

57

http://oracle.com/partners/goto/hub-ecemea






http://www.slideshare.net/oracle_imc_team










https://www.facebook.com/OPN.PartnerHub.Migration.Center

https://www.facebook.com/OPN.PartnerHub.Migration.Center

http://linkedin.com/groups/Oracle-Partner-Hub-Migration-Center-4535240











