indexes: structure, splits and free space management internals

2014 © Trivadis

BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA

2014 © Trivadis

Indexes: Structure, Splits and Free Space Management Internals

OakTable World, 29 September 2014

Christian Antognini

29 September 2014Indexes: Structure, Splits and Free Space Management Internals

2014 © Trivadis

Indexes: Structure, Splits and Free Space Management Internals29 September 2014

@ChrisAntognini

Senior principal consultant, trainer and partnerat Trivadis in Zurich (CH) christian.antognini@trivadis.com http://antognini.ch

Focus: get the most out of Oracle Database Logical and physical database design Query optimizer Application performance management

Author of Troubleshooting Oracle Performance Apress, 2008/2014

OakTable Network, Oracle ACE Director

2014 © Trivadis

AGENDA

1. Concepts

2. Index Keys

3. Splits

4. Free Space

5. Reorganizations

6. Bitmap

2014 © Trivadis

Concepts

2014 © Trivadis

Indexes: Structure, Splits and Free Space Management Internals5

Structure of a B-Tree

29 September 2014

12 29 42 86

2 ROWID

8 ROWID

10 ROWID

11 ROWID

12 ROWID

18 ROWID

20 ROWID

24 ROWID

26 ROWID

29 ROWID

34 ROWID

38 ROWID

42 ROWID 51 ROWID

55 ROWID

73 ROWID

86 ROWID

99 ROWID

left<24right≥24

root block

2014 © Trivadis

NULL values are stored in the index only when at least one of the indexed columns is not NULL Exception: bitmap indexes always store indexed data

For unique indexes the uniqueness is guaranteed only if at least one of the indexed columns is not NULL

NULL values are stored after not NULL values in the index

29 September 2014

2014 © Trivadis

Oracle B-tree indexes can become “unbalanced” over time and need to be rebuilt

Deleted space in an index is “deadwood” and over time requires the index to be rebuilt

If an index reaches “x” number of levels, it becomes inefficient and requires the index to be rebuilt

If an index has a poor clustering factor, the index needs to be rebuilt

To improve performance, indexes need to be regularly rebuilt

29 September 2014

Source: Oracle B-Tree Index Internals: Rebuilding The Truth, Richard Foote

2014 © Trivadis

Index Keys

2014 © Trivadis

Internal Key

Internally a unique key is needed to sort the indexed columns

For unique indexes the internal key is the same as the index key

For non-unique indexes the ROWID of the indexed row is added to the index key to make it unique For global indexes an extended ROWID is used, otherwise a restricted

ROWID is enough

Therefore The index entries are sorted in a consistent way Even for non-unique indexed the order of the index entries changes only

when the indexed table is reorganized (i.e. not if the index is rebuilt!)

29 September 2014

2014 © Trivadis

Branch Block Key

In branch blocks the internal key is truncated after the first byte that differs from the last key of the left leaf block

All keys are preceded by a length field 1 byte for data up to 127 bytes Otherwise 2 bytes

- The high bit of the first byte is set (0x80)- The remaining bits of the first byte and the second byte contain the length

Partially stored keys are terminated by 0xFE Nulls have a column length of 0xFF, trailing nulls are stored

29 September 2014

RDBALen

Key 1DataKey 1

LenKey X

DataKey X

2014 © Trivadis

Leaf Block Key

All keys are preceded by a length field 1 byte for data up to 127 bytes Otherwise 2 bytes

- The high bit of the first byte is set (0x80)- The remaining bits of the first byte and the second byte contain the length

Nulls have a column length of 0xFF, trailing nulls are stored

For global indexes an extended ROWID is used, otherwise a restricted ROWID is enough

29 September 2014

Flag LockLen

Key 1DataKey 1

LenROWID

ROWID(optional)

2014 © Trivadis

Splits

2014 © Trivadis

Splits

Two split methods are implemented

50:50 split The keys are evenly distributed over two blocks Based on size, not number of keys

99:1 split (a.k.a. 90:10) The new key is the right-most of the index A new leaf block containing only the new key is added Useful for increasing values like sequences and timestamps

A change in the structure of an index (like a split) is not rollbacked if the transaction that caused it is rollbacked

29 September 2014

2014 © Trivadis

Splits

29 September 2014

2014 © Trivadis

Splits

29 September 2014

2014 © Trivadis

Splits

29 September 2014

#3#2 #4

2014 © Trivadis

Splits

29 September 2014

#3#2 #4 #5

2014 © Trivadis

Splits

29 September 2014

2014 © Trivadis

Free Space

2014 © Trivadis

Reminder

PCTFREE is only used when the index is created (or rebuilt) An update is implemented as a delete followed by an insert The free space in leaf blocks is strongly dependent on the type of split (50:50

or 99:1)

PCTUSED cannot be used for indexes

29 September 2014

Free Space

29 September 2014

#4#2 #3

Free Space

29 September 2014

#3#2 #4

Reorganizations

If the free space is not automatically reused, it should be manually reclaimed This is unusual, therefore such an operation should only be performed if it is

really necessary

To manually reclaim free space two possibilities exists Rebuild: the index is recreated Coalesce: free space in contiguous blocks is merged and, therefore, some

blocks are unchained

Unnecessary rebuild/coalesce may lead to sub-optimal data density in leaf blocks Too much free space, i.e. too low density Too much block splits, i.e. too high density

29 September 2014

Rebuild vs. Coalesce

29 September 2014

Pro Cons

Rebuild

Doesn’t lock table (online rebuilds only)

Reclaim all the free space

Locks table during rebuild (offline rebuilds only)

Doubles space usage temporarily

The index is completely recreated (to do so large sorts may be performed)

Coalesce

Doesn’t lock table

Current index structure is reused

Doesn’t reclaim all the free space

Can generate lot of redo

Bitmap

A bitmap index is a regular B-tree, only the internal key differs from the non-bitmap index!

The key is composed by Indexed column(s) Start/End ROWID Bitmap representing the rows that have the key

- To save space it is “compressed”

29 September 2014

Text Text

Summary

B-tree indexes cannot become “unbalanced”

Deleted space in an index can be reused

Indexes do not need to be regularly rebuilt

Questions and answers ...

BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA

Christian Antognini

Senior Principal Consultant

christian.antognini@trivadis.com

indexes: structure, splits and free space management internals

Technology

front splits fast - how to learn the splits: four exercises...

haskell internals

indexes à arbres et indexes à hachage

spamcheetah internals

front splits fast how to learn the splits-how flexibility...

function-based indexes partitioned indexes

aix internals

module 3: creating and tuning indexes. planning indexes...

twig internals - maksym moskvychevtwig internals maksym...

ducted and wall splits - wall splits & ducted systems

nginx internals

symfony internals

solaris internals, 2nd edition - solaris internals and...

using changelogs for efficient search and content...

asyncio internals

samsung mini splits

achieving the splits

plpgsql internals

sga internals

chapter 14-1 chapter outline types of single-level ordered...