balanced search trees coursenotes

7/30/2019 Balanced Search Trees Coursenotes

1/57

http://algs4.cs.princeton.edu

Algorithms ROBERT SEDGEWICK | KEVIN WAYNE

3.3 BALANCED SEARCH TREES

2-3 search trees

red-black BSTs

B-trees


2/57

2

Symbol table review

Challenge. Guarantee performance.

This lecture. 2-3 trees, left-leaning red-black BSTs, B-trees.


3/57


2-3 search trees

red-black BSTs

B-trees



4/57

Allow 1 or 2 keys per node.

2-node: one key, two children.

3-node: two keys, three children.

Perfect balance. Every path from root to null link has same length.

2-3 tree

4

S XA C PH

R

M

L

3-node

E J

2-node

null link


5/57

Allow 1 or 2 keys per node.

2-node: one key, two children.

3-node: two keys, three children.


Symmetric order. Inorder traversal yields keys in ascending order.

2-3 tree

5

between E and J

larger than Jsmaller than E

S XA C PH

R

M

L

E J


6/57

Search.

Compare search key against keys in node.

Find interval containing search key.

Follow associated link (recursively).

2-3 tree demo

6

search for H

S XA C PH

R

M

L

E J


7/57

Insertion into a 3-node at bottom.

Add new key to 3-node to create temporary 4-node.

Move middle key in 4-node into parent.

Repeat up the tree, as necessary.

If you reach the root and it's a 4-node, split it into three 2-nodes.

2-3 tree demo

7

S XA C PH

E R

Linsert L


8/57

2-3 tree construction demo

8

insert S

S


9/57

S XA C

2-3 tree construction demo

9

2-3 tree

PH

E R

L


10/57

Splitting a 4-node is a local transformation: constant number of operations.

b c d

a e

betweena andb

lessthan a

betweenb andc

betweend ande

greaterthan e

betweenc andd

betweena andb

lessthan a

betweenb andc

betweend ande

greaterthan e

betweenc andd

b d

a c e

10

Local transformations in a 2-3 tree


11/57

Invariants. Maintains symmetric order and perfect balance.

Pf. Each transformation maintains symmetric order and perfect balance.

11

Global properties in a 2-3 tree

b

right

middle

left

right

left

b db c d

a ca

a b c

d

ca

b d

a b cca

root

parent is a 2-node

parent is a 3-node

c e

b d

c d e

a b

b c d

a e

a b d

a c e

a b c

d e

ca

b d e


12/57

12

2-3 tree: performance


Tree height.

Worst case:

Best case:


13/57

13

2-3 tree: performance


Tree height.

Worst case: lgN. [all 2-nodes]

Best case: log3N .631 lgN. [all 3-nodes]

Between 12 and 20 for a million nodes.

Between 18 and 30 for a billion nodes.

Guaranteed logarithmic performance for search and insert.


14/57

ST implementations: summary

14

constants depend upon implementation


15/57

15

2-3 tree: implementation?

Direct implementation is complicated, because:

Maintaining multiple node types is cumbersome.

Need multiple compares to move down tree.

Need to move back up the tree to split 4-nodes.

Large number of cases for splitting.

Bottom line. Could do it, but there's a better way.


16/57


2-3 search trees

red-black BSTs

B-trees



17/57


2-3 search trees

red-black BSTs

B-trees



18/57


19/57

A BST such that:

No node has two red links connected to it.

Every path from root to null link has the same number of black links.

Red links lean left.

19

An equivalent definition

"perfect black balance"

X

SH

P

J R

E

A

M

C

L

c ree


20/57

Key property. 11 correspondence between 23 and LLRB.

20

Left-leaning red-black BSTs: 1-1 correspondence with 2-3 trees

X

SH

P

J R

E

A

M

C

L

XSH P

J RE

A

M

C L

redblack tree

horizontal red links

2-3 tree

E J

H L

M

R

P S XA C


21/57

Search implementation for red-black BSTs

Observation. Search is the same as for elementary BST (ignore color).

Remark. Most other ops (e.g., floor, iteration, selection) are also identical.

21

but runs faster because of better balance

X

SH

P

J R

E

A

M

C

L

ree


22/57

Red-black BST representation

Each node is pointed to by precisely one link (from its parent)

can encode color of links in nodes.

22

null links are black

J

G

E

A D

C

hh.left.coloris RED

h.right.coloris BLACK


23/57

Elementary red-black BST operations

Left rotation. Orient a (temporarily) right-leaning red link to lean left.

Invariants. Maintains symmetric order and perfect black balance.

23

greater

than S

x

h

S

between

E and S

less

than E

E

rotate E left

(before)


24/57


Left rotation. Orient a (temporarily) right-leaning red link to lean left.


24

greater

than S

less

than E

x

h E

between

E and S

S

rotate E left

(after)


25/57


Right rotation. Orient a left-leaning red link to (temporarily) lean right.


25

rotate S right

(before)

greater

than S

less

than E

h

x E

between

E and S

S


26/57


Right rotation. Orient a left-leaning red link to (temporarily) lean right.


26

rotate S right

(after)

greater

than S

h

x

S

between

E and S

less

than E

E


27/57

Color flip. Recolor to split a (temporary) 4-node.



27

greater

than S

between

E and S

between

A and E

less

than A

Eh

SA

flip colors

(before)


28/57

Color flip. Recolor to split a (temporary) 4-node.



28

Eh

SA

flip colors

(after)

greater

than S

between

E and S

between

A and E

less

than A


29/57

Basic strategy. Maintain 1-1 correspondence with 2-3 trees by

applying elementary red-black BST operations.

Insertion in a LLRB tree: overview

29

E

A

E

R

S

R

S

A

C

E

R

S

C

A

add newnode here

right link redso rotate left

insert C


30/57

Warmup 1. Insert into a tree with exactly 1 node.

Insertion in a LLRB tree

30

red link tonew node

containingaconverts 2-node

to 3-node

search endsat this null link

b

a

b

root

root

left

search endsat this null link

attached new nodewith red link

rotated leftto make a

legal 3-node

a

b

a

a

b

root

root

right


31/57

Case 1. Insert into a 2-node at the bottom.

Do standard BST insert; color new link red.

If new red link is a right link, rotate left.

Insertion in a LLRB tree

31

E

A

E

R

S

R

S

A

C

E

R

S

C

A

add newnode here

right link redso rotate left

insert C


32/57


33/57


34/57


35/57

Red-black BST construction demo

35

S

insert S


36/57

Red-black BST construction demo

36

C

A

E

X

S

P

M

R

red-black BST

L

H


37/57

Insertion in a LLRB tree: Java implementation

Same code for all cases.

Right child red, left child black: rotate left.Left child, left-left grandchild red:rotate right.

Both children red: flip colors.

37

insert at bottom

(and color it red)

split 4-node

balance 4-node

lean left

only a few extra lines of code provides near-perfect balance

flipcolors

rightrotate

leftrotate

h

h

h


38/57

Insertion in a LLRB tree: visualization

38

255 insertions in ascending order


39/57

39


255 insertions in descending order


40/57

40


255 random insertions


41/57

41

Balance in LLRB trees

Proposition. Height of tree is 2 lgNin the worst case.

Pf.Every path from root to null link has same number of black links.

Never two red links in-a-row.

Property. Height of tree is ~1.00 lgNin typical applications.


42/57

ST implementations: summary

42

* exact value of coefficient unknown but extremely close to 1


43/57

Xerox PARC innovations. [1970s]

Alto.GUI.

Ethernet.

Smalltalk.

InterPress.

Laser printing.

Bitmapped display.

WYSIWYG text editor.

...

War story: why red-black?

43

A DIClIROlV1ATIC FUAl\lE\V()HK Fo n BALANCED TREES

Leo J. Guibas.Xerox Palo Alto Research Center,Palo Alto, California, andCarnegie-Afellon University

andRobert Sedgewick*Program in Computer ScienceBrown UniversityProvidence, R. I.

ABSTUACT

I() this paper we present a uniform framework for the implementat ionand s tudy of halanced tree algorithms. \Ve show how to imhcd in thisframework the best known halanced tree tecilIl iques and thell usc theframework to deVl'lop new which per form the update andrebalancing in one pas s, Oil the way down towards a leaf. \Veconclude with a study of performance issues and concurrent updating.

O. Introduction

the way down towards a leaf. As we will see, this has a number ofsignificant advantages ovcr the older methods. We shall cxamine anumhcr of variations on a common theme and exhibit fullimplementat ions which are notable for their brcvity. Oneimp1cn1entation is exatnined carefully, and some properties about itsbehavior are proved.]n both sections 1 and 2 par ti cular a ttent ion is paid to practicalimplementation issues, and cOlnplcte impletnentations are given forall of the itnportant algorithms. '1l1is is significant because onemeasure under which balanced tree algorithtns can differ greatly is

Xerox Alto


44/57

War story: red-black BSTs

44

Telephone company contracted with database provider to build real-time

database to store customer information.

Database implementation.

Red-black BST search and insert; Hibbard deletion.

Exceeding height limit of 80 triggered error-recovery process.

Extended telephone service outage.

Main cause = height bounded exceeded!

Telephone company sues database provider.

Legal testimony:

allows for up to 240 keys

Hibbard deletion

was the problem


45/57


2-3 search trees red-black BSTs

B-trees



46/57



B-trees



47/57

47

File system model

Page. Contiguous block of data (e.g., a file or 4,096-byte chunk).

Probe. First access to a page (e.g., from disk to memory).

Property. Time required for a probe is much larger than time to access

data within a page.

Cost model. Number of probes.

Goal. Access data using minimum number of probes.

slow fast


48/57

B-tree. Generalize 2-3 trees by allowing up toM- 1 key-link pairs per node.

At least 2 key-link pairs at root.At leastM/2 key-link pairs in other nodes.

External nodes contain client keys.

Internal nodes contain copies of keys to guide search.

48

B-trees (Bayer-McCreight, 1972)

choose M as large as possible sothat M links fit in a page, e.g., M = 1024

Anatomy of a B-tree set (M = 6)

2-node

external3-node external 5-node (full)

internal 3-node

external 4-node

all nodes except the root are 3-, 4- or 5-nodes

* B C

sentinel key

D E F H I J K M N O P Q R T

* D H

* K

K Q U

U W X Y

each red key is a copy

of min key in subtree

client keys (black)are in external nodes


49/57

Start at root.

Find interval for search key and take corresponding link.Search terminates in external node.

* B C

searching for E

D E F H I J K M N O P Q R T

* D H

* K

K Q U

U W X

search forE inthis external node

follow this link becauseE is between * andK

follow this link becauseE

is betweenD

andH

Searching in a B-tree set (M = 6)

49

Searching in a B-tree


50/57

Search for new key.

Insert at bottom.Split nodes withMkey-link pairs on the way up the tree.

50

Insertion in a B-tree

* A B C E F H I J K M N O P Q R T

* C H

* K

K Q U

U W X

* A B C E F H I J K M N O P Q R T U W X

* C H K Q U

* A B C E F H I J K M N O P Q R T U W X

* H K Q U

* B C E F H I J K M N O P Q R T U W X

* H K Q U

new key (A) causesoverflow and split

root split causesa new root to be created

new key (C) causesoverflow and split

Inserting a new key into a B-tree set

inserting A


51/57

Proposition. A search or an insertion in a B-tree of order MwithNkeys

requires between logM-

1Nand logM/2Nprobes.

Pf. All internal nodes (besides root) have between M/ 2 andM- 1 links.

In practice. Number of probes is at most 4.

Optimization. Always keep root page in memory.

51

Balance in B-tree

M = 1024; N = 62 billionlog M/2 N 4


52/57

52

Building a large B tree

full page splits intotwo half -full pages

then a new key is addedto one of them

full page, about to split

white: unoccupied portion of page

black: occupied portion of page

each line shows the resultof inserting one key

in some page


53/57

53

Balanced trees in the wild

Red-black trees are widely used as system symbol tables.

Java: java.util.TreeMap, java.util.TreeSet.C++ STL: map, multimap, multiset.

Linux kernel: completely fair scheduler, linux/rbtree.h.

B-tree variants. B+ tree, B*tree, B# tree,

B-trees (and variants) are widely used for file systems and databases.

Windows: HPFS.

Mac: HFS, HFS+.

Linux: ReiserFS, XFS, Ext3FS, JFS.

Databases: ORACLE, DB2, INGRES, SQL, PostgreSQL.


54/57

54

Red-black BSTs in the wild

Common sense. Sixth sense.Together they're the

FBI's newest team.


55/57

Red-black BSTs in the wild

55


56/57



B-trees


Algorithms S |


57/57


Algorithms ROBERT SEDGEWICK | KEVIN WAYNE



B-trees

balanced search trees coursenotes

Documents