241-303 discrete maths: trees/6 1 discrete maths objective – –introduce more unusual tree...

241-303 Discrete Maths: Trees/6 1

Discrete Maths

• Objective– introduce more unusual tree algorithms and techniques

• you already know about binary (search) trees

241-303, Semester 1 2014-2015

6. Trees


Overview

1. Uses of Trees

2. Huffman Codes

3. (Rooted) Tree Terminology

4. Spanning Trees

5. Minimal Spanning Trees

6. Game Trees

7. More Information


1. Uses of Trees

Davenport

S. Williams

Bartoli

V. Williams

S. Williams

V. Williams

S. Williams

Wimbledon Womens Tennis


Organizational Chart

President

Vice-Presidentfor Academics

Vice-Presidentfor Admin.

Dean ofEngineering

Dean ofBusiness

Head of CoE Head of EE Head of AC.

PlanningOfficer

PurchasesOfficer

. . . .

. . . .

. . . .


Saturated HydrocarbonsH

HH

H

C

HH C

HH C

HH CH C

H

H HH C

H

HC

H

H

C

H

Butane

Isobutane

• Non-rooted (free) trees– a free tree is a graph with no cycles


A Computer File System

/

usr bin tmp

bin ad spool ls mail who junk

ed vi exs opr uucp

printer


2. Huffman Codes• A Huffman code represents characters by variable-length bit s

trings– there are many different Huffman codes

• By comparison, ASCII uses fixed-length codes.– Character ASCII code

A 1000001 B 1000010 : :

S 1010011 : :


A Huffman Code in Tree Form

Root

1 0

01

01

01

A

O

R

ST

• Character Code A 1 O 00 R 010 S 0110 T 0111

where do thesecome from?


Huffman uses Less Bits

• ASCII coding of "RATS":– 1010010 1000001 1010100 1010011– 4*7 = 28 bits

• Huffman Coding (as in the tree):– 010 1 0111 0110– 12 bits

useful for network comms, text compression, etc.


Constructing a Huffman Code

• A Huffman code is based on a table giving the frequency of the characters in the 'language'.

• The 'language' can be a subset of a full language– e.g. business English, computer language keyword

s, the months of the year, a language made up of only {A, O, R, S, T}


Letter Frequencies for English

• The most common letters, in full English, in order of frequency, are usually ETAOINSHRDLU.

• This varies depending on the data source – e.g. novels, textbooks, cartoo

ns


Example: The Haddock Language

• The Haddock language contains only 5 characters:– !, @, #, $, %– we can make words like:

• !!!@@ $%$# @@@@@@@@@

• We must create a frequency table by analysing existing examples of Haddock writing.

continued


• The frequency table we create:– Character Frequency

! 2/32 @ 3/32 # 7/32 $ 8/32 % 12/32

• We work with the numerators of the fractions.


Informal Algorithm• 1. Make a sequence of the frequencies:

{ 2, 3, 7, 8, 12}

• 2. Repeatedly replace the smallest two frequencies by a sum until only 2 elements are left:– {2,3,7,8,12} ==> {2+3, 7, 8, 12}

{5,7,8,12} ==> {5+7, 8, 12}{8,12,12} ==> {8+12, 12}{12,20}

continued


• 3. Build a tree by expanding the sequence back to its full size.– Start with the 2 elements as a 2 branch tree.

01

2012

==> 1 0

0112

128

==> 1 0

01

128

01

75

==>

continued


• 4. Replace each frequency by the character having that frequency:

==> 1 0

01

128

01

701

32

1 0

01

%$

01

#01

@!continued


Huffman Code for Haddock

• Character Code ! 111 @ 110 # 10 $ 01 % 00

• The higher frequency letters have shorter Huffman codes.


Another Possible Answer

• Why? There are two "12" nodes (see slide 12), either of which can be expanded back to "5" and "7".

1 0

0112

8 01

701

32

continued


Huffman Code for Haddock v.2

• Character Code ! 0011 @ 0010 # 000 $ 01 % 1

• The higher frequency letters have shorter Huffman codes.


Huffman Algorithm

• Tree huffman(FreqSeq f, int size){ if (size == 2){ let f1, f2 be the frequencies in f return baseTree; else { let fi, fj be the smallest freqs in f; replace fi and fj in f with fi+fj; Tree t1 = huffman(f, size-1); Tree t = replace "fi+fj" vertex in t1 with extTree return t; }}

01

f2f1baseTree

01

fjfiextTree

call: Tree t = huffman( {2,3,7,8,12}, 5);


3. (Rooted) Tree Terminology

• e.g. Part of the ancient Greek god family:

Uranus

Aphrodite Kronos Atlas Prometheus

Eros Zeus Poseidon Hades Ares

Apollo Athena Hermes Heracles

levels

0

1

2

3::


Some Definitions

• Let T be a tree with root v0.

• Suppose that x, y, z are verticies in T.

• (v0, v1,..., vn) is a simple path in T (no loops).

• a) vn-1 is the parent of vn.

• b) v0, ..., vn-1 are ancestors of vn

• c) vn is a child of vn-1

continued


• d) If x is an ancestor of y, then y is a descendant of x.

• e) If x and y are children of z, then x and y are siblings.

• f) If x has no children, then x is a terminal vertex (or a leaf).

• g) If x is not a terminal vertex, then x is an internal (or branch) vertex.

continued


• h) The subtree of T rooted at x is the graph with vertex set V and edge set E– V contains x and all the descendents of x– E = {e | e is an edge on a simple path from x to

some vertex in V}

• i) The length of a path is the number of edges it uses, not verticies.

continued


• j) The level of a vertex x is the length of the simple path from the root to x.

• k) The height of a vertex x is the length of the simple path from x to the farthest leaf– the height of a tree is the height of its root

• l) A tree where every internal vertex has exactly m children is called a full m-ary tree.


Applied to the Example

• The root is Uranus.• A simple path is {Uranus, Aphrodite, Eros}

• The parent of Eros is Aphrodite.• The ancestors of Hermes are Zeus, Kronos, and U

ranus.• The children of Zeus are Apollo, Athena, Hermes,

and Heracles.

continued


• The descendants of Kronos are Zeus, Poseidon, Hades, Ares, Apollo, Athena, Hermes, and Heracles.

• The leaves (terminal verticies) are Eros, Apollo, Athena, Hermes, Heracles, Poseidon, Hades, Ares, Atlas, and Prometheus.

• The branches (internal verticies) are Uranus, Aphrodite, Kronos, and Zeus.

continued


• The subtree rooted at Kronos:

Kronos

Zeus Poseidon Hades Ares

Apollo Athena Hermes Heracles

continued


• The length of the path {Uranus, Aphrodite, Eros} is 2 (not 3).

• The level of Ares is 2.

• The height of the tree is 3.


Some Properties of Trees

• 1) A tree with n verticies has n-1 edges.– e.g.

• Proof:– ignore the root;– pair each vertex with the edge above it;– so n-1 verticies use n-1 edges (all of them)


• 2) A full m-ary tree with i internal verticies has a total of m*i + 1 verticies.– e.g.

A full 2-ary tree(binary tree)

4 internal verticiesA total of 9 verticies

continued


• Proof– ignore the root;– every remaining vertex is the child of an interna

l vertex;– each internal vertex has m children;– there are i internal verticies, so there is a total o

f m*i verticies in the tree;– add back the root, to get m*i + 1


4. Spanning Trees

• A spanning tree T is a subgraph of a graph G which contains all the verticies of G.

• Example graph G:

a b

dc

e f

hg

continued


• One possible spanning tree:

a b

dc

e f

hg

the tree isdrawn withthick lines


4.1. Example: IP Multicasting

• A network of computers and routers:

sourcecomputer

router

continued


• How can a packet (message) be sent from the source computer to every other computer?

• The inefficient way is to use broadcasting– send a copy along every link, and have each router d

o the same

– each router and computer will receive many copies of the same packet

– loops may mean the packet never disappears!

continued


• IP multicasting is an efficient solution– send a single packet to one router– have the router send it to 1 or more routers in su

ch a way that a computer never receives the packet more than once

• This behaviour can be represented by a spanning tree.

continued


• The spanning tree for the network:

sourcecomputer

router

the tree isdrawn withthick lines


4.2. Finding a Spanning Tree

• There are two main types of algorithms:– breadth-first search– depth-first search


4.3. Breadth-first Search

• Process all the verticies at a given level before moving to the next level.

• Example graph G (again):

a b

dc

e f

hg


Informal Algorithm• 1) Put the verticies into an ordering

– e.g. {a, b, c, d, e, f, g, h}

• 2) Select a vertex, add it to the spanning tree T: e.g. a

• 3) Add to T all edges (a,X) and X verticies that do not create a cycle in T– i.e. (a,b), (a,c), (a,g)

T = {a, b, c, g}

continued

a

b c g


• Repeat step 3 on the verticies just added, these are on level 1– i.e. b: add (b,d)

c: add (c,e) g: nothing

T = {a,b,c,d,e}

• Repeat step 3 on the verticies just added, these are on level 2– i.e. d: add (d,f)

e: nothingT = {a,b,c,d,e,f}

a

b c g

d e

a

b c g

d e

fcontinued

level 1

level 2


• Repeat step 3 on the verticies just added, these are on level 3– i.e. f: add (f,h)

T = {a,b,c,d,e,f,h}

• Repeat step 3 on the verticies just added, these are on level 4– i.e. h: nothing, so stop

a

b c g

d e

f

h

continued

level 3


• Resulting spanning tree:

a b

dc

e f

hg

a differentspanning treefrom the earliersolution


Breadth-first Algorithm• Tree bfs(Verts vsG, Edges edsG)

// input ordered verts and edges of graph G{ Vert v1 = firstVert(VsG); // first vert in G Seq levelVs = ( v1 ); // verts at curr level Verts vsT = { v1 }; // verts in tree Edges edsT = {}; // edges in tree while (1) { for each x in levelVs for each y in vsG - vsT if (x,y) is an edge in edsG add (x,y) to edsT; add y to vsT; if no edges added to vsT return tree made from vsT and edsT; levelVs = childrenOf( levelVs ); }} continued


• Typical call:Verts vsG = {a, b, c, d, e, f, g, h};

Edges edsG = { (a,b), (a,c), (a,g), (b,d), (b,g), (c,d), (c,e), (d,f), (e,g),

(f,h) };

Tree spanTree = bfs( vsG, edsG );:


4.4. Depth-first Search• Process all the verticies down one path, the

n backtrack (go back) to verticies along other paths.

• Example graph G (again):

a b

dc

e f

hg


Informal Algorithm• 1) Put the verticies into an ordering

– e.g. {a, b, c, d, e, f, g, h}

• 2) Select a vertex, add it to the spanning tree T: e.g. a

• 3) Add the edge (a,X) where X is the smallest vertex in the ordering, and does not make a cycle in T– i.e. (a,b), T = {a, b}

continued

a

b


• 4) Repeat step 3 with the new vertex, until there is no possible new vertex– i.e. add the edges (b,d) (d,c) (c,e) (e,f) (f,h)

T = {a,b,d,c,e,f,h}

• 5) At this point, there is no (h,X), so backtrack to a vertex that does have another edge:– parent of h == f

but there is no new (f,X) to add, so backtrack

continued

a

b

d

c

e

f

h


– parent of f == e– there is an (e,g) to add, so repeat step 3 wi

th e

• 6) After g is added, there are no further verticies to add, so stop.

a

b

d

c

e

f

h

g

continued


• Resulting spanning tree:

a b

dc

e f

hg

a differentspanning treefrom thebreadth-firstsolution


4.5. Another Backtracking Problem

• Find a subset of {31,27,15,11,7,5} with a sum that equals 39.– one answer is {27,7,5}

• How?: build up the sum by adding integers. An integer is included if the sum remains less than 39.– backtracking is required to try different integers wh

en the sum doesn't reach 39.

Summing a Subset


Backtracking Diagram

{} sum = 0

{31} sum = 31

{31,7} sum = 38

{31,5} sum = 36

{27} sum = 27

{27,11} sum = 38

{27,7} sum = 34

{27,7,5} sum = 39


Prolog Code and Query• sum(0, _, []).

sum(Sum, List, [X|Used]) :- Sum > 0, select(List, X, List1), Sum1 is Sum - X, sum(Sum1, List1, Used).

select([X|L], X, L).select([Y|L], X, [Y|L1]) :- select(L, X, L1).

• ?- sum(39, [31,27,15,11,7,5], Used).Used = [27,7,5]

It can calculatesums of any set.


Amzi-Prolog Execution

; means "giveanother answer"


5. Minimal Spanning Tree• A minimal spanning tree T is a subgraph of

a weighted graph G which contains all the verticies of G and whose edges have the minimum summed weight.

• Example weighted graph G:

A B

CD

EF

3

4

5

163

2

6

2


• A minimal spanning tree (weight = 12):

• A non-minimal spanning tree (weight = 20):

A B

CD

EF

3

4

5

163

2

2

6

A B

CD

EF

3

4

5

163

2

6

2


5.1. Prim's Algorithm

• Prim's algorithm finds a minimal spanning tree T by iteratively adding edges to T.

• At each iteration, a minimum-weight edge is added that does not create a cycle in the current T.


Informal Algorithm• For the graph G.• 1) Add any vertex to T

– e.g A, T = {A}

• 2) Examine all the edges leaving {A} and add the vertex with the smallest weight.– edge weight

(A,B) 4(A,C) 2(A,E) 3

– add edge (A,C), T becomes {A,C}

continued

A B

CD

EF

3

4

5

163

2

6

2


• 3) Examine all the edges leaving {A,C} and add the vertex with the smallest weight.– edge weight edge weight

(A,B) 4 (C,D) 1(A,E) 3 (C,E) 6(C,F) 3

– add edge (C,D), T becomes {A,C,D}

continued


• 4) Examine all the edges leaving {A,C,D} and add the vertex with the smallest weight.– edge weight edge weight

(A,B) 4 (D,B) 5(A,E) 3 (C,E) 6(C,F) 3 (D,F) 6

– add edge (A,E) or (C,F), it does not matter– add edge (A,E), T becomes {A,C,D,E}

continued


• 5) Examine all the edges leaving {A,C,D,E} and add the vertex with the smallest weight.– edge weight edge weight

(A,B) 4 (D,B) 5(C,F) 3 (D,F) 6(E,F) 2

– add edge (E,F), T becomes {A,C,D,E,F}

continued


• 6) Examine all the edges leaving {A,C,D,E,F} and add the vertex with the smallest weight.– edge weight edge weight

(A,B) 4 (D,B) 5

– add edge (A,B), T becomes {A,B,C,D,E,F}

• All the verticies of G are now in T, so we stop.

continued


• Resulting minimum spanning tree (weight = 12):

A B

CD

EF

3

4

5

163

2

2

6


Pseudocode

• Tree prim(Graph G, int numVerts){ Tree T = anyVert(G); for i = 1 to numVerts-1{ Edge e = an edge of minimum weight to a

vertex in T and not forming a cycle in T when added to T;

T = T with e added; } return T}


5.2. A Greedy Algorithm

• Prim's algorithm is an example of a greedy algorithm– its choice at each iteration does not depend on a

ny previous choices

• A greedy algorithm does the best thing at this time without considering the past (or the future).


Greed is not Always Best

• An example where greed is not the best approach:– a greedy shortest-path algorithm can lead to a n

on-optimal solution (not the best)

– the algorithm selects an edge with the minimum weight connected to the last vertex added

continued


• Find the shortest path a --> z.• The greedy algorithm produces (a,c,z), but t

he shortest is (a,b,z)

a

b

c

z

2 4

61

8


5.3. Kruskal's Minimum Spanning Tree Algorithm

• At the start, the minimum spanning tree T consists of all the verticies of the weighted graph G, but no edges.

• At each iteration, add an edge e to T having minimum weight that does not create a cycle in T.

• When T has n-1 edges, stop.


Pseudocode

• Tree kruskal(Graph G, int numVerts){ Tree T = allVerts(G); for i = 1 to numVerts-1 { Edge e = an edge of minimum weight in G and not forming a

cycle in T when added to T;

T = T with e added; } return T}


Example• Graph G:

ab c d

ef g h

i j k l

2 3 13

4

3

4 2

1

3

2 3

4 3

3 1

3

iter edge 1 (c,d) 2 (k,l) 3 (b,f) 4 (c,g) 5 (a,b) 6 (f, j) 7 (b,c) 8 (j,k) 9 (g,h) 10 (i, j) 11 (a,e)

continued


• Minimum spanning tree (weight = 24):

ab c d

ef g h

i j k l

2 3 13

4

3

4 2

1

3

2 3

4 3

3 1

3


5.4. Difference between Prim and Kruskal

• Prim's algorithm chooses an edge that must already be connected to a vertex in the minimum spanning tree T.

• Kruskal's algorithm can choose an edge that may not already be connected to a vertex in T.


6. Game Trees

• Trees can be used to develop game-playing strategies– used in many AI (Artificial Intelligence) game

programs where the human is competing against the 'clever' machine


6.1. Nim

• In Nim there are two piles of stones.• Each player takes 1 or more stones from on

e pile.• The player who removes the last stone loses

.

• For example, the pile (3,2):


Part of a Nim Game Tree

(3,2)

(2,2) (1,2) (0,2) (3,1) (3,0)

(1,2) (0,2) (2,1) (2,0) (0,2) (1,1) (1,0)

(1,1) (0,1) (2,0)

(1,0) (0,1)

(0,0) (0,0)

(0,0) (1,0) (0,0)

(0,0)

(0,1) (1,0)

(0,0) (0,0)

(2,0) (1,0) (0,0)

(1,0) (0,0) (0,0)

(0,0)

. . . . . . . . . . . . . . . . . . . .

. . . .. . . .

1st playergoes now

2nd playergoes now


Analysing the Game Tree

• Label each terminal vertex with the first player's score:– if the first player won, write "1"– if the first player lost, write "0"

continued


– means that it is the 1st player’s turn– means 1st player wins since there are no stones left– so assign it

– means that it is the 2nd player’s turn– means 2nd player wins since there are no stones left.

So the 1st player loses.– so assign it

Graphically( 0, 0 )

( 0, 0 )

1

0


• Case 1. Label a box:

• The box is the first player, who will move to the most valuable child (to win).

• So label the box with the maximum child value (e.g. 1).

0 1 0

continued

max

Moving up the Tree


• Case 2. Label an oval

• The oval is the second player, who will move to the least valuable child (so the first player loses).

• So label the box with the minimum child value (e.g. 0).

1 1 0

min


The Minimax Algorithm• In many games:

– the first player chooses the most valuable position to move to; and

– the second player chooses the least valuable position for the first player

• The minimax algorithm:– first player seeks the max of the child values– second player seeks the min of the child values


. . . .

Nim Game Tree Analysis (part)

1

1 0 0 0 0

1 1 1 1 1 1 0

0 1 0

0 0

0 0

1 0 1

0

1 1

1 1

1 0 1

1 0 0

1

. . . . . . . . . . . . . . . . . . . .

. . . .

1st player

2nd player

continued

max

min

max

min

max


• The root value (the starting position) contains a "1"– this means that the first player will definitely wi

n if he/she chooses the most valuable child position to move to


Larger Game Trees

• Games like Chess or Go generate game trees that are too large to analyse using a simple minimax algorithm.

• Instead the game tree is generated to a certain level, and then an evaluation function is used to 'calculate' the scores at that level for the first player.

• Minimax is then used to work back to the root.


Tic-Tac-Toe

. . . .. . . .

X X

X

X X X X XO

OO

OO

Game tree generated to level 2 (with symmetric positions ignored).


The Evaluation Function

• For a given position P:– eval(P) = numWinsX - numWinsO;

– numWinsX = number of rows, columns, and diagonals containing an X that the X player might be able to complete in the future

– numWinsO = same, but for O

continued


• Example position:

• eval(P) = numWinsX - numWinsO= 2 - 1 = 1

• If eval(P) is positive, then X has more ways to win than O.

X O


Analyse the Tic-Tac-Toe Tree

0 1 0 0 -1 -1 -2 0 -1 0 1 2

1

-1 -2 1

• This means that the best move for the first player (X) is the center square.

max

min


Alpha-Beta Pruning• Evaluating a game tree with a large level takes too muc

h time.

• Alpha-beta pruning reduces the number of positions (verticies) that need to be evaluated, and still gives the right answer.

• There are two parts to alpha-beta pruning:– alpha cutoff– beta cutoff


Alpha Cutoff

• An example:

3 2 6 4 1 ? ? ?

x

2 y ?

?

A

B C D

MLKJIHGFE

alphacutoff continued

Evaluate children left-to-right.

max

min min min

max


• In A, x must be >= 2– the lower bound for A, the alpha value for A

• In C, y must be <= 1– since y <= x, it cannot affect x, so there is no need to evalu

ate J– also the C subtree can be ignored (cutoff)

• Alpha cutoff occurs when a grandchild of A (e.g. I) has a value <= the alpha value of A.


Beta Cutoff• An example:

3 2 6 8 ? ? ? ? ?

y ?6

x

max

minA1

B1 C1 D1

E1 F1 G1 H1 I1 J1 K1 L1 M1

betacutoff continued

max max

min


• In A1, x must be <= 6– the upper bound for A1, the beta value for A1

• In C1, y must be >= 8– since y >= x, it cannot affect x, so there is no need t

o evaluate I1 or J1– also the C1 subtree can be ignored (cutoff)

• Beta cutoff occurs when a grandchild of A1 (e.g. H1) has a value >= the beta value of A1.


7. More Information

• DM: chapter 7

• Rosen: chapter 8

241-303 discrete maths: trees/6 1 discrete maths objective – –introduce more unusual tree...

Documents