a generic top-down dynamic-programming approach to prefix-free coding
DESCRIPTION
A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding. Yalin Zhang [email protected]. Huffman Coding. Given a probability distribution over a set of n words, find a minimal-cost prefix-free coding for these words. Input: alphabet , probability Output: code - PowerPoint PPT PresentationTRANSCRIPT
![Page 2: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/2.jpg)
Huffman CodingGiven a probability distribution over a set of
n words, find a minimal-cost prefix-free coding for these words.
Input: alphabet , probability Output: code Cost =
![Page 3: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/3.jpg)
Huffman CodingComplexity greedy algorithm for binary case.Complexity P is sorted, r-ary case.
What about extra constraints or generalization?
![Page 4: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/4.jpg)
Variations of Huffman CodingLength-limited codingUnequal-cost codingMixed-radix codingReserved-length codingOne-ended coding
![Page 5: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/5.jpg)
Variations of Huffman Coding
Standard binary tree
Mix-radix tree Reserved-length tree
General approach: Dynamic Programming to build an optimal coding tree.
![Page 6: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/6.jpg)
Top-Down DP for GMR codingGeneralized-Mixed-Radix problemBoth the arity of an internal node and the
length of an edge depends on its level.Given a sequence of arities R = (r1, r2, . . .)
and a sequence of edge lengths C = (c1, c2 . . .), a generalized mixed radix (GMR) tree T satisfying R, C is a tree in which internal node v at level i − 1 has at most ri children and and the length of an edge from v to any of its children is ci
![Page 7: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/7.jpg)
Top-Down DP for GMR codingThe depth of a node on level i of the tree will
be thesum of the lengths of the edges on the path from the root to level i.
Cost(T) =
The problem to be solved is, given P and R,C, find a min-cost tree T∗ with n leaves Cost(T∗) = min {Cost(T) | T has n leaves}
![Page 8: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/8.jpg)
Top-Down Dynamic ProgramIn , ≥ ≥ … ≥ ≥ 0;For i > n, = 0;There is a full tree T∗ with the same cost as
the optimal tree for n leaves. T∗ has n′ leaves where
max(n, r (T∗)) ≤ n′ ≤ n + r (T∗) − 1ℓ ℓThe cost of a tree is fully determined by the
number of leaves on each level.
![Page 9: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/9.jpg)
Top-Down Dynamic ProgramBuild optimal trees top-down, starting with a
root;Expand tree level-by-level by making some
nodes on the bottom level internal.Until a tree whose bottom level contains no
internal nodes.
![Page 10: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/10.jpg)
Top-Down Dynamic Programi-level signature is an ordered pair
in which m = b = |{}| Partial cost of T is
![Page 11: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/11.jpg)
Top-Down Dynamic ProgramP={1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}
![Page 12: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/12.jpg)
Top-Down Dynamic ProgramLet T′ be an (i − 1)-level tree with. Expand T′ to a full i-level tree by adding all nodes on level i.(m’, b’)->(m,b), where m = m’ + b’ri – b(0, 1)->…->(n’, 0)
![Page 13: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/13.jpg)
Top-Down Dynamic ProgramObjective: Complexity? Speedup by batching approach
![Page 14: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/14.jpg)
Batching for Speed up
![Page 15: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/15.jpg)
Batching for Speed up If
Else
![Page 16: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/16.jpg)
Batching for Speed up For fixed i, we can fill all valid signatures (m, b), where m+b=d, in time.d is bounded by n.We can fill all valid signatures in level i in
time.The overall time complexity is
![Page 17: A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding](https://reader035.vdocuments.mx/reader035/viewer/2022062520/568163a5550346895dd4abcb/html5/thumbnails/17.jpg)
Results