ma 252: data structures and algorithms · 2012. 1. 16. · • e.g., insertion sort, merge sort,...
TRANSCRIPT
![Page 1: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/1.jpg)
MA 252: Data Structures and Algorithms
Lecture 9http://www.iitg.ernet.in/psm/indexing_ma252/y12/index.htmlhttp://www.iitg.ernet.in/psm/indexing_ma252/y12/index.html
Partha Sarathi Mandal
Dept. of Mathematics, IIT Guwahati
![Page 2: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/2.jpg)
• We are lucky if the partitioning is balance i.e.,
(n/2, n/2).
• We are unlucky if the partitioning is imbalance
i.e., (1, n-1) or (n-1, 1).
More about quicksort
i.e., (1, n-1) or (n-1, 1).
P. S. Mandal, IITG
![Page 3: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/3.jpg)
More intuition
• Suppose we alternate lucky, unlucky, lucky, unlucky, lucky, ….
• L(n)= 2U(n/2) + Θ(n) lucky
• U(n)= L(n –1) + Θ(n) unlucky• U(n)= L(n –1) + Θ(n) unlucky
Solving:
L(n) = 2(L(n/2 –1) + Θ(n/2)) + Θ(n)
= 2L(n/2 –1) + Θ(n)
= Θ(nlg n)
How can we make sure we are usually lucky ?
P. S. Mandal, IITG
![Page 4: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/4.jpg)
Randomized quicksort
• IDEA: Partition around a random element.
• Running time is independent of the input order.
• No assumptions need to be made about the • No assumptions need to be made about the input distribution.
• No specific input elicits the worst-case behavior.
• The worst case is determined only by the output of a random-number generator.
P. S. Mandal, IITG
![Page 5: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/5.jpg)
Pseudocode for
Randomized quicksort
P. S. Mandal, IITG
![Page 6: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/6.jpg)
Randomized quicksort analysis
• Let T(n) =the random variable for the running time of randomized quicksort on an input of size n, assuming random numbers are independent.
• For k= 0, 1, …, n–1, define the indicator random
variable:variable:
• E[Xk] = 0.Pr{Xk=0} + 1.Pr{Xk=1} = Pr{Xk = 1} = 1/n, since all splits are equally likely, assuming elements are distinct.
P. S. Mandal, IITG
![Page 7: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/7.jpg)
Analysis (continued)
P. S. Mandal, IITG
![Page 8: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/8.jpg)
Calculating expectation
P. S. Mandal, IITG
![Page 9: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/9.jpg)
Calculating expectation
P. S. Mandal, IITG
![Page 10: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/10.jpg)
Calculating expectation
P. S. Mandal, IITG
![Page 11: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/11.jpg)
Calculating expectation
P. S. Mandal, IITG
![Page 12: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/12.jpg)
Calculating expectation
P. S. Mandal, IITG
![Page 13: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/13.jpg)
Exercise
(The k = 0, 1 terms can be absorbed in the Θ(n) )
Exercise: Exercise:
Prove: E[T(n)] ≤ anlg n for constant a > 0.
– Choose a large enough so that anlgn dominates
E[T(n)] for sufficiently small n ≥ 2.
Use fact:
P. S. Mandal, IITG
![Page 14: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/14.jpg)
Substitution method
P. S. Mandal, IITG
![Page 15: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/15.jpg)
Substitution method
P. S. Mandal, IITG
![Page 16: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/16.jpg)
Substitution method
P. S. Mandal, IITG
![Page 17: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/17.jpg)
Substitution method
P. S. Mandal, IITG
![Page 18: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/18.jpg)
Quicksort in practice
• Quicksort is a great general-purpose sorting
algorithm.
• Quicksort is typically over twice as fast as
merge sort.merge sort.
• Quicksort can benefit substantially from code
tuning.
• Quicksort behaves well even with caching and
virtual memory.
P. S. Mandal, IITG
![Page 19: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/19.jpg)
How fast can we sort?
• All the sorting algorithms we have seen so far are comparison sorts: only use comparisons to determine the relative order of elements.
• E.g., insertion sort, merge sort, quicksort, heapsort.heapsort.
• The best worst-case running time that we’ve seen for comparison sorting is O(nlg n).
Is O(nlg n) the best we can do ?
• Decision tree scan help us answer this question.
P. S. Mandal, IITG
![Page 20: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/20.jpg)
Decision-tree example
• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.
• The left subtree shows subsequent comparisons if ai ≤ aj.
• The right subtree shows subsequent comparisons if ai ≥ aj.
P. S. Mandal, IITG
![Page 21: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/21.jpg)
Decision-tree example
• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.
• The left subtree shows subsequent comparisons if ai ≤ aj.
• The right subtree shows subsequent comparisons if ai ≥ aj.
P. S. Mandal, IITG
![Page 22: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/22.jpg)
Decision-tree example
• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.
• The left subtree shows subsequent comparisons if ai ≤ aj.
• The right subtree shows subsequent comparisons if ai ≥ aj.
P. S. Mandal, IITG
![Page 23: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/23.jpg)
Decision-tree example
• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.
• The left subtree shows subsequent comparisons if ai ≤ aj.
• The right subtree shows subsequent comparisons if ai ≥ aj.
P. S. Mandal, IITG
![Page 24: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/24.jpg)
Decision-tree example
• Each leaf contains a permutation ⟨π(1), π(2),…, π(n)⟩
to indicate that the ordering aπ(1) ≤ aπ(2) ≤ … ≤ aπ(n) has
been established.
P. S. Mandal, IITG
![Page 25: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/25.jpg)
Decision-tree model
A decision tree can model the execution of any comparison sort:
• One tree for each input size n.
• View the algorithm as splitting whenever it compares two elements.compares two elements.
• The tree contains the comparisons along all possible instruction traces.
• The running time of the algorithm = the length of the path taken.
• Worst-case running time = height of tree.
P. S. Mandal, IITG
![Page 26: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/26.jpg)
Lower bound for decision-tree sorting
Theorem: Any decision tree that can sort nelements must have height Ω(nlg n).
Proof: The tree must contain ≥ n! leaves, since there are n! possible permutations. A height-h binary tree has ≤ 2h leaves. Thus, n! ≤ 2h.
∴
tree has ≤ 2h leaves. Thus, n! ≤ 2h.
∴ h ≥ lg (n!) (lg is monotonically increasing)
≥ lg ((n/e)n) (Stirling’s formula)
= nlg n – nlg e
= Ω(nlg n)
P. S. Mandal, IITG
![Page 27: MA 252: Data Structures and Algorithms · 2012. 1. 16. · • E.g., insertion sort, merge sort, quicksort, heapsort. • The best worst-case running time that we’ve seen for comparison](https://reader033.vdocuments.mx/reader033/viewer/2022051811/601c231b243b1f7f6f57845e/html5/thumbnails/27.jpg)
Lower bound for comparison sorting
Corollary: Heapsort and merge sort are
asymptotically optimal comparison sorting
algorithms.
P. S. Mandal, IITG