Design and Analysis of AlgorithmsNon-comparison sort (sorting in linear time)

Haidong XueSummer 2012, at GSU

Comparison based sorting

• Algorithms that determine sorted order based only on comparisons between the input elements

Algorithm Worst Time Expected Time Extra Memory Stable

Insertion sort O(1) (in place) Can be

Merge sort O(n) Can be

Quick sort O(1) (in place) Can be

Heap sort O(1) (in place) No

What is the lower bound?

Lower bounds for comparison based sorting

<?

<? <?

<?

Y N

Y

……

…………………………..

Done

N

Done

Y

For n element array, how many possible inputs are there?

Factorial of n ----- n!

What is the shortest tree can have n! leaves?

A perfect tree,

As a result ……

h≥ lg (𝑛! )=Ω(𝑛𝑙𝑔𝑛)

Sorting in linear time

• Can we sort an array in linear time?• Yes, but not for free• E.g. sort cards with 13 slots• What if there are more than one elements in

the same slot?

Counting Sort

• Input: array A[1, … , n]; k (elements in A have values from 1 to k)

• Output: sorted array AAlgorithm: 1. Create a counter array C[1, …, k]2. Create an auxiliary array B[1, …, n]3. Scan A once, record element frequency in C4. Calculate prefix sum in C5. Scan A in the reverse order, copy each element to B at the

correct position according to C.6. Copy B to A

Counting Sort2 5 3 6 2 3 7 3A:

C: 1 2 3 4 5 6

0 2 3 0 1 1

7

1

1 2 3 4 5 6 7 8

B:1 2 3 4 5 6 7 8

2 55 76 8Position indicator: 0

3

4

7

7

3

3

2

1

6352

2 650

Analysis of Counting Sort

• Input: array A[1, … , n]; k (elements in A have values from 1 to k)

• Output: sorted array AAlgorithm: 1. Create a counter array C[1, …, k]2. Create an auxiliary array B[1, …, n]3. Scan A once, record element frequency in C4. Calculate prefix sum in C5. Scan A in the reverse order, copy each element to B at the

correct position according to C.6. Copy B to A

Time

O(n)

O(k)

O(n)

O(n+k)=O(n) (if k=O(n))

Space

O(k)

O(n)

O(n)

O(n+k)=O(n) (if k=O(n))

Radix-Sort

• Input: array A[1, … , n]; d (number of digit a element has)

• Output: sorted array AAlgorithm: for each digit{ use a stable sort to sort A on a digit}

T(n)=O(d(n+k))

SummaryAlgorithm Worst Time Expected Time Extra Memory Stable

Insertion sort O(1) (in place) Yes

Merge sort O(n) Yes

Quick sort O(1) (in place) Yes

Heap sort O(1) (in place) No

Counting sort Yes

Design strategies:

Divide and conquer

Employ certain special data structure

Tradeoff between time and space

Knowledge treeAlgorithms

Analysis DesignAlgorithms for

classic problemsClassic data

structure

Asymptotic notations

Probabilistic analysis

Sorting Shortest path

Matrix multiplication …

Divide & Conquer

GreedyDynamic

Programming

O(), o(), (), (), ()

Heap,Hashing,

Binary Tree,RBT,….

Quicksort,Heapsort,

Mergesort,…

… … …… … …

… … … … … … … … … … … ……