Chapter 7 SortingChapter 7 SortingPart I

7.1 Motivation7.1 Motivationlist: a collection of records.keys: the fields used to

distinguish among the records.One way to search for a record

with the specified key is to examine the list in left-to-right or right-to-left order.◦Sequential search.

Sequential SearchSequential Search

To search the key ‘22’:

int SeqSearch(int a[], int n, key) { int i; for (i=0; i<n && a[i] != key; i++) ; if (i >= n) return -1; return i; }

int SeqSearch(int a[], int n, key) { int i; for (i=0; i<n && a[i] != key; i++) ; if (i >= n) return -1; return i; }

23

0

2

1

7

215

342

412

5

The search makes n key comparisons when it is unsuccessful.

Analysis of Time Analysis of Time ComplexityComplexityWorst case:

◦O(n) when the search is unsuccessful.

◦Each element is examined exactly once.

Average case:◦When the search is successful, the

number of comparison depends on the position of the search key.

2

11

2

)1(

1

n

n

nnni

ni

Binary SearchBinary Searchint BinarySearch(int a[], int n, in key){ int left = 0, right = n-1; while (left <= right) { int middle = (left + right) / 2; if (key < a[middle]) right = middle - 1; else if (key > a[middle]) left = middle + 1; else return middle; } return -1;}

int BinarySearch(int a[], int n, in key){ int left = 0, right = n-1; while (left <= right) { int middle = (left + right) / 2; if (key < a[middle]) right = middle - 1; else if (key > a[middle]) left = middle + 1; else return middle; } return -1;}

2

0

7

112

215

323

442

5

left right

middle

To find 23, middle

found.

Binary SearchBinary Search

Even when the search is unsuccessful, the time complexity is still O(log n).◦Something is to be gained by

maintaining the list in an order manner.

Sorting MethodsSorting MethodsInternal:

◦Can be carried out in memory. Insertion sort. Quick sort. Merge sort. Heap sort. Radix sort.

External:◦The dataset is much more bigger so

the data cannot be fully carried out in memory.

7.2 INSERTION SORT7.2 INSERTION SORT

IdeaIdeaConsider how to insert a new

integer into a sorted array so that all the elements in the array are sorted.

0

0

1

1

5

2

6

3

7

4

9

5

11

6

12

7

23

8 9

8

9 11 12 238

Insertion into a Sorted ListInsertion into a Sorted ListSuppose a is an integer array with n

elements. void Insert(int key, int a[], int i);

◦ Insert a new value, key, into the first i integers in a, where the first i integers should be sorted.void Insert(int key, int a[], int i){ int j; for (j = i-1; j >= 0 && key < a[j]; j--) a[j+1] = a[j]; a[j+1] = key;}

void Insert(int key, int a[], int i){ int j; for (j = i-1; j >= 0 && key < a[j]; j--) a[j+1] = a[j]; a[j+1] = key;}

Insertion SortInsertion SortAt the ith iteration of this algorithm,

the first i elements in the original array will be sorted.

12

0

1

1

6

2

0

3

7

4

23

5

11

6

9

7

5

8

i = 1

1

12a:

temp:

1

i = 2

6

void InsertionSort(int key, int a[], int i){ for (j = 1; j < n; j++) int temp = a[j] Insert(temp, a, j-1);}

void InsertionSort(int key, int a[], int i){ for (j = 1; j < n; j++) int temp = a[j] Insert(temp, a, j-1);}

Analysis of InsertionSort()Analysis of InsertionSort()Worst case

◦As each new record is inserted into the sorted part of the list, the entire sorted part is shifted right by one position.

j 0 1 2 3 4

- 5 4 3 2 1

1 4 5 3 2 1

2 3 4 5 2 1

3 2 3 4 5 1

4 1 2 3 4 5

Analysis of InsertionSort()Analysis of InsertionSort()In worst case, InsertSort() makes O(i) comparison before a[i] is inserted.

For i=1, 2, …, n-1. The time complexity of InsertionSort() is

◦In fact, insertion sort is about the fastest sorting method for small n (n ≦ 30).

)(2

)1( 21

1

nOnn

iOn

i

VariationsVariationsBinary Insertion Sort:

◦To reduce the number of comparison.◦Therefore, apply binary search in

Insert().Linked Insertion Sort

◦The elements of the list are represented as a linked list. The number of record move can become

zero because only link fields require adjustment.

7.3 QUICK SORT7.3 QUICK SORT

IntroductionIntroductionQuick sort has the best average

behavior among the sorting methods.

Concept:a:

pivot

Find a pivot from a.

pivot

a’: ≧pivot≦ pivot

Quick Sort Quick Sort

DefinitionDefinitionleft and right are used to indicate the

scope of the array.◦ left should be less than right; if not, return

directly.a[left] is initially chosen as pivot.

4 1 5 1 2 6 4

left right

pivot

i j

i: examined as index to scan the array from left to right. j: examined as index to scan the array from right to left.

Initially, i = left, j=right+1.

ExampleExample

4 1 5 1 2 6 4

left right

pivot

i j

5 > pivot and should go to the other side.

2 < pivot and should go to the other side.

Interchange a[i] and a[j]

2 5

Stop.

a[j] will eventually stop at a position where a[j] < pivot.

Interchange a[j] and pivot.

1 44

AlgorithmAlgorithm

void QuickSort(int a[], int left, int right) { if (left < right) { pivot = a[left]; i = left; j = right+1; while (i < j) { for (i++; i<=j and a[i] < pivot; i++) ; for (j++; i>=j and a[i] >= pivot; i++) ; if (i < j) interchages a[i] amd [i]; } QuickSort(a, left, j-1); QuickSort(a, j+1 right;); } }

void QuickSort(int a[], int left, int right) { if (left < right) { pivot = a[left]; i = left; j = right+1; while (i < j) { for (i++; i<=j and a[i] < pivot; i++) ; for (j++; i>=j and a[i] >= pivot; i++) ; if (i < j) interchages a[i] amd [i]; } QuickSort(a, left, j-1); QuickSort(a, j+1 right;); } }

Analysis of QuickSort()Analysis of QuickSort()Best case

◦If the pivot is correctly positioned (left size = right size), the time complexity is

)log(

)1(log

......

)4/(42))4/(22/(2

.constant somefor ),2/(2)(

2

nnO

nTncn

ntcnnTcncn

cnTcnnT

Analysis of QuickSort()Analysis of QuickSort()Worst case:

◦Consider a list is stored.

The smallest one is always chosen as pivot.

n iterations are required to reach base case (left >= right). Each iteration takes O(n) time.

The time complexity is O(n2);

1 642 5

Lemma 7.1Lemma 7.1Let Tavg(n) be the expect time for

function QuickSort() to sort a list with n records. Then there exists a constant k such that Tavg(n)≦knlogen for n≧2.

Lemma 7.1Lemma 7.1Proof:

◦Assume pivot is at j. ◦The expected time to sort two sides

is

◦The average time is

)()1( jnTjT avgavg

n

javgavgavg jnTjT

ncnnT

1

))()1((1

)(

jj-1 n-j

where c is a constant.

1

0

1

))((2

))0(...)2()1(())1(...)1()0((1

))()1((1

n

javg

avgavgavgavgavgavg

n

javgavg

jTn

TnTnTnTTTn

jnTjTn

1

01

.2for )(2

))()1((1

)(n

javg

n

javgavgavg njT

ncnjnTjT

ncnnT

)(2 and 2for ln)(

prove toinduction Use

.constant somefor )1( and )0( Assume

cbknnknnT

bbTbT

avg

avgavg

Induction base: for n = 2

Induction hypothesis:◦Assume that

Induction step

2ln222)2( kbcTavg

.1for ln)( mnnknnTavg

1

2

1

2

1

0

ln24

)(24

)(2

)(

m

j

m

javg

m

javgavg

jjm

k

m

bcmjT

mm

bcm

jTm

cmmT

m

avg xdxxm

k

m

bcmmT

jjj

2ln

24)(

, offunction increasingan is ln Since

42

ln

4ln

2

1ln

4ln

2

1

2

1ln

2

1ln

2

1.vLet

1.lnLet

. Use

22

2

22

2

222

2

mmmxxxxdxx

xxxxdxxxdxxx

xvxdxd

dxx

duxu

vduuvudv

mm

2for ,ln2

ln4

ln24

)(2

mmkm

kmmkm

m

bcm

xdxxm

k

m

bcmmT

m

avg