sorting algorithms bubble sort merge sort quick sort randomized quick sort

Sorting Algorithms

Bubble SortMerge Sort

Quick SortRandomized Quick Sort

2

Overview

Bubble Sort

Divide and Conquer

Merge Sort

Quick Sort

Randomization

3

Sorting

Sorting takes an unordered collection and makes it an ordered one.

512354277 101

1 2 3 4 5 6

5 12 35 42 77 101

1 2 3 4 5 6

4

"Bubbling Up" the Largest Element Traverse a collection of elements

Move from the front to the end “Bubble” the largest value to the end using

pair-wise comparisons and swapping

512354277 101

1 2 3 4 5 6

5




512354277 101

1 2 3 4 5 6

Swap42 77

6




512357742 101

1 2 3 4 5 6

Swap35 77

7




512773542 101

1 2 3 4 5 6

Swap12 77

8




577123542 101

1 2 3 4 5 6

No need to swap

9




577123542 101

1 2 3 4 5 6

Swap5 101

10




77123542 5

1 2 3 4 5 6

101

Largest value correctly placed

11

The “Bubble Up” Algorithm

index <- 1last_compare_at <- n – 1

loop exitif(index > last_compare_at) if(A[index] > A[index + 1]) then Swap(A[index], A[index + 1]) endif index <- index + 1endloop

12

No, Swap isn’t built in.

Procedure Swap(a, b isoftype in/out Num)

t isoftype Num

t <- a

a <- b

b <- t

endprocedure // Swap

13

Items of Interest

Notice that only the largest value is correctly placed

All other values are still out of order So we need to repeat this process

77123542 5

1 2 3 4 5 6

101

Largest value correctly placed

14

Repeat “Bubble Up” How Many Times?

If we have N elements…

And if each time we bubble an element, we place it in its correct location…

Then we repeat the “bubble up” process N – 1 times.

This guarantees we’ll correctly place all N elements.

15

“Bubbling” All the Elements

77123542 51 2 3 4 5 6

101

5421235 771 2 3 4 5 6

101

42 5 3512 771 2 3 4 5 6

101

42 35 512 771 2 3 4 5 6

101

42 35 12 5 771 2 3 4 5 6

101

N -

1

16

Reducing the Number of Comparisons

12 35 42 77 1011 2 3 4 5 6

5

77123542 51 2 3 4 5 6

101

5421235 771 2 3 4 5 6

101

42 5 3512 771 2 3 4 5 6

101

42 35 512 771 2 3 4 5 6

101

17

Reducing the Number of Comparisons

On the Nth “bubble up”, we only need to do MAX-N comparisons.

For example: This is the 4th “bubble up” MAX is 6 Thus we have 2 comparisons to do

42 5 3512 771 2 3 4 5 6

101

18

N is … // Size of Array

Arr_Type definesa Array[1..N] of Num

Procedure Swap(n1, n2 isoftype Num)

temp isoftype Num

temp <- n1

n1 <- n2

n2 <- temp

endprocedure // Swap

Putting It All Together

19

The Truth

NOBODY EVER uses Bubble Sort (except for Ezra)

NOBODY

NOT EVER

Because it is EXTREMELY INEFFICIENT

20

This is how it goes…

21

Comparison (semi-log y axis)

22

Let’s forget about Bubble Sort

23

Divide and Conquer

1. Base Case, solve the problem directly if it is small enough

1. Divide the problem into two or more similar and smaller subproblems

1. Recursively solve the subproblems

1. Combine solutions to the subproblems

24

Divide and Conquer - Sort

Problem: Input: A[left..right] – unsorted array of integers

Output: A[left..right] – sorted in non-decreasing

order

25

Divide and Conquer - Sort1. Base case

at most one element (left ≥ right), return

2. Divide A into two subarrays: FirstPart, SecondPart Two Subproblems:

sort the FirstPart sort the SecondPart

3. Recursively sort FirstPart sort SecondPart

4. Combine sorted FirstPart and sorted SecondPart

26

This reminds me of…

That’s easy. Mariah Carey.

As a singing star, Ms. Carey has perfected the “wax-on” wavemotion--a clockwise sweep of her hand used to emphasize lyrics.

The Maria “Wax-on” Angle:

(,t)The Siren of Subquadratic SortsThe Siren of Subquadratic Sorts

27

How To Remember Merge Sort?

Just as Mariah recursively moves Just as Mariah recursively moves her hands into smaller circles, so her hands into smaller circles, so too does merge sort recursively too does merge sort recursively split an array into smaller split an array into smaller segments. segments.

(,t)

We need two such recursions, We need two such recursions, one for each half of the split one for each half of the split array.array.

28

Overview

Divide and Conquer

Merge Sort

Quick Sort

Randomization

29

Merge Sort: Idea

Merge

Recursively sort

Divide intotwo halves

FirstPart SecondPart

FirstPart SecondPart

A

A is sorted!

30

Merge Sort: Algorithm

Merge-Sort (A, left, right)

if left ≥ right return

else

middle ← b(left+right)/2

Merge-Sort(A, left, middle)

Merge-Sort(A, middle+1, right)

Merge(A, left, middle, right)

Recursive Call

31A[middle]A[middle]A[left]A[left]

SortedSorted FirstPartFirstPart

Sorted Sorted SecondPartSecondPart

Merge-Sort: Merge

A[right]A[right]

mergemerge

A:A:

A:A:

SortedSorted

32

6 10 14 223 5 15 28

L:L: R:R:

Temporary ArraysTemporary Arrays

5 15 28 30 6 10 145

Merge-Sort: Merge Example

2 3 7 8 1 4 5 6A:A:

33


3 5 15 28 30 6 10 14

L:L:

A:A:

3 15 28 30 6 10 14 22

R:R:

i=0 j=0

k=0

2 3 7 8 1 4 5 6

1

34


1 5 15 28 30 6 10 14

L:L:

A:A:

3 5 15 28 6 10 14 22

R:R:

k=1

2 3 7 8 1 4 5 6

2

i=0 j=1

35


1 2 15 28 30 6 10 14

L:L:

A:A:

6 10 14 22

R:R:

i=1

k=2

2 3 7 8 1 4 5 6

3

j=1

36


1 2 3 6 10 14

L:L:

A:A:

6 10 14 22

R:R:

i=2 j=1

k=3

2 3 7 8 1 4 5 6

4

37


1 2 3 4 6 10 14

L:L:

A:A:

6 10 14 22

R:R:

j=2

k=4

2 3 7 8 1 4 5 6

i=2

5

38


1 2 3 4 5 6 10 14

L:L:

A:A:

6 10 14 22

R:R:

i=2 j=3

k=5

2 3 7 8 1 4 5 6

6

39


1 2 3 4 5 6 14

L:L:

A:A:

6 10 14 22

R:R:

k=6

2 3 7 8 1 4 5 6

7

i=2 j=4

40


1 2 3 4 5 6 7 14

L:L:

A:A:

3 5 15 28 6 10 14 22

R:R:2 3 7 8 1 4 5 6

8

i=3 j=4

k=7

41


1 2 3 4 5 6 7 8

L:L:

A:A:

3 5 15 28 6 10 14 22

R:R:2 3 7 8 1 4 5 6

i=4 j=4

k=8

42

Merge(A, left, middle, right)1. n1 ← middle – left + 1

2. n2 ← right – middle

3. create array L[n1], R[n2]

4. for i ← 0 to n1-1 do L[i] ← A[left +i]

5. for j ← 0 to n2-1 do R[j] ← A[middle+j]6. k ← i ← j ← 0

7. while i < n1 & j < n2 8. if L[i] < R[j] 9. A[k++] ← L[i++]10. else11. A[k++] ← R[j++]

12. while i < n1

13. A[k++] ← L[i++]

14. while j < n2

15. A[k++] ← R[j++]n = n1+n2

Space: n

Time : cn for some constant c

43

6 2 8 4 3 7 5 16 2 8 4

3 7 5 1

Merge-Sort(A, 0, 7)

DivideA:

44

6 2 8 4

3 7 5 1

6 2

8 4

Merge-Sort(A, 0, 3) , divideA:

Merge-Sort(A, 0, 7)

45

3 7 5 1

8 4

6 26 2

Merge-Sort(A, 0, 1), divideA:

Merge-Sort(A, 0, 7)

46

3 7 5 1

8 4

6

2

Merge-Sort(A, 0, 0) , base caseA:

Merge-Sort(A, 0, 7)

47

3 7 5 1

8 4

6 2

Merge-Sort(A, 0, 0), returnA:

Merge-Sort(A, 0, 7)

48

3 7 5 1

8 4

6

2

Merge-Sort(A, 1, 1), base case

A:

Merge-Sort(A, 0, 7)

49

3 7 5 1

8 4

6 2


Merge-Sort(A, 0, 7)

50

3 7 5 1

8 4

2 6

Merge(A, 0, 0, 1)A:

Merge-Sort(A, 0, 7)

51

3 7 5 1

8 4

2 6


Merge-Sort(A, 0, 7)

52

3 7 5 1

8 4

2 6

Merge-Sort(A, 2, 3)

48

, divideA:

Merge-Sort(A, 0, 7)

53

3 7 5 1

4

2 6

8

Merge-Sort(A, 2, 2), base caseA:

Merge-Sort(A, 0, 7)

54

3 7 5 1

4

2 6

8


Merge-Sort(A, 0, 7)

55

4

2 6

8

Merge-Sort(A, 3, 3), base caseA:

Merge-Sort(A, 0, 7)

56

3 7 5 1

4

2 6

8


Merge-Sort(A, 0, 7)

57

3 7 5 1

2 6

4 8

Merge(A, 2, 2, 3)A:

Merge-Sort(A, 0, 7)

58

3 7 5 1

2 6

4 8


Merge-Sort(A, 0, 7)

59

3 7 5 1

2 4 6 8

Merge(A, 0, 1, 3)A:

Merge-Sort(A, 0, 7)

60

3 7 5 1

2 4 6 8


Merge-Sort(A, 0, 7)

61

3 7 5 1

2 4 6 8

Merge-Sort(A, 4, 7)A:

Merge-Sort(A, 0, 7)

62

1 3 5 7

2 4 6 8

A:

Merge (A, 4, 5, 7)

Merge-Sort(A, 0, 7)

63

1 3 5 7

2 4 6 8


Merge-Sort(A, 0, 7)

64

1 2 3 4 5 6 7 8

Merge(A, 0, 3, 7)A:

Merge-Sort(A, 0, 7)

Merge-Sort(A, 0, 7), done!

65

Merge-Sort Analysis

cn

2 × cn/2 = cn

4 × cn/4 = cn

n/2 × 2c = cn

log n levels

• Total running time: (nlogn)• Total Space: (n)

Total: cn log n

n

n/2 n/2

n/4 n/4 n/4 n/4

2 2 2

66

Merge-Sort Summary

Approach: divide and conquer

Time Most of the work is in the merging Total time: (n log n)

Space: (n), more space than other sorts.

67

Overview

Divide and Conquer

Merge Sort

Quick Sort

68

Quick Sort Divide:

Pick any element p as the pivot, e.g, the first element

Partition the remaining elements into FirstPart, which contains all elements < p

SecondPart, which contains all elements ≥ p

Recursively sort the FirstPart and SecondPart

Combine: no work is necessary since sorting is done in place

69

Quick Sort

x < p p p ≤ x

PartitionFirstPart SecondPart

p

pivot

A:

Recursive call

x < p p p ≤ x

SortedFirstPart

SortedSecondPart

SortedSorted

70

Quick Sort

Quick-Sort(A, left, right)

if left ≥ right return

else

middle ← Partition(A, left, right)

Quick-Sort(A, left, middle–1 )

Quick-Sort(A, middle+1, right)

end if

71

Partition

p

p x < p p ≤ x

p p ≤ xx < p

A:A:

A:A:

A:A:p

72

Partition Example

A:A: 4 8 6 3 5 1 7 2

73

Partition Example

A:A: 4 8 6 3 5 1 7 2

i=0i=0

j=1j=1

74

Partition Example

A:A:

j=1j=1

4 8 6 3 5 1 7 2

i=0i=0

8

75

Partition Example

A:A: 4 8 6 3 5 1 7 26

i=0i=0

j=2j=2

76

Partition Example

A:A: 4 8 6 3 5 1 7 2

i=0i=0

383

j=3j=3

i=1i=1

77

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=1i=1

5

j=4j=4

78

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=1i=1

1

j=5j=5

79

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=2i=2

1 6

j=5j=5

80

Partition Example

A:A: 4 3 8 5 7 2

i=2i=2

1 6 7

j=6j=6

81

Partition Example

A:A: 4 3 8 5 7 2

i=2i=2

1 6 22 8

i=3i=3

j=7j=7

82

Partition Example

A:A: 4 3 2 6 7 8

i=3i=3

1 5

j=8j=8

83

Partition Example

A:A: 4 1 6 7 8

i=3i=3

2 542 3

84

A:A: 3 6 7 81 542

x < 4x < 4 4 4 ≤≤ x x

pivot incorrect position

Partition Example

85

Partition(A, left, right)

1. x ← A[left]

2. i ← left

3. for j ← left+1 to right

4. if A[j] < x then

5. i ← i + 1

6. swap(A[i], A[j])

7. end if

8. end for j

9. swap(A[i], A[left])

10. return in = right – left +1 Time: cn for some constant c Space: constant

86

4 8 6 3 5 1 7 22 3 1

5 6 7 8

4

Quick-Sort(A, 0, 7)Partition

A:

87

2 3 1

5 6 7 8

4

2

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 2)

A:

, partition

88

2

5 6 7 8

4

1

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 0) , base case, return

89

2

5 6 7 8

4

1

33

Quick-Sort(A, 0, 7)Quick-Sort(A, 1, 1) , base case

90

5 6 7 8

42

1 3

2

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 2, 2), returnQuick-Sort(A, 0, 2), return

91

42

1 3

5 6 7 8

6 7 85

Quick-Sort(A, 0, 7)Quick-Sort(A, 2, 2), returnQuick-Sort(A, 4, 7) , partition

92

4

5

6 7 8

7 8

66

2

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 5, 7) , partition

93

4

5

6

7 8

87

2

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 6, 7) , partition

94

4

5

6

7

2

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 7, 7)

8

, return, base case

8

95

4

5

6 87

2

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 6, 7) , return

96

4

5

2

1 3


6 87

97

42

1 3


5 6 87

98

42

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 7) , done!

5 6 87

99

Quick-Sort: Best Case Even Partition

Total time: (nlogn)

cn

2 × cn/2 = cn

4 × c/4 = cn

n/3 × 3c = cn

log n levels

n

n/2 n/2

n/4

3 3 3

n/4n/4n/4

100

cn

c(n-1)

3c

2c

n

n-1

n-2

3

2

c(n-2)

Happens only if input is sortd input is reversely sorted

Quick-Sort: Worst Case Unbalanced Partition

Total time: (n2)

101

Randomized Quick Sort

102

Quick-Sort: an Average Case Suppose the split is 1/10 : 9/10

Quick-Sort: an Average Case

cn

cn

cn

≤cn

n

0.1n 0.9n

0.01n 0.09n 0.09n

Total time: (nlogn)

0.81n

2

2

log10n

log10/9n

≤cn

103

Quick-Sort Summary

Time Most of the work done in partitioning. Average case takes (n log(n)) time. Worst case takes (n2) time

Space Sorts in-place, i.e., does not require additional

space

104

Summary Divide and Conquer

Merge-Sort Most of the work done in Merging (n log(n)) time (n) space

Quick-Sort Most of the work done in partitioning Average case takes (n log(n)) time Worst case takes (n2) time (1) space

105

Quiz 6 (Show output)public static void main(String[ ] args) {

int[ ] array = {4,18,16,3,5,21,7};recursiveQuickSort(array, 0, array.length-1);System.out.println("The following array should be sorted: ");printList(array);System.exit(0);

}

public static void recursiveQuickSort(int[ ] list, int first, int last) {if(first < last){

int p = partition(list, first, last);printList(list);recursiveQuickSort(list, first, p-1);recursiveQuickSort(list, p+1, last);

}}

Assume that printlist(list) prints the list separated by commas.

sorting algorithms bubble sort merge sort quick sort randomized quick sort

Documents

end bubble

largest value

largest element

collection of elements

pairwise comparisons

placed slide

overview bubble sort

unordered collection