sorting algorithms merge sort quick sort hairong zhao new jersey institute of technology

Sorting Algorithms

Merge Sort Quick Sort

Hairong Zhaohttp://web.njit.edu/~hz2

New Jersey Institute of Technology

2

Overview Divide and Conquer

Merge Sort

Quick Sort

3

Divide and Conquer1. Base Case, solve the problem directly

if it is small enough

2. Divide the problem into two or more similar and smaller subproblems

3. Recursively solve the subproblems

4. Combine solutions to the subproblems

4

Divide and Conquer - SortProblem: Input: A[left..right] – unsorted array of integers Output: A[left..right] – sorted in non-decreasing

order

5

Divide and Conquer - Sort1. Base case

at most one element (left ≥ right), return

2. Divide A into two subarrays: FirstPart, SecondPart Two Subproblems:

sort the FirstPart sort the SecondPart

3. Recursively sort FirstPart sort SecondPart

4. Combine sorted FirstPart and sorted SecondPart

6


Merge Sort

Quick Sort

7

Merge Sort: Idea

Merge

Recursively sort

Divide intotwo halves FirstPart SecondPart

FirstPart SecondPart

A

A is sorted!

8

Merge Sort: AlgorithmMerge-Sort (A, left, right)

if left ≥ right return

else

middle ← b(left+right)/2

Merge-Sort(A, left, middle)

Merge-Sort(A, middle+1, right)

Merge(A, left, middle, right)

Recursive Call

9A[middle]A[middle]A[left]A[left]

SortedSorted FirstPartFirstPart

Sorted Sorted SecondPartSecondPart

Merge-Sort: Merge

A[right]A[right]

mergemerge

A:A:

A:A:

SortedSorted

10

6 10 14 223 5 15 28L:L: R:R:

Temporary ArraysTemporary Arrays

5 15 28 30 6 10 145

Merge-Sort: Merge Example

2 3 7 8 1 4 5 6A:A:

11


3 5 15 28 30 6 10 14

L:L:

A:A:

3 15 28 30 6 10 14 22R:R:

i=0 j=0

k=0

2 3 7 8 1 4 5 6

1

12


1 5 15 28 30 6 10 14

L:L:

A:A:

3 5 15 28 6 10 14 22R:R:

k=1

2 3 7 8 1 4 5 6

2

i=0 j=1

13


1 2 15 28 30 6 10 14

L:L:

A:A:

6 10 14 22R:R:

i=1

k=2

2 3 7 8 1 4 5 6

3

j=1

14


1 2 3 6 10 14

L:L:

A:A:

6 10 14 22R:R:

i=2 j=1

k=3

2 3 7 8 1 4 5 6

4

15


1 2 3 4 6 10 14

L:L:

A:A:

6 10 14 22R:R:

j=2

k=4

2 3 7 8 1 4 5 6

i=2

5

16


1 2 3 4 5 6 10 14

L:L:

A:A:

6 10 14 22R:R:

i=2 j=3

k=5

2 3 7 8 1 4 5 6

6

17


1 2 3 4 5 6 14

L:L:

A:A:

6 10 14 22R:R:

k=6

2 3 7 8 1 4 5 6

7

i=2 j=4

18


1 2 3 4 5 6 7 14

L:L:

A:A:

3 5 15 28 6 10 14 22R:R:

2 3 7 8 1 4 5 6

8

i=3 j=4

k=7

19


1 2 3 4 5 6 7 8

L:L:

A:A:

3 5 15 28 6 10 14 22R:R:

2 3 7 8 1 4 5 6

i=4 j=4

k=8

20

Merge(A, left, middle, right)1. n1 ← middle – left + 12. n2 ← right – middle3. create array L[n1], R[n2]4. for i ← 0 to n1-1 do L[i] ← A[left +i]5. for j ← 0 to n2-1 do R[j] ← A[middle+j]6. k ← i ← j ← 07. while i < n1 & j < n2 8. if L[i] < R[j] 9. A[k++] ← L[i++]10. else11. A[k++] ← R[j++]12. while i < n1

13. A[k++] ← L[i++]14. while j < n2

15. A[k++] ← R[j++] n = n1+n2

Space: n

Time : cn for some constant c

21

6 2 8 4 3 7 5 16 2 8 4 3 7 5 1

Merge-Sort(A, 0, 7)Divide

A:

22

6 2 8 4

3 7 5 1

6 2 8 4

Merge-Sort(A, 0, 3) , divideA:

Merge-Sort(A, 0, 7)

23

3 7 5 1

8 4

6 26 2

Merge-Sort(A, 0, 1), divideA:

Merge-Sort(A, 0, 7)

24

3 7 5 1

8 4

6

2

Merge-Sort(A, 0, 0) , base caseA:

Merge-Sort(A, 0, 7)

25

3 7 5 1

8 4

6 2

Merge-Sort(A, 0, 0), returnA:

Merge-Sort(A, 0, 7)

26

3 7 5 1

8 4

6

2

Merge-Sort(A, 1, 1), base caseA:

Merge-Sort(A, 0, 7)

27

3 7 5 1

8 4

6 2


Merge-Sort(A, 0, 7)

28

3 7 5 1

8 4

2 6

Merge(A, 0, 0, 1)A:

Merge-Sort(A, 0, 7)

29

3 7 5 1

8 42 6


Merge-Sort(A, 0, 7)

30

3 7 5 1

8 4

2 6

Merge-Sort(A, 2, 3)

48

, divideA:

Merge-Sort(A, 0, 7)

31

3 7 5 1

4

2 6

8


Merge-Sort(A, 0, 7)

32

3 7 5 1

4

2 6

8


Merge-Sort(A, 0, 7)

33

4

2 6

8


Merge-Sort(A, 0, 7)

34

3 7 5 1

4

2 6

8


Merge-Sort(A, 0, 7)

35

3 7 5 1

2 6

4 8

Merge(A, 2, 2, 3)A:

Merge-Sort(A, 0, 7)

36

3 7 5 1

2 6 4 8


Merge-Sort(A, 0, 7)

37

3 7 5 1

2 4 6 8

Merge(A, 0, 1, 3)A:

Merge-Sort(A, 0, 7)

38

3 7 5 12 4 6 8Merge-Sort(A, 0, 3), return

A:

Merge-Sort(A, 0, 7)

39

3 7 5 1

2 4 6 8Merge-Sort(A, 4, 7)

A:

Merge-Sort(A, 0, 7)

40

1 3 5 7

2 4 6 8A:Merge (A, 4, 5, 7)

Merge-Sort(A, 0, 7)

41

1 3 5 72 4 6 8Merge-Sort(A, 4, 7), return

A:

Merge-Sort(A, 0, 7)

42

1 2 3 4 5 6 7 8Merge(A, 0, 3, 7)

A:

Merge-Sort(A, 0, 7)Merge-Sort(A, 0, 7), done!

43

Merge-Sort Analysis cn

2 × cn/2 = cn

4 × cn/4 = cn

n/2 × 2c = cn

log n levels

• Total running time: (nlogn)• Total Space: (n)

Total: cn log n

n

n/2 n/2

n/4 n/4 n/4 n/4

2 2 2

44

Merge-Sort SummaryApproach: divide and conquerTime

Most of the work is in the merging Total time: (n log n)

Space: (n), more space than other sorts.

45


Merge Sort

Quick Sort

46

Quick Sort Divide:

Pick any element p as the pivot, e.g, the first element

Partition the remaining elements into FirstPart, which contains all elements < pSecondPart, which contains all elements ≥ p

Recursively sort the FirstPart and SecondPart

Combine: no work is necessary since sorting is done in place

47

Quick Sort

x < p p p ≤ x

PartitionFirstPart SecondPart

ppivot

A:

Recursive call

x < p p p ≤ x

SortedFirstPart

SortedSecondPart

SortedSorted

48

Quick SortQuick-Sort(A, left, right)if left ≥ right return

else middle ← Partition(A, left,

right) Quick-Sort(A, left, middle–1 ) Quick-Sort(A, middle+1, right)

end if

49

Partition

p

p x < p p ≤ x

p p ≤ xx < p

A:A:

A:A:

A:A:p

50

Partition Example

A:A: 4 8 6 3 5 1 7 2

51

Partition Example

A:A: 4 8 6 3 5 1 7 2

i=0i=0

j=1j=1

52

Partition Example

A:A:

j=1j=1

4 8 6 3 5 1 7 2

i=0i=0

8

53

Partition Example

A:A: 4 8 6 3 5 1 7 26

i=0i=0

j=2j=2

54

Partition Example

A:A: 4 8 6 3 5 1 7 2

i=0i=0

383

j=3j=3

i=1i=1

55

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=1i=1

5

j=4j=4

56

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=1i=1

1

j=5j=5

57

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=2i=2

1 6

j=5j=5

58

Partition Example

A:A: 4 3 8 5 7 2

i=2i=2

1 6 7

j=6j=6

59

Partition Example

A:A: 4 3 8 5 7 2

i=2i=2

1 6 22 8

i=3i=3

j=7j=7

60

Partition Example

A:A: 4 3 2 6 7 8

i=3i=3

1 5

j=8j=8

61

Partition Example

A:A: 4 1 6 7 8

i=3i=3

2 542 3

62

A:A: 3 6 7 81 542

x < 4x < 4 4 ≤ x4 ≤ x

pivot incorrect position

Partition Example

63

Partition(A, left, right)1. x ← A[left]2. i ← left3. for j ← left+1 to right4. if A[j] < x then 5. i ← i + 16. swap(A[i], A[j])7. end if8. end for j9. swap(A[i], A[left])10. return i

n = right – left +1 Time: cn for some constant c Space: constant

64

4 8 6 3 5 1 7 22 3 1 5 6 7 84

Quick-Sort(A, 0, 7)Partition

A:

65

2 3 1

5 6 7 84

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 2)

A:

, partition

66

2

5 6 7 84

1

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 0) , base case, return

67

2

5 6 7 84

1

33

Quick-Sort(A, 0, 7)Quick-Sort(A, 1, 1) , base case

68

5 6 7 842 1 3

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 2, 2), returnQuick-Sort(A, 0, 2), return

69

42 1 3

5 6 7 86 7 85

Quick-Sort(A, 0, 7)Quick-Sort(A, 2, 2), returnQuick-Sort(A, 4, 7) , partition

70

4

5

6 7 87 866

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 5, 7) , partition

71

4

5

6

7 887

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 6, 7) , partition

72

4

5

6

7

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 7, 7)

8

, return, base case

8

73

4

5

6 87

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 6, 7) , return

74

4

5

2 1 3


6 87

75

42 1 3


5 6 87

76

42 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 7) , done!

5 6 87

77

Quick-Sort: Best Case Even Partition

Total time: (nlogn)

cn

2 × cn/2 = cn

4 × c/4 = cn

n/3 × 3c = cn

log n levels

n

n/2 n/2

n/4

3 3 3

n/4n/4n/4

78

cn

c(n-1)

3c

2c

n

n-1

n-2

3

2

c(n-2)

Happens only if input is sortd input is reversely sorted

Quick-Sort: Worst Case Unbalanced Partition

Total time: (n2)

79

Quick-Sort: an Average Case Suppose the split is 1/10 : 9/10

Quick-Sort: an Average Case

cn

cn

cn

≤cn

n

0.1n 0.9n

0.01n 0.09n 0.09n

Total time: (nlogn)

0.81n

2

2

log10n

log10/9n

≤cn

80

Quick-Sort Summary Time

Most of the work done in partitioning. Average case takes (n log(n)) time. Worst case takes (n2) time

Space Sorts in-place, i.e., does not require additional

space

81

Summary Divide and Conquer

Merge-Sort Most of the work done in Merging (n log(n)) time (n) space

Quick-Sort Most of the work done in partitioning Average case takes (n log(n)) time Worst case takes (n2) time (1) space

82

Homework1. What is the running time of Merge-Sort if the

array is already sorted? What is the best case running time of Merge-Sort?

2. Demonstrate the working of Partition on sequence (13, 5, 14, 11, 16, 12, 1, 15). What is the value of i returned at the completion of Partition?

sorting algorithms merge sort quick sort hairong zhao new jersey institute of technology

Documents