CHAPTER 09

Compiled by: Dr. Mohammad Omar AlhawaratSorting & Searching

2

Content

Sorting: Bubble Sort. Insertion Sort. Selection Sort. Merge Sort. Quicksort

Searching: Sequential Search. Binary Search. Hashing.

3

Sorting

4

Sorting

Definition: Rearranging the values into a specific order: (Ascending OR Descending).

Sorting is important and is required in many Applications, i.e., Searching.

one of the fundamental problems in computer science can be solved in many ways:

fast/slow use more/less memory depends on data utilize multiple computers / processors, ...

5

Sorting

Comparison-based sorting: determining order by comparing pairs of elements.

An internal sort requires that the collection of data fit entirely in the computer’s main memory.

We can use an external sort when the collection of data cannot fit in the computer’s main memory all at once but must reside in secondary storage such as on a disk.

We will analyze only Comparison-based and internal sorting algorithms.

6

Bubble Sort

Idea: Repeatedly pass through the array Swaps adjacent elements that are out of

order

Easy to implement, but slow O(N2)

7

Example

8

Example

9

Example

10

Example

11

Bubble Sort – Analysis

Worst-case: O(n2) Array is in reverse order:

Average-case: O(n2) We have to look at all possible initial data organizations.

So, Bubble Sort is O(n2)

12

Insertion Sort

Insertion sort is a simple sorting algorithm that is appropriate for small inputs.

The list is divided into two parts: sorted and unsorted.

In each pass, the first element of the unsorted part is picked up, transferred to the sorted sublist, and inserted at the appropriate place.

A list of n elements will take at most n-1 passes to sort the data.

13

Example

14

Insertion Sort – Analysis

Worst-case: O(n2) Array is in reverse order:

Average-case: O(n2) We have to look at all possible initial data organizations.

So, Insertion Sort is O(n2)

15

Selection Sort

Idea: Find the smallest element in the array

Exchange it with the element in the first position

Find the second smallest element and exchange it with the element in the second position

Continue until the array is sorted

16

Example

1329648

8329641

8349621

8649321

8964321

8694321

9864321

9864321

17

Selection Sort – Analysis

Worst-case: O(n2) Array is in reverse order:

Average-case: O(n2) We have to look at all possible initial data organizations.

So, Selection Sort is O(n2)

18

Merge sort

Idea: Is based on “Merging” idea where two sorted

lists are combined in the right order.

The start point is to consider each element in the list as an ordered small list.

The result is a list of two-element sorted lists.

Repeatedly combine the ordered list until having one list

19

Merging Algorithm

Merging two ordered lists:1. Access the first item from both lists2. While neither sequence is finished

1. Compare the current items of both2. Copy smaller current item to the output3. Access next item from that input sequence

3. Copy any remaining from first sequence to output

4. Copy any remaining from second to output

20

Example of Merging

21

Example: Merge sort

22

Merge sort – Analysis

Worst-case: O(N LogN) Array is in reverse order:

Average-case: O(N LogN) We have to look at all possible initial data organizations.

So, Merge sort Sort is O(N LogN)

But, merge sort requires an extra array whose size equals to the size of the original array.

23

Quicksort

Idea: Repeatedly partition the data into two halves. Only the element in the middle is sorted. After (Log2N) repetitions then the data is sorted.

Advantage: One of the practically best sorting Algorithms [O(N Log2N)] in the average case.

Drawbacks: O(N2) in the worst case.

24

Searching

25

Introduction to Search Algorithms

Search: locate an item in a list (array, vector, etc.) of information

Three algorithms: Linear search (Also known as: Sequential

Search) Binary search Hashing

26

Linear Search Example

Following Array contains:

Searching for the value 11, linear search examines 17, 23, 5, and 11

Searching for the value 7, linear search examines 17, 23, 5, 11, 2, 29, and 3

17 23 5 11 2 29 3

27

Linear Search Tradeoffs

Benefits Easy algorithm to understand Array can be in any order

Disadvantage Inefficient O(N) (slow): for array of N

elements, examines N/2 elements on average for value that is found in the array, N elements for value that is not in the array

28

Binary Search Algorithm

1. Divide a sorted array into three sections: middle element elements on one side of the middle element elements on the other side of the middle element

2. If the middle element is the correct value, done. Otherwise, go to step 1, using only the half of the array that may contain the correct value.

3. Continue steps 1 and 2 until either the value is found or there are no more elements to examine.

29

Ignoring one-half of the data when the data is sorted.

Binary Search Algorithm

30

Binary Search Example

If the following Array contains:

Searching for the value 11, binary search examines 11 and stops

Searching for the value 7, binary search examines 11, 3, 5, and stops

2 3 5 11 17 23 29

31

Binary Search Example

32

Binary Search Example

33

Binary Search Tradeoffs

Benefit Much more efficient than linear search(For array of N elements, performs at mostlog2N comparisons) O(log2N)

Disadvantage Requires that array elements be sorted

34

Time Complexity Summary

Worst Case Average Case

O(N2) O(N2) Bubble Sort

O(N2) O(N2) Insertion Sort

O(N2) O(N2) Selection Sort

O(N LogN) O(N LogN) Merge Sort

O(N LogN) O(N LogN) Heap Sort

O(N2) O(N LogN) Quick Sort

O(N) O(N) Sequential Search

O(LogN) O(LogN) Binary Search

O(N) O(LogN) Binary search Tree

35

Sorting

36

Hashing

Hashing can be classified as one of the searching techniques that is usually used with external storage as Hard disk drive (HDD).

Hashing, is an information retrieval strategy for providing efficient access to information based on a key.

One usage is indexing databases. In such case, the location of a record in a database is linked to the key/index of that record.

Information can usually be accessed in constant time.

37

Concept of Hashing

The information to be retrieved is stored in a hash table which is best thought of as an array of m locations, called buckets

The mapping between a key and a bucket is called the hash function

The time to store and retrieve data is proportional to the time to compute the hash function (constant)

38

Hashing function

The ideal function, termed a perfect hash function, would distribute all elements across the buckets such that no collisions ever occurred

h(v) = f(v) mod m

Knuth (1973) suggests using as the value for m a prime number

39

Determines position of key in the array

Assume table (array) size is N

Function f(x) maps any key x to an integer between 0 and N−1

For example, assume that N=15, that key x is a non-negative integer between 0 and MAX_INT, and hash function f(x) = x Mod 15.

Hash Function

40

Let f(x) = x Mod 15. Then,if x =25 129 35 2501 47 36 f(x) = 10 9 5 11 2 6

Storing the keys in the array is straightforward:

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14_ _ 47 _ _ 35 36 _ _ 129 25 2501 _ _ _

Thus, delete and find can be done in O(1), and also insert, except…

Hash Function

41

Hash Function

What happens when you try to insert: x = 65 ?x = 65f(x) = 5

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14_ _ 47 _ _ 35 36 _ _ 129 25 2501 _ _ _ 65(?)

This is called a collision.

42

Handling Collisions

A collision occurs when two different keys hash to the same value:

Ex.: For TableSize = 17, the keys 18 and 35 hash to the same value 18 mod 17 = 1 and 35 mod 17 = 1

Cannot store both data records in the same slot in array!

Resolution: Separate Chaining (Closed Addressing): Use a

dictionary data structure (such as a linked list) to store multiple items that hash to the same slot

Closed Hashing (Open Addressing): search for empty slots and store item in first empty slot that is found

Multi-Hash functions: use another hash function to resolve the collision.

43

Separate Chaining

Let each array element be the head of a chain.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 47 65 36 129 25 2501 35

Where would you store: 29, 16, 14, 99, 127 ?

44

Separate Chaining

Let each array element be the head of a chain:

Where would you store: 29, 16, 14, 99, 127 ?

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 47 65 36 127 99 25 2501 14 35 129 29

New keys go at the front of the relevant chain.

45

Closed Hashing

The hash table should be large enough to include all possible keys:

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 47 65 36 127 99 25 2501 14

Where would you store: 29, 60, 24, 97?

46

Closed Hashing

The hash table should be large enough to include all possible keys:

Where would you store: 29, 60, 24, 97?

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 29 16 47 60 65 36 127 97 99 25 2501 24 14

New keys go at the next free bucket in the hash table.

47

Factors affecting efficiency

Choice of hash function Collision resolution strategy

Hashing offers excellent performance for insertion and retrieval of data.

48

Questions ?