cfe2 ch12 final

C++ for Everyone by Cay HorstmannCopyright © 2012 by John Wiley & Sons. All rights reserved

Chapter Twelve: Sorting and Searching

Slides by Evan Gallagher


• To compare the selection sort and merge sort algorithms• To study the linear search and binary search algorithms• To appreciate that algorithms for the same task can differ

widely in performance• To understand the big-Oh notation• To be able to estimate and compare the performance of

algorithms• To write code to measure the running time of a program

Chapter Goals


Selection Sort

The selection sort algorithm

sorts a sequence by repeatedly finding the smallest element

of the unsorted tail region and moving it to the front.

11 9 17 5 12


Selection Sort

The selection sort algorithm

sorts a sequence by repeatedly finding the smallest element

of the unsorted tail region and moving it to the front.

Now the first element is in the correct place.

5 9 17 11 12


Selection Sort

Now do the same for the second position.Find the minimum in the unsorted tail region.

5 9 17 11 12


Selection Sort

There is no need to swap this time.The minimum in the tail portion is already greater.

So we do nothing,and the first and second elements are in order.

5 9 17 11 12


Selection Sort

Repeat this process of finding the minimum in the tail portionand either swapping it into place or doing nothing.

5 9 17 11 12


Selection Sort

Repeat this process of finding the minimum in the tail portionand either swapping it into place or doing nothing.

5 9 11 17 12


Even when the unsorted region only two elements long, keep to the same successful strategy.

5 9 11 17 12

Selection Sort


Even when the unsorted region only two elements long, keep to the same successful strategy.

5 9 11 12 17

Selection Sort


That leaves you with an unprocessed region of length 1,but of course a region of length 1 is always sorted.

5 9 11 12 17

Selection Sort


That leaves you with an unprocessed region of length 1,but of course a region of length 1 is always sorted.

You are done.

5 9 11 12 17

Selection Sort


#include <cstdlib>#include <ctime>#include <iostream>using namespace std;

/**Gets the position of the smallest element in an array range.@param a the array@param from the beginning of the range@param to the end of the range@return the position of the smallest element inthe range a[from]...a[to]

*/int min_position(int a[], int from, int to){

int min_pos = from;for (int i = from + 1; i <= to; i++){

if (a[i] < a[min_pos]) { min_pos = i; }}return min_pos;

}

ch12/selsort.cpp

Selection Sort


/**Swaps two integers.@param x the first integer to swap@param y the second integer to swap

*/void swap(int& x, int& y){

int temp = x;x = y;y = temp;

}

ch12/selsort.cpp

Selection Sort


/**Sorts an array using the selection sort algorithm.@param a the array to sort@param size the number of elements in a

*/void selection_sort(int a[], int size){

int next; // The next position to be set to the minimumfor (next = 0; next < size - 1; next++){

// Find the position of the minimumint min_pos = min_position(a, next, size - 1);if (min_pos != next){ swap(a[min_pos], a[next]);}

}}

ch12/selsort.cpp

Selection Sort


/**Prints all elements in an array.@param a the array to print@param size the number of elements in a

*/void print(int a[], int size){

for (int i = 0; i < size; i++){ cout << a[i] << " ";}cout << endl;

}

ch12/selsort.cpp

Selection Sort


int main(){

srand(time(0));const int SIZE = 20;int values[SIZE];for (int i = 0; i < SIZE; i++){ values[i] = rand() % 100;}print(values, SIZE);selection_sort(values, SIZE);print(values, SIZE);return 0;

}

ch12/selsort.cpp

Selection Sort


If speed was not an issue for us,we could stop the discussion of sorting right here.

However, the selection sort algorithm shows disappointingperformance when run on large data sets,

and it is worthwhile to study better sorting algorithms.

Selection Sort


Profiling A Program Run

.


To measure the performance of a program,one could simply run it

and measure how long it takes by using a stopwatch.

click

click

Go!



However, most of our programs run very quickly,and it is not easy to time them accurately in this way.

click

click

Go!



Instead we use the time function.



#include <ctime>

is required



int now = time(0);

This call sets now to the number of secondsthat have elapsed since January 1, 1970.



int now = time(0); int later = time(0);

We only care about the differencebetween the start and end times.



Profiling the Selection Sort Algorithm

By measuring the time just before and after the sorting,

int before = time(0);selection_sort(values, size);int after = time(0);cout << "Elapsed time = “ << after - before << " seconds" << endl;



By measuring the time just before and after the sorting,you don’t count the time it takes to initialize the array

or the time during which the program waitsfor the user to provide inputs.

int before = time(0);selection_sort(values, size);int after = time(0);cout << "Elapsed time = “ << after - before << " seconds" << endl;


Here are the results of some sample runs.



These measurements were obtained on a Pentium processorwith a clock speed of 1.67 GHz running Linux.



On another computer, the actual numbers will differ,but the relationship between the numbers will be the same.



As you can see,doubling the size of the data set

more than doubles the time needed to sort it..



Analyzing the Performance of the Selection Sort Algorithm

Let’s do some analysis…



Let us count the number of operations that a program must carry out to sort a sequence using the selection sort algorithm.

Actually, we don’t know how many machine operationsare generated for each C++ instruction

or which of those instructions are more time-consuming than others, but we can make a simplification.



Simply count how often an element is visited.

Each visit requires about the sameamount of work by other operations,

such as incrementing subscriptsand comparing values.



Let n be the size of the array.



To set the first element from the unsorted portion of the array:

You must find the smallest of n numbers.This takes n visits.

Then you must (or might) swap the elements.This takes two visits.



To set the second element from the unsorted portion of the array:

You must find the smallest of n – 1 numbers.This takes n – 1 visits.




To set the third element from the unsorted portion of the array:

You must find the smallest of n – 2 numbers.This takes n – 2 visits.




The process continues until…

…there are only two elements in the unsorted region.



To set the last two elements:

There is no searching, there are just two elements.This takes zero visits.

Then you must (or might) swap the last two elements.This takes two visits.



Therefore, the total number of visits is



This is a quadratic equation in n.

That explains why the graph looks

approximately like a parabola.



Now simplify the analysis further:

Just ignore the lower-level terms!



We are really only interested inhow the algorithm performs

as n increases.

We need only look for the factor thatmost influences the increase in time.

We can discount constant multipliersand look for the largest exponent.

Here the most important factor is the n2



We will simply say:

“The number of visits is of order n2”.

Computer scientists often use big-Oh notation:

The number of visits is O(n2).



To turn an exact expression such as

into big-Oh notation,

locate the fastest-growing term, n2,and ignore its constant coefficient,

½ in this case, no matter how large or small it may be:

O(n2)



The sad fact remains thatdoubling the size of the array causes

a fourfold increase in the time required for sorting it.

When the size of the sequence increases by a factor

of 100, the sorting time increases by a factor of 10,000!



To sort a sequence of a million

entries (for example, to create a telephone directory), takes 10,000 times as long

as sorting 10,000 entries.

If 10,000 entries can be sorted in about a second (as in our

example), then sorting one million entries requires almost three hours.


click

Go!




day 1



day 2



day 3



day 4



day 5



day 6



day 7



day 8



day 9



day 10



day 11



day 12



day 13


click


day 14


A Fortnight!


day 14


Merge Sort

Merge Sort


Merge Sort

The basic idea behind merge sort is very simple:

wishful thinking!


Merge Sort

We will simply hope that the first half

of the array is already perfectly sorted,

and the second half is too.


Now it is an easy matter to merge the two sorted sequencesinto a sorted sequence, simply by taking a new element from

either the first or the second subarray andchoosing the smaller of the elements each time:

Merge Sort


Merge Sort

Now it is an easy matter to merge the two sorted sequencesinto a sorted sequence, simply by taking a new element from

either the first or the second subarray andchoosing the smaller of the elements each time:


Merge Sort

This process continuesuntil…



Merge Sort



There is only one element left (in either side),

Merge Sort



and the array is completely sorted.

Merge Sort


Merge Sort

Recall that the basic idea behind merge sort is

wishful thinking!


Merge Sort

The two half arrays will hardly ever be already sorted.

Someone had to sort each half.


Merge Sort

But if you think about it,what really happed was:

the array was divided into halves,then each half was sorted (by magic),


Merge Sort

But if you think about it,what really happed was:

the array was divided into halves,then each half was sorted (by magic),

and then the two sorted halves were merged together.


Merge Sort

If this process were to continue,then eventually there would besubarrays of only one element

which are very easy to sort and merged together


Merge Sort

If this process were to continue,then eventually there would besubarrays of only one element

which are very easy to sort and merged together

and then the two-length subarrays could be merged together

and then the four-length subarrays could be merged together

and then the eight-length subarrays could be merged together

and then the. . .


Merge Sort

and the entire array will be sortedwhen we get back to merging the first two halves:

Merge Sort!

Yes, this is a recursive algorithm.

At the recursive step,the merge sort is applied to each half problem,

recursively sorting each half and putting them back together.

The end test is an array of only one element.


We pass the same array on each call

and use indicesas the left and right endpointsof the subarray to be sorted.

Merge Sort

0 9


Using 0 and 9 (the whole array),one of the subarrays will be:

5 to 9:

Merge Sort

0 9


Using 0 and 9 (the whole array),one of the subarrays will be:

5 to 9:

Merge Sort

5 9


The end test of a one element arrayoccurs when the endpoints are the same index:

Merge Sort

77


7


Merge Sort

7



Merge Sort

7


/**Sorts the elements in a range of an array.@param a is the array with the elements to sort@param from start of the range to sort@param to end of the range to sort

*/void merge_sort(int a[], int from, int to){

if (from == to) { return; }int mid = (from + to) / 2;// Sort the first half and the second halfmerge_sort(a, from, mid);merge_sort(a, mid + 1, to);merge(a, from, mid, to);

}

Merge Sort


The merging part is longer,but straightforward.

The code for merge sort:

Merge Sort


ch12/mergesort.cpp

Merge Sort#include <cstdlib>#include <ctime>#include <iostream>using namespace std;

/** Merges two adjacent ranges in an array. @param a the array with the elements to merge @param from the start of the first range @param mid the end of the first range @param to the end of the second range*/

void merge(int a[], int from, int mid, int to){ int n = to - from + 1; // Size of the range to be merged

// Merge both halves into a temporary array b . // We allocate the array dynamically because its size is only // known at runtime—see Section 7.4 int* b = new int[n];


ch12/mergesort.cpp

Merge Sort

int i1 = from; // Next element to consider in the first half

int i2 = mid + 1; // Next element to consider in the second half

int j = 0; // Next open position in b


ch12/mergesort.cpp

Merge Sort

// As long as neither i1 nor i2 is past the end, // move the smaller element into b while (i1 <= mid && i2 <= to) { if (a[i1] < a[i2]) { b[j] = a[i1]; i1++; } else { b[j] = a[i2]; i2++ j++; } }


ch12/mergesort.cpp

Merge Sort

// Note that the body of only one of the // two while loops below is executed

// Copy any remaining entries of the first half while (i1 <= mid) { b[j] = a[i1]; i1++; j++; }

// Copy any remaining entries of the second half while (i2 <= to) { b[j] = a[i2]; i2++; j++; }


ch12/mergesort.cpp

Merge Sort

// Copy back from the temporary array for (j = 0; j < n; j++) { a[from + j] = b[j]; }

// The temporary array is no longer needed delete[] b;}


ch12/mergesort.cpp

Merge Sort

/** Sorts the elements in a range of an array. @param a the array with the elements to sort @param from start of the range to sort @param to end of the range to sort*/void merge_sort(int a[], int from, int to){ if (from == to) { return; } int mid = (from + to) / 2; // Sort the first half and the second half merge_sort(a, from, mid); merge_sort(a, mid + 1, to); merge(a, from, mid, to);}


ch12/mergesort.cpp

Merge Sort

/** Prints all elements in an array. @param a the array to print @param size the number of elements in a*/void print(int a[], int size){ for (int i = 0; i < size; i++) { cout << a[i] << " "; } cout << endl;}


ch12/mergesort.cpp

Merge Sort

int main(){ srand(time(0)); const int SIZE = 20; int values[SIZE]; for (int i = 0; i < SIZE; i++) { values[i] = rand() % 100; } print(values, SIZE); merge_sort(values, 0, SIZE - 1) print(values, SIZE); return 0;}


Analyzing the Merge Sort Algorithm

.



The merge sort algorithm looksmuch more complicated

than the selection sort algorithm,

and it appears that it may welltake much longer

to carry out these repeated subdivisions.


Here are the results of some sample runscomparing selection sort and merge sort.



You’ll need to look closely – there’s merge sort.



Note that the graph does not have a parabolic shape.

Instead, it appears as if the running time grows approximately linearly with the size of the sequence.




To understand why the merge sort algorithmis such a tremendous improvement,

let us estimate the number of sequence element visits.



First, we tackle the merge process that happens

after the first and second halves have been sorted.

merge(a, from, mid, to);



Each step in the merge process adds one more element to b.

There are n elements in b.

That element may come from the first or second half of a,and in most cases the elements from the two halves

must be compared to see which one to take.

Count that as 3 visits per element(one for b and one each for the two halves of a),

or 3n visits total.



3n (for comparisons)



Then you must

copy back from b to a.

Using two visits per elementyields another 2n visits.



3n (for comparisons) 2n (for copying) -------- 5n (for the whole merging process)



Now for the merge sort portion.

merge_sort(a, from, mid);merge_sort(a, mid + 1, to);

Letting T(n) denote the number of visits required to sort a range of n elements,

the two recursive calls will each require T(n/2) visits.



Now for the merge sort portion.

merge_sort(a, from, mid);merge_sort(a, mid + 1, to);

Letting T(n) denote the number of visits required to sort a range of n elements,

the two recursive calls will each require T(n/2) visits.

But…



What if n is not an even number?



What if n is not an even number?

We can (and will) make the assumption thatn is some power of 2, say m

(this will help later on).

n = 2m

Now every subsequence is divisible by 2.



To understand the relationship,

evaluate T(n/2),using the same formula:



Or:



Do that process again:

evaluate T(n/4),using the same formula:



Or:



This generalizes from 2, 4, 8, to arbitrary powers of 2:



Letting k be the power of 2:



Remember that it’s-going-to-be-helpful-laterassumption we made that n was a power of 2?

n = 2m



Let k = m



Let k = mBecause k = 2m, you have m = log2(n).



To establish the growth order,you drop the lower order term n



To establish the growth order,you drop the lower order term n

and you are left with:



You will also drop constant term 5:



It is customary to drop the base of the logarithmbecause all logarithms are related by a constant factor.



So the Merge Sort algorithm is an

O( n log( n ) )

algorithm.

So much better than Selection Sort’s O(n2)

How much better?



To sort 1,000,000 integers,

if it took Selection Sort a bit less than three hours O(n2)

on the same machine,

it would take Merge Sort about three minutes! O( n log( n ) )


Searching

Searching for an element in a sequenceis an extremely common task.

As with sorting,the right choice of algorithmscan make a big difference.


Linear Search

What’s Bjarne Stroustrup’s phone number?

Here’s the phone book – look it up!

(He’s under the C++’s, not the S’s!)

It’s easy because the elementsin a phone book are sorted by name.


Linear Search

Whose phone number is 399-728-9011?

Here’s the phone book – look it up!

Not so easy!


Linear Search

So get started.

Until you find the number:

Open the phone book to the first page,go to the first entry,

then the second entry,then the third…

go through all of them on page 1

if it’s not there, you’ll need to turn the page


Linear Search

On page 2:

go to the first entry,then the second entry,

then the third…all of them on page 2

if it’s not there, you’ll need to turn the page


Linear Search

On page 3:

go to the first entry, then the second…


Linear Search

THIS IS GOING TO TAKE FOREVER!

Or maybe I’ll get lucky.


Linear Search

This process is called a linear or sequential search.

A linear search examines all values in a sequenceuntil it finds a match or reaches the end.

Here’s the code:


Linear Search

int linear_search(int a[], int size, int value) { for (int i = 0; i < size; i++) { if (a[i] == value) { return i; } } return -1; }

-1 is returned to indicate that the value was not found.

ch12/lsearch.cpp(excerpt)


Analyzing the Linear Search Algorithm

.


Analyzing the Linear Search Algorithm

If you assume that the value is present in the array, then the average search visits n/2 elements.

In the worst case, when the value it is not present,then all n elements must be inspected to verify the absence.

Either way, a linear search is an O(n) algorithm.


Binary Search

If you were told that the values in an array were already sorted,

(sounds a little bit like what we assumed at first in the Merge Sort!)

would you still consider using a linear search?


Binary Search

Consider this array:


Binary Search

Is 123 in the array?


Binary Search

Is 123 in the first half of this array?


Binary Search

Consider the last element in the first half (at index 3)?What is the relationship between 123

and the last value in the first half array?


Binary Search

Consider the last element in the first half (at index 3)?What is the relationship between 123

and the last value in the first half array?

?

123


Binary Search

Given that 123 is greater than 100,then because the data is sorted,123 must be in the upper half.

?

123


Binary Search

Repeat the process on the upper half array.

123


Binary Search

Consider the last index in the lower halfof the array from index 4 to 7.

That’s index 5:

?

123


Binary Search

123 is less than 290so it must be in the left half of this subarray

(or not in the array at all)

?

123


Binary Search

123 is less than 290so it must be in the left half of this subarray

(or not in the array at all)

123


Binary Search

Consider the last index in the lower halfof the very short subarray from index 4 to 5.

That at index 4:

?

123


Binary Search

123 is greater than 115 so you must look atthe very, very short array index 5:

?

123


Binary Search

There’s only one element in this subarrayand 290 is not 123.

123 is not found.

?

123


Binary Search

There’s only one element in this subarrayand 290 is not 123.

123 is not found.

Not found

123


Binary Search

Did we visit all n elements? No.

Not found

123


Binary Search

Did we visit all n elements? No.

Binary search cuts the search in half each time.We do not visit every element.

Of course, this is only possible when thevalues in the array already sorted.

Not found

123


Binary Search

Here is the code for binary search:


Linear Search

int binary_search(int a[], int from, int to, int value){ if (from > to) { return -1; }

int mid = (from + to) / 2; if (a[mid] == value) { return mid; } else if (a[mid] < value) { return binary_search(a, mid + 1, to, value); } else { return binary_search(a, from, mid - 1, value); }}

ch12/bsearch.cpp (excerpt)


Analyzing the Binary Search Algorithm

.



Use the same technique as in the analysis of merge sortto determine the number of element visits required.



Because you look at the middle element,which counts as one comparison,

and then search either the left or the rightsubsequence, you have



As before, to understand the relationship,

evaluate T(n/2),using the same formula, you get:



Again, evaluate T(n/4),

using the same formula, and continue…



This generalizes to:



As in the analysis of merge sort,you make the simplifying assumption

that n is a power of 2, n = 2 m,where m = log2(n).

Then you obtain



So binary search is an O(log(n)) algorithm.



Binary search is much faster than linear search,

but

is it worthwhile to sort a sequence firstand then use a binary search?

Sorting takes time, too!



Is it worth the time to sort?

It depends.



If you will only search the sequence once,then it is more efficient to pay

for an O(n) linear search than for anO(n log(n)) sort plus an O(log(n)) binary search.

But if one must make a number of searchesin the same sequence,

then sorting before the searchesis definitely worthwhile.


Chapter Summary


End Chapter Twelve

Slides by Evan Gallagher

cfe2 ch12 final

Technology

john wiley sons

cay horstmann copyright

cay horstmanncopyright

selection sortthere

selection sortnow

unsorted tail region

cay horstmannslides

unsorted region