invitation to computer science 5 th edition

Invitation to Computer Science 5th Edition

Chapter 3

The Efficiency of Algorithms

Invitation to Computer Science, 5th Edition

Objectives

In this chapter, you will learn about:

• Attributes of algorithms

• Measuring efficiency

• Analysis of algorithms

• When things get out of hand

2


Introduction

• An important part of computer science– Finding algorithms to solve problems of interest

• Are some algorithms better than others?

3


Attributes of Algorithms

• We expect correctness from our algorithms

• Program maintenance– Fixes errors uncovered through repeated use – Extends program to meet new requirements

• Ease of understanding– Desirable characteristic of an algorithm

• Elegance– Algorithmic equivalent of style

4


Attributes of Algorithms (continued)

• Efficiency– Term used to describe an algorithm’s careful use of

resources

• Benchmarks– Useful for rating one machine against another and

for rating how sensitive a particular algorithm is with respect to variations in input on one particular machine

5


Measuring Efficiency

• Analysis of algorithms– The study of the efficiency of algorithms– An important part of computer science

6


Sequential Search

• Search for NAME among a list of n names

• Start at the beginning and compare NAME to each entry until a match is found

7


Figure 3.1 Sequential Search Algorithm

8


Figure 3.2 Number of Comparisons to Find NAME in a List of n Names Using Sequential Search

9


Order of Magnitude - Order n

• Order of magnitude n– Anything that varies as a constant times n (and

whose graph follows the basic shape of n)

• Sequential search– An Θ(n) algorithm in both the worst case and the

average case

10


Figure 3.3 Work = 2n

11


Figure 3.4 Work = cn for Various Values of c

12


Figure 3.5 Growth of Work = cn for Various Values of c

13


Selection Sort

• Selection sort algorithm– Sorts in ascending order

• Subtask within selection sort– Task of finding the largest number in a list

• When selection sort algorithm begins– The largest-so-far value must be compared to all the

other numbers in the list– If there are n numbers in the list, n – 1 comparisons

must be done

14


Figure 3.6 Selection Sort Algorithm

15


Figure 3.7 Comparisons Required by Selection Sort

16


Selection Sort (continued)

• Selection sort algorithm – Does n exchanges, one for each position in the list to

put the correct value in that position

• Space efficiency of the selection sort– Original list occupies n memory locations– Storage is needed for:

• The marker between the unsorted and sorted sections

• Keeping track of the largest-so-far value and its location in the list

17


Figure 3.8 An Attempt to Exchange the Values at X and Y

18


Figure 3.9 Exchanging the Values at X and Y

19


Order of Magnitude - Order n2

• Order of magnitude n2, or Θ(n2)– An algorithm that does cn2 work for any constant c

• Selection sort– An Θ(n2) algorithm (in all cases)

• Sequential search – An Θ(n) algorithm (in the worst case)

20


Figure 3.10 Work 5 cn2 for Various Values of c

21


Figure 3.11 A Comparison of n and n2

22


Figure 3.12 For Large Enough n, 0.25n2 Has Larger Values Than 10n

23


Order of Magnitude - Order n2

(continued)• Part of the job of program documentation

– To make clear any assumptions or restrictions about the input size the program was designed to handle

• Comparing algorithm efficiency – Only makes sense if there is a choice of algorithms

for the task at hand

24


Figure 3.13 A Comparison of Two Extreme Q(n2) and Q(n) Algorithms

25


Analysis of Algorithms

• Data cleanup algorithms– The Shuffle-Left Algorithm– The Copy-Over Algorithm – The Converging-Pointers Algorithm

26


The Shuffle-Left Algorithm

• Scans list from left to right

• When a zero is found, copy each remaining data item in the list one cell to the left

• Value of legit– Originally set to the length of the list– Is reduced by 1 every time a 0 is encountered

27


Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup

28


The Shuffle-Left Algorithm (continued)

• To analyze time efficiency:– Identify the fundamental units of work the algorithm

performs

• Copying numbers– Best case occurs when the list has no 0 values

because no copying is required– Worst case occurs when the list has all 0 values

29


The Shuffle-Left Algorithm (continued)

• Worst case – Occurs when the list has all 0 values– An Θ(n2) algorithm

• Space-efficient – Only requires four memory locations to store the

quantities n, legit, left, and right

30


The Copy-Over Algorithm

• Scans the list from left to right, copying every legitimate (non-zero) value into a new list that it creates

• Every list entry is examined to see whether it is 0

• Every non-zero list entry is copied once

• Best case – Occurs if all elements are 0– Θ(n) in time efficiency– No extra space is used

31


Figure 3.15 The Copy-Over Algorithm for Data Cleanup

32


The Copy-Over Algorithm (continued)

• Worst case – Occurs if there are no 0 values in the list– Algorithm copies all n non-zero elements into the

new list and doubles the space required– Θ(n) in time efficiency

• Time/space tradeoff– You gain something by giving up something else

33


The Converging-Pointers Algorithm

• Swap zero values from left with values from right until pointers converge in the middle

• Best case – A list containing no 0 elements

• Worst case– A list of all 0 entries– Θ(n) in time efficiency– Is space-efficient

34


Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup

35


Figure 3.17 Analysis of Three Data Cleanup Algorithms

36


The Converging-Pointers Algorithm (continued)

• In an Θ(n) algorithm:– The work is proportional to n

• In an Θ(n2) algorithm:– The work is proportional to the square of n

37


Binary Search

• Procedure– First looks for NAME at roughly the halfway point in

list– If name equals NAME, search is over– If NAME comes alphabetically before name at

halfway point, search is narrowed to the front half of list

– If NAME comes alphabetically after name at halfway point, search is narrowed to the back half of the list

– Algorithm halts when NAME is found

38


Figure 3.18 Binary Search Algorithm (list must be sorted)

39


Binary Search (continued)

• Logarithm of n to the base 2– Number of times a number n can be cut in half and

not go below 1

• As n doubles:– lg n increases by only 1, so lg n grows much more

slowly than n

• Worst case and average case– Θ(lg n) comparisons

• Works only on a list that has already been sorted

40


Figure 3.19 Binary Search Tree for a 7-Element List

41


Figure 3.20 Values for n and lg n

42


Figure 3.21 A Comparison of n and lg n

43


Pattern-Matching

• Usually involves a pattern length that is short compared to the text length– That is, when m is much less than n

• Best case– Θ(n)

• Worst case– Θ(m 3 n)

44


When Things Get Out of Hand

• Polynomially bound algorithms– Work done is no worse than a constant multiple of n2

• Graph– A collection of nodes and connecting edges

• Hamiltonian circuit– A path through a graph that begins and ends at the

same node and goes through all other nodes exactly once

45


Figure 3.22 Order-of-Magnitude Time Efficiency Summary

46


Figure 3.23 Four Connected Cities

47


Figure 3.24 Hamiltonian Circuits among All Paths from A in Figure 3.23 with Four Links

48


When Things Get Out of Hand (continued)

• Exponential algorithm– An Θ(2n) algorithm

• Brute force algorithm– One that beats the problem into submission by trying

all possibilities

• Intractable problem– No polynomially bounded algorithm exists

• Approximation algorithms– Do not solve the problem, but provide a close

approximation to a solution

49


Figure 3.25 Comparisons of lg n, n, n2, and 2n

50


Figure 3.26 Comparisons of lg n, n, n2, and 2n for Larger Values of n

51


Figure 3.27 A Comparison of Four Orders of Magnitude

52


Summary of Level 1

• Level 1 (Chapters 2 and 3) explored algorithms

– Chapter 2

• Pseudocode

• Sequential, conditional, and iterative operations

• Algorithmic solutions to various practical problems

– Chapter 3

• Desirable properties for algorithms

• Time and space efficiencies of a number of algorithms

53


Summary

• Desirable attributes in algorithms– Correctness– Ease of understanding (clarity)– Elegance– Efficiency

• Efficiency– An algorithm’s careful use of resources is extremely

important

54

invitation to computer science 5 th edition

Documents