invitation to computer science 6th edition chapter 3 the efficiency of algorithms

Invitation to Computer Science 6th Edition

Chapter 3

The Efficiency of Algorithms

Invitation to Computer Science, 6th Edition

Objectives

In this chapter, you will learn about:

• Attributes of algorithms

• Measuring efficiency

• Analysis of algorithms

• When things get out of hand

Introduction

• An important part of computer science– Finding algorithms to solve problems of interest

• Are some algorithms better than others?

Attributes of Algorithms

• We expect correctness from our algorithms

• Program maintenance– Fixes errors uncovered through repeated use – Extends program to meet new requirements

• Ease of understanding– Desirable characteristic of an algorithm

• Elegance– Algorithmic equivalent of style

Attributes of Algorithms (continued)

• Efficiency– Term used to describe an algorithm’s careful use of

resources

• Benchmarks– Useful for rating one machine against another and

for rating how sensitive a particular algorithm is with respect to variations in input on one particular machine

Measuring Efficiency

• Analysis of algorithms– The study of the efficiency of algorithms– An important part of computer science

• Efficiency measured as a function relating size of input to time or space used

• For one input size, best case, worst case, and average case behavior must be considered

• The notation captures the order of magnitude of the efficiency function

Invitation to Computer Science, 6th Edition 7

Sequential Search

• Search for NAME among a list of n names

• Start at the beginning and compare NAME to each entry until a match is found

Figure 3.1 Sequential Search Algorithm

Figure 3.2 Number of Comparisons to Find NAME in a List of n Names Using Sequential Search

Order of Magnitude - Order n

• Order of magnitude n– Anything that varies as a constant times n (and

whose graph follows the basic shape of n)

• Sequential search– An Θ(n) algorithm in both the worst case and the

average case

Figure 3.3 Work = 2n

Figure 3.4 Work = cn for Various Values of c

Figure 3.5 Growth of Work = cn for Various Values of c

Selection Sort

• Selection sort algorithm– Sorts in ascending order

• Subtask within selection sort– Task of finding the largest number in a list

• When selection sort algorithm begins– The largest-so-far value must be compared to all the

other numbers in the list– If there are n numbers in the list, n – 1 comparisons

must be done

Figure 3.6 Selection Sort Algorithm

Figure 3.7 Comparisons Required by Selection Sort

Selection Sort (continued)

• Selection sort algorithm – Does n exchanges, one for each position in the list to

put the correct value in that position

• Space efficiency of the selection sort– Original list occupies n memory locations– Storage is needed for:

• The marker between the unsorted and sorted sections

• Keeping track of the largest-so-far value and its location in the list

Figure 3.8 An Attempt to Exchange the Values at X and Y

Figure 3.9 Exchanging the Values at X and Y

Count comparisons of largest so far against other values

Find Largest, given m values, does m-1 comparisons

Selection sort calls Find Largest n times,

Each time with a smaller list of values

Cost = n-1 + (n-2) + … + 2 + 1 = n(n-1)/2

Time efficiency

Comparisons: n(n-1)/2

Exchanges: n (swapping largest into place)

Overall: (n2), best and worst cases

Space efficiency

Space for the input sequence, plus a constant number of local variables

Order of Magnitude – Order n2

All functions with highest-order term cn2 have similar shape

An algorithm that does cn2 work for any constant c is order of magnitude n2, or (n2)

Order of Magnitude – Order n2

Anything that is (n2) will eventually have larger values than anything that is (n), no matter what the constants are

An algorithm that runs in time (n) will outperform one that runs in (n2)

Order of Magnitude - Order n2

(continued)• Order of magnitude n2, or Θ(n2)

– An algorithm that does cn2 work for any constant c

• Selection sort– An Θ(n2) algorithm (in all cases)

• Sequential search – An Θ(n) algorithm (in the worst case)

Figure 3.10 Work 5 cn2 for Various Values of c

Figure 3.11 A Comparison of n and n2

Figure 3.12 For Large Enough n, 0.25n2 Has Larger Values Than 10n

Order of Magnitude - Order n2

(continued)• Part of the job of program documentation

– To make clear any assumptions or restrictions about the input size the program was designed to handle

• Comparing algorithm efficiency – Only makes sense if there is a choice of algorithms

for the task at hand

Figure 3.13 A Comparison of Two Extreme Q(n2) and Q(n) Algorithms

Analysis of Algorithms

Multiple algorithms for one task may be compared for efficiency and other desirable attributes

Data cleanup problem

Search problem

Pattern matching

Analysis of Algorithms

Given a collection of numbers, find and remove all zeros

Possible algorithms

Shuffle-left

Copy-over

Converging-pointers

The Shuffle-Left Algorithm

• Scans list from left to right

• When a zero is found, copy each remaining data item in the list one cell to the left

• Value of legit– Originally set to the length of the list– Is reduced by 1 every time a 0 is encountered

Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup

The Shuffle-Left Algorithm (continued)

• To analyze time efficiency:– Identify the fundamental units of work the algorithm

performs• Examinations

• Copying numbers– Best case occurs when the list has no 0 values

because no copying is required– Worst case occurs when the list has all 0 values

Time efficiency Count examinations of list values and copies Best case

No copies, n examinations (n)

Worst case Shift at each pass, n passes n2 copies plus n examinations (n2)

• Space-efficient

– n slots for n values, plus a few local variables

– Only requires four memory locations to store the quantities n, legit, left, and right

The Copy-Over Algorithm

• Scans the list from left to right, copying every legitimate (non-zero) value into a new list that it creates

• Every list entry is examined to see whether it is 0

• Every non-zero list entry is copied once

Figure 3.15 The Copy-Over Algorithm for Data Cleanup

The Copy-Over Algorithm

Use a second list Copy over each nonzero element in turn

Time efficiency Count examinations and copies

Best case

All zeros

n examinations and 0 copies

The Copy-Over Algorithm (continued)

Time efficiency (continued) Worst case

No zeros

n examinations and n copies

Space efficiency 2n slots for n values, plus a few extraneous variables

The Copy-Over Algorithm (continued)

Time/space tradeoff

Algorithms that solve the same problem offer a tradeoff

One algorithm uses more time and less memory

Its alternative uses less time and more memory

The Converging-Pointers Algorithm

• Swap zero values from left with values from right until pointers converge in the middle

Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup

The Converging-Pointers Algorithm

Time efficiency

Count examinations and swaps

Best case

n examinations, no swaps

The Converging-Pointers Algorithm (continued)

Time efficiency (continued)

Worst case

n examinations, n swaps

Space efficiency

n slots for the values, plus a few extra variables

Figure 3.17 Analysis of Three Data Cleanup Algorithms

The Converging-Pointers Algorithm (continued)

• In an Θ(n) algorithm:– The work is proportional to n

• In an Θ(n2) algorithm:– The work is proportional to the square of n

Binary Search Algorithm

Given ordered data

Search for NAME by comparing to middle element

If not a match, restrict search to either lower or upper half only

Each pass eliminates half the data

Binary Search

• Procedure– First looks for NAME at roughly the halfway point in

list– If name equals NAME, search is over– If NAME comes alphabetically before name at

halfway point, search is narrowed to the front half of list

– If NAME comes alphabetically after name at halfway point, search is narrowed to the back half of the list

– Algorithm halts when NAME is found

Figure 3.18 Binary Search Algorithm (list must be sorted)

Figure 3.19 Binary Search Tree for a 7-Element List

Binary Search Algorithm (continued)

Time Efficiency Best case

1 comparison

Worst case

lg n comparisons lg n: The number of times n can be divided by two

before reaching 1

(lg n)

Binary Search (continued)

• Logarithm of n to the base 2– Number of times a number n can be cut in half and

not go below 1

• As n doubles:– lg n increases by only 1, so lg n grows much more

slowly than n

• Worst case and average case– Θ(lg n) comparisons

• Works only on a list that has already been sorted

Figure 3.20 Values for n and lg n

Figure 3.21 A Comparison of n and lg n

Binary Search Algorithm (continued)

Tradeoff

Sequential search

Slower, but works on unordered data

Binary search

Faster (much faster), but data must be sorted first

Figure 2.16 Final Draft of the Pattern-Matching Algorithm

Pattern-Matching Algorithm

Analysis involves two measures of input size

m: length of pattern string

n: length of text string

Unit of work

Comparison of a pattern character with a text character

Pattern-Matching

• Usually involves a pattern length that is short compared to the text length– That is, when m is much less than n

Time Efficiency Best case

Pattern does not match at all n - m + 1 comparisons (n)

Pattern-Matching

Time Efficiency Worst case

Pattern almost matches at each point (m)(n - m + 1) comparisons (m x n)

• Text: AAAAAAAAA

• Pattern: AAAB

Figure 3.22 Order-of-Magnitude Time Efficiency Summary

When Things Get Out of Hand

Polynomially bound algorithms

Work done is no worse than a constant multiple of n2

Intractable algorithms

Run in worse than polynomial time

Examples

Hamiltonian circuit

Bin-packing

When Things Get Out of Hand

• Graph– A collection of nodes and connecting edges

• Hamiltonian circuit– A path through a graph that begins and ends at the

same node and goes through all other nodes exactly once

Figure 3.23 Four Connected Cities

Figure 3.24 Hamiltonian Circuits among All Paths from A in Figure 3.23 with Four Links

When Things Get Out of Hand (continued)

• Exponential algorithm– An Θ(2n) algorithm

• Brute force algorithm– One that beats the problem into submission by trying

all possibilities

• Intractable problem– No polynomially bounded algorithm exists

• Approximation algorithms– Do not solve the problem, but provide a close

approximation to a solution

Figure 3.27 A Comparison of Four Orders of Magnitude

Figure 3.25 Comparisons of lg n, n, n2, and 2n

Figure 3.26 Comparisons of lg n, n, n2, and 2n for Larger Values of n

Summary of Level 1

• Level 1 (Chapters 2 and 3) explored algorithms

– Chapter 2

• Pseudocode

• Sequential, conditional, and iterative operations

• Algorithmic solutions to various practical problems

– Chapter 3

• Desirable properties for algorithms

• Time and space efficiencies of a number of algorithms

Summary

• Desirable attributes in algorithms– Correctness– Ease of understanding (clarity)– Elegance– Efficiency

• Efficiency– An algorithm’s careful use of resources is extremely

important

invitation to computer science 6th edition chapter 3 the efficiency of algorithms

Documents

dce.kar.nic.indce.kar.nic.in/files...

simulation in a big data world - b. nikolic · simulation...

search algorithms winter semester 2004/2005 22 nov 2004 6th...

invitation- 6th govt. law college international law summit

invitation · - invitation - 2012 european challenge & 2012...

introduction to convex hull...

programming in python › ~harold › courses › old ›...

cesi - julien vaubourg · 2020. 6. 21. · inviter...

algorithm discovery and design - fordham...

educator guide - welcome | duke tip · student invitation...

1st international conference of nutrition 2017€¦ · 1st...

american society of exercise physiologistsamerican society...

hash tables - lehman...

invitation to computer science 6th edition

invitation story luxury wedding invitation designs

invitation to 6th chongqing international neurology · web...

chapter 3: the efficiency of algorithms invitation to...

vinay kumar madhadi 09/23/2009 csc-8320. outline part 1 :...

an invitation is an invitation is an invitation ... · an...

6th aquarius/sac-d science meeting 19-21 july 2010 seattle,...