csc 1700 – analysis of algorithms: lower bounds€¦ · · 2016-12-13csc 1700 – analysis of...

Villanova University Department of Computing Sciences

CSC 1700 – Analysis of Algorithms: Lower

Bounds

Professor Henry CarterFall 2016


Extra Credit

• Puzzle questions requiring the application of you algorithms knowledge

• 20 possible points, 4 questions

• Replaces the next two lowest quiz grades (after dropped quizzes)

• You may collaborate with one partner and submit your solutions jointly (cite your collaborator)

• Due Monday, Dec. 12

2


Recap

• Iterative improvement algorithms take a feasible solution and iteratively improve it until optimized

• Simplex method: linear equations

• Ford-Fulkerson: network flow

• Bipartite matching: matching cardinality or matching weight

3


How low can you go?

• Algorithmic power explored so far has been powerful

• Several implicit limitations have been encountered

• These limitations established through lower-bound proofs (big-Ω)

4


Proving the lower bound

• Trivial limits

• Problem reduction

• Adversarial argument

• Information-theoretic

5


Trivial Limits

• A simple lower bound can be established based on the size of the input and output

• Every input element must be read

• Every output element must be written

6


Examples

• Enumerating permutations of n elements

• Evaluating a polynomial

• N x N matrix multiplication

7


Tight Bounds

• The tightness of the bound indicates how close the lower bound is to the actual known lower bound

• If a lower bound is known along with an algorithm with the same efficiency, the bound is "tight"

• Examples:

‣ Polynomial evaluation

‣ N x N matrix multiplication

8


Problem Reduction

• If we have a lower bound for problem Q, it can be applied to other problems

• Reducing Q to P implies that the worst-case lower bound for Q also applies to P

• Since the worst-case Q requires Ω(n) operations no matter how you try to solve it, reducing the problem to P and solving P also faces the same lower bound

9


Common problems for reduction

10

Problem Lower Bound Tightness

sorting Ω(n log n) yes

Searching a sorted array

Ω(log n) yes

Element uniqueness Ω(n log n) yes

n-digit integer multiplication

Ω(n) unknown

n x n matrix multiplication

Ω(n2) unknown


Euclidean Minimum Spanning Tree

11


(Matrix) Multiplication

• Integer multiplication:

• Matrix multiplication

12

x · y =(x+ y)2 � (x� y)2

4x

2 = x · x

X =

0 AAT 0

�Y =

0 BT

B 0

�


Adversarial Arguments

• An adversarial player generates inputs during execution to incur maximum comparisons

• The adversary must remain consistent with previous choices

• Determine how many comparisons the adversary can force

• Example: merging two sorted arrays

13


Information-Theoretic Arguments

• Determine the minimum number of questions required to gather the necessary "information"

• Example: guessing a number between 1 and n

• Enumerate possible execution paths with decision trees

• The maximum and average heights of the tree correspond to the worst-case and average number of questions (comparisons)

14


Example tree: Insertion Sort

15


Sorting Lower Bound

• Lower bound on height:

• Stirling's formula: n! ≈ (2πn)1/2 (n/e)n

• Average height:

16


Example tree: Searching

17


Example tree: Searching

18


Searching Lower Bound

• Representation-dependent

• Ternary decision trees:

‣ Not tight

• Binary decision trees:

19


Practice: finding the median of three elements

• What is the information-theoretic lower bound using comparison-based algorithms?

• Draw a decision tree

• Is the bound illustrated by your decision tree tight? Can we get a better algorithm?

20


Recap

• Algorithmic power is broad but limited

• Lower bounds determine whether an algorithm can be improved by more than a constant factor

• Established through input size, problem reduction, adversarial generation, and information-theoretic decision trees

21


Next Time...

• Levitin Chapter 11.3

‣ Remember, you need to read it BEFORE you come to class!

• Homework:

‣ 11.1: 1, 2, 3ab

‣ 11.2: 1, 3a

22

csc 1700 – analysis of algorithms: lower bounds€¦ · · 2016-12-13csc 1700 – analysis of...

Documents