evaluate algorithms
Post on 10-Apr-2018
225 Views
Preview:
TRANSCRIPT
-
8/8/2019 Evaluate Algorithms
1/36
1
Algorithm and Complexity
Sheng Zhong
Computer Sci and Engr DeptSUNY Buffalo
-
8/8/2019 Evaluate Algorithms
2/36
2
Algorithm
The core of computer science is algorithm.
An algorithm is a finite sequence of
precise instructions for performing acomputation or for solving a problem.
Example:
Algorithm: go to NSC building; find room 210;find the 2nd row; sit in the 4th seat from the left;
calculate x=5+3.
-
8/8/2019 Evaluate Algorithms
3/36
3
Example non-algorithm
Algorithm cant be vague: go to NSC building;find room 210; find the 2nd row; sit in the 4th seat;calculate x=5+3.
The 4th seat from the left or from the right?
Doing any step of algorithm cant require
creativity or non-trivial thinking: go to NSCbuilding; find room 210; find the 2nd row; sit in
the 4th seat from the left; determine whetherP=NP and give a proof.
It requires creativity to solve the P vs. NP problem.
-
8/8/2019 Evaluate Algorithms
4/36
4
Example of non-algorithm
Algorithm cant be infinite: go to NSC
building; find room 210; find the 2nd row; sit
in the 4
th
seat from the left; calculatex=5+3, calculate x=5+4, calculate
x=5+5, ... calculate x=5+n,
Note that, if the above sequence of
instructions stops at calculatingx=5+1000,000,000, it is still an algorithmbecause it is finite.
-
8/8/2019 Evaluate Algorithms
5/36
5
Algorithm vs. program
A computer program is actually an algorithmwritten in a specific programming language. The advantage is that it can be understood and
executed by a computer.
The disadvantage is that it is hard for human beingsto understand the algorithm.
So usually algorithms are written in pseudo-codeor technical English, which is easier tounderstand. Translating an algorithm to a program is called
implementation of the algorithm in thecorresponding programming language.
-
8/8/2019 Evaluate Algorithms
6/36
6
How to write an algorithm
Below is an algorithm for finding the
maximum element in a finite sequence.
procedure max (a1, a2, , an: integers)max := a1for i:=2 to n
if max
-
8/8/2019 Evaluate Algorithms
7/36
7
Example: binary search
To demonstrate how algorithms are
designed, now we consider a problem of
searching: given a sequence of integers a1,a2, , an , where a1< a2
-
8/8/2019 Evaluate Algorithms
8/36
8
The idea
What if we start our search from the middle ofthe sequence?
Lets say we compare ai with x.
Well be very happy if we are lucky enough to find ai=x, in which case we are done.
If we find ai x, we know that all integers to the right of
ai are all greater than x, and so we only need to lookat those to the left of ai .
-
8/8/2019 Evaluate Algorithms
9/36
9
Example of binary search
Lets say we are searching for 13 in
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19,20, 22.
First, we compare 13 with the number inthe middle, 10.
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18,
19, 20, 22.We find that 10
-
8/8/2019 Evaluate Algorithms
10/36
10
Example of binary search
Now our remaining sequence is
12, 13, 15, 16, 18, 19, 20, 22.
Again we compare 13 with the number inthe middle, 16.
12, 13, 15, 16, 18, 19, 20, 22.
This time we find that 16>13 and hence weonly need to look at the numbers to the left.
-
8/8/2019 Evaluate Algorithms
11/36
11
Example of binary search
Now our remaining sequence is
12, 13, 15
We compare 13 with the integer in themiddle, 13, and find that 13=13. So we are
done! This is exactly the one we need. It is
the 10th in the original sequence.
-
8/8/2019 Evaluate Algorithms
12/36
12
Binary search algorithm
Formally we can write the algorithm as follows:procedure binary search(x: integer, a1, a2, , an :increasing integers)
i:=1 {i is the left endpoint of search interval}
j:=n {j is the right endpoint of search interval}
while iam then i:=m+1
else if x
-
8/8/2019 Evaluate Algorithms
13/36
13
Other algorithm examples
The textbook has a few more examples of
algorithms like bubble sort, greedy change
making, etc. You should read them and understand howthey work.
-
8/8/2019 Evaluate Algorithms
14/36
14
Can algorithms solve all problems?
Computer algorithms are so useful. Theyhave applications in almost all aspects oftodays society. But can they solve all the problems for us?
Are they omnipotent?
The answer is NO. It suffices to observe that the set of algorithms
is countable, but the set of problems to solveis uncountable. We simply dont have enoughalgorithms to solve all the problems.
-
8/8/2019 Evaluate Algorithms
15/36
15
Example unsolvable problem
A famous problem that cant solved by algorithms is theTuring halting problem. The problem has the description of a Turing machine (you can
temporarily understand it as a computer program) as an input.
It asks whether this Turing machine will halt. (That is, whetherthe computer program will terminate after running for some time,or it will run forever.)
Unfortunately, there is no algorithm that can solve this problemfor us.
WARNING: The proof in the textbook is NOT rigorous.You should skip it. You should be able to see a rigorous proof in your theory class
(CSE 396).
-
8/8/2019 Evaluate Algorithms
16/36
16
Evaluation of algorithm
Suppose you are given an algorithm for solving acertain problem. How can you evaluate it? You can evaluate its running time (i.e., time
complexity).
You can evaluate the space it requires (i.e., spacecomplexity).
You can evaluate the amount of communications itneeds (i.e., communication complexity), the number ofrandom bits it needs,
For each of the above metric, we can evaluatethe algorithm in the average case and in theworst case. In computer science, people are normally more
interested in the worst case.
-
8/8/2019 Evaluate Algorithms
17/36
17
How to measure complexity
How do we measure complexities? For example,consider the worst-case time complexity of analgorithm.
Ideally, we should give the number of seconds thealgorithm requires in the worst case.
Unfortunately, this number can be significantlyaffected by the kind of computer you use. A supercomputer can be millions times faster than an
embedded computer. So, we count the number of basic operations (e.g.,
addition, multiplication, comparison, etc.) in stead.
-
8/8/2019 Evaluate Algorithms
18/36
18
Example complexity
What is the worst case time complexity of thealgorithm below?procedure factorial (x:integer)r:=1;for i:=1 to x
r:=r * i{r is the output}
We can easily see it requires x multiplications
(and also some other operations). Note that thisvaries with different values of input. But we dont want the complexity of an algorithm to
be a variable.
-
8/8/2019 Evaluate Algorithms
19/36
19
Complexity as a function
In general, the running time (and space,
amount of communications, ) of analgorithm vary greatly with different inputs.
Specifically, with larger inputs, we may needmore running time, more space
So, instead of using a variable to describe the
complexity, we use a function whoseargument is the length of input.
-
8/8/2019 Evaluate Algorithms
20/36
20
Example complexity
Lets look at the worst case time complexity ofthe algorithm below again.procedure factorial (x:integer)
r:=1;for i:=1 to x
r:=r * i
{r is the output}
Suppose the input has n bits. Then we needabout2n multiplications in the worst case. Here 2n is a function in n that we use to describe the
complexity.
-
8/8/2019 Evaluate Algorithms
21/36
21
Growth of function
Suppose we have three algorithms, and their worst casetime complexities are n+10, n2, and n2-1, respectively.How do they compare with each other? When n is small (say, n=2), n+10 can be larger than n2 and n2-1,
but we dont care because for smaller inputs we always havesufficient time to run the algorithm.
When n is large, n+10 is much smaller than n2 and n2-1, andthere is no significant difference between n2 and n2-1. This iswhat we really care about.
In general, when we consider complexities, we often look
at how fast a function grows, rather than look at thefunction itself. This called the asymptotic complexity of the function.
-
8/8/2019 Evaluate Algorithms
22/36
22
Big O notation
To specify the asymptotic complexity, we often use thebig O notation:
Let f and g be functions from R+ to R. We sayf(x)=O(g(x)) if there are constants C and k such that,
for all x>k, |f(x)| !!!! C|g(x)|. In the above inequality, the symbols of absolute values
can often be eliminated because we often restrict ourattention to positive valued functions. Even if a function is not positive everywhere, it should always be
positive except for a few small values of x. Here we use x as the name of argument variable. Note
that it is just n (the length of input) when we discusscomplexities.
-
8/8/2019 Evaluate Algorithms
23/36
23
Example big O
Suppose f(x)=2x3+8x. Show that f(x)=O(x3).
Proof: For each x>2, we have that
|f(x)| = 2x3+8x= 4 ( x3+2x)
< 4 x3
= 4 | x3 |.So f(x)= O(x3).
-
8/8/2019 Evaluate Algorithms
24/36
24
Equality symbol in big O
Note that f(x)=O(g(x)) is NOT a real
equality.
For example, you cant rewrite it asO(g(x))=f(x).
It does NOT mean f(x) is equal to a functioncalled O(g(x)).
It is written this way just for easiness. You must be careful with this notation.
-
8/8/2019 Evaluate Algorithms
25/36
25
Big O of polynomial function
Theorem: Suppose f(x) is a degree-n polynomial.Then f(x) = O(xn).
Proof: Since f(x) is degree-n polynomial, we can write
it as f(x)=anxn+an-1xn-1++a1x+a0.
So, for x>1 we have
|f(x)| = |anxn+an-1x
n-1++a1x+a0|
! |an| xn
+|an-1|xn-1
++|a1|x+|a0|= xn (|an| +|an-1| / x++|a1| / x
n-1+|a0| / xn)
! |xn |(|an| +|an-1|++|a1|+|a0|).
Hence, f(x)= O(xn).
-
8/8/2019 Evaluate Algorithms
26/36
26
Example polynomial big O
Suppose f(n)=1+2++n. Show that
f(n)=O(n2).
Proof. First, we note that f(n) is the sum ofan arithmetic sequence. Hence,f(n)=n+n(n-1)/2=n(n+1)/2.
Next, applying the theorem we just studied,
we get that f(n)=O(n2).
-
8/8/2019 Evaluate Algorithms
27/36
27
Big O of logarithm
It is easy to show the following fact:
For k "0, xk log x= O(xk+1).
For k "0, xk= O(xk log x). In particular, we have
log x= O(x).
1= O(log x).
-
8/8/2019 Evaluate Algorithms
28/36
28
Big O of sum
Theorem: Suppose f1 (x)=O(g1 (x)) and f2(x)=O(g2 (x)).Then
f1(x)+f2(x)=O(max(|g1(x)|,|g2(x)|))Proof: By definition of big O, we know there exist constants
k1, C1, k2, C2 , such thatfor all x>k1, |f1 (x)| !C1 |g1 (x)|;for all x>k2, |f2 (x)| !C2 |g2 (x)|.When x> both k1 and k2,|f1(x)+f2(x)| ! |f1(x)|+|f2(x)| ! C1 |g1 (x)|+ C2 |g2 (x)|
! C1 max(|g1(x)|,|g2(x)|)+ C2 max(|g1(x)|,|g2(x)|)! (C1 + C2 ) max(|g1(x)|,|g2(x)|),which means f1(x)+f2(x)=O(max(|g1(x)|,|g2(x)|)).
-
8/8/2019 Evaluate Algorithms
29/36
29
Example big O of sum
What is the big O of f(x)=x2+x2logx ?
Using the rule we just learned, f(x)= O(max(x2,x2logx))=O(x2logx).
What is the big O of f(x)=x3+x2logx ?
Using the rule we just learned, f(x)= O(max(x3,
x2logx))=O(x3).
-
8/8/2019 Evaluate Algorithms
30/36
30
Big O of product
We can also easily prove the followingtheorem:
Suppose f1 (x)=O(g1 (x)) and f2(x)=O(g2(x)). Then
f1(x)f2(x)=O(g1(x)g2(x)).
Example: f1 (x)= x2 +x+1, f2 (x)= x
3+2x+1.
So we have f1 (x)=O(x2
) and f2(x)=O(x3
).Then,
f1(x)f2(x)=O(x5).
-
8/8/2019 Evaluate Algorithms
31/36
31
Example big O
Use big O to estimate
f(x)= (xlogx+x2) (x+logx).
First, using the rule for sum, we havexlogx+x2= O(xlogx,x2)=x2.
Second, using the rule for sum, we have
x+logx= O(x,logx)=x.
Finally, using the rule for product, we have
f(x)=O(x3).
-
8/8/2019 Evaluate Algorithms
32/36
32
Big Omega notation
Intuitively, f(x)=O(g(x)) means f(x) grows
no faster than g(x) asymptotically.
But what if we want to say f(x) grows noslower than g(x) asymptotically?
We use so called big Omega notation.
We say f(x)=#$%$&''()*(%$&'+,$*$&''-
. /&012345(6)784(39%(&+(,$&':(;4(%4
-
8/8/2019 Evaluate Algorithms
33/36
33
Big Theta notation
Now we have ways to say f(x) grows no faster orno slower than g(x) asymptotically.
What if we want to say f(x) grows exactly as fast as
g(x) asymptotically? We use so called big Theta notation.
We say f(x)=>$%$&''()*(*$&'+,$%$&''(07?(
%$&'+,$*$&''-
. /&012345(6)784(39%(&+(,$39% &(@A'(07?(39%(&(@A(+(,$39% &':(;4(%4
-
8/8/2019 Evaluate Algorithms
34/36
34
Symmetry of big Theta
By the definition of big Theta, clearly we
have f(x)= >(g(x)) if and only if g(x)=
>(f(x)).
We also have that f(x)=>$%$&''()*(07?(
973B()*(*$&'+(#$%$&''(07?(%$&'+(
#$*$&''-
-
8/8/2019 Evaluate Algorithms
35/36
35
Frequently used results
In the textbook, you can find a number of
frequently used results on big O, bigOmega, and big Theta notations.
You should memorize these results, so thatyou can use them to solve problems.
-
8/8/2019 Evaluate Algorithms
36/36
36
Homework 6
Rosen 3.2: Questions 10, 20, 30.
Rosen 3.3: Questions 10, 26.
top related