02_the role of algorithms in computing
TRANSCRIPT
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 1/37
What is an Algorithm
An algorithm is all about the procedure with
the input and output
1
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 2/37
The Sort Problems
Input
A sequence of numbers <a1, a2, ,an>
Output A permutation (reordering) of the input sequence
such that a1a2an
2010/11/30The Role of Algorithms in Computing 2
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 3/37
Correct Algorithms
A correct algorithm
Always produces the correct output for any input
For example: Input <31,41,59,26,41,58>
>>>>>A correct algorithm<<<<<
<26,31,41,41,58,59>
2010/11/30The Role of Algorithms in Computing 3
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 4/37
What is an Algorithm?
A detail step by step instruction
Describe the instruction in words
What kind of computer programming languages? Doesnt matter
What kind of human spoken languages?
The language you like, even Chinese, Martian
Language
2010/11/30The Role of Algorithms in Computing 4
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 5/37
How to Solve the Sort
Algorithms? Selection Sortvoid selectionSort(int numbers[], int array_size)
{
int i, j, min;
for (i = 0; i < array_size-1; i++)
{
min = i;
for (j = i+1; j < array_size; j++)
{
if (numbers[j] < numbers[min])
min = j;
}
if(min!=i) swap(numbers[x],numbers[i]);
}
}2010/11/30The Role of Algorithms in Computing 5
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 6/37
How to Solve the Sort
Algorithms? II MergeSort
Separate the list into halves
Sort the two halves recursively Merge the two sorted halves into one sorted list
2010/11/30The Role of Algorithms in Computing 6
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 7/37
Which one is better?
Better means
Time
Space Generally focusing on the worst-case number
of comparisons required to solve the
problem.
2010/11/30The Role of Algorithms in Computing 7
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 8/37
Time Needed
Selection Sort
n-1, n-2, n-3, ., 1
O(c1n2) MergeSort
T(n)=T(n/2) +T(n/2) + cn
T(n)=O(c2nlog2n)
2010/11/30The Role of Algorithms in Computing 8
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 9/37
Time Needed
MergeSort Wins
And Space?
2010/11/30The Role of Algorithms in Computing 9
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 10/37
Think about
Two machine
Machine A: 109 per second instructions
MachineB: 106
per seconds instructions Supposed c1=2, c2=50
We want to sort 102 integers
Which one runs faster? We want to sort 106 integers
Which one runs faster?
2010/11/30The Role of Algorithms in Computing 10
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 11/37
Problems in the Computer?
The Human Genome Project
Identify 100,000 genes
Determine 3 billion chemical based pairs Genes search between different human
2010/11/30The Role of Algorithms in Computing 11
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 12/37
Problems in the Computer?
Internet and the Web
IP route method.
Search Engine Compress the data
How to store the password (RSA)
NetworkSecurity
Lots of problems
2010/11/30The Role of Algorithms in Computing 12
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 13/37
Problems in the Computer?
Manufacturing
Resources price
Human power price Transportation price
Market
The place to set the
Factories
Warehouse
Stores
2010/11/30The Role of Algorithms in Computing 13
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 14/37
Applications in Computer
Hardware
GUI
Object-OrientedSystem WAN, LAN
2010/11/30The Role of Algorithms in Computing 14
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 15/37
Thinks About
Program P1 solves a problem in n days.
Program P2 solves the same problem in 2n
seconds.
Q: Which one you will use?
A: Program P1 runs faster forn
> 20.
2010/11/30The Role of Algorithms in Computing 15
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 16/37
Motivation
The rate of growth affects more.
We need notation to capture the concept of
rate of growth when we measure the timeand space efficiency of algorithms
2010/11/30The Role of Algorithms in Computing 16
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 17/37
Asymptotic Notation
Edmund Landau
1877~1938
Inventor of the asymptotic notation Donald E. Knuth
1938 ~
Turing Award, 1974.
Father of the analysis of algorithms
Popularizing the asymptotic notation
2010/11/30The Role of Algorithms in Computing 17
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 18/37
Asymptotic Notation
2010/11/30The Role of Algorithms in Computing 18
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 19/37
Using Asymptotic Notation in
Sentences Let n be the number of input integers
Program 1 takes time O(n)
It takes time O(n) for program 1 to solve theproblem
The time required by program 2 is O(2n)
O(n) reads?
Big-Oh of n
Order n
2010/11/30The Role of Algorithms in Computing 19
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 20/37
Meaning
Sentence
Program p2 takes time O(n) to solve the problem.
Means The time needed by p2 is a function f(n) with
f(n)=O(n)
2010/11/30The Role of Algorithms in Computing 20
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 21/37
Meaning 2
Sentence
The comparison-based sort methods can not be
solved in O(n) time. Means
The comparison-based sort problems can not besolved in h(n) time, for any function h(n) with
h(n)=O(n)
2010/11/30The Role of Algorithms in Computing 21
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 22/37
f(n)=O(g(n))
Intuitive meaning
f(n) does not grow faster then g(n)
Comments f(n)=O( g(n) ) roughly means f(n) g(n) in terms of
rate of growth
= is not the meaning of equation. Its more like
We do not write O(g(n))=f(n)
2010/11/30The Role of Algorithms in Computing 22
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 23/37
Formal Definition of O
For any two functions f(n) and g(n), we write
f(n)=O(g(n))
If there exists positive constants n0 and c suchthat the inequality f(n) c g(n)
holds for each integer n n0
2010/11/30The Role of Algorithms in Computing 23
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 24/37
In Other Words
The definition of f(n)=O(g(n)) states that
there exists a positive constant c such that
the value of f(n) is upper-bounded by cg(n)for all sufficiently large positive integers n.
2010/11/30The Role of Algorithms in Computing 24
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 25/37
Questions
n=O(n)?
999n=O(n)?
5n2=O(n3-n2)? n2=O(n)?
2010/11/30The Role of Algorithms in Computing 25
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 26/37
n=O(n)?
Yes, for n0=1 and c=1
n 1n , for n 1
2010/11/30The Role of Algorithms in Computing 26
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 27/37
999n=O(n)?
Yes, for n0=1, c=999
999n 999n , for n 1
2010/11/30The Role of Algorithms in Computing 27
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 28/37
5n2=O(n3-n2)?
Yes, let n0=2, c=5
5n25(n3-n2)
5n2 5n3-5n2
10n2 5n3
2 n
2010/11/30The Role of Algorithms in Computing 28
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 29/37
n2=O(n)?
Is n can be the upper-bound of n2
Instinctive feeling: NO
How to proof? Contradiction Method
2010/11/30The Role of Algorithms in Computing 29
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 30/37
Proof
Assume for a contradiction that there exists
constants c and n0 such that
n2
cn holds for any integer n with n n0. n is an
arbitrary integer strictly larger than
max(n0,c ).
For instance, let n = 1+max(n0,c)
n2>cn (n>c), contradiction.
2010/11/30The Role of Algorithms in Computing 30
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 31/37
Other Notations
Big-Oh only gives the upper bounds. We need
other asymptotic notations.
f(n)=O(g(n)) f(n) g(n) in rate of growth
f(n)=(g(n)) f(n) g(n) in rate of growth
f(n)=(g(n)) f(n) g(n) in rate of growth
f(n)=o(g(n)) f(n) g(n) in rate of growth
f(n)= (g(n)) f(n) g(n) in rate of growth
2010/11/30The Role of Algorithms in Computing 31
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 32/37
Omega
Definition [Omega]
f(n) = (g(n)) iff there exist positive constants
c and n0
such that f(n) cg(n) for all n, n n0.
The function g(n) is only a lower bound on f(n).
f(n) = (g(n)) to be informative, g(n) should be
as large a function of n as one can come up
with for which f(n) =
(g(n)) is true.
2010/11/30CH1: Basic Concepts 32
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 33/37
Theta
Definition [Theta]:
f(n) = (g(n)) iff there exist positive constants
c1, c
2and n
0such that c
1g(n) f(n) c
2g(n) for
all n, n n0.
Lower bound and upper bound on f(n)
The theta notation is more precise than
both ³big oh´ and omega notations.
2010/11/30CH1: Basic Concepts 33
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 34/37
Comparing Two Algorithms
We say that Algorithm A is no worse than
Algorithm B in terms of worst-case time
complexity if there exists a positive functionf(n) such that
Algorithm A runs in time O(f(n)) and
AlgorithmB runs in time(f(n)) in the worst case
2010/11/30The Role of Algorithms in Computing 34
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 35/37
Comparing Two Algorithms (2)
Algorithm A is strictly better than algorithm B
in terms of worst-case time complexity if
there exists a positive function f(n) such that Algorithm A runs in time O(f(n)) and
AlgorithmB runs in time (f(n)) in the worst case
OR
Algorithm A runs in time o(f(n)) and
AlgorithmB runs in time (f(n)) in the worst case
2010/11/30The Role of Algorithms in Computing 35
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 36/37
Tightness in Analysis
Supposed we figure out that the time
complexity of a algorithm is O(f(n)). We say
that the analysis is tight if the algorithm runsin(f(n)) in the worst case.
In others words, if the time complexity of the
algorithm is O(f(n)) and the analysis is tight,
then the time complexity of the algorithm is(f(n))
2010/11/30The Role of Algorithms in Computing 36
8/8/2019 02_The Role of Algorithms in Computing
http://slidepdf.com/reader/full/02the-role-of-algorithms-in-computing 37/37
Time Optimal Algorithm
We say that Algorithm A is a optimal
algorithm for a problem P in terms of worst-
case time complexity if Algorithm A runs in time O(f(n)) and
Any algorithm that solves the problem P requirestime(f(n)) in the worst case.
2010/11/30The Role of Algorithms in Computing 37