analyzing algorithms growth of functions
Post on 22-Jan-2016
51 Views
Preview:
DESCRIPTION
TRANSCRIPT
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 1
Analyzing AlgorithmsAnalyzing AlgorithmsGrowth of FunctionsGrowth of FunctionsAnalyzing AlgorithmsAnalyzing AlgorithmsGrowth of FunctionsGrowth of Functions
(Chapters 1 & 2)
Mihaela IridonMihaela Iridon, Ph.D.
mihaela@engr.smu.edu
CSE 5350/7350Introduction to Algorithms
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 2
What means analyzing What means analyzing algorithms?algorithms?• Predicting the required resources• What do we measure?
– Computational time– Memory– Communication bandwidth– Other
• Model constructs:– Technology– Resources (hardware & software)– Associated costs– Assumptions (1 processor, RAM-model: sequential
operations)
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 3
Tools used for algorithm Tools used for algorithm analysisanalysis• Mathematical tools:
– Discrete combinatorics– Probability theory– Ability to single out the predominant
operations (most significant terms in a formula)
• Modeling and simulation tools• Software utilities, benchmarking
programs
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 4
TerminologyTerminology
• Input Size– In general the time to execute a set of
operations is dependent on the size of the input
– Depends on the problem definition– = the number of items in the input– Could be more than one number (e.g. a graph)
• Running Time– = the number of primitive operations (steps)
executed– Machine-independent term
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 5
Example (Example (11) – Insertion Sort () – Insertion Sort (In In PlacePlace))
/// <summary> /// Sorts the input list of integers by using the Insertion Sorting algorithm /// (see Cormen textbook, Chapter 1.1) /// </summary> /// <param name="input">Input list of integers (to be sorted) – input list will be modified</param> public static void InsertionSort(List<int> input) { if (input == null) return; //NULL input if (input.Count < 2) return; //one-element array; nothing to sort
int i=0, key=0; for (int j = 1; j < input.Count; j++) { key = input[j];
//insert input[j] into the sorted sequence input[0..j-1] i = j-1; while (i > -1 && input[i] > key) { input[i+1] = input[i]; i--; } input[i+1] = key; } }
/// <summary> /// Sorts the input list of integers by using the Insertion Sorting algorithm /// (see Cormen textbook, Chapter 1.1) /// </summary> /// <param name="input">Input list of integers (to be sorted) – input list will be modified</param> public static void InsertionSort(List<int> input) { if (input == null) return; //NULL input if (input.Count < 2) return; //one-element array; nothing to sort
int i=0, key=0; for (int j = 1; j < input.Count; j++) { key = input[j];
//insert input[j] into the sorted sequence input[0..j-1] i = j-1; while (i > -1 && input[i] > key) { input[i+1] = input[i]; i--; } input[i+1] = key; } }
Cost # of Times executedC1 n
C2 n-1
0 n-1C4 n-1C5 Σ(j=1..n-1)tj
C6 Σ(j=1..n-1)(tj – 1)C7 Σ(j=1..n-1)(tj – 1)
C8 n-1
Cost # of Times executedC1 n
C2 n-1
0 n-1C4 n-1C5 Σ(j=1..n-1)tj
C6 Σ(j=1..n-1)(tj – 1)C7 Σ(j=1..n-1)(tj – 1)
C8 n-1
1
1
n
jjt
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 6
Example (Example (11) – Insertion Sort () – Insertion Sort (In In PlacePlace))• The running time depends on the input
value– If the input is already sorted then the body of the while loop does
not execute and the best case scenario/running time for insertion sort is:
T(n) = c1 n + c2 (n-1) + c4 (n-1) + c5 (n-1) + c8 (n-1) = (c1 + c2 + c4 + c5 + c8) * n – (c2 + c4 + c5 +
c8) = a * n + b linear function of n
– If the input is in reverse sorted order: (worst case scenario)
T(n) = c1 n + c2 (n-1) + c4 (n-1) + c5 (n(n+1)/2 - 1) +
c6 [n(n-1)/2] + c7 [n(n-1)/2] + c8 (n-1) = a * n2 + b * n + c quadratic function
of n
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 7
Worst-case & Average-case Worst-case & Average-case AnalysisAnalysis
• Worst-case running time:– The longest running time for any input of size n
(i.e. the longest path in the execution)
– Upper bound on the running time for any input– Occurs fairly often– The average-case ~ the worst-case
• Average-case running time:– Difficult to define what average input means– Example for Insertion Sort: On average, half the
elements in an array A1 ... Aj-1 are less than an element Aj, and half are greater.
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 8
Order/Rate of GrowthOrder/Rate of Growth
• Simplifying abstraction• Consider only the leading term• Ignore the leading term’s constant
coefficient• Worst-case running time for
Insertion Sort is Θ(n2) (theta of n-squared)
• An algorithm with Θ(n2) will run faster than one with Θ(n3)
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 9
Divide-and-Conquer AlgorithmsDivide-and-Conquer Algorithms
• Incremental approach (e.g. Insertion Sort)
• Divide-and-conquer approach (e.g. recursive algorithms such as Merge Sort)– [Divide] Break the problem into related
or similar sub-problems of smaller size;– [Conquer] Solve the sub-problems– [Combine] Combine the solutions
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 10
Analyzing divide-and-conquer Analyzing divide-and-conquer algorithmsalgorithms• Recursive approach use recurrence
equation (recurrence) to describe the running time
• T(n) = running time on a problem of size n• If n= small (n <= c) then Θ(1)• Otherwise, divide problem in a sub-
problems, each of size n / b• D(n) = time to divide• C(n) = time to combine• T(n) = a*T(n/b) + D(n) + C(n) (when n > c)
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 11
Merge Sort (1)Merge Sort (1)
public static void MergeSort2(List<int> input, int startIx, int endIx)
{
if (input == null) return;
if (startIx == endIx) return; //stop condition
int middle = (int) Math.Floor((endIx + startIx) / 2.0);
MergeSort2(input, startIx, middle);
MergeSort2(input, middle + 1, endIx);
Combine2(input, startIx, middle, endIx);
}
public static void MergeSort2(List<int> input, int startIx, int endIx)
{
if (input == null) return;
if (startIx == endIx) return; //stop condition
int middle = (int) Math.Floor((endIx + startIx) / 2.0);
MergeSort2(input, startIx, middle);
MergeSort2(input, middle + 1, endIx);
Combine2(input, startIx, middle, endIx);
}
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 12
Merge Sort (2)Merge Sort (2)public static void Combine2(List<int> input, int i1, int i2, int i3)
{
if (input == null || input.Count == 0) return;
List<int> result = new List<int>(i3 - i1 + 1); //not 100% in-place
int ix1 = i1, ix2 = i2+1;
while (result.Count < i3 - i1 + 1)
{
while (ix1 < i2 + 1 && (ix2 == (i3 + 1) || input[ix1] < input[ix2]))
result.Add(input[ix1++]);
while (ix2 < i3 + 1 && (ix1 == (i2 + 1) || input[ix1] > input[ix2]))
result.Add(input[ix2++]);
}
for (int j = i1; j <= i3; j++)
input[j] = result[j - i1];
}
public static void Combine2(List<int> input, int i1, int i2, int i3)
{
if (input == null || input.Count == 0) return;
List<int> result = new List<int>(i3 - i1 + 1); //not 100% in-place
int ix1 = i1, ix2 = i2+1;
while (result.Count < i3 - i1 + 1)
{
while (ix1 < i2 + 1 && (ix2 == (i3 + 1) || input[ix1] < input[ix2]))
result.Add(input[ix1++]);
while (ix2 < i3 + 1 && (ix1 == (i2 + 1) || input[ix1] > input[ix2]))
result.Add(input[ix2++]);
}
for (int j = i1; j <= i3; j++)
input[j] = result[j - i1];
}
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 13
Analyzing Merge SortAnalyzing Merge Sort
• [Divide] D(n) = Θ(1)• [Conquer] 2 * T(n/2)• [Combine] C(n) = Θ(n)
• T(n) = Θ(1) if n=12T(n/2) + Θ(n) if n≥1= Θ(n * log2 n) better than Insertion
Sort
{
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 14
Growth of FunctionsGrowth of Functions
• Algorithms efficiency• Compare relative performance of
alternative algorithms• Analysis for large input size: e.g. n ∞• Asymptotic efficiency of algorithms:
– Input size is large enough to make only the order of growth of the running time relevant
– How the running time increases with the size of the input in the limit, as the size of the input increases without bound
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 15
Asymptotic NotationAsymptotic Notation
• Notations used to describe the asymptotic running time of an algorithm
• Are defined in terms of functions whose domains are the set N = {0, 1, 2, …}
• Convenient for describing the worst-case running-time function T(n)
• Notation abused vs. misused
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 16
ΘΘ-notation-notation• Θ: asymptotically bounds a function from
above and below
f(n) = Θ(g(n)) indicates f(n) Θ(g(n))
org(n) is an asymptotically tight bound for f(n)
})()()(0
0,,:)({))((
021
021
nnngcnfngc
nccnfng
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 17
OO-notation-notation• O: asymptotic upper bound
f(n) = Θ(g(n)) f(n) = O(g(n))• The Θ-notation is stronger than the O-notation• Example: n = O(n2)• E.g.: worst-case for insertion sort = O(n2)• Notation abuse: The running time of insertion sort is
O(n2). – The running time depends on the particular input of size n. – It is true only for the worst-case scenario (i.e. no matter what
particular input of size n is chosen for each value of n)
})()(0
0,:)({))((
0
0
nnncgnf
ncnfngO
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 18
ΩΩ-notation-notation• Ω: asymptotic lower bound
• Theorem f(n) and g(n), f(n) = Θ(g(n)) iff
f(n) = O(g(n)) and f(n) = Ω(g(n))
• Used to bound the best-case running time• E.g.: best-case for insertion sort is Ω(n)• Worst-case for insertion sort is Ω(n2)
})()(0
0,:)({))((
0
0
nnnfncg
ncnfng
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 19
Graphical ComparisonGraphical Comparison
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 20
oo-notation-notation
• Upper bound that is not asymptotically tight
• 2n2 = O(n2) is asymptotically tight• 2n = O(n2) is not asymptotically tight
• 2n = o(n2), but 2n2 o(n2)
})()(0
0,0:)({))((
0
0
nnncgnf
ncnfngo
0)(
)(lim
ng
nfn
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 21
-notation-notation
• Lower bound that is not asymptotically tight
• f(n) (g(n)) iff g(n) o(f(n))
• E.g.: n2/2 = (n), but n2/2 = (n2)
})()(0
0,0:)({))((
0
0
nnnfncg
ncnfng
)(
)(lim
ng
nfn
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 22
Comparison of FunctionsComparison of Functions
• Transitivity (for all five notations):If f(n) = X(g(n)) and g(n) = X(h(n)) f(n)
= X(h(n))• Reflexivity (for the big-X notations)
f(n) = X(f(n))• Symmetry:
f(n) = Θ(g(n)) iff g(n) = Θ(f(n))• Transpose symmetry:
f(n) = O(g(n)) iff g(n) = Ω(n)f(n) = o(g(n)) iff g(n) = (n)
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 23
Analyzing Algorithms - Analyzing Algorithms - AddendumAddendum
•Finding the largest clique in a graph•Parsing an object model
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 24
Finding the largest cliqueFinding the largest clique• Graph: G(V,E)• A graph or undirected graph G is an
ordered pair G: = (V,E) that is subject to the following conditions:
– V is a set, whose elements are called vertices or nodes, – E is a set of pairs (unordered) of distinct vertices, called
edges or lines.
• A clique in an undirected graph G is a set of vertices V such that for every two vertices in V, there exists an edge connecting the two. (Complete sub-graph)
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 25
The Clique ProblemThe Clique Problem
• determining whether a graph contains a clique of at least a given size k.
• Verification of actual clique : trivial• The clique problem is in NP (non-
deterministic polynomial time).• NP-complete • The corresponding optimization
problem, the maximum clique problem, is to find the largest clique in a graph.
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 26
Brute force approach to the clique Brute force approach to the clique problemproblem• Examine each sub-graph with at least k
vertices and check to see if it forms a clique• Number of cases to inspect:
• A clique C=(vi1, vi2, .., vin) exists only when its n sub-cliques each of size n-1 exist.
• Event-raising mechanism to increment a counter of sub-cliques using the threshold graph
• Space required to build all cliques until G=completely connected: 12..
432
n
n
nnnn n
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 27
Sorting an object modelSorting an object model
• Input:– A collection of property paths
• Output:– A sorted collection of property paths
Input Sample (excerpt)Input Sample (excerpt)• Company.Principals[0].References[0].ResidentialInfos[0].Address.AddressType• Company.Principals[0].References[0].ResidentialInfos[1].Address.AddressType• Company.Principals[0].References[1].ResidentialInfos[0].Address.AddressType• Company.Principals[0].References[1].ResidentialInfos[1].Address.AddressType• Company.Principals[1].References[0].ResidentialInfos[0].Address.AddressType• Company.Principals[1].References[0].ResidentialInfos[1].Address.AddressType• Company.Principals[1].References[1].ResidentialInfos[0].Address.AddressType• Company.Principals[1].References[1].ResidentialInfos[1].Address.AddressType• Company.Principals[0].References[0].ResidentialInfos[0].Address.StreetAddressLine1• Company.Principals[0].References[0].ResidentialInfos[1].Address.StreetAddressLine1• Company.Principals[0].References[1].ResidentialInfos[0].Address.StreetAddressLine1• Company.Principals[0].References[1].ResidentialInfos[1].Address.StreetAddressLine1• Company.Principals[1].References[0].ResidentialInfos[0].Address.StreetAddressLine1• Company.Principals[1].References[0].ResidentialInfos[1].Address.StreetAddressLine1• Company.Principals[1].References[1].ResidentialInfos[0].Address.StreetAddressLine1• Company.Principals[1].References[1].ResidentialInfos[1].Address.StreetAddressLine1• Company.Principals[0].References[0].ResidentialInfos[0].Address.StreetAddressLine2• Company.Principals[0].References[0].ResidentialInfos[1].Address.StreetAddressLine2• Company.Principals[0].References[1].ResidentialInfos[0].Address.StreetAddressLine2• Company.Principals[0].References[1].ResidentialInfos[1].Address.StreetAddressLine2• Company.Principals[1].References[0].ResidentialInfos[0].Address.StreetAddressLine2• Company.Principals[1].References[0].ResidentialInfos[1].Address.StreetAddressLine2• Company.Principals[1].References[1].ResidentialInfos[0].Address.StreetAddressLine2• Company.Principals[1].References[1].ResidentialInfos[1].Address.StreetAddressLine2
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 28
Object ModelObject Model
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 29
CompanyPrincipals[]FinancialStatements[]TradeLines[]Contacts[]Declarations[]Documents[]Addresses[]CompanyInfoBusinessAddressMailingAddressRelationshipSummary
PrincipalEmploymentInfos[]IncomeInfos[]TradeLines[]References[]Assets[]IdentificationInfos[]ResidentialInfos[]ContactInfoPersonalInfoPrincipalTypeEmployedByCompanyYearsAsOwnerIndividualOrJointTypePersonType
AddressAddressTypeStreetAddressLine1StreetAddressLine2CityCountyStateZipcodeCountry
1 1..*
1..*
1
Output sample (sorted object Output sample (sorted object model)model)• Company.Principals[0].EmploymentInfos[0].EmployerAddress.AddressType
• Company.Principals[0].EmploymentInfos[0].EmployerAddress.StreetAddressLine1• Company.Principals[0].EmploymentInfos[0].EmployerAddress.StreetAddressLine2• Company.Principals[0].EmploymentInfos[0].EmployerAddress.City• Company.Principals[0].EmploymentInfos[0].EmployerAddress.County• Company.Principals[0].EmploymentInfos[0].EmployerAddress.State• Company.Principals[0].EmploymentInfos[0].EmployerAddress.ZipCode• Company.Principals[0].EmploymentInfos[0].EmployerAddress.Country• Company.Principals[0].EmploymentInfos[0].EmployerName• Company.Principals[0].EmploymentInfos[0].EmploymentType• Company.Principals[0].EmploymentInfos[0].OccupationType• Company.Principals[0].EmploymentInfos[0].YearsOfEmployment• Company.Principals[0].EmploymentInfos[0].MonthsOfEmployment• Company.Principals[0].EmploymentInfos[0].Title• Company.Principals[0].EmploymentInfos[0].Department• Company.Principals[0].EmploymentInfos[1].EmployerAddress.AddressType• Company.Principals[0].EmploymentInfos[1].EmployerAddress.StreetAddressLine1• Company.Principals[0].EmploymentInfos[1].EmployerAddress.StreetAddressLine2• Company.Principals[0].EmploymentInfos[1].EmployerAddress.City• Company.Principals[0].EmploymentInfos[1].EmployerAddress.County• Company.Principals[0].EmploymentInfos[1].EmployerAddress.State• Company.Principals[0].EmploymentInfos[1].EmployerAddress.ZipCode• Company.Principals[0].EmploymentInfos[1].EmployerAddress.Country• Company.Principals[0].EmploymentInfos[1].EmployerName
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 30
Data GraphData Graph
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 31
ROOT ApplicationNumber
ConversationLogs Comment
DateTimeStamp
IncludeInUDR
Company Principals EmploymentInfos
FinancialStatements
IncomeInfos
TradeLines
Data StructuresData Structures
CSE 5350 - Fall 2007Analyzing AlgorithmsGrowth of Functions Slide 32
Node
Text : stringCount : intCrtIndex : intIsLast : boolNextNode : NodeRightNode : Node
top related