Data Structure:1 Recurrence Relation

Solving Recurrences Using Induction We consider first a recursive algorithm that compute ! Algorithm :1 Factorial Problem: Determine ! = . . when . Inputs: a nonnegative integer . Outputs: ! int fact (int n) { If (n==0) return 1; else return ( ); }

To gain insight into the efficiency of this algorithm, lets determine how many times this function does multiplication instruction for each value of . For a given , the number of multiplications done is the number done when ( ) is computed plus the one multiplication done when is multiplied by . If we represent the number of multiplication done for a given value of by , we have established that

= + Multiplications Multiplications In recursive call at top level An equation such as this is called a recurrence equation because the value of function at is given in terms of the value of the function at a smaller value of . A recurrence by itself does not represent a unique function. We must also have a starting point, which is called initial condition. In this algorithm, no multiplications are done when = .,

= .

We can compute for larger values of as follows: = + = + = = + = + = = + = + =

Continuing in this manner gives us more and more values of ,but it does not enable us to compute for arbitrary without starting at . We need an explicit expression for . Such an expression is called a solution to the recurrence equation . Recall that it is not possible to find a solution using induction. Induction can only verify that a candidate solution is correct. We can obtain a candidate solution to this recurrence by inspecting the first few values. An inspection of the values just computed indicates that

= Is the solution. Now that we have a candidate solution, we can use induction to try to prove that it is correct. Induction base: For = ,

= . Induction hypothesis: Assume, for arbitrary positive integer , that

= .

Induction step: We need to show that = + .

If we insert + in the recurrence, we get + = + + = + = + .

This completes the induction proof that our candidate solution is correct. There are two steps in the analysis of a recursive algorithm. The first step is determining the recurrence; the second step is solving it. Our purpose here is to show how to solve recurrences. Determining the recurrences of the recursive algorithm here is done when we discuss the algorithms. Therefore , in we take the recurrences as given. We now present more examples of solving recurrences using induction. Ex: Consider the recurrence

= + for > , a power of 2 =

The first few values are =

+ = + = + =

=

+ = + = + = =

+ = + = + =

=

+ = + = + = . It appears that

= + . We use induction to prove that this is correct. Induction base: For = ,

= = + . Induction hypothesis: Assume , for an arbitrary > and a power of 2, that

= + .

Induction step: Because the recurrence is only for power of 2, the next values to consider after is . Therefore, we need to show that

= + .

If we insert in the recurrence, we get = ()/ + = + = + + = + + = + .

Ex.: Consider the recurrence = / for > , a power of 2

= . The first few values are

= / = = = / = = = / = = = / = =

It appears =

We use induction to prove that it is correct. Induction base: For = ,

= = = . Induction hypothesis: assume, for an arbitrary > and a power of 2, that

= .

Induction step: We need to show that = ().

If we insert in the recurrence, we get = ()/ = = = + = + = ().

This completes the induction proof. Finally because () = .() = () () = ()

,we observe that = ()

Or = = .

For many recurrence relation, we have no obvious candidate solution. Therefore the method of induction cannot be applied in such a case.

Solving Recurrences Using the Characteristic Equation We develop a technique for determining the solutions to a large class of recurrences. 1. Homogeneous linear Recurrences Definition: A recurrence of the form

+ + + = Where and the terms are constants, is called a homogeneous linear recurrence equation with constant coefficients.

Ex.: The Fibonacci sequence is defined as: = + ; = , =

The foregoing equation may be written as: = ,

So the Fibonacci sequence is a linear homogeneous equation.

Ex. Suppose we have the recurrence + = ; = , =

By assuming a solution of the form =

We find that + = + =0

Or + =

The roots are = , . Therefore

= , = = Are the solution to the recurrence. We write the general solution as:

= + Using the boundary conditions , the solution becomes

=

Theorem:1 Let the homogeneous linear recurrence equation with constant coefficients

+ + + = be given. If its characteristic equation

+ + + =

has distinct solutions , , , , then the only solutions to the recurrence are = + + ,

where the terms are arbitrary constants. Ex.: We solve the recurrence that generates the Fibonacci sequence

= , = , = Obtain the characteristic equation as:

= The roots are = , = + which are distinct.

Applying the theorem:1 , we write the solution as:

= + + Using the two initial conditions , one may evaluate

= , = . In the foregoing discussion, we observe that if the roots of characteristic equation are distinct , then the solution is governed by Theorem:1 In case some roots are same we may write the solution based on the following theorem. Theorem:2 Let be a root of multiplicity of the characteristic equation for a homogeneous linear recurrence with constant coefficients. Then

= , = , = , , = Are all solutions to the recurrence. Therefore , a term for each of these solutions is included in the general solutions to the recurrence.

Ex. We solve the recurrence + = ; = , = , = .

The characteristic equation in this case is given by

+ = . The roots are

= , = . Applying the theorem:2 , the solution may be written as:

= + + . Using the initial conditions, the general solution may be written as:

= + because = , = , = .

Nonhomogeneous Linear Recurrences Definition A recurrence of the form

+ + + = () where and the terms are constants and () is a function other than the zero function, is called a nonhomogeneous linear recurrence equation with constant coefficients. We next develop a method for solving the common special case

+ + + = = () where is a constant and () is a polynomial in . Ex. The recurrence

= is an example where = , = , () = . If = + then following recurrence is obtained

= + .

Theorem:3 A nonhomogeneous linear recurrence of the form

+ + + = () Can be transformed into a homogeneous linear recurrence that has the characteristic equation + + + + = , Where is the degree of . Notice that the characteristic is composed of two parts: 1. The characteristic equation for the corresponding homogeneous recurrence 2. A term obtained from the nonhomogeneous part of the recurrence If there is more than one term like () on the right side, each one contributes a term to the characteristic equation.

Ex. We solve the recurrence = ( + ), = , =

The characteristic equation for the corresponding homogeneous equation: = .

For the nonhomogeneous part = ( + )

Therefore we have = = . So the term from the nonhomogeneous part is

+ = + Therefore applying theorem:3, the characteristic equation becomes

=0 The solutions are = , = . Therefore the solution is

= + + . Using the initial conditions, we find

= + .

Ex. We solve the recurrence = ; =

The characteristic equation from the homogeneous recurrence may be written as:

= . Obtain a term from the nonhomogeneous part of recurrence

= = () The term is

+ Applying theorem:3 to obtain the characteristic equation:

= The solution is = and of multiplicity 3. Therefore the general solution is

= + + . From the recurrence , using the initial condition, we may generate two more initial conditions:

= , = . Using these conditions, we find the general solution as:

= ( )

Change of Variables ( Domain transformation) Sometimes a recurrence that is not in the form that can be solved using Theorem:3 but can be solved by performing a change of variables to transform it into a new recurrence that is in that form. The technique is illustrated in the following examples. In these examples, we use () for the original recurrence, because is used for the new recurrence. The notation () means the same things as -- namely , that a unique number is associated with each value of . Ex.: We solve the recurrence

=

+ , = and a power of 2. We solved this equation earlier using a different method. Here we use the change of variable technique. It is to be noted that the form of the recurrence is not in the form so as to apply theorem:3 because of the term . We can transform it into a recurrence that is in that form as follows: First set

= , which means that = . Second , substitute for in the recurrence to obtain

=

+ = +

Next , set =

So as to obtain the new recurrence = + .

This new recurrence is in the form that we can use Theorem:3. Therefore, applying that theorem, we can determine the solution as:

= + . Using back substitution and also the initial condition, we find that

= + . Ex.: We solve the recurrence

() =

+ , = a power of . Following the method described earlier , we may write

=

+ . Next, set

=

To obtain = + .

Now apply Theorem:3 to obtain = + + .

Now back substitution yields = + + = + + .

Next, use the initial condition = . Therefore the solution becomes:

= .

Solving recurrence by substitution Sometimes a recurrence can be solved using a technique called substitution. The following examples illustrate the substitution method. Ex.: We solve the recurrence

= + , = . In a sense, substitution is the opposite of induction. That is, we start at and work backward

= + = + = + . . .

= + = .

We then substitute each equality into the previous one, as follows: = + = + + = + + + . . . = + + + + + = + + + ( ) + ( ) + = = ( + )

=

Ex.: We solve the recurrence

= + , = First, work backward from :

= + = + = + . .

= + = .

Then substitute each equation into previous one:

= + = + + = + + + . . = + + + + + = +

+ +

+

+

=

=

for large .

Slide Number 1Slide Number 2Slide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Slide Number 20Slide Number 21Slide Number 22