fixed points and contractive transformations - rice university · fixed points of contractive maps...

31
Fixed Points and Contractive Transformations Ron Goldman Department of Computer Science Rice University

Upload: doliem

Post on 30-May-2019

227 views

Category:

Documents


0 download

TRANSCRIPT

Fixed Points and Contractive Transformations

Ron GoldmanDepartment of Computer ScienceRice University

Applications

Computer Graphics• Fractals• Bezier and B-Spline Curves and Surfaces

Root Finding• Newton’s Method• Trivial Fixed Point Method

Solving Large Systems of Linear Equations -- Relaxation Methods• Jacobi• Gauss-Seidel

Existence and Uniqueness Theorem for Ordinary Differential Equations

Reconstruction of Curves and Surfaces from Point Clouds

Part I: Theory

Fixed Points of Functions

Fixed Point

F(P) = P

Example

F(x) = x2

F(0) = 0 and F(1) =1

Fixed Points by Iteration

Iteration

T1 = F(T0 )

T2 = F2 (T0 ) = F(T1)

↓ ↓ ⇒ F(T ) = F∞+1(T0 ) = T ( fixed point)

T = F∞(T0 )

Another Iteration

S1 = F(S0 )

S2 = F 2(S0 ) = F(S1)

↓ ↓ ⇒ F(S) = F∞+1(S0 ) = S ( fixed point)

S = F∞(S0 )

Observations• Iteration Might Not Converge• If Iteration Converges → Fixed Point • Different Starting Values May Generate Different Fixed Points

Contractive Transformation

Contractive Map (T)

Dist T(P),T(Q)( ) ≤ sDist(P,Q)         0 < s <1

Examples

Uniform Scaling -- 0 < s <1

Image of Three Points -- Triangle Shrinks on All Sides

Necessary Condition Necessary and Sufficient Conditions

Det(T ) < 1

Eigenvalues(T ) <1

Lemma

T Contractive

⇒ T Continuous

Iterations and Transformations

Iteration -- Arbitrary Transformations

• Either Converges to a Fixed Point or Diverges

• Possibly Many Distinct Fixed Points

• Starting Point Matters

Iteration -- Contractive Transformations

• Always Converges to a Fixed Point

• Fixed Point is Unique

• Starting Point does not Matter

Fixed Points of Contractive Maps

Theorem 1: Suppose that T is a Contractive and Pn +1 = T(Pn ) for all n ≥ 0.If P = Limn→∞Pn exists, then P is a fixed point of T.

Proof: T(P) = T Limn→∞Pn( ) = Limn→∞T(Pn ) = Limn→∞Pn+1 = P.

Theorem 2: T Contractive ⇒ Fixed Point is Unique.

Proof: If P and Q are both fixed points of T, then

T(P) = P and T(Q) = Q

Dist T(P),T(Q)( ) = Dist(P,Q).

Hence T is not contractive. Contradiction.

Cauchy Sequences

DefinitionA sequence Pn{ } is Cauchy if Dist(Pn +m , Pn ) < ε for all n > N .

IntuitionA sequence of points Pn{ } is Cauchy if the points get closer and closer as n gets larger and larger.

Cauchy’s Theorem (Completeness)Every Cauchy sequence converges.

Triangular Inequality

P1

P2

P3

Dist(P1,P3 ) ≤ Dist(P1,P2)+ Dist(P2,P3 )

Dist(P1,Pn ) ≤ Dist(P1,P2)+ Dist(P2,P3 )+L+ Dist(Pn−1,Pn )

Contractive Maps and Cauchy Sequences

Theorem 3 Suppose that T is a Contractive Map, and Pn +1 = T(Pn ) for all n ≥ 0 .Then Pn{ } is a Cauchy Sequence for Any Choice of P0 .

Pr oof : Dist Pn+1, Pn( ) = Dist T(Pn ),T(Pn−1 )( ) ≤ sDist Pn , Pn−1( ) = sDist T (Pn−1),T(Pn−2 )( )

M ≤ snDist P1, P0( )

Dist Pn+m+1,Pn( ) ≤ Dist Pn+m+1,Pn+m( )+L+ Dist Pn+1,Pn( ) ≤ (sn+m +L+ sn )Dist P1,P0( )

≤ sn Dist P1,P0( )1− s

< ε

Trivial Fixed Point Theorem

Trivial Fixed Point Theorem

Suppose that

T is a Contractive Map

Pn +1 = T(Pn ) for all n ≥ 0 .

Then Limn→∞Pn is the Unique Fixed Point of T for Any Choice of P0 !

Proof: The Sequence Pn{ } is Cauchy. (Theorem 3)

Therefore, Limn→∞Pn Exists. (Cauchy’s Theorem)

Hence, Limn→∞Pn is a Fixed Point of T. (Theorem 1)

Moreover, this Fixed Point is Unique. (Theorem 2)

Part II: Applications

Equation Solving

Finding Roots of Real Valued Functions

• Guess and Iterate

-- Newton’s Method

-- Trivial Fixed Point Method

Solving Large Systems of Linear Equations

• Relaxation Methods

-- Jacobi

-- Gauss-Seidel

Generating Curves and Surfaces

Ordinary Differential Equations

• Existence and Uniqueness Theorem

• Solutions by Iteration

Fractals

• Iterated Function Systems

• Fractal Algorithm

Root Finding by Newton’s Method

ProblemSolve F(x) = 0

Newton’s Method1. Replace F(x) by

G(x) = x − F(x) / ′ F (x)• F(x) has a root

⇔ G(x) has a Fixed Point•

F(x*) = 0 ⇔ G(x*) = x * 2. Select an initial guess x0 for x *.3. Compute

xk+1 = G(xk ) = xk − F(xk ) / ′ F (xk ) for k = 1,K,n .4. Stop when xk +1 − xk < ε

Observations• May Diverge --

′ F (xk ) ≈ 0 • May Converge to Different Roots for Different Initial Guesses

Finding a Fixed Point by Iteration

y = x

y = G(x)

x0•

• •

•x1

• •

•x2

• ••

•x3

y = G(x0 )x1 = G(x0 )

Fixed PointG( ˆ x ) = ˆ x

•̂ x

Root Finding by Trivial Fixed Point Method

ProblemSolve F(x) = 0

Fixed Point Method

1. Replace F(x) by

G(x) = x + F(x)

• F(x) has a root ⇔ G(x) has a Fixed Point

• F(x*) = 0 ⇔ G(x*) = x *

2. Select any(!) initial guess x0 for x *.

3. Compute xk +1 = G(xk ) for k = 1,K,n .4. Stop when xk +1 − xk < ε

ObservationWorks when G(x) is a Contractive Map.

Root Finding for Differentiable Functions

Mean Value Theorem

G(b) – G(a) ≤ ′ G (c)(b − a) a ≤ c ≤ b

Observations

• ′ G (c) < 1⇒G(x) is Contractive

• ′ G (c) < 1/ 2 ⇒ Fixed Point Method is Faster than Bisection Method

Example

• Find a root of F(x) = cos(x) − x

• Find a fixed point of G(x) = cos(x)

Solving Large Systems of Linear Equations

System of Linear Equations

M11x1 + M12x2 +L+ M1nxn = b1M21x1 + M22x2 +L+ M2nxn = b2

M M ⇔ xi =bi

Mii−

MijMiij≠i

∑ x j 1≤ i ≤ n

Mn1x1 + Mn2x2 +L+ Mnnxn = bn

Matrix Form

M11 M12 L M1nM21 M22 L M2n

M M O M

Mn1 Mn2 L Mnn

M1 2 4 4 4 4 3 4 4 4 4

x1x2M

xn

X{

=

b1b2M

bn

B{

⇔ M ∗X = B

Relaxation Methods

Jacobi Relaxation

xip =

biMii

−MijMiij≠i

∑ x jp−1 1≤ i ≤ n

Gauss-Seidel Relaxation

xip =

biMii

−MijMiij=1

i−1

∑ x jp −

MijMiij=i+1

n

∑ x jp−1 1≤ i ≤ n

Jacobi Relaxation

System of Linear Equations

M11x1 + M12x2 +L+ M1nxn = b1M21x1 + M22x2 +L+ M2nxn = b2 M M ⇔ M ∗X = B ⇔ B− M ∗X = 0Mn1x1 + Mn2x2 +L+ Mnnxn = bn .

Equivalent System of Linear Equations

M11x1 = b1 −M12x2 −L− M1nxnM22x2 = b2 −M21x1 −L− M2nxn M M M ⇔ D∗ X = B − (M − D)∗XMnnxn = bn −Mn1x1 −L

D =

M11 0 L 00 M22 L 0M O O M

0 L 0 M NN

Jacobi Relaxation (continued)

Equivalent System of Linear Equations

x1 = b1M11

− M12M11

x2 −L− M1nM11

xn

x2 = b2M22

− M21M22

x1 −L− M2nM22

xn

M M M ⇔ X = D−1∗B −D−1∗ (M − D)∗X

xn =bn

Mnn−

Mn1Mnn

x1 −L

Fixed Point

T(X ) = D−1B + (I −D−1M )X

T(X ) = X (fixed point)

Gauss-Seidel Relaxation

System of Linear Equations

M11x1 + M12x2 +L+ M1nxn = b1M21x1 + M22x2 +L+ M2nxn = b2 M M ⇔ M ∗X = B ⇔ B− M ∗X = 0Mn1x1 + Mn2x2 +L+ Mnnxn = bn .Mn1x1 + Mn2x2 +L+ Mnnxn = bn .

.

Equivalent System of Linear Equations

M11x1 = b1 −M12x2 −L− M1nxnM21x1 + M22x2 = b2 − M23x1 −L− M2nxn M M ⇔ L∗ X = B − (M − L)∗ XMn1x1 +L+ Mnnxn = bn

L =

M1,1 0 L 0M2,1 M2,2 L 0

M O O M

Mn,1 L Mn−1,n−1 Mn,n

Gauss-Seidel Relaxation (continued)

Equivalent System of Linear Equations

L∗ X = B − (M − L)∗ X ⇔ X = L−1∗B− L−1∗ (M − L)∗ X

Fixed Point

T(X ) = L−1B+ L−1(L − M )X

T(X ) = X (fixed point)

Convergence

Diagonally Dominant

• Both methods converge for any initial guess when M is diagonally dominant

| Mii |≥ | Mijj≠i∑ | (diagonally dominant)

Fixed Point

M ∗X = B⇔Q∗X = (Q −M )X + B⇔ X = (I −Q−1M )X +Q−1B

T(X ) = (I −Q−1M )X +Q−1B

T(X ) = X (fixed point)

Contractive Map• T is a contractive map when

-- M is diagonally dominant-- Q is the diagonal part of M (Jacobi)-- Q is the lower triangular part of M (Gauss-Seidel)

Ordinary Differential Equations

General Case

y(n) = an−1(x)y(n−1) +L+ a1(x)y' + a0(x)y +b(x) (Differential Equation)

y(0) = c0 ,K,y(n−1)(0) = cn−1 (Initial Conditions)

Example 1

• y'= −y• y(0) =1

   

⇔ y = e−x

Example 2

• y' '= −y• y(0) =1 y'(0) = 0

⇔ y = cos(x)

Contractive Map for Ordinary Differential Equations

Ordinary Differential Equation

y(n) = an−1(x)y(n−1) +L+ a1(x)y' + a0(x)y +b(x) (Differential Equation)

y(0) = c0 ,K,y(n−1)(0) = cn−1 (Initial Conditions)

Ordinary Differential Equation

y(n) = F(x,y,y',K,y(n−1)) (Differential Equation)

y(0) = c0 ,K,y(n−1)(0) = cn−1 (Initial Conditions)

Contractive Map

• T ( f )(x) = c0 +c1x +L+cn−1xn−1

(n −1)!

+ L F tn, f (tn ), f ' (tn ),K, f (n −1)(tn )

0tn −1∫0

t1∫0x∫ dtnLdt1

• Fixed Point of T = Solution of ODE

Examples

Example 1

• y'= −y

• y(0) =1

T( f )(x) =1+ − f (x)0x∫ dx

T(e−x ) =1+ −e−t0x∫ dt = e−x

Example 2

y''= −y

y(0) =1 y' (0) = 0

T( f )(x) =1+ − f (u)0t∫0

x∫ dudt

T cos(x)( ) =1+ −cos(u)0t∫0

x∫ du dt = cos(x)

Solving Ordinary Differential Equations

Ordinary Differential Equation

y(n) = an−1(x)y(n−1) +L+ a1(x)y' + a0(x)y +b(x) (Differential Equation)

y(0) = c0 ,K,y(n−1)(0) = cn−1 (Initial Conditions)

Fixed Point Method

1. Select any continuous function

f0(x) as the initial guess.

2. Compute

fk +1 = T ( fk )

k =1,K,n.

3. Stop when

fk +1− fk < ε .

Spaces and Functions

Euclidian Space Function Space

Points Continuous Functions on Closed Intervals

Euclidian Distance

| f − g |∞= Max | f − g |

Cauchy Sequences of Points Converge Cauchy Sequences of Functions Converge

Contractive Maps Integrals

Fixed Points Solutions of ODE’s

Trivial Fixed Point Theorem Trivial Fixed Point Theorem

Root Finding Algorithm Algorithm for Solving ODE’s• Start with Any Point and Iterate • Start with Any Function and Iterate