the kolmogorov superposition theoremjaa5/files/presentations/defenseslides.pdf · desert view b2...
TRANSCRIPT
![Page 1: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/1.jpg)
The Kolmogorov Superposition Theorem:A framework for multivariate computation
Jonas Actor
Rice University
7 May 2018
Contact: [email protected]
![Page 2: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/2.jpg)
Curse of Dimensionality
10 nodes 102 nodes 103 nodes
Cost of simulation / computation grows exponentially
1
![Page 3: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/3.jpg)
A Question 1
Do functions of three variables exist at all?
1Polya and Szego, Problems and Theorems of Analysis, 1925 (German), transl. 1945, reprinted 1978
2
![Page 4: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/4.jpg)
A Question, Restated
Can functions of three variables be expressed usingfunctions of only two variables?
Example:Cardano’s Formula for roots of a cubic equation
3
![Page 5: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/5.jpg)
A Better Question
Hilbert’s 13th Problem (1900)
Can the solution x to the 7th degree polynomial equation
x7 + ax3 + bx2 + cx + 1 = 0
be represented by a finite number of compositions of bivariate continuous(algebraic) functions using the three variables a, b, c?
4
![Page 6: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/6.jpg)
Answer: Arnol’d 1957 2
Any continuous function of three variables can beexpressed using continuous functions of only two
variables.
2Arnol’d, Dokl. Akad. Nauk SSSR 114:5, 1957
5
![Page 7: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/7.jpg)
The Next Question
Can continuous functions of three variables be expressedusing functions of only one variable and addition?
6
![Page 8: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/8.jpg)
Answer: Kolmogorov 19573
Any continuous f : [0, 1]n → R can be written as
f (x1, . . . , xn) =2n∑q=0
χ
(n∑
p=1
ψp,q(xp)
)
3Kolmogorov, Dokl. Akad. Nauk SSSR 114:5, 1957
7
![Page 9: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/9.jpg)
What’s going on here?
f (x1, . . . , xn) =2n∑q=0
χ
(n∑
p=1
ψp,q(xp)
)
8
![Page 10: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/10.jpg)
What’s going on here?
f (x1, . . . , xn) =2n∑q=0
χ
(n∑
p=1
ψp,q(xp)
)
Original function f : [0, 1]n → R
8
![Page 11: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/11.jpg)
What’s going on here?
f (x1, . . . , xn) =2n∑q=0
χ
(n∑
p=1
ψp,q(xp)
)
Inner function ψp,q : [0, 1]→ R
8
![Page 12: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/12.jpg)
What’s going on here?
f (x1, . . . , xn) =2n∑q=0
χ
(n∑
p=1
ψp,q(xp)
)
Addition
8
![Page 13: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/13.jpg)
What’s going on here?
f (x1, . . . , xn) =2n∑q=0
χ
(n∑
p=1
ψp,q(xp)
)
Outer function χ : R→ R
8
![Page 14: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/14.jpg)
What’s going on here?
f (x1, . . . , xn) =2n∑q=0
χ
(n∑
p=1
ψp,q(xp)
)
Function composition
8
![Page 15: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/15.jpg)
Implication
Ψq(x1, . . . , xn) =n∑
p=1
ψp,q(xp)
(independent of f )
KST: f 7−→ χ
9
![Page 16: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/16.jpg)
No Free Lunch. . .
Traded smoothness for variables:
• No KST4 for ψp,q ∈ C 1([0, 1])
• Current ψp,q ∈ Holder([0, 1]) → bad for computation
• Possible ψp,q ∈ Lip([0, 1])
4Vituskin, DAN, 95:701–704, 1954.
10
![Page 17: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/17.jpg)
Smoothness
Goal: Construct Lipschitz functions ψp,q
11
![Page 18: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/18.jpg)
Rest of the Talk
1 Constructive KST
2 The Fridman Strategy
3 Reparameterization Approach
4 Conclusions and Outstanding Tasks
12
![Page 19: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/19.jpg)
Table of Contents
1 Constructive KST
2 The Fridman Strategy
3 Reparameterization Approach
4 Conclusions and Outstanding Tasks
![Page 20: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/20.jpg)
An Analogy
f (x , y) = elevation of Grand Canyon at (x , y)
https://viewer.nationalmap.gov/basic/
13
![Page 21: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/21.jpg)
An Analogy
f (Thor’s Temple) = ?
14
![Page 22: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/22.jpg)
An Analogy
A B
1
2
Ψq(Thor’s Temple) = A2f (Thor’s Temple) = χ(A2)
Bright Angel Point A1 Kaibab Nat’l Forest B1Desert View B2 Thor’s Temple A2Grand Canyon Lodge A1 Walhalla’s Overlook A2
Elevation (ft) A1 8200 B1 8000A2 9000 B2 7600
Locations real; elevations are not.
15
![Page 23: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/23.jpg)
An Analogy
A B
1
2
Ψq(Thor’s Temple) = A2f (Thor’s Temple) = χ(A2)
Bright Angel Point A1 Kaibab Nat’l Forest B1Desert View B2 Thor’s Temple A2Grand Canyon Lodge A1 Walhalla’s Overlook A2
Elevation (ft) A1 8200 B1 8000A2 9000 B2 7600
15
![Page 24: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/24.jpg)
An Analogy
A B
1
2
Ψq(Thor’s Temple) = A2
f (Thor’s Temple) = χ(A2)Bright Angel Point A1 Kaibab Nat’l Forest B1Desert View B2 Thor’s Temple A2Grand Canyon Lodge A1 Walhalla’s Overlook A2
Elevation (ft) A1 8200 B1 8000A2 9000 B2 7600
15
![Page 25: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/25.jpg)
An Analogy
A B
1
2
Ψq(Thor’s Temple) = A2
f (Thor’s Temple) = χ(A2)
Bright Angel Point A1 Kaibab Nat’l Forest B1Desert View B2 Thor’s Temple A2Grand Canyon Lodge A1 Walhalla’s Overlook A2
Elevation (ft) A1 8200 B1 8000A2 9000 B2 7600
15
![Page 26: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/26.jpg)
An Analogy
A B
1
2
16
![Page 27: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/27.jpg)
An Analogy
A B
1
2
16
![Page 28: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/28.jpg)
An Analogy
A B
1
2
16
![Page 29: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/29.jpg)
Decomposition Process
Strategy
• Leave gaps between squares → gives room to vary continuously
• Duplicate and slightly shift the squares to cover the gaps
• Define functions ψp,q as we refine squares
Decisions
• How big to make the gaps
• How to assign values to functions
17
![Page 30: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/30.jpg)
Sprecher’s Reduction5
Define 1 function instead of 2n2 + n functions:
ψp,q(xp) = λp ψ(xp + qε)
λ1, . . . , λn integrally independent
5Sprecher, Trans. AMS, 115:340–355, 1965
18
![Page 31: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/31.jpg)
Requirements (1/2)
Refinement:Squares Sk get smaller as k →∞
More Than Half:Each point in at least n+1
2n+1 sets of shifted squares
19
![Page 32: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/32.jpg)
Requirements (2/2)
Disjoint Image:Image of squares under Ψq disjoint
Monotonicity:Function ψ strictly monotonic increasing
Bounded Slope:Function ψ Lipschitz
20
![Page 33: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/33.jpg)
Table of Contents
1 Constructive KST
2 The Fridman Strategy
3 Reparameterization Approach
4 Conclusions and Outstanding Tasks
![Page 34: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/34.jpg)
Background
First proof of Lipschitz inner KST functions6
Predominant approach for Lipschitz KST functions
Never successfully implemented
6Dokl. Akad. Nauk SSR, 177:5:1019–1022, 1967.
21
![Page 35: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/35.jpg)
Defining our Squares
Squares: Cartesian product of 1D family of intervals
22
![Page 36: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/36.jpg)
Space Decomposition
5 copies of the same family of intervals, shifted by 14
-1 0 1 2
14
23
![Page 37: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/37.jpg)
Space Decomposition
Cartesian product for each shifted family of intervals
14
24
![Page 38: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/38.jpg)
Space Decomposition
Each refinement level k ∈ N,break largest intervals roughly in half
k = 0
k = 1
k = 2
......
......
...
25
![Page 39: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/39.jpg)
Space Decomposition
Breaks are copied in each shifted family of intervals . . .
-1 0 1 2
14
26
![Page 40: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/40.jpg)
Space Decomposition
. . . and each shifted family of squares.
14
27
![Page 41: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/41.jpg)
Requirements on Breaks
Every x ∈ [0, 1] in at least All But One of shifted families
14
-1 0 1 2
All But One on intervals ⇐⇒ More Than Half on squares
28
![Page 42: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/42.jpg)
Choosing Breaks
Keep inserting breaks −→ lose All But One Condition
14
-1 0 1 2
X
29
![Page 43: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/43.jpg)
Choosing Breaks
Solution: plug the gaps that cause problems
14
-1 0 1 2
X
30
![Page 44: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/44.jpg)
Inner Function ψ
At each refinement level k :
• Assign a value of ψk on left endpoint of each interval
• Value is fixed for all future k
ψk
31
![Page 45: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/45.jpg)
Inner Function ψ
ψk constant on intervals, linear on gaps
ψk
large plugs → small gaps → steep ψk
32
![Page 46: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/46.jpg)
Fixing Values: Disjoint Image Condition
For any two squares S , S ′ ∈ Sk ,
Ψ(S) ∩Ψ(S ′) = ∅.
33
![Page 47: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/47.jpg)
Disjoint Image Condition
Sk11 Sk
21
Sk12 Sk
22
Ψ(Sk11) Ψ(Sk
21) Ψ(Sk12) Ψ(Sk
22)
34
![Page 48: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/48.jpg)
Disjoint Image Condition
Don’t know ψ at level k
Can’t enforce Disjoint Image Condition without ψ
Two remedies:
• Fridman’s Disjoint Image Condition
• Conservative Disjoint Image Condition
35
![Page 49: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/49.jpg)
Fridman’s Disjoint Image Condition
For any two squares S , S ′ ∈ Sk ,
Ψk(S) ∩Ψk(S ′) = ∅.
Not Ψ(S), Ψ(S ′)
36
![Page 50: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/50.jpg)
Fridman’s Disjoint Image Condition
For any two squares S , S ′ ∈ Sk ,
Ψk(S) ∩Ψk(S ′) = ∅.
Not Ψ(S), Ψ(S ′)
36
![Page 51: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/51.jpg)
Fridman’s Disjoint Image Condition
Sk11 Sk
21
Sk12 Sk
22
Ψk(Sk11) Ψk(Sk
21) Ψk(Sk12) Ψk(Sk
22)
37
![Page 52: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/52.jpg)
Fridman’s Disjoint Image Condition
Ψ(x) =n∑
p=1
λp ψ(xp + qε)
λ1, . . . , λn integrally independent
ψk rational values at left endpoints
Fridman’s Condition is always met!Does not enforce disjointness in limit
38
![Page 53: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/53.jpg)
Conservative Disjoint Image Condition
Use Lipschitz bound to define intervals ∆ki that are
guaranteed to contain Ψ(Ski )
Ψk(Ski ) ⊆ Ψ(Sk
i ) ⊆ ∆ki .
39
![Page 54: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/54.jpg)
Conservative Disjoint Image Condition
For any two squares Ski , S
ki′ ∈ Sk
and corresponding intervals ∆ki ,∆
ki′ ,
∆ki ∩∆k
i′ = ∅.
40
![Page 55: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/55.jpg)
Conservative Disjoint Image Condition
Sk11 Sk
21
Sk12 Sk
22
Ψ(Sk11) Ψ(Sk
21) Ψ(Sk12) Ψ(Sk
22)
∆k11 ∆k
21 ∆k12 ∆k
22
41
![Page 56: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/56.jpg)
Conservative Disjoint Image Condition
Requires removing at least half of each interval
Points not contained in enough shifted copies of Sk toreconstruct χ
42
![Page 57: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/57.jpg)
Implications
Choose 2 of 3:
• (Conservative) Disjoint Image Condition
• All But One Condition
• Bounded Slope Condition
43
![Page 58: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/58.jpg)
Table of Contents
1 Constructive KST
2 The Fridman Strategy
3 Reparameterization Approach
4 Conclusions and Outstanding Tasks
![Page 59: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/59.jpg)
Reasoning
KST inner functions are strictly monotonic increasing
. . . which are of Bounded Variation
. . . so they define rectifiable curves
. . . which have Lipschitz reparameterizations.
44
![Page 60: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/60.jpg)
Reasoning
KST inner functions are strictly monotonic increasing
. . . which are of Bounded Variation
. . . so they define rectifiable curves
. . . which have Lipschitz reparameterizations.
44
![Page 61: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/61.jpg)
Reasoning
KST inner functions are strictly monotonic increasing
. . . which are of Bounded Variation
. . . so they define rectifiable curves
. . . which have Lipschitz reparameterizations.
44
![Page 62: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/62.jpg)
Reasoning
KST inner functions are strictly monotonic increasing
. . . which are of Bounded Variation
. . . so they define rectifiable curves
. . . which have Lipschitz reparameterizations.
44
![Page 63: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/63.jpg)
Approach
1 Find (non-Lipschitz) function ψ
2 Define reparameterization σ
3 Create Lipschitz ψ from ψ using σ
4 Construct squares Sk that meet conditions
45
![Page 64: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/64.jpg)
Holder Continous ψ
• Define iteratively: ψ = limk→ ψk
• Fix values at points with k digits in base γ expansion
• Small increase for most points• Large increase for expansions ending in γ − 1
• Linearly interpolate between fixed values
46
![Page 65: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/65.jpg)
Koppen’s KST Inner Function ψ7
47
![Page 66: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/66.jpg)
Lipschitz Reparameterization
Reparameterization σ : [0, 1]→ [0, 1]
σ(x) =arclength of ψ from 0 to x
total arclength of ψ from 0 to 1
ψ(x) = ψ(σ−1(x))
48
![Page 67: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/67.jpg)
Lipschitz Reparameterization ψ7
49
![Page 68: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/68.jpg)
Correct Squares
ψ = ψ ◦ σ−1
Intervals for ψ are Sk
Intervals for ψ are Sk = σ(Sk)
ψ satisfies Disjoint Image Condition on Sk
ψ satisfies Disjoint Image Condition on Sk
50
![Page 69: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/69.jpg)
Comparing Sk and Sk
0.20 0.25 0.30 0.35 0.40
0.0
0.1
0.2
0.3
S2
S2
51
![Page 70: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/70.jpg)
Largest Interval Size for Sk
52
![Page 71: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/71.jpg)
Reparameterized Intervals
Refinement Condition:Intervals shrink at O(2−k) not O(γ−k)
All But One Condition:Gaps between intervals get smaller under σ
53
![Page 72: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/72.jpg)
Takeaway
Function ψ = ψ ◦ σ−1 is a Lipschitz KST inner function
Squares Sk = σ(Sk) are KST squares
54
![Page 73: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/73.jpg)
Table of Contents
1 Constructive KST
2 The Fridman Strategy
3 Reparameterization Approach
4 Conclusions and Outstanding Tasks
![Page 74: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/74.jpg)
Review
• Motivated why KST is interesting, hard
• Illustrated how KST works
• Outlined Fridman Strategy
• Showed where Fridman Strategy fails
• Posed new reparameterization approach
• Justified reparameterization meets requirements
55
![Page 75: The Kolmogorov Superposition Theoremjaa5/files/Presentations/DefenseSlides.pdf · Desert View B2 Thor’s Temple A2 Grand Canyon Lodge A1 Walhalla’s Overlook A2 Elevation (ft) A1](https://reader034.vdocuments.mx/reader034/viewer/2022042108/5e87c598c7ea2b3c253fcc23/html5/thumbnails/75.jpg)
Outstanding Tasks
Complete proofs for reparameterization argument for ψ
Framework for computing outer function χ
• Requires squares at each level k
• Requires final Ψ
First Application: Image compression
56