ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · ontheconstructionof...

87
On the construction of minimax-distance (sub-)optimal designs Luc Pronzato UniversitØ Cte d’Azur, CNRS, I3S, France

Upload: others

Post on 08-May-2020

24 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

On the construction ofminimax-distance (sub-)optimal

designs

Luc Pronzato

Université Côte d’Azur, CNRS, I3S, France

Page 2: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

1) Introduction & motivation

Objective:Approximation/interpolation of a function f : x ∈X ⊂ Rd −→ R,

(with X compact: typically, X = [0, 1]d)à Choose n points Xn = {x1, . . . , xn} ∈X n (the design)

where to evaluate f (no repetition)

Design criterion = minimax distanceà minimize ΦmM(Xn) = maxx∈X mini=1,...,n ‖x− xi‖ (`2-distance)

= maxx∈X d(x,Xn)= dH(X ,Xn) (Hausdorff distance, `2)= dispersion of Xn in X (Niederreiter, 1992, Chap. 6)

X∗n an optimal n-point design Ô ΦmM-efficiency EffmM(Xn) = Φ∗mM,nΦmM (Xn) ∈ (0, 1]

with Φ∗mM,n = ΦmM(X∗n )

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 2 / 41

Page 3: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

1) Introduction & motivation

Objective:Approximation/interpolation of a function f : x ∈X ⊂ Rd −→ R,

(with X compact: typically, X = [0, 1]d)à Choose n points Xn = {x1, . . . , xn} ∈X n (the design)

where to evaluate f (no repetition)

Design criterion = minimax distanceà minimize ΦmM(Xn) = maxx∈X mini=1,...,n ‖x− xi‖ (`2-distance)

= maxx∈X d(x,Xn)= dH(X ,Xn) (Hausdorff distance, `2)= dispersion of Xn in X (Niederreiter, 1992, Chap. 6)

X∗n an optimal n-point design Ô ΦmM-efficiency EffmM(Xn) = Φ∗mM,nΦmM (Xn) ∈ (0, 1]

with Φ∗mM,n = ΦmM(X∗n )

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 2 / 41

Page 4: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

d = 2, n = 7

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 3 / 41

Page 5: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

Why ΦmM? two good reasons (at least) to minimize ΦmM(Xn):

¬ Suppose f ∈ RKHS H with kernel K (x, y) = C(‖x− y‖), then∀x ∈X , |f (x)− η̂n(x)| ≤ ‖f ‖H ρn(x) where

η̂n(x) = BLUP based on the f (xi ), i = 1, . . . , nρ2n(x) = “kriging variance" at x

see, e.g., Vazquez and Bect (2011); Auffray et al. (2012)

Schaback (1995) à supx∈X ρn(x) ≤ S[ΦmM(Xn)]for some increasing function S[·] (depending on K )

­ X∗n has no (or few) points on the boundary of X

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 4 / 41

Page 6: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

Why ΦmM? two good reasons (at least) to minimize ΦmM(Xn):

¬ Suppose f ∈ RKHS H with kernel K (x, y) = C(‖x− y‖), then∀x ∈X , |f (x)− η̂n(x)| ≤ ‖f ‖H ρn(x) where

η̂n(x) = BLUP based on the f (xi ), i = 1, . . . , nρ2n(x) = “kriging variance" at x

see, e.g., Vazquez and Bect (2011); Auffray et al. (2012)

Schaback (1995) à supx∈X ρn(x) ≤ S[ΦmM(Xn)]for some increasing function S[·] (depending on K )

­ X∗n has no (or few) points on the boundary of X

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 4 / 41

Page 7: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

Evaluation of ΦmM(Xn)? Not considered here!

To evaluate ΦmM(Xn) = maxx∈X mini=1,...,n ‖x− xi‖ = maxx∈X d(x,Xn)we need to find x∗ = arg maxx∈X d(x,Xn)

Key idea: replace arg maxx∈X d(x,Xn) by arg maxx∈XQ d(x,Xn)for a suitable finite XQ ∈X Q

Replacing XQ by a regular grid, or first Q points of a Low DiscrepancySequence in X , is not accurate:

à ΦmM(Xn; XQ) ≤ ΦmM(Xn) (optimistic result)requires Q = O(1/εd ) to have ΦmM(Xn) < ΦmM(Xn; XQ) + ε

For d . 5, use tools from algorithmic geometry (Delaunay triangulation orVoronoï tessellation) Þ exact result

For larger d , use MCMC with XQ = adaptive grid (LP, 2017a)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 5 / 41

Page 8: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

Evaluation of ΦmM(Xn)? Not considered here!

To evaluate ΦmM(Xn) = maxx∈X mini=1,...,n ‖x− xi‖ = maxx∈X d(x,Xn)we need to find x∗ = arg maxx∈X d(x,Xn)

Key idea: replace arg maxx∈X d(x,Xn) by arg maxx∈XQ d(x,Xn)for a suitable finite XQ ∈X Q

Replacing XQ by a regular grid, or first Q points of a Low DiscrepancySequence in X , is not accurate:

à ΦmM(Xn; XQ) ≤ ΦmM(Xn) (optimistic result)requires Q = O(1/εd ) to have ΦmM(Xn) < ΦmM(Xn; XQ) + ε

For d . 5, use tools from algorithmic geometry (Delaunay triangulation orVoronoï tessellation) Þ exact result

For larger d , use MCMC with XQ = adaptive grid (LP, 2017a)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 5 / 41

Page 9: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

Bounds on Φ∗mM,n = ΦmM(X∗n ) when X = [0, 1]d

Lower bound: the n balls B(xi ,Φ∗mM,n) cover X

⇒ nVd (Φ∗mM,n)d ≥ vol(X ) (= 1), with Vd = vol[B(0, 1)] = πd/2/Γ(d/2 + 1)

R∗n = (nVd )−1/d ≤ Φ∗mM,n

Upper bound: use any design!

md -point regular grid in X :Φ∗mM,md ≤

√d

2m :

Take m = bn1/dc, so that md ≤ n and Φ∗mM,n ≤ Φ∗mM,md , therefore

Φ∗mM,n ≤ R∗n =√

d2bn1/dc

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 6 / 41

Page 10: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

Bounds on Φ∗mM,n = ΦmM(X∗n ) when X = [0, 1]d

Lower bound: the n balls B(xi ,Φ∗mM,n) cover X

⇒ nVd (Φ∗mM,n)d ≥ vol(X ) (= 1), with Vd = vol[B(0, 1)] = πd/2/Γ(d/2 + 1)

R∗n = (nVd )−1/d ≤ Φ∗mM,n

Upper bound: use any design!

md -point regular grid in X :Φ∗mM,md ≤

√d

2m :

Take m = bn1/dc, so that md ≤ n and Φ∗mM,n ≤ Φ∗mM,md , therefore

Φ∗mM,n ≤ R∗n =√

d2bn1/dc

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 6 / 41

Page 11: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

d = 2

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41

Page 12: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

d = 5

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41

Page 13: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

d = 10

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41

Page 14: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

d = 20

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41

Page 15: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

­ Minimization of ΦmM(Xn) with respect to Xn ∈X n for a given n

® n is not fixed (nmin ≤ n ≤ nmax, we may stop before nmax evaluations of f )

How to obtain good “anytime designs”, such thatall nested designs Xn have a high efficiency EffmM(Xn), nmin ≤ n ≤ nmax

¯ Design measures that minimize a regularized version of ΦmM

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41

Page 16: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

­ Minimization of ΦmM(Xn) with respect to Xn ∈X n for a given n® n is not fixed (nmin ≤ n ≤ nmax, we may stop before nmax evaluations of f )

How to obtain good “anytime designs”, such thatall nested designs Xn have a high efficiency EffmM(Xn), nmin ≤ n ≤ nmax

¯ Design measures that minimize a regularized version of ΦmM

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41

Page 17: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

1) Introduction

­ Minimization of ΦmM(Xn) with respect to Xn ∈X n for a given n® n is not fixed (nmin ≤ n ≤ nmax, we may stop before nmax evaluations of f )

How to obtain good “anytime designs”, such thatall nested designs Xn have a high efficiency EffmM(Xn), nmin ≤ n ≤ nmax

¯ Design measures that minimize a regularized version of ΦmM

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41

Page 18: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed

2) Minimization of ΦmM(Xn), Xn ∈X n, n fixed

General global optimization method (e.g., simulated annealing): notpromising2.1) k-means and centroids2.2) Stochastic gradient

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 9 / 41

Page 19: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed

2) Minimization of ΦmM(Xn), Xn ∈X n, n fixed

General global optimization method (e.g., simulated annealing): notpromising2.1) k-means and centroids2.2) Stochastic gradient

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 9 / 41

Page 20: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

2.1/ k-means and centroids

Minimize the L2 energy functional

E2(Tn,Xn) =∫

X

( n∑i=1

ICi (x) ‖x− xi‖2)

dx =n∑

i=1

∫Ci

‖x− xi‖2 dx

where Tn = {Ci , i = 1, . . . , n} is a tessellation of XICi = indicator function of Ci

Then (Du et al., 1999):Ci = V(xi ) = Voronoï region for the site xi , for all i

(⇒ E2(Tn,Xn) =∫

X d2(x,Xn) dx)simultaneously xi = centroid of Ci (center of gravity) for all i :

xi = (∫Cix dx)/vol(Ci )

Þ such a Xn should thus perform reasonably well in terms of space-filling(Lekivetz and Jones, 2015)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 10 / 41

Page 21: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

2.1/ k-means and centroids

Minimize the L2 energy functional

E2(Tn,Xn) =∫

X

( n∑i=1

ICi (x) ‖x− xi‖2)

dx =n∑

i=1

∫Ci

‖x− xi‖2 dx

where Tn = {Ci , i = 1, . . . , n} is a tessellation of XICi = indicator function of Ci

Then (Du et al., 1999):Ci = V(xi ) = Voronoï region for the site xi , for all i

(⇒ E2(Tn,Xn) =∫

X d2(x,Xn) dx)simultaneously xi = centroid of Ci (center of gravity) for all i :

xi = (∫Cix dx)/vol(Ci )

Þ such a Xn should thus perform reasonably well in terms of space-filling(Lekivetz and Jones, 2015)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 10 / 41

Page 22: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

Lloyd’s method (1982): (= fixed-point iterations)

Þ Move each xi to the centroid of its own Voronoï cell, repeat . . .

à Algorithmic geometry (Voronoï tessellation) if d very small,use a finite set XQ otherwise

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 11 / 41

Page 23: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

30 points from Sobol’ LDS

k-means clustering (30 clusters) of 1,000 point from Sobol’ LDS

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 12 / 41

Page 24: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

k-means clustering (30 clusters) of 1,000 point from Sobol’ LDS

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 12 / 41

Page 25: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

However. . . minimax-optimal design is related to the construction of a centroidaltessellation for

Eq(Tn,Xn) =∫

X

( n∑i=1

ICi (x) ‖x− xi‖q

)dx =

n∑i=1

∫Ci

‖x− xi‖q dx

for q →∞à use Chebyshev centers

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 13 / 41

Page 26: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

However. . . minimax-optimal design is related to the construction of a centroidaltessellation for

Eq(Tn,Xn) =∫

X

( n∑i=1

ICi (x) ‖x− xi‖q

)dx =

n∑i=1

∫Ci

‖x− xi‖q dx

for q →∞à use Chebyshev centers

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 13 / 41

Page 27: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

Variant of Lloyd’s method:

0) Select X (1)n and ε� 1, set k = 1

1) Compute the Voronoï tessellation {Vi , i = 1, . . . , n} of X (or XQ) based onX (k)

n

2) For i = 1, . . . , nä determine the smallest ball B(ci , ri ) enclosing Vi (= convex QP problem)ä replace xi by ci in X (k)

n (Chebyshev center of Vi)3) if ΦmM(X(k)

n )− ΦmM(X(k+1)n ) < ε, then stop; otherwise k ← k + 1, return to

step 1

Þ Move each xi to the Chebyshev center of its own Voronoï cell, repeat . . .

[ΦmM(X(k)n ) decreases monotonically, convergence to a local minimum (or a saddle point)]

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 14 / 41

Page 28: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 15 / 41

Page 29: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

Determination of the smallest enclosing ball containing Z = {z1, . . . , zN}(vertices of a Voronoï cell, points of XQ closest to xi):

⇔ minimize f (c) = maxi=1,...,N ‖zi − c‖2 with respect to c ∈ Rd

Direct problem = convex QPTake any c0 ∈ Rd , minimize ‖c− c0‖2 + t

with respect to (c, t) ∈ Rd+1,subject to ‖zi − c0‖2 − 2(zi − c0)>(c− c0) ≤ t , i = 1, . . . ,N

(N linear constraints)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 16 / 41

Page 30: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

Determination of the smallest enclosing ball containing Z = {z1, . . . , zN}(vertices of a Voronoï cell, points of XQ closest to xi):

⇔ minimize f (c) = maxi=1,...,N ‖zi − c‖2 with respect to c ∈ Rd

Direct problem = convex QPTake any c0 ∈ Rd , minimize ‖c− c0‖2 + t

with respect to (c, t) ∈ Rd+1,subject to ‖zi − c0‖2 − 2(zi − c0)>(c− c0) ≤ t , i = 1, . . . ,N

(N linear constraints)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 16 / 41

Page 31: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

Determination of the smallest enclosing ball containing Z = {z1, . . . , zN}

Dual problem = similar to an optimal design problem:maximize trace[V(ξ)], with ξ a prob. measure on Z,

V(ξ) = covariance matrix for ξcenter of the ball = c(ξ) =

∫Z z ξ(dz)

Þ Algorithms of the exchange-type (Yildirim, 2008)(≈ Fedorov algorithm for D-optimal design: optimal step length is available)

Þ One can remove inessential points from Z: (LP, 2017b)à Combine this with the use of a standard QP solver for the direct problem

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 17 / 41

Page 32: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids

Determination of the smallest enclosing ball containing Z = {z1, . . . , zN}

Dual problem = similar to an optimal design problem:maximize trace[V(ξ)], with ξ a prob. measure on Z,

V(ξ) = covariance matrix for ξcenter of the ball = c(ξ) =

∫Z z ξ(dz)

Þ Algorithms of the exchange-type (Yildirim, 2008)(≈ Fedorov algorithm for D-optimal design: optimal step length is available)

Þ One can remove inessential points from Z: (LP, 2017b)à Combine this with the use of a standard QP solver for the direct problem

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 17 / 41

Page 33: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient

2.2/ Stochastic gradient

d is large: Lloyd’s algorithm cannot be used (computational geometryis too complicated, regular grids or LDS are not dense enough)

minimize Eq∗(Xn) =

∫X

( n∑i=1

IVi (x) ‖x− xi‖q

)dx

with Vi = Voronoï region for the site xi

Þ Stochastic gradient algorithm:(MacQueen, 1967) for q = 2, (Cardot et al., 2012) for q = 1

0) k = 1, X (1)n , set ni,0 = 0 for all i = 1, . . . , n

1) sample X uniformly distributed in X

2) find i∗ = arg mini=1,...,n ‖X − x(k)i ‖, ni∗,k ← ni∗,k + 1 [← X ∈ cell V∗

i ]

3) x(k+1)i∗ = x(k)

i∗ − γi∗,k q‖X − x(k)i∗ ‖

q−2 (x(k)i∗ − X )︸ ︷︷ ︸

=gradient

, k ← k + 1,

return to step 1, stop when k = K

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 18 / 41

Page 34: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient

2.2/ Stochastic gradient

d is large: Lloyd’s algorithm cannot be used (computational geometryis too complicated, regular grids or LDS are not dense enough)

minimize Eq∗(Xn) =

∫X

( n∑i=1

IVi (x) ‖x− xi‖q

)dx

with Vi = Voronoï region for the site xi

Þ Stochastic gradient algorithm:(MacQueen, 1967) for q = 2, (Cardot et al., 2012) for q = 1

0) k = 1, X (1)n , set ni,0 = 0 for all i = 1, . . . , n

1) sample X uniformly distributed in X

2) find i∗ = arg mini=1,...,n ‖X − x(k)i ‖, ni∗,k ← ni∗,k + 1 [← X ∈ cell V∗

i ]

3) x(k+1)i∗ = x(k)

i∗ − γi∗,k q‖X − x(k)i∗ ‖

q−2 (x(k)i∗ − X )︸ ︷︷ ︸

=gradient

, k ← k + 1,

return to step 1, stop when k = K

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 18 / 41

Page 35: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient

Typical choice for γi∗,k = c/nαi∗,k , with α ∈ (1/2, 1]and consider X̂n = 1

K∑K

k=1 X (k)n when α < 1

Little information to store (no grid or other finite approximation of X )Þ can also be used with large d

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 19 / 41

Page 36: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient

Example: n = 10 dall methods are initialized at the same random design, 100 repetitionsk-means and Lloyd’s method with Chebyshev centers use 2d+8 points

from a LDS (Sobol’)

d = 2, n = 20 (R∗n ≈ 0.1262, R∗n ≈ 0.1768)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 20 / 41

Page 37: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient

Example: n = 10 dall methods are initialized at the same random design, 100 repetitionsk-means and Lloyd’s method with Chebyshev centers use 2d+8 points

from a LDS (Sobol’)

d = 3, n = 30 (R∗n ≈ 0.1996, R∗n ≈ 0.2887)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 20 / 41

Page 38: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient

Example: n = 10 dall methods are initialized at the same random design, 100 repetitionsk-means and Lloyd’s method with Chebyshev centers use 2d+8 points

from a LDS (Sobol’)

d = 4, n = 40 (R∗n ≈ 0.2668, R∗n = 0.5)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 20 / 41

Page 39: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient

Example:

d = 10, n = 100 (R∗n ≈ 0.5746, R∗n ≈ 1.5811)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 20 / 41

Page 40: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs

3) Nested designs

à obtain a high ΦmM-efficiency EffmM(Xn) = Φ∗mM,nΦmM (Xn) for all Xn, nmin ≤ n ≤ nmax

[EffmM(Xn) ∈ (0, 1]]

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 21 / 41

Page 41: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.1/ Coffee-house design

3.1/ Coffee-house design

x1 at the centre of X , then xn+1 furthest point from Xn for all n ≥ 1(called coffee-house design (Müller, 2007, Chap. 4))

Guarantees EffmM(Xn) = Φ∗mM,nΦmM (Xn) ≥

12 and EffMm(Xn) = ΦMm(Xn)

Φ∗Mm,n≥ 1

2 for all n

with ΦMm(Xn) = mini 6=j∈{1,...,n} ‖xi − xj‖ the maximin-distance criterion,and Φ∗Mm,n its optimal (maximum) value

Proof. (Gonzalez, 1985)by construction:ΦMm(Xn+1) , minxi 6=xj∈Xn+1 ‖xi − xj‖ = d(xn+1,Xn) = ΦmM(Xn)X∗n a ΦmM-optimal design: the n balls B(x∗i ,ΦmM(X∗n )), x∗i ∈ X∗n , cover X⇒ one of them contains 2 points xi , xj in Xn+1 for any Xn+1 (n + 1 points)⇒ ΦMm(Xn+1) ≤ ‖xi − xj‖ ≤ 2ΦmM(X∗n )⇒ Φ∗Mm,n+1 ≤ 2ΦmM(X∗n ) ≤ 2ΦmM(Xn) = ΦMm(Xn+1)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 22 / 41

Page 42: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.1/ Coffee-house design

3.1/ Coffee-house design

x1 at the centre of X , then xn+1 furthest point from Xn for all n ≥ 1(called coffee-house design (Müller, 2007, Chap. 4))

Guarantees EffmM(Xn) = Φ∗mM,nΦmM (Xn) ≥

12 and EffMm(Xn) = ΦMm(Xn)

Φ∗Mm,n≥ 1

2 for all n

with ΦMm(Xn) = mini 6=j∈{1,...,n} ‖xi − xj‖ the maximin-distance criterion,and Φ∗Mm,n its optimal (maximum) value

Proof. (Gonzalez, 1985)by construction:ΦMm(Xn+1) , minxi 6=xj∈Xn+1 ‖xi − xj‖ = d(xn+1,Xn) = ΦmM(Xn)X∗n a ΦmM-optimal design: the n balls B(x∗i ,ΦmM(X∗n )), x∗i ∈ X∗n , cover X⇒ one of them contains 2 points xi , xj in Xn+1 for any Xn+1 (n + 1 points)⇒ ΦMm(Xn+1) ≤ ‖xi − xj‖ ≤ 2ΦmM(X∗n )⇒ Φ∗Mm,n+1 ≤ 2ΦmM(X∗n ) ≤ 2ΦmM(Xn) = ΦMm(Xn+1)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 22 / 41

Page 43: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.1/ Coffee-house design

3.1/ Coffee-house design

x1 at the centre of X , then xn+1 furthest point from Xn for all n ≥ 1(called coffee-house design (Müller, 2007, Chap. 4))

Guarantees EffmM(Xn) = Φ∗mM,nΦmM (Xn) ≥

12 and EffMm(Xn) = ΦMm(Xn)

Φ∗Mm,n≥ 1

2 for all n

with ΦMm(Xn) = mini 6=j∈{1,...,n} ‖xi − xj‖ the maximin-distance criterion,and Φ∗Mm,n its optimal (maximum) value

Proof. (Gonzalez, 1985)by construction:ΦMm(Xn+1) , minxi 6=xj∈Xn+1 ‖xi − xj‖ = d(xn+1,Xn) = ΦmM(Xn)X∗n a ΦmM-optimal design: the n balls B(x∗i ,ΦmM(X∗n )), x∗i ∈ X∗n , cover X⇒ one of them contains 2 points xi , xj in Xn+1 for any Xn+1 (n + 1 points)⇒ ΦMm(Xn+1) ≤ ‖xi − xj‖ ≤ 2ΦmM(X∗n )⇒ Φ∗Mm,n+1 ≤ 2ΦmM(X∗n ) ≤ 2ΦmM(Xn) = ΦMm(Xn+1)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 22 / 41

Page 44: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.1/ Coffee-house design

X = [0, 1]2, n = 7

1

2

3

4

5

6

7

EffmM(Xn), n = 1 . . . , 50

0 5 10 15 20 25 30 35 40 45 500.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Regular construction à largefluctuations of EffmM(Xn)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 23 / 41

Page 45: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.1/ Coffee-house design

X = [0, 1]2, n = 7

1

2

3

4

5

6

7

EffmM(Xn), n = 1 . . . , 50

0 5 10 15 20 25 30 35 40 45 500.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Regular construction à largefluctuations of EffmM(Xn)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 23 / 41

Page 46: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.2/ Submodularity and greedy algorithms

3.2/ Submodularity and greedy algorithms

XQ = {x(1), . . . , x(Q)} a finite set with Q points in X(regular grid, first Q points of a LDS — Halton, Sobol’ . . . )

ψ: 2XQ −→ R a set function (to be maximized)non-decreasing: ψ(A ∪ {x}) ≥ ψ(A ) for all A ⊂XQ and x ∈XQ

Definition 1:ψ is submodular iff ψ(A ) +ψ(B) ≥ ψ(A ∪B) +ψ(A ∩B) for all A ,B ⊂XQ

Equivalently, Definition 1’ (diminishing return property):ψ is submodular iff ψ(A ∪ {x})− ψ(A ) ≥ ψ(B ∪ {x})− ψ(B) for allA ⊂ B ⊂XQ and x ∈XQ \B

(a sort of concavity property for set functions)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 24 / 41

Page 47: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.2/ Submodularity and greedy algorithms

3.2/ Submodularity and greedy algorithms

XQ = {x(1), . . . , x(Q)} a finite set with Q points in X(regular grid, first Q points of a LDS — Halton, Sobol’ . . . )

ψ: 2XQ −→ R a set function (to be maximized)non-decreasing: ψ(A ∪ {x}) ≥ ψ(A ) for all A ⊂XQ and x ∈XQ

Definition 1:ψ is submodular iff ψ(A ) +ψ(B) ≥ ψ(A ∪B) +ψ(A ∩B) for all A ,B ⊂XQ

Equivalently, Definition 1’ (diminishing return property):ψ is submodular iff ψ(A ∪ {x})− ψ(A ) ≥ ψ(B ∪ {x})− ψ(B) for allA ⊂ B ⊂XQ and x ∈XQ \B

(a sort of concavity property for set functions)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 24 / 41

Page 48: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.2/ Submodularity and greedy algorithms

Greedy Algorithm:1 set A = ∅2 while |A | < k

find x in XQ such that ψ(A ∪ {x}) is maximalA ← A ∪ {x}

3 end while4 return Ak = A

Denote ψ∗k = maxB⊂XQ , |B|≤k ψ(B)

Theorem (Nemhauser, Wolsey & Fisher, 1978): When ψ is non-decreasing andsubmodular, then for all k ∈ {1, . . . ,Q} the algorithm returns a set Ak such that

ψ(Ak)− ψ(∅)ψ∗k − ψ(∅) ≥ 1− (1− 1/k)k ≥ 1− 1/e > 0.6321

Bad news: we maximize −ΦmM which is non-decreasing but not submodularà no guaranteed efficiency for sequential optimization

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 25 / 41

Page 49: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.2/ Submodularity and greedy algorithms

Greedy Algorithm:1 set A = ∅2 while |A | < k

find x in XQ such that ψ(A ∪ {x}) is maximalA ← A ∪ {x}

3 end while4 return Ak = A

Denote ψ∗k = maxB⊂XQ , |B|≤k ψ(B)

Theorem (Nemhauser, Wolsey & Fisher, 1978): When ψ is non-decreasing andsubmodular, then for all k ∈ {1, . . . ,Q} the algorithm returns a set Ak such that

ψ(Ak)− ψ(∅)ψ∗k − ψ(∅) ≥ 1− (1− 1/k)k ≥ 1− 1/e > 0.6321

Bad news: we maximize −ΦmM which is non-decreasing but not submodularà no guaranteed efficiency for sequential optimization

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 25 / 41

Page 50: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.2/ Submodularity and greedy algorithms

Greedy Algorithm:1 set A = ∅2 while |A | < k

find x in XQ such that ψ(A ∪ {x}) is maximalA ← A ∪ {x}

3 end while4 return Ak = A

Denote ψ∗k = maxB⊂XQ , |B|≤k ψ(B)

Theorem (Nemhauser, Wolsey & Fisher, 1978): When ψ is non-decreasing andsubmodular, then for all k ∈ {1, . . . ,Q} the algorithm returns a set Ak such that

ψ(Ak)− ψ(∅)ψ∗k − ψ(∅) ≥ 1− (1− 1/k)k ≥ 1− 1/e > 0.6321

Bad news: we maximize −ΦmM which is non-decreasing but not submodularà no guaranteed efficiency for sequential optimization

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 25 / 41

Page 51: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

3.3/ Submodular alternatives to minimax

A) Covering measure, c.d.f. and dispersion [SIAM UQ, Lausanne, 2016]For any r ≥ 0, any Xn ∈X n, define the covering measure of Xn by

ψr (Xn) = vol{X ∩ [∪ni=1B(xi , r)]} à non-decreasing and submodular

Maximizing ψr (Xn) is equivalent to maximizingFXn (r) = ψr (Xn)/vol(X ) = µL{X ∩[∪n

i=1B(xi ,r)]}µL(X )

which can be considered as a c.d.f., with FXn (r) ∈ [0, 1], increasing in r ,and FXn (0) = 0, FXn (r) = 1 for any r ≥ ΦmM(Xn)

Take any probability measure µ on X (e.g., with finite support XQ)à define FXn (r) = µ{X ∩ [∪n

i=1B(xi , r)]}as a function of r Ô forms a c.d.f.,as a function of Xn Ô non-decreasing and submodular

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 26 / 41

Page 52: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

3.3/ Submodular alternatives to minimax

A) Covering measure, c.d.f. and dispersion [SIAM UQ, Lausanne, 2016]For any r ≥ 0, any Xn ∈X n, define the covering measure of Xn by

ψr (Xn) = vol{X ∩ [∪ni=1B(xi , r)]} à non-decreasing and submodular

Maximizing ψr (Xn) is equivalent to maximizingFXn (r) = ψr (Xn)/vol(X ) = µL{X ∩[∪n

i=1B(xi ,r)]}µL(X )

which can be considered as a c.d.f., with FXn (r) ∈ [0, 1], increasing in r ,and FXn (0) = 0, FXn (r) = 1 for any r ≥ ΦmM(Xn)

Take any probability measure µ on X (e.g., with finite support XQ)à define FXn (r) = µ{X ∩ [∪n

i=1B(xi , r)]}as a function of r Ô forms a c.d.f.,as a function of Xn Ô non-decreasing and submodular

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 26 / 41

Page 53: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

3.3/ Submodular alternatives to minimax

A) Covering measure, c.d.f. and dispersion [SIAM UQ, Lausanne, 2016]For any r ≥ 0, any Xn ∈X n, define the covering measure of Xn by

ψr (Xn) = vol{X ∩ [∪ni=1B(xi , r)]} à non-decreasing and submodular

Maximizing ψr (Xn) is equivalent to maximizingFXn (r) = ψr (Xn)/vol(X ) = µL{X ∩[∪n

i=1B(xi ,r)]}µL(X )

which can be considered as a c.d.f., with FXn (r) ∈ [0, 1], increasing in r ,and FXn (0) = 0, FXn (r) = 1 for any r ≥ ΦmM(Xn)

Take any probability measure µ on X (e.g., with finite support XQ)à define FXn (r) = µ{X ∩ [∪n

i=1B(xi , r)]}as a function of r Ô forms a c.d.f.,as a function of Xn Ô non-decreasing and submodular

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 26 / 41

Page 54: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Which r should we take in FXn (r)?A positive linear combination of non-decreasing submodular functions isnon-decreasing and submodular

à Consider Ψb,B,q(Xn) =∫ B

b rq FXn (r) dr , for B > b ≥ 0, q > 0Ô guaranteed efficiency bounds when maximizing with a greedy algorithm

Justification:

Ψ0,B,q(Xn) = Bq+1

q+1 FXn (B)− 1q+1

∫ B0 rq+1 FXn (dr)

Take any B ≥ ΦmM(Xn) Ô FXn (B) = 1Maximizing Ψ0,B,q(Xn) for B large enough ⇔ minimizing

∫ B0 rq+1 FXn (dr)

⇔ minimizing[∫ B

0 rq+1 FXn (dr)]1/(q+1)

and[∫ B

0 rq+1 FXn (dr)]1/(q+1)

→ ΦmM(Xn) as q →∞

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 27 / 41

Page 55: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Which r should we take in FXn (r)?A positive linear combination of non-decreasing submodular functions isnon-decreasing and submodular

à Consider Ψb,B,q(Xn) =∫ B

b rq FXn (r) dr , for B > b ≥ 0, q > 0Ô guaranteed efficiency bounds when maximizing with a greedy algorithm

Justification:

Ψ0,B,q(Xn) = Bq+1

q+1 FXn (B)− 1q+1

∫ B0 rq+1 FXn (dr)

Take any B ≥ ΦmM(Xn) Ô FXn (B) = 1Maximizing Ψ0,B,q(Xn) for B large enough ⇔ minimizing

∫ B0 rq+1 FXn (dr)

⇔ minimizing[∫ B

0 rq+1 FXn (dr)]1/(q+1)

and[∫ B

0 rq+1 FXn (dr)]1/(q+1)

→ ΦmM(Xn) as q →∞

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 27 / 41

Page 56: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

ImplementationEasy when

X approximated by XQ = {s1, . . . , sQ} ∈X Q , µ = 1Q∑Q

j=1 δsj

Xn ∈XQn

(inter-distances ‖si − sj‖ are only computed once)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 28 / 41

Page 57: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Ex: X = [0, 1]2, XQ = grid with Q = 33× 33 = 1089 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)

à EffmM(Xn) as a function of n

EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - -

Centered L2 discrepancies

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 29 / 41

Page 58: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Ex: X = [0, 1]2, XQ = grid with Q = 33× 33 = 1089 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)

à EffmM(Xn) as a function of n

EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - Xnmax with Ψb,B,q(·)

Centered L2 discrepancies

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 29 / 41

Page 59: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Ex: X = [0, 1]2, XQ = grid with Q = 33× 33 = 1089 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)

à EffmM(Xn) as a function of n

EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - First nmax points of Sobol’ LDS

Centered L2 discrepancies

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 29 / 41

Page 60: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Ex: X = [0, 1]2, XQ = grid with Q = 33× 33 = 1089 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)

à EffmM(Xn) as a function of n

EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - Centered L2 discrepancies

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 29 / 41

Page 61: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Ex: X = [0, 1]3, XQ = grid with Q = 113 = 1331 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)

à EffmM(Xn) as a function of n

EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - -

Centered L2 discrepancies

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 30 / 41

Page 62: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Ex: X = [0, 1]3, XQ = grid with Q = 113 = 1331 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)

à EffmM(Xn) as a function of n

EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - Centered L2 discrepancies

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 30 / 41

Page 63: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Large d (d > 3, say): we cannot use a regular grid XQ

Þ adaptive grid with MCMC: illustration for d = 2 (Q ≈ nmaxd)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 31 / 41

Page 64: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Large d (d > 3, say): we cannot use a regular grid XQ

Þ adaptive grid with MCMC: illustration for d = 2 (Q ≈ nmaxd)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 31 / 41

Page 65: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Ex: X = [0, 1]10, XQ = adaptive grid with Q = 1000 pointsnmin = 30, nmax = 100, q = 2 in Ψb,B,q(·)

à EffmM(Xn) = R∗nΦmM (Xn) as a function of n

EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - -

Centered L2 discrepancies

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 32 / 41

Page 66: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Ex: X = [0, 1]10, XQ = adaptive grid with Q = 1000 pointsnmin = 30, nmax = 100, q = 2 in Ψb,B,q(·)

à EffmM(Xn) = R∗nΦmM (Xn) as a function of n

EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - Centered L2 discrepancies

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 32 / 41

Page 67: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

B) Lq relaxation

Approximate X by XQ with Q elements sk , k = 1, . . . , q, q > 0, minimize

Φq,Q(Xn) ,

1Q

Q∑k=1

(1n

n∑i=1‖sk − xi‖−q

)−11/q

For any Xn, Φq,Q(Xn)→ ΦmM(Xn; XQ), q →∞

where ΦmM(Xn; XQ) = maxx∈XQ d(x,Xn)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 33 / 41

Page 68: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

B) Lq relaxation

Approximate X by XQ with Q elements sk , k = 1, . . . , q, q > 0, minimize

Φq,Q(Xn) ,

1Q

Q∑k=1

(1n

n∑i=1‖sk − xi‖−q

)−11/q

For any Xn, Φq,Q(Xn)→ ΦmM(Xn; XQ), q →∞

where ΦmM(Xn; XQ) = maxx∈XQ d(x,Xn)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 33 / 41

Page 69: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Efficiency:

If X∗n,q minimizes Φq,Q(·), then

EffmM(X∗n,q; XQ) ≥ (nQ)−1/q

Φq,Q(·) is non-increasingΨ(·) = 1

n Φqq,Q(·) is supermodular

[ongoing joint work with João Rendas (CNRS, I3S, UCA) & Céline Helbert (ÉcoleCentrale Lyon)]

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 34 / 41

Page 70: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design

Efficiency:

If X∗n,q minimizes Φq,Q(·), then

EffmM(X∗n,q; XQ) ≥ (nQ)−1/q

Φq,Q(·) is non-increasingΨ(·) = 1

n Φqq,Q(·) is supermodular

[ongoing joint work with João Rendas (CNRS, I3S, UCA) & Céline Helbert (ÉcoleCentrale Lyon)]

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 34 / 41

Page 71: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

4) Measures minimizing regularized dispersion— joint work with Anatoly Zhigljavsky (LP & AZ, 2017)

For a n-point design, Lq relaxation:

Φq,Q(Xn) ,

1Q

Q∑k=1

(1n

n∑i=1‖sk − xi‖−q

)−11/q

, q > 0

For a design measure ξ, integral version:

φq(ξ) ,[∫

X

(∫X

‖s− x‖−q ξ(dx))−1

µ(ds)]1/q

, q > 0

with µ uniform prob. measure on X (µ(X ) = 1)

Th 1: φqq(·), q > 0, is convex, and is strictly convex when 0 < q < d

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 35 / 41

Page 72: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

4) Measures minimizing regularized dispersion— joint work with Anatoly Zhigljavsky (LP & AZ, 2017)

For a n-point design, Lq relaxation:

Φq,Q(Xn) ,

1Q

Q∑k=1

(1n

n∑i=1‖sk − xi‖−q

)−11/q

, q > 0

For a design measure ξ, integral version:

φq(ξ) ,[∫

X

(∫X

‖s− x‖−q ξ(dx))−1

µ(ds)]1/q

, q > 0

with µ uniform prob. measure on X (µ(X ) = 1)

Th 1: φqq(·), q > 0, is convex, and is strictly convex when 0 < q < d

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 35 / 41

Page 73: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

q ≥ d

φq(ξ) > 0 for any discrete measure ξφq(ξ) = 0 for any ξ equivalent to the Lebesgue measure on X

. . . not very interesting

0 < q < d

(Strict) convexity of φqq(·) Þ “equivalence theorem”

Th 2: ξq,∗ minimizes φq(·) iff ∀y ∈X , d(ξq,∗, y) ≤ φqq(ξq,∗)

where d(ξ, y) =∫

X

{‖y− x‖−q [∫

X ‖x− z‖−q ξ(dz)]−2}

µ(dx)= directional derivative of φq

q(·) at ξ in the direction of δy

ξq,∗ is unique and d(ξq,∗, y) = φqq(ξq,∗) for ξq,∗-almost all y ∈X

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 36 / 41

Page 74: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

q ≥ d

φq(ξ) > 0 for any discrete measure ξφq(ξ) = 0 for any ξ equivalent to the Lebesgue measure on X

. . . not very interesting

0 < q < d

(Strict) convexity of φqq(·) Þ “equivalence theorem”

Th 2: ξq,∗ minimizes φq(·) iff ∀y ∈X , d(ξq,∗, y) ≤ φqq(ξq,∗)

where d(ξ, y) =∫

X

{‖y− x‖−q [∫

X ‖x− z‖−q ξ(dz)]−2}

µ(dx)= directional derivative of φq

q(·) at ξ in the direction of δy

ξq,∗ is unique and d(ξq,∗, y) = φqq(ξq,∗) for ξq,∗-almost all y ∈X

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 36 / 41

Page 75: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Two distinct situations

0 < q ≤ d − 2

ξq,∗ may be singular

Ex: X = Bd (0, 1); ξq,∗ = δ0 is optimal

max{0, d − 2} < q < d

Th 3: ξq,∗ does not possess atoms in the interior of X

Þ Minimization of Φq,Q(Xn): take q > d − 2 to be space-filling

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 37 / 41

Page 76: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Two distinct situations

0 < q ≤ d − 2

ξq,∗ may be singular

Ex: X = Bd (0, 1); ξq,∗ = δ0 is optimal

max{0, d − 2} < q < d

Th 3: ξq,∗ does not possess atoms in the interior of X

Þ Minimization of Φq,Q(Xn): take q > d − 2 to be space-filling

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 37 / 41

Page 77: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Construction of ξq,∗?

Discretize X (again): replace µ by µQ = 1Q∑Q

k=1 δsk (grid or LDS)

φqq(ξ;µQ) = trace[M−1(ξ)]

with M(ξ) =∫

X diag{Q ‖x− sk‖−q, k = 1, . . . ,Q} ξ(dx) (Q × Q-dimensional)Þ an A-optimal design problem: multiplicative, or vertex-direction, algorithm

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 38 / 41

Page 78: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Ex: X = Bd (0, 1), make use of symmetry(only consider distributions of the radii)

φqq(ξ) function of q for ξ = δ0 (. . .), ξ = µ (- - -) and ξ = ξq,∗ (—)

d = 3

0 0.5 1 1.5 2 2.5 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

d = 5

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.2

0.4

0.6

0.8

1

1.2

1.4

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 39 / 41

Page 79: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

µ(r) uniform on Bd (0, r), d = 3

efficiency φqq(ξq,∗)φq

q(µ(r)) of µ(r) function of rq = 0.5, q = 1.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.88

0.9

0.92

0.94

0.96

0.98

1

optimal r function of q

0 0.5 1 1.5 2 2.5 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 39 / 41

Page 80: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

d = 3, optimal density of radii for ξq,∗ (with respect to ϕ(r) = drd−1)

q = 2, q = 2.1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

q = 2.25, q = 2.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 39 / 41

Page 81: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

d = 3, optimal density of radii for ξq,∗ (with respect to ϕ(r) = drd−1)

q = 2, q = 2.1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

q = 2.25, q = 2.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

Minimization of Φq,Q(Xn):take q > d − 2 to be space-filling,no point near the border of X

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 39 / 41

Page 82: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Conclusions

Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient

(without any evaluation of ΦmM(Xn))

Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))

Use an adaptive grid XQ (MCMC) if d is largeConsider projections on lower dimensional subspaces?Which submodular alternative is best?

What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41

Page 83: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Conclusions

Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient

(without any evaluation of ΦmM(Xn))Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))

Use an adaptive grid XQ (MCMC) if d is largeConsider projections on lower dimensional subspaces?Which submodular alternative is best?

What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41

Page 84: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Conclusions

Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient

(without any evaluation of ΦmM(Xn))Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))

Use an adaptive grid XQ (MCMC) if d is large

Consider projections on lower dimensional subspaces?Which submodular alternative is best?

What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41

Page 85: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Conclusions

Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient

(without any evaluation of ΦmM(Xn))Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))

Use an adaptive grid XQ (MCMC) if d is largeConsider projections on lower dimensional subspaces?

Which submodular alternative is best?

What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41

Page 86: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

4) Measures minimizing regularized dispersion

Conclusions

Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient

(without any evaluation of ΦmM(Xn))Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))

Use an adaptive grid XQ (MCMC) if d is largeConsider projections on lower dimensional subspaces?Which submodular alternative is best?

What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41

Page 87: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France

References

References I

Auffray, Y., Barbillon, P., Marin, J.-M., 2012. Maximin design on non hypercube domains and kernel interpolation. Statistics and Computing 22 (3),703–712.

Cardot, H., Cénac, P., Monnez, J.-M., 2012. A fast and recursive algorithm for clustering large datasets. Comput. Statist. Data Anal. 56 (6), 1434–1449.

Du, Q., Faber, V., Gunzburger, M., 1999. Centroidal Voronoi tessellations: applications and algorithms. SIAM Review 41 (4), 637–676.

Gonzalez, T., 1985. Clustering to minimize the maximum intercluster distance. Theoretical Computer Science 38, 293–306.

Janson, S., 1986. Random coverings in several dimensions. Acta Mathematica 156 (1), 83–118.

Janson, S., 1987. Maximal spacings in several dimensions. The Annals of Probability 15 (1), 274–280.

Lekivetz, R., Jones, B., 2015. Fast flexible space-filling designs for nonrectangular regions. Quality and Reliability Engineering International 31 (5), 829–837.

Lloyd, S., 1982. Least squares quantization in PCM. IEEE Transactions on Information Theory 28 (2), 129–137.

MacQueen, J., 1967. Some methods for classification and analysis of multivariate observations. In: LeCam, L., Neyman, J. (Eds.), Proceedings of the FifthBerkeley Symposium on Mathematical Statistics and Probability. Vol. 1. pp. 281–297.

Müller, W., 2007. Collecting Spatial Data. Springer, Berlin, [3rd ed.].

Nemhauser, G., Wolsey, L., Fisher, M., 1978. An analysis of approximations for maximizing submodular set functions–I. Mathematical Programming14 (1), 265–294.

Niederreiter, H., 1992. Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia.

Pronzato, L., 2017a. Minimax and maximin space-filling designs: some properties and methods for construction. Journal de la Société Française deStatistique 158 (1), 7–36.

Pronzato, L., 2017b. On the elimination of inessential points in the smallest enclosing ball problem. Optimization Methods and SoftwareTo appear.

Pronzato, L., Zhigljavsky, A., 2017. Measures minimizing regularized dispersion. Submitted.

Schaback, R., 1995. Error estimates and condition numbers for radial basis function interpolation. Advances in Computational Mathematics 3 (3), 251–264.

Vazquez, E., Bect, J., 2011. Sequential search based on kriging: convergence analysis of some algorithms. Proc. 58th World Statistics Congress of the ISI,August 21-26, Dublin, Ireland, arXiv preprint arXiv:1111.3866v1.

Yildirim, E., 2008. Two algorithms for the minimum enclosing ball problem. SIAM Journal on Optimization 19 (3), 1368–1391.

Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 41 / 41