optimal design of groundwater remediation systems with

Optimal Design of GroundwaterRemediation Systems with Sampling

MethodsC. T. Kelley

Department of Mathematics

Center for Research in Scientific Computation

North Carolina State University

Raleigh, North Carolina, USA

Chinese University of Hong Kong

Hong Kong, January 25, 2006

Supported by NSF, ARO, DOEd.

C. T. Kelley – p.1

Outline

• Collaborators

• General formulation and example problem• Formulation• Optimization Landscapes• Results

• Implicit Filtering

• Community problems

• Conclusions

C. T. Kelley – p.2

Collaborators

• IFFCO developers from NCSU Math:Tony Choi, Owen Eslinger, Paul Gilmore,Alton Patrick, Vincent Bannister

• NCSU Math: Corey Winton, Dan Finkel, Jörg Gablonsky,Katie Fowler , Chris Kees, Jill Reese, Todd Coffey

• Other Places:

• Boeing: Andrew Booker, John Dennis

• UNC: Casey Miller, Matt Farthing, Glenn Williams

• ERDC: Stacy Howington

• Univ. Trier: Astrid Battermann

• Mich Tech: Alex Mayer

C. T. Kelley – p.3

What’s the problem.

• Control flow of contaminants in groundwater.• Keep plume on site.• Keep concentrations at acceptable levels.• Minimize cost, mass of contaminant,

contaminant concentration . . .

• Control flow and pressure.• Municipal water supplies.• Agriculture.

C. T. Kelley – p.4

Many approaches

• Tightly coupled simulation/optimization (Shoemaker)

• GAs (Mayer, Pinder, Minkser, Yeh . . . )

• Surrogates: response surface, neural nets

Our long-term objectives:

•• Examine many formulation, simulator, optimizercombinations in a portable way.

• Build testbed for both groundwater and optimizationcommunities.

• Design new approaches.

Today: one problem/simulator/optimizer triple.C. T. Kelley – p.5

What we do.

• Black-box optimization:Use accepted, widely-used, production 3D simulators.• Improved portability/documentation relative to

research codes.• Community might listen to us.• No guarantee of differentiability wrt design

variables.

• Put problems/solutions on the web.http://www4.ncsu.edu/˜ctk/community.html

C. T. Kelley – p.6

Flow in the saturated zone

Ss∂h∂ t

= ∇ · (K∇h)+S ,

Data:

• BC, IC, spatial domain Ω• Ss (specific storage coefficient)

• K (hydraulic conductivity)

• S is the souce/sink term,computed from the design variables.

Output: h (hydraulic head)Typical simulators: ADH, FEMWATER, MODFLOW.

C. T. Kelley – p.7

Species Transport

∂θC∂ t

= ∇ · (θD ·∇C)−∇ · (θvC)+SC.

Data: porosity θ , interphaseDesign: S C mass sources/sinks

• C is concentration, solution of PDE;

• v is velocity, computed from h;

• D is the dispersion tensor, computed from h.

Typical simulator: MT3D

C. T. Kelley – p.8

Computing the fluid velocity v

Darcy’s law says

θv =kµ

(∇p+ρg∇z)

• p = ρg(h− z): fluid pressure

• k: intrinsic permeability; µ : dynamic viscosity

• ρ : density; g: gravitational acceleration

• ∇z: vector in vertical direction

C. T. Kelley – p.9

What’s D

Di j = δi jαt |v|+(αl−αt)viv j

|v|+δi jτD∗

• αl, αt : longitudinal/transverse dispersivities

• τ : tortuosity of the porous medium

• D∗: free liquid diffusivity.

C. T. Kelley – p.10

Design variables

Number and location of wells, pumping rates.Pumping rates and well locations go in the source term forflow

∫

ΩS (t)dΩ =

n

∑i=1

Qi

and for concentration∫

ΩS

C(t)dΩ =n

∑i=1

C(xi)Qi.

Examples:

• Sum of δ functions at well locations.

• Well model with well diameter, well type, ...

C. T. Kelley – p.11

Example: Hydraulic Capture

Minimize total cost:

f T (Q) =n

∑i=1

c0db0i + ∑

Qi<−10−6

c1|Qim|b1(zgs−hmin)b2

︸ ︷︷ ︸

f c

+

∫ t f

0

(

∑i,Qi<−10−6

c2Qi(hi− zgs)+ ∑i,Qi>10−6

c3Qi

)

dt

︸ ︷︷ ︸

f o

,

to keep a contaminant inside a “capture zone”.Ω = [0,1000]× [0,1000]

C. T. Kelley – p.12

Notation

• (xi,yi) are well locations.

• Qi is pumping rate(> 0 for injection, < 0 for extraction.

• di is depth of well i

• hi is head at well i (MODFLOW)

• zgs is elevation of ground surface

• Qm is design pumping rate.

• hmin is minimum allowable pumping rate.

C. T. Kelley – p.13

Boundary conditions: Unconfined aquifer

∂h∂x

∣∣∣∣x=0

=∂h∂y

∣∣∣∣y=0

=∂h∂ z

∣∣∣∣z=0

= 0, t > 0

K∂h∂ z

(x,y,z = h, t > 0) =−1.903×10−8 (m/s).

h(1000,y,z, t > 0) = 20−0.001y(m),h(x,1000,z, t > 0) = 20−0.001x(m),h(x,y,z,0) = hs.

C. T. Kelley – p.14

Constraints I

Simple bounds:

Qemax ≤ Qi ≤ Qimax, i = 1, ...,n

Limits on the pumps.Simple linear inequality:

∑i

Qi ≥ QmaxT ,

limit on total net extraction rate.

C. T. Kelley – p.15

Constraints II

Keep wells away from Dirichlet boundary

0≤ xi,yi ≤ 800.

Bounds on h

hmin ≤ hi ≤ hmax, i = 1, ...,n

No dry holes.Velocity Highly nonlinear function of well locations.50×50×10 grid.

C. T. Kelley – p.16

Formulation Decisions I

• Contain plume: constrain velocity at zone boundary.Test velocity at five downstream locations.Approximate velocity with difference of h.Five new constraints.Need only flow code. Better simulations in progress.

• Implicit filtering deals with bounds naturally.

• Treat constraints as yes/no for sampling method• Stratify by cost.• Avoid simulator if infeasible wrt cheap (linear)

constraints.

• Well is de-installed if pumping rate is suff small.

C. T. Kelley – p.17

Formulation Decisions II

• Discontinuous objective.• 50×50×10 grid. Wells must be on grid nodes.

Move to nearest.• Remove well from array (di = 0) if pumping rate is

too small.

• Treat head constraint and linear constraintsas hidden or yes-no.

• Initial iterate: two extraction, two injection

C. T. Kelley – p.18

Initial iterate

No wells

0 100 200 300 400 500 600 700 800 900 10000

100

200

300

400

500

600

700

800

900

1000

24

23

22.5

22

21.5

21

20.5

20

19.5

Four well configuration

0 100 200 300 400 500 600 700 800 900 10000

100

200

300

400

500

600

700

800

900

1000

20

20.5

21

22

22.5

22

23

21

20.5

22

22.5

C. T. Kelley – p.19

Initial/Final Plumes

Initial Plume Plume after 5 years

C. T. Kelley – p.20

Results

Optimal configuration

0 100 200 300 400 500 600 700 800 900 10000

100

200

300

400

500

600

700

800

900

1000

22

20.5

19.5

20 19.5

20

Cost of Optimization

0 50 100 150 200 250 300 350 4002

3

4

5

6

7

8

9x 10

4

Calls to MODFLOW

Fun

ctio

n V

alue

C. T. Kelley – p.21

Landscapes

Vary (x1,y1) near initial iterate

400450

500550

600650

700

100

150

200

250

300

350

4007.88

7.9

7.92

7.94

7.96

7.98

8

8.02

x 104

xy

Fun

ctio

n V

alue

Vary pumping rate initial iterate

−1.5−1

−0.50

0.51

1.5

x 10−3

−1.5

−1

−0.5

0

0.5

1

1.5

x 10−33

4

5

6

7

8

x 104

well 4

Well 1F

unct

ion

Val

ue

C. T. Kelley – p.22

Other Approaches

100

101

102

103

2

3

4

5

6

7

8

9

10

11

12x 10

4

Calls to MODFLOW

Fun

ctio

n V

alue

HC

IFFCO DIRECT−L Nomad2N Nomad2N+1GA DE APPS

C. T. Kelley – p.23

Wait a minute!

• Optimal point has one well, we start with four.Was this fair?

• How does performance depend on initial iterate?

• Do some methods benefit from special choices?

• How can you construct a “rich” set of initial iterates fortesting?

• We’re trying:• Use DIRECT to find feasible points.• Use statistics to identify clusters.• Sample wisely within the clusters.

C. T. Kelley – p.24

Optimization strategy

minx∈D

f (x)

• Conventional gradient-based methods can fail if f is• multi-modal,• non-convex,• discontinuous,• non-deterministic, or if

• D is not determined by smooth inequalities.

Sampling methods attempt to address these problems.

C. T. Kelley – p.25

Stencil-based sampling methods

• Begin with a base point x.

• Examine points on a stencil;reject or adjust points not in D .

• Determine location of next stencil.

• If f (x) is smallest, shrink the stencil.

Examples: Coordinate Search, Nelder-Mead,Hooke-Jeeves, (P)MDS, GPS, Implicit Filtering

This is not global optimization.

C. T. Kelley – p.26

Example: coordinate search

Sample f at x on a stencil centered at x, scale=h

S(x,h) = x±hei

• Move to the best point.

• If x is the best point, reduce h.

Necessary Conditions: No legal direction points downhill(which is why you reduce h).

C. T. Kelley – p.27

What if x is the best point?Smooth Objective

If f (x)≤minz∈S(x,h) f (z) (stencil failure)then‖∇ f (x)‖= O(h)

So, if (xn,hn) are the points/scales generated by coordinatesearch and f has bounded level sets, then

• hn→ 0 (finitely many grid points/level) and therefore

• any limit point of xn is a critical point of f .

Not a method for smooth problems.

C. T. Kelley – p.28

Model Problemmotivated by the landscapes.

minRN

f

f = fs +φ

• fs smooth, easy to minimize; φ noise

• N is small, f is typically costly to evaluate.

• f has multiple local minimawhich trap most gradient-based algorithms.

C. T. Kelley – p.29

Convergence?

Stencil failure implies that

‖∇ fs(xn)‖= O

(

hn +‖φ‖S(xn,hn)

hn

)

where‖φ‖S(x,h) = max

z∈S|φ(z)|.

C. T. Kelley – p.30

Bottom line

So, if (xn,hn) are the points/scales generated by coordinatesearch, f has bounded level sets, and

limn→∞

(hn +h−1n ‖φ‖S(x,hn)) = 0

then

• hn→ 0 (finitely many grid points/level) and therefore

• any limit point of xn is a critical point of f .

Analysis for Hooke-Jeeves, MPS, GPS is similar.Nelder-Mead is different.

C. T. Kelley – p.31

Implicit Filtering

Accelerate coordinate search with a quasi-Newton method.imfilter(x, f , pmax,τ ,hn,amax)

for k = 0, . . . dofdquasi(x, f , pmax,τ ,hn,amax)

end forpmax, τ , amax are termination parameters

fdquasi = finite difference quasi-Newton method using acentral difference gradient ∇h f .

C. T. Kelley – p.32

fdquasi(x, f , pmax,τ ,h,amax)

p = 1

while p≤ pmax and ‖∇h f (x)‖ ≥ τh docompute f and ∇h f

terminate with success on stencil failure

update the model Hessian H if appropriate; solve

Hd =−∇h f (x)

use a backtracking line search, with at most amax backtracks,

to find a step length λterminate with failure on > amax backtracks

x← x+λd; p← p+1

end whileif p > pmax report iteration count failure

C. T. Kelley – p.33

Implicit Filtering: Start

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

C. T. Kelley – p.34

Implicit Filtering: Move

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

C. T. Kelley – p.34

Implicit Filtering: Move

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

C. T. Kelley – p.34

Implicit Filtering: Stencil Failure

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

C. T. Kelley – p.34

Implicit Filtering: Shrink/Move

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

C. T. Kelley – p.34

Implicit Filtering: Termination

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

C. T. Kelley – p.34

Basic Convergence Theorem

Let (xn,hn) be the sequence from implicit filtering.If

• ∇ fs is Lipschitz continuous.

• limn→∞(hn +h−1n ‖φ‖S(x,hn)) = 0

• fdquasi terminates with success for infinitely many n.

then any limit point of xn is a critical point of fs.Convergence rates need more.

C. T. Kelley – p.35

Hidden Constraints

A hidden constraint is violated if the call to f fails.One can (we do) treat all but bound constraints as hidden.What to do?

• Assign a large value.

• Assign a value of infinity and reject the sample. OK forHJ/MDS, bad for IF.

• Assign a value a bit higher than the nearby points.

• Reject and use least squares to compute ∇h f .• Coming in new version.

C. T. Kelley – p.36

NCSU fortran implementation: IFFCO

• Naturally parallel; but watch out for load balancing.

• Use best value in stencil + quasi-Newton search.

• Quasi-Newton model Hessian essential in practice.

• Termination• fdquasi: stencil failure, small gradient, amax, pmax• overall: list of scales, budget, target

• parameters: IFFCO has reasonable defaults

• hidden constraints: f does not return a valueIFFCO is prepared

• MATLAB: imfil.m new version coming soon

C. T. Kelley – p.37

How to get the software

• IFFCO: Implicit Filtering For Constrained Optimization

• New version released May, 2001MPI/PVM/Serial

• ftp to ftp.math.ncsu.edu inFTP/kelley/iffco/IFFCO.tar.gz or email toTim_Kelley@ncsu.eduhttp://www4.ncsu.edu/˜ctkhttp://www4.ncsu.edu/˜ctk/iffco.html

C. T. Kelley – p.38

Community Problems

• Suite of problems in groundwater remediation3D, flow+transport, varying difficulty.

• We provide or point to simulators/optimization codesthat will produce a formulation and a solution.

• No pretense that formulation or solution is bestpossible.

• Portable, good testbed for optimization codes.

C. T. Kelley – p.39

How to get the Community Problems

• Constantly updated onhttp://www4.ncsu.edu/˜ctk/community.html

• Packages include problems, makefiles, IFFCOexample.You need to get the simulators; we tell you how.

• Tested on• g77: Solaris, Red Hat 7.3,8.0, MAC OSX, IBM-SP• MPI: IBM-SP, several linux clusters

• Three problems in place (only MODFLOW).

• New problems under construction.

• Massive comparison in progressGA, NOMAD, Boeing DE, DIRECT, APPS

C. T. Kelley – p.40

Conclusions

• Optimal design of groundwater remediation problems• Formulation: constraints

specification of problemchoice of simulators

• Community problems• Solution: we like sampling methods

• Sampling methods• Variants of coordinate search• Implicit filtering

C. T. Kelley – p.41