inverse problems in radiative transfer … · inverse problems are ubiquitous in all areas of...

Inverse Problems in Radiative Transfer

Kyle J. Daun

AbstractInverse problems are ubiquitous in all areas of radiative heat transfer. They canbroadly be categorized as inverse design problems, with the goal of inferring adesign configuration that satisfies an engineering requirement, and parameterestimation problems, in which an unknown parameter or set of parameters isinferred from measurement data. Both problem types are mathematicallyill-posed, due to the fact that the available information is either barely adequateor inadequate to identify a unique or stable solution. This chapter reviews themathematical properties of inverse problems, along with inverse analysisschemes that have been used to solve inverse problems that arise in radiativetransfer. This is followed by a summary of inverse design and parameter estima-tion problems reported in the literature, along with detailed case studies for aninverse boundary condition design problem and a parametric estimation problemfocused on inferring the soot aggregate size distribution from light scatteringmeasurements.

Contents1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Overview of Inverse Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.1 What is Inverse Analysis? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Types of Inverse Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Solution Methods for Inverse Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.1 Linear Regularization Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 Nonlinear Programming Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.3 Metaheuristic Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

K.J. Daun (*)Department of Mechanical and Mechatronics Engineering, University of Waterloo, Waterloo, ON,Canadae-mail: [email protected]

# Springer International Publishing AG 2017F.A. Kulacki (ed.), Handbook of Thermal Science and Engineering,DOI 10.1007/978-3-319-32003-8_64-1

1

mailto:[email protected]

3.4 Bayesian Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.5 Does Least-Squares Minimization Constitute Inverse Analysis? . . . . . . . . . . . . . . . . . . . . . 22

4 Radiant Enclosure Design Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.1 Linear Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.2 Nonlinear Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.3 Case Study: Inverse Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5 Parameter Estimation Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345.1 Linear Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.2 Nonlinear Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385.3 Case Study: Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6 Conclusions and Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 Cross-References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

1 Introduction

Inverse problems can be defined as problems of inference, in which the goal is toinfer some unknown quantity from specified information. They can be sub-categorized as parameter estimation problems, in which the objective is to infer aparameter or parameter distribution from indirect measurement data, or inversedesign problems, in which the goal is to infer a design configuration that satisfiesan imposed design objective. Inverse problems are distinct from generic inferenceproblems because they are mathematically ill-posed, which makes them challengingto solve.

In the context of thermal sciences, early work focused mainly on the inverse heatconduction problem (Beck 1968), in which the objective is to infer a surface heatflux, or some other remote quantity, from the time-response of subsurface thermo-couples. Likewise, the earliest examples of inverse analysis applied to thermalradiation were parameter estimation problems, and focused on inferring the propertyor distribution of a property within a participating medium from intensity measure-ments made at the periphery; the first studies were carried out by Özişik’s group inthe late 1980s (Ho and Özşik 1988), although flame tomography (Santoro et al.1981) and X-ray based medical tomography (Cormack 1973) considerably predatethis work. The radiant enclosure design problem, in which the goal is usually to inferan enclosure design that produces a desired heat flux and temperature distribution ona particular surface (called the “design surface”), was conceived in mid- to late 1990s(Fedorov et al. 1998; Jones 1999; Harutunian et al. 1995), and further developed byHowell’s group at the University of Texas at Austin in the early 2000s.

Subsequent interest in inverse problems of both types has exploded: two majortextbooks on thermal radiation have devoted chapters on inverse analysis (Howellet al. 2016; Modest 2013), and many articles have been published on the topic.An important caveat, however, is that a majority of these papers are limited to“numerical experiments,” while only a small fraction feature physical experimenta-tion or promote schemes that have otherwise been implemented in the physicalworld. While it may still be argued that these problems are interesting and worthy of

2 K.J. Daun

study from a theoretical standpoint, many of these papers exclude a thoroughdiscussion of the mathematical properties underlying inverse problems.

This chapter presents a pedagogical review of inverse analysis and how it appliesto radiative transfer. It begins with general overview of inverse problems, mathe-matical ill-posedness, and solution techniques that have been used to solve inverseproblems involving thermal radiation. Subsequent sections focus on the radiantenclosure design problem and the parameter estimation problem, with detailedexamples to highlight the mathematical properties of both problem classes.

2 Overview of Inverse Analysis

2.1 What is Inverse Analysis?

The common objective of all inverse problems is to infer some unknown property orattribute from specified, albeit indirect, information. While it may be tempting tosimply classify inverse problems as “problems of inference,” by this definition onecould argue that virtually every problem is an inverse problem, since the goal of mostproblems in science and engineering is to infer something from some specifiedinformation. Taking things to a logical extreme: if one looks out the window andsee blue sky and sunshine, one could infer that it is not raining. Does this constitutean inverse problem? Clearly not.

A key property of inverse problems is that they aremathematically ill-posed. Thisterm originates from Hadamard (1923), who formally defined well-posed problemsas those that: have a solution, which is unique, and stable to small perturbations tothe input data. Problems that violate the: (i) existence, (ii) uniqueness, and (iii)stability criteria are thus mathematically ill-posed. In all cases, ill-posedness arisesfrom an information deficit.

Parameter estimation problems are often ill-posed when the informationcontained in the measured data is barely adequate to specify a unique solution,making it highly sensitive to measurement nose (violating the stability criterion), oris altogether insufficient to specify a unique solution, in which case multiplescenarios exist that could explain the data (violating the uniqueness criterion). Asan example of parameter inference, Bohren and Huffman (1983) present an exampleof a knight hunting dragons in a forest. The knight could easily anticipate thefootprints made by different species of dragon, if this information is known aheadof time. Inferring the species of dragon from the footprints may be more difficult,however, particularly if different species of dragons leave similar footprints (Fig. 1).The problem is even harder if the footprints are smeared in the mud, which violatesthe stability criterion since a small change to the footprint may suggest a differentspecies of dragon. The problem becomes impossible to solve if the footprint isindistinguishable for two different species of dragon. In this example, predictingthe footprint left by a dragon is the forward, well-posed problem, while inferringthe dragon species from the footprint is the inverse, ill-posed problem. Note that, ifeach candidate species of dragon left a distinct footprint, the problem would not be

Inverse Problems in Radiative Transfer 3

mathematically ill-posed, and therefore would not be classified as an inverseproblem.

Inverse design problems often have multiple solutions that produce the desiredoutcome, violating the uniqueness criterion, and, quite often, have no feasible solitonwhatsoever, which violates the existence criterion. As a trivial example, shoppingfor orange juice could be cast as a design problem, in which the objective is topurchase 1 L of orange juice for the lowest possible cost. The problem is straight-forward unless the grocery store stocks two competing brands of orange juice at thesame cost, which violates the uniqueness criterion. On the other hand, the store maybe sold out of orange juice, which violates the existence criterion.

The ill-posedness of both examples is caused by an information deficit, and,without exception, inverse analysis techniques address this ill-posedness by intro-ducing more information into the analysis. In the case of parameter estimationproblems, the additional information takes the form of some prior knowledge ofthe unknown parameters (e.g., the species of dragon likely to inhabit the forest),while in design problems, the designer may specify additional desired attributes thattilts the outcome toward one of multiple candidate solutions (e.g., one brand oforange juice tastes better).

In a more technical context, consider again the inverse heat conduction andcomputed tomography examples mentioned at the beginning of this chapter. Bothare governed by integral equations of the first kind (IFKs),

g sð Þ ¼ðf tð Þk s, tð Þdt (1)

Fig. 1 A knight hunting dragons in a forest can be conceived as an inverse problem. Predicting thefootprint left by a species of dragon is the well-posed forward problem, but inferring the speciesfrom a footprint is mathematically ill-posed if the footprints left by different species are similar. Thedifficulty of the problem is exacerbated if the footprints are smeared in mud, which is conceptuallysimilar to measurement noise

4 K.J. Daun

where g(s) is the measurement data, f(t) is the unknown parameter, and k(s,t) is thekernel function. Predicting g(s) from a specified f(t) by integration is the well-posedforward problem. Inferring f(t) from g(s) requires deconvolution or “unfolding” ofthe integral equation (Wing and Zhart 1991), which is the ill-posed inverse problem.The ill-posedness arises from the “smoothing” or “blending” action of the kernel. Inthe forward problem, variations in f(t) are “blended out” by the integration into muchsmaller variations in g(s). While the exact solution for f(t) can, in principle, berecovered by deconvolution, if g(s) is perturbed in some way (e.g., with measure-ment noise, or by discretizing the problem), these perturbations tend to overwhelmthe small variations in f(t) that correspond to the “true” problem physics, and areamplified into large artifacts by the deconvolution process.

2.2 Types of Inverse Problems

Understanding the type and fundamental properties of the inverse problem is anessential first step, since this determines the appropriate solution methodology.Inverse problems that can be expressed as an IFK like Eq. 1 are “linear,” sincef and g are linearly-related. Most often these problems are solved in matrix form,Ax = b, where x and b are discrete approximations of f and g, respectively, and theelements of A correspond to k in some way. In the simplest representation

bi ¼ f sið Þ ¼ðg tð Þk si, tð Þdt �

Xnj¼1

Aijg tj� � ¼Xn

j¼1

Aijxj (2)

which is written for each element of b to form an (n � n) matrix equation. (For nowassume that the number of unknowns in x equals the number of knowns in b, so A isa square matrix.) Writing the integral equation in discrete form facilitates solutionand also explicates the underlying ill-posedness of the problem via the properties ofA. Specifically, the smoothing properties of kmakes A ill-conditioned, which can beobserved through a singular value decomposition, A = UΣVT, where U and V areorthonormal matrices whose column vectors form a basis for the data space andsolution space, respectively, and Σ is a diagonal matrix containing the singularvalues, arranged in descending order, i.e., Σ = diag[σ1, σ2, σ3, . . ., σn],σ1 � σ2 � . . . � σn. Discretizing a linear IFK usually produces an A matrix havingsingular values that decay continuously over several orders of magnitude. Anexample singular value spectrum is shown in Fig. 2. In this scenario, an “exact”solution can be recovered if the “exact” data is known and the linear system isconsistent, i.e.,

xexact ¼Xnj¼1

uTj bexact

σjvj (3)


The summation builds xexact in a way analogous to a Fourier series; the firstsummation terms correspond to the lowest frequency components, while the“sharpest” features are resolved with the latter terms. Hansen (1999) further eluci-dates the relationship between Fourier series and SVD. Even though the singularvalues become very small as j increases, the magnitudes of the terms in thenumerator, the “Fourier coefficients,” |uj

Tbexact|, decay faster for a smoothly-varyingx, and consequently the summation converges. This is called the “discrete Picardcriterion.” If the data is perturbed with an error vector, b = bexact + δb, however,

x ¼Xnj¼1

uTj bexact þ δbð Þσj

vj

|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}xexact

¼Xnj¼1

uTj bexact

σjvj

|fflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflffl}xexact

þXnj¼1

uTj δbσj

vj

|fflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflffl}δx

(4)

In this case, the discrete Picard criterion is violated because the singular valuesdecay faster than the |uj

Tδb| terms, which makes x large in magnitude (due to ||δx||)and oscillatory. Equivalently, using the properties of vector and matrix norms, it canbe shown that

δxk kxk k � δbk k

xk k Ak k A�1�� ¼ δbk k

xk k Cond Að Þ (5)

where Cond(A) = σmin/σmax = σ1/σn is the condition number, which is large fordiscrete ill-posed problems. (||�|| denotes the Euclidean norm.)

An interesting scenario arises when there are more unknowns than equations, i.e.,if x�ℜn and b � ℜm, A�ℜm�n andm< n. In this case, A is rank-deficient and the

100

0 5 10 15 20i

25 30 35 40

10−5

10−10σi

10−15

10−20

Fig. 2 A singular valuespectrum from a discretizedintegral equation of the firstkind. The singular valuesdecay continuously overseveral orders of magnitude,indicating the ill-posedness ofthe problem

6 K.J. Daun

singular value decomposition produces U�ℜm�n, V�ℜn�n, and Σ�ℜm�n.Equivalently, one can conceive of a matrix problem having n equations andn unknowns but only m < n independent equations, but in this case, U�ℜn�n andS�ℜn�n, and the last n-m column vectors of U and the last n-m singular values arezero, or “null.” Equation 3 becomes

x ¼Xmj¼1

uTj b

σjvj

|fflfflfflfflfflffl{zfflfflfflfflfflffl}xr

þXnj¼mþ1

Cjvj

|fflfflfflfflfflffl{zfflfflfflfflfflffl}xn

(6)

where {Cj, j = m + 1. . .n} is a set of arbitrary constants. The solution has twocomponents: xr and xn, which belong to the range and nullspace of A, respectively.The first component is also the x having the smallest Euclidean norm that solvesAx = b, while the second component is formed by a linear combination of the lastm-n column vectors of V. This scenario violates Hadamard’s uniqueness criteria,since there exists an infinite set of candidate solutions for x that satisfy Ax = b. Inthis scenario, A is rank-deficient and has a nontrivial null-space. In contrast to theill-conditioned but full-rank case, in which the matrix equation provides barelyenough information to specify a unique (albeit unstable) solution, in the rank-deficient case, the matrix equation provides no information about a key componentof the physical solution, xn.

A convenient geometric interpretation for these scenarios comes from plotting theresidual norm squared, ||Ax � b||2, for a two-dimensional case; the contours areellipsoids centered on the exact solution, xexact = A�1bexact, or more generallyxexact = A#bexact, where A# is the pseudoinverse of A. The principal axes of theellipse are aligned with the column vectors of V and have lengths inversely propor-tional to the singular values, as shown for a well-conditioned case in Fig. 3a. In anill-conditioned case, shown in Fig. 3b, the ellipse axis corresponding to the smallestsingular value is very long, and the square of the residual norm has a very long,shallow topography surrounding xexact. In this scenario, any candidate solutionx along the “valley” can be substituted into the measurement equation to obtain asmall residual. In the context of parameter estimation, this residual may be smallerthan measurement noise, while in the case of inverse design, these points mayrepresent multiple candidate solutions. Conversely, if one were to contaminateb with small perturbations, the resulting solutions are widely-distributed within theelliptical contours of the residual norm squared, highlighting the violation ofHadamard’s stability criterion. (In fact, these ellipses correspond to confidenceintervals via the chi-squared statistic.) Finally, the A matrix is rank-deficient if itsrows are identical, as shown in Fig. 3c. In contrast to Fig. 3a, b, ||Ax � b||2 has atrough shape, and is invariant in the v2 direction. (The contours can be conceived asellipses with one infinite principal axis that corresponds to σ2 = 0.) As definedabove, the xr solution component is obtained from the first summation term, in thiscase xr = u1

Tb/σ1�v1. Adding xn = C�v2 to xr defines a locus of solutions aligned


with the “bottom” of the valley. In a topographic sense, the “floor” of this valley isperfectly flat, highlighting the violation of Hadamard’s uniqueness criterion. Whilethese results apply in two dimensions, they can be conceptually extended to n-dimensional hyperspace, in which case the “exact” solution is surrounded by n-dimensional hyperellipsoids.

These figures also highlight a key property of linear inverse problems: with theexception of the rank-deficient case, the square of the residual norm is convex,meaning that it has one global minimum. This also is often, but not always, thecase for nonlinear inverse problems. Nonlinear problems arise when the unknown

0.25

0.2a b

c

0.15

0.1

0.05

0X2

X2

X2

X1 X1

X1

−0.05

−0.1

−0.15

−0.2

0.3

1.2

1

0.8

0.6

0.4

0.2

0

0 1

0.4

V2/σ2

V2

σ1

Xr

v1u1Tb

V1/σ1

0.5

5

4

3

2

1

0

0.6 0.7−0.25

−1

−2

−3

−4

−4−5

−2

−0.2

−0.4

−0.6

−0.8−0.5 0.5

0 2 4

Fig. 3 Contour plots of ||Ax � b||22 for: (a) a well-conditioned problem; (b) an ill-conditioned

problem; and (c) a rank-deficient problem. Contours are ellipses with principal axes aligned in thedirections of the V column vectors and lengths proportional to the inverse of the correspondingsingular values. In the well-posed and ill-conditioned cases, xexact is marked with a cross,surrounded by trial solutions generated by perturbing bexact with a randomly generated vector δbhaving the same expected value. (Note the difference in axis scale.) The solution to the rank-deficient problem is the dotted line defined by the sum of xr = (ui

Tb/σ1)v1 and xn = C�v2

8 K.J. Daun

variables cannot be expressed as a linear function of the known variables; forexample, nonlinear IFKs have the form

g sð Þ ¼ðf tð Þk s, t, f tð Þ½ �dt (7)

and may be discretized as b = a(x). Figure 4a shows that, in contrast to the linearcase, the contours are not ellipses. Nevertheless, the function is convex over theplotted region, indicating that Hadamard’s uniqueness criterion is satisfied, but thestability criterion is violated. On the other hand, Fig. 4b shows a scenario in whichthe residual norm is nonconvex; instead of a global minimum, there are multiple localminima. This may violate Hadamard’s uniqueness criterion, if multiple minima existthat represent alternative viable solutions to the governing equations. The convexity/nonconvexity of ||a(x) � b||2 strongly influences the solution technique that shouldbe used, as discussed in the next section.

3 Solution Methods for Inverse Problems

Just as there are many types of inverse problems, an equally large and diverse suiteof analysis tools have been developed for solving them. All inverse problems derivetheir ill-posedness from an information deficit, and likewise all inverse analysistechniques address this ill-posedness by introducing additional information about theexpected or desired solution attributes into the solution procedure. This informationis called “prior information” since it is known prior to the inference procedure. Sincethe information contained in the ill-posed governing equations is barely adequate orinadequate to specify a unique solution, the information added during the solutionscheme strongly influences the recovered solution. Accordingly, it is crucial for the

1

1.5

2.5

3.5

4.5

5.5

2

3

4

5

6

0

1

2

3

−4 −3 −2 −1 0 1 2 3 4

4

5

6

X1

X2

X2

X1

1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3

a b

Fig. 4 (a) A nonlinear inverse function that violates Hadamard’s stability criterion, but is convexover the function domain. (Note that the contours of ||A(x) � b||2 are not ellipses.) In somescenarios, it is possible to have a nonconvex residual norm, like the one shown in (b). In thisfunction, the two local minima have the same value, and violate Hadamard’s uniqueness criterion


analyst to be fully aware of how the solution schemes introduce prior informationinto the analysis.

It is also important to use “the right tool for the job,”which, in this context, meansselecting an algorithm that is compatible with the nature of the ill-posedness, size ofthe problem, and number and type of variables. At a minimum, the followingquestions should be considered:

• What is the nature of the ill-posedness? Is ||Ax � b||2 or ||A(x) � b||2 convex ornonconvex? (Linear inverse problems are always convex, as are many nonlinearinverse problems.)

• What additional information can be added to mitigate the ill-posedness?• Is uncertainty quantification important? (It usually is important, but often

ignored.)• How many variables does the problem have? Are these variables continuous,

discrete, or a mixture of both types?• How costly are the model equations to solve, and what is the computational

budget for solving the problem? Can the problem be solved deterministically, orstochastically using a Monte Carlo method?

By skipping this essential step, the analyst risks selecting an inappropriateanalytical tool, leading to excessive computational time and questionable results.

Inverse solution methods can be broadly categorized as: linear regularizationschemes, nonlinear programming methods, and metaheuristics. Bayesian methodsfor solving stochastic inverse problems are also considered.

3.1 Linear Regularization Techniques

These are the oldest class of techniques for solving linear inverse problems; most ofthese approaches exploit the spectral structure of the coefficient matrix. Threetechniques are briefly summarized: truncated singular value decomposition(TSVD); Tikhonov regularization; and conjugate gradient regularization. A moredetailed description is available elsewhere in the literature, e.g., Hansen (1999).

The singular value decomposition described in the previous section presents anobvious regularization approach: if the first summation terms in Eq. 3 fill out the“main part” of the solution, while the ill-posedness is associated with the smallsingular values and the high-frequency solution components represented by the lastsummation terms, why not ignore these last terms? In truncated singular valuedecomposition (TSVD), the last p summation terms are excluded from the recon-struction, so the regularized solution is given by

xp ¼Xn�p

j¼1

uTj b

σjvj (8)

10 K.J. Daun

Prior information is added in an implicit way. Progressively truncating the higher-order summation terms promotes a solution that becomes less oscillatory (i.e.,neighboring elements in x do not vary significantly) and small in magnitude. Theamount of prior information introduced into the analysis is controlled by p, which iscalled a regularization parameter. Since the summation terms relate to progressivelyhigher frequency components, TSVD can also be interpreted as a bandpass filter.

Tikhonov regularization (Tikhonov and Arsenin 1977) is based on minimizinga quadratic function

F xð Þ ¼ Ax� bk k2 þ λ2 Lxk k2 (9)

where L is a smoothing matrix and λ is the regularization parameter. Minimizing thefirst term promotes a small residual of the measurement equations, while minimizingthe second term, the “regularizing” or “penalty” term, promotes a desired solutionattribute that depends on the choice of L. In standard, or zeroth-order Tikhonov, L ischosen as the identity matrix, and minimizing the second term promotes a solutionthat has a small norm. In first-order Tikhonov regularization, L is a discrete approx-imation of the derivative operator,

L ¼1 �1 0 � � � 0

0 1 �1 ⋱ ⋮⋮ ⋱ ⋱ ⋱ 0

0 � � � 0 1 �1

2664

3775�ℜ n�1ð Þ�n (10)

so ||Lx||2 is minimized by a spatially-smooth solution. Because the function isquadratic, the minimum can be found by setting ∇F(x) = 0 and solving the normalequations

ATAþ λ2LTL� �

x ¼ ATb (11)

although this should not be done in practice because Cond(ATA) = Cond(A)2.Instead, the problem can be rewritten as a linear least-squares minimization problem

xλ ¼ arg minx F xð Þf garg minxA

λL

� �x� b

0

� ��2

( )(12)

and solved through singular-value decomposition. (The notation arg minx[F(x)]means “the value of x that minimizes F(x).”)

There are a multitude of ways to conceptualize how Tikhonov regularizationincorporates prior information into the analysis. The least-squares function in Eq. 12explicitly shows that the regularizing term introduces an additional set of equations,Lx = 0, into the analysis, the influence of which relative to Ax = b is determined byλ. In terms of topography, adding ||Lx||2 to ||Ax � b||2 “steepens the valley” thatsurrounds xexact, which reduces the magnitude of δx. Making λ too large pushes xλ


away from xexact and toward the minimizer of ||Lx||. Finally, in the case of zeroth-order Tikhonov, it can be shown that

xλ ¼Xnj¼1

f j λð Þ uTj b

σjvj, f j λð Þ ¼ σ2j

σ2j þ λ2(13)

where {fj(λ), j = 1. . .n} are the filter factors. If λ << σj, fj(λ) = 1, while if λ >> σj,fj = 0, highlights the relationship between Tikhonov regularization and TSVD.Similar relations can be derived for other choices of L (Hansen 1999).

Conjugate gradient (CG) regularization also works by minimizing a residualnorm, F(x) = ||Ax � b||2, which is equivalent to solving the normal equations

ATAx ¼ ATb (14)

While Tikhonov regularization does this in one step, conjugate gradient regular-ization carries out the minimization iteratively. Starting from the null vectorx0 = {0}, each successive iteration is updated according to

xkþ1 ¼ xk þ αkpk (15)

where αk is the step size and pk is the search direction. There are many possibleschemes for choosing αk and pk; the conjugate gradient scheme generates a sequenceof “noninterfering” search directions that are mutually-conjugate with respect toATA, i.e., they satisfy

pkT

ATA� �

pk ¼ 0, i 6¼ j (16)

The step size is then found by setting @f(xk + αkpk)/ @αk = 0, which can be doneanalytically for a quadratic function.

In the case of a well-conditioned matrix, and using exact arithmetic, exactlyn steps are required to arrive at x = A�1b because the search directions arenoninterfering; in other words, progress made in one search direction is not“undone” in a subsequent search direction. If x0 = 0, it can also be shown thatsolution norm, ||xk||, increases monotonically with each iteration, while the residualnorm ||Axk � b|| drops monotonically with increasing k. For ill-conditioned matricesand a perturbed b = bexact + δb, however, the CG iteration schemes will converge tox = xexact + δx, where δx is large. Instead, as a regularization scheme CG exploitssemiconvergence; during the first few iterations, xk approaches xexact, but withsubsequent iterations, xk hones onto xexact + δx, so iterations are terminated before||Ax � b|| is minimized and the iteration number, k, is the regularization parameter.This property is due to the fact that the search directions are generated in a sequencethat roughly matches the order of column vectors inV for an SVD ofATA, so each xk

is conceptually similar to the TSVD solution that would be obtained by excludingthe last n-k summation terms in Eq. 3. Like TSVD, x becomes progressively largerand more oscillatory with each successive iteration, so in CG, the prior

12 K.J. Daun

information is introduced implicitly by truncating the number of regularization steps.CG regularization is particularly well-suited to large-scale linear problems, since itinvolves repeated vector products and avoids matrix decomposition or inversion.

In all of the above linear regularization schemes, the analyst must determine theamount of prior information with which to supplement Ax= b via the regularizationparameter. This must be done with care: sufficient regularization must be used toovercome the ill-posedness of the model equations, but using too much regulariza-tion overwhelms the often meagre information contained in these equations. To thisend a number of parameter-choice methods have been developed to elucidate thetradeoff between a solution that minimizes ||Ax � b|| and one that complies withprior knowledge. These include: the L-curve curvature method (Hansen and O’Leary1993); Morozov’s discrepancy principle (Morozov 1968); and generalized cross-validation (Golub et al. 1979).

3.2 Nonlinear Programming Methods

The linear regularization techniques presented in the previous chapter can onlybe applied directly to linear problems. In many scenarios, however, the knownsand unknowns are related in a nonlinear way, and the objective is often to solvea nonlinear least-squares problem of the form

x ¼ arg minx F xð Þ½ � ¼ arg minx1

2f xð Þk k2

� �¼ 1

2

Xmj¼1

aj xð Þ � bj� 2

(17)

where f(x) = a(x) � b is the residual vector. In problems where F(x) is convex, thesolution can be found iteratively through nonlinear programming. This procedurefollows Eq. 15, starting from some initial guess x0. At each iteration, the searchdirection pk is chosen based on the local curvature of F(xk), and a step size is thenselected, either through univariate minimization (a “line search”) or bya nonconvergent series (e.g., αk = α0k�1.) These algorithms are classified as non-linear programming (NLP) methods (Bertsekas 1999; Gill et al. 1986).

The most obvious choice for pk is the direction of steepest descent, i.e.,pk = �∇F(xk), but this approach often requires a large number of iterations becauseprogress in successive search directions partially “cancels out” progress made inprevious iterations, as shown in Fig. 5a. A better choice considers both the first- andsecond-order curvature of f(xk). A locally-quadratic model of F(x) can beconstructed through a Taylor-series expansion,

F xk þ pk� � � F xk

� �þ pkT

∇F xk� �þ 1

2pk

TT∇2F xk� �

pk (18)

where ∇2F(xk) is the Hessian matrix,


∇2F xk� � ¼

@2F=@x21 @2F=@x1@x2 � � � @2F=@x1@xn@2F=@x2@x1 @2F=@x22 � � � @2F=@x2@xn⋮ ⋱ ⋮@2F=@xn@x1 @2F=@xn@x2 � � � @2F=@x2n

2664

3775 (19)

Taking the gradient of Eq. 19, neglecting derivatives higher than second-order,and setting ∇F(xk + pk) = 0 results in

Fig. 5 Minimization of a convex nonlinear function by nonlinear programming (NLP): (a) steepestdescent; (b) Newton’s method; (c) conjugate gradient minimization. Steepest descent requires themost iterations because progress made in one search direction is cancelled out due to “back-tracking.” Newton’s method uses the fewest steps but requires the second-order sensitivities. Thesearch directions in conjugate gradient are mutually-conjugate and noninterfering. In the case of ann-dimensional linear problem, Newton’s method and CG would require one step and n steps,respectively

14 K.J. Daun

∇2F xk� �

pk ¼ �∇F xk� �

(20)

This direction is called “Newton’s direction,” and the corresponding solutionscheme is called Newton’s method. (The search direction is self-scaling, so αk = 1.)This approach generally requires the fewest steps to reach a stationary point of F(x),as shown in Fig. 5b, and only one step if a(x) = Ax and F(x) is quadratic.

A major drawback of Newton’s method is the computational expense needed tocalculate the second-order sensitivities. While this can sometimes be done analyti-cally, more often they must be approximated by finite differences, and if F(x) iscostly to evaluate, the expense of calculating the Hessian at every iteration negatesthe advantage of needing fewer iterations. Accordingly, two techniques have beendeveloped that only use first-order sensitivities to approximate the Hessian: thequasi-Newton method (sometimes called the variable metric method), and theGauss–Newton method. In the quasi-Newton method, the gradient information is“built up” over successive iterations to develop an approximation for the Hessian,∇2F(xk) � Bk, and the search direction is then found by solving Bkpk = �∇F(xk).The algorithm is initiated from B0 = I so p0 = �∇F(xk), i.e., the steepest descentdirection, but in most cases, Bk quickly converges to the Hessian in only a fewiterations. In contrast to Newton’s method, a separate scheme (e.g., nonconvergentseries, line-search) must also be used to select αk at each iteration.

The Gauss–Newton method exploits the special structure of the least-sum-of-squares function, Eq. 17. It can be shown that

∇F xð Þ ¼ J xð ÞTf xð Þ (21)

where J(x) is the Jacobian matrix,

J xð Þ ¼@f 1=x1 @f 1=x2 � � � @f 1=xn@f 2=x1 @f 2=x2 � � � @f 2=xn⋮ ⋮ ⋮

@f m=x1 @f m=x2 � � � @f m=xn

2664

3775 (22)

The second-order sensitivities are given by

∇2F xð Þ ¼ J xð ÞTJ xð Þ þXmj¼1

f j xð Þ∇2f j xð Þ � J xð ÞTJ xð Þ (23)

since the second-order terms vanish close to x*. In this case, Newton’s direction isapproximated by solving

J xk� �T

J xk� �

pk ¼ �J xk� �T

f xk� �

(24)

which is the Gauss–Newton method. A comparison of Eqs. 24 and 14 shows thatthese are the normal equations found by setting the gradient of ||J(x)p � f(x)||2 equalto zero, and in the linear case, a(x) = Ax and J(x) = A. In the case of ill-posed


problems, J(x)TJ(x) is usually ill-conditioned or singular, especially near the mini-mum of F(x). Alternatively, in the first few steps, Eq. 24 may return a poor choice forpk since F(x) may not resemble a quadratic function far away from its minimum. Inthese scenarios, calculation of the search direction can be stabilized using zeroth-order Tikhonov regularization

J xk� �T

J xk� �þ λI

h ipk ¼ �J xk

� �Tf xk� �

(25)

where λ is usually chosen heuristically. This is the Levenberg–Marquardt method. Itis important to note that, in contrast to Tikhonov regularization, the regularizationparameter does not affect the curvature of F(x) by adding information to theproblem, and therefore does not actually regularize the problem (Aster et al. 2013).

Finally, the linear conjugate gradient algorithm described above can also beapplied to nonlinear problems in which the search directions are generated byreplacing f=Ax� b in the linear problem with ∇F(x). Because F(x) is not generallyquadratic, more than n steps are required to minimize F(x), but typically the numberof steps is O (n) (Fig. 5c). As with the linear case, key advantages of nonlinearconjugate gradient minimization is that it only requires calculation of vector prod-ucts, so it is well-suited for large-scale problems.

3.3 Metaheuristic Methods

Nonlinear programming is well-suited for methods in which ||a(x) � b||2 is convex(i.e., one global minimum, cf. Fig. 4a) and the variables are nondiscrete. Somenonlinear inverse problems involve nonconvex functions, like Fig. 4b, or havediscrete variables. A suite of techniques called “metaheuristic methods” have beendeveloped for solving these problems. In contrast to nonlinear programming tech-niques, which are derived from rigorous mathematical analysis often based ona quadratic model of the objective function, and have a deterministic outcome,metaheuristic algorithms are heuristically-derived and include a stochastic compo-nent to prevent the algorithm from becoming stuck in a local minimum. Many ofthese algorithms are inspired by physical or biological processes. This section brieflysummarizes four types of metaheuristics that have been used to solve inverseproblems in radiation heat transfer: simulated annealing, genetic algorithms, Taboosearch, and particle swarm optimization.

Simulated annealing (SA) (Kirkpatrick et al. 1983) is motivated by therearrangement of atoms and molecules during phase change (e.g., the annealing ofmetals). At each iteration, a candidate update is generated, xk+1,c, often by samplingfrom an n-dimensional normal distribution centered on xk. The candidate point isaccepted with a probability determined by the Metropolis criterion (Metropoliset al. 1953)

16 K.J. Daun

P xkþ1, c� � / exp �ΔF xkþ1, c, xk� �

Tk

� �(26)

where ΔF = F(xk) � F(xk+1,c) is the improvement realized by the new pointand Tk is an annealing temperature. The acceptance of xk+1 is determined bycomparing P(xk+1,c) with a random number drawn from a uniform distributionbetween 0 and 1. Progress of the SA algorithm mirrors the rearrangement of atomsduring the annealing of metals: the random thermal motion of atoms allows themto move into higher energy states at high temperatures, but this becomes lessprobable at lower temperatures. Thus, if the metal is quenched quickly, the atomsbecome locked in high-energy configurations (dislocations), while quenching themetal slowly allows the atoms to form a low-energy crystal lattice. The algorithmperformance depends on the annealing temperature and how it changes with itera-tion: dropping the temperature slowly increases the likelihood of identifying a goodlocal minimum, but the temperature must decay quickly enough for computationalexpediency. An example SA optimization path is shown in Fig. 6.

This algorithm highlights some key differences between NLP and metaheuristics:(i) In contrast to NLP algorithms, which always generate a downhill direction,according to Eq. 26, an uphill direction may be accepted with a probabilitythat increases with Tk; (ii) while NLP algorithms generate xk+1 deterministically,in SA both the candidate update xk+1,c and its acceptance are generatedrandomly; and (iii) while the convergence properties of NLP algorithms are wellcharacterized, the performance of the SA algorithm depends strongly on the choice

3ba

2

1

0

−1

−2

−3−3 −2 −1 0

X2

X1 X1

1 2 3

3

2

1

0

−1

−2

−3−3 −2 −1 0

X2

1 2 3

Fig. 6 Minimization of a nonconvex function by simulated annealing. In contrast to NLP methods,which are deterministic, metaheuristic algorithms contain a random element and can occasionallyaccept an uphill direction. (a) shows that “fast annealing” becomes trapped in a shallow localminimum, while in (b), the temperature is dropped more slowly, which prevents the algorithm fromgetting trapped in the local minimum


of heuristics, i.e., the annealing schedule and those that control how the candidateupdates are generated. These properties broadly apply to all metaheuristics.

Genetic algorithms (Mitchell 1996) are inspired by evolution in the natural world,i.e., “survival of the fittest.” The first step is to encode the variables onto a chromo-some: in the case of continuous variables, this is often done using a binary repre-sentation, so each bit corresponds to a “gene” on the chromosome and a variable isrepresented by multiple genes. The algorithm then proceeds as follows:

1. An initial population of candidate solutions is generated, usually by randomsampling, and the corresponding functions F(x) are evaluated. The size of thepopulation depends on the size of the solution space.

2. A mating pool consisting of the best candidates in the population is selected andcandidates are organized into mating pairs at random. New chromosomes aregenerated through a crossover operation of genes, which represents the exchangeof genetic material during reproduction.

3. An optional “mutation step” can take place, in which a small subset of genes onthe chromosomes are randomly perturbed.

These steps continue until a computational budget is exceeded. Many variationsof the above steps have been developed that can improve solution quality andcomputational efficiency, depending on the attributes of the inverse problem.

The underlying strategy of the tabu search algorithm (Glover 1986) is to “remem-ber” previous search paths, and to avoid returning to recently-visited subdomains(i.e., around a previously-visited local minimum). The algorithm performsa sequence of local or “neighborhood” downhill searches, where neighborhooddefines a subset of solution space in which the candidate solutions share the samecore attributes. A candidate list of potential solutions within the neighborhood isgenerated, and the best of these candidates is chosen as the solution. Since thissolution may be worse than the incumbent solution, it is possible to make “uphill”steps, which allows an algorithm to escape from a local minimum. A memorystructure is used to keep track of previous solutions, which are excluded from thecurrent search for a set period of iterations. (These directions become “tabu,” whichgives the algorithm its name.)

In particle swarm optimization (PSO) (Kennedy and Eberhart 1995), a populationof candidate solutions, each of which is called a “particle,” is dispersed over thesearch space. The particles move through the search space with a trajectoryinfluenced by the best position visited by the particle, but also the best positionsvisited by other members of the population. The communication of informationbetween the individual particles results in a “swarm behavior.” Information betweenthe particles is communicated through a social network structure, and the intercon-nectedness of the structure determines how quickly the particles swarm, and thequality of the final solution. A high degree of interconnectedness causes rapidswarming, but may also lead to an incomplete search of the problem space and

18 K.J. Daun

a shallow local minimum. On the other hand, a less interconnected structure canprovide a more thorough search and a higher quality solution, but is more time-consuming.

As a final note, it is important to debunk a commonly-held belief that meta-heuristics algorithms return a global minimum. They are often categorized as “globalsearch algorithms” because, unlike NLP algorithms, they contain mechanisms toavoid becoming trapped in local minima. Nevertheless, there is no way to guaranteethat the solution returned by a metaheuristics algorithm is the true global minimum;in general, the only scenario in which the recovered minimum can be established asthe global minimum is if the function can be shown to be convex.

3.4 Bayesian Inference

The previous sections showed how linear regularization, NLP, and metaheuristicsalgorithms can be used to recover a solution x that best satisfies some specified b,generally by minimizing ||a(x) � b||2. Because inverse problems are ill-posed,however, many candidate solutions exist that solve the model equations within thetolerances prescribed by measurement noise or design flexibility. In this context,least-squares minimization is deficient since it only provides a single answer to theinverse problem, and little to no information about the existence of other candidatesolutions. An awareness of multiple candidate solutions is important in inversedesign problems, since these represent feasible design alternatives, and parameterestimation problems, since the existence of multiple solutions implies uncertainty inthe recovered variables.

Bayesian inference (von Toussaint 2011; Kaipio and Somersalo 2005) directlyaddresses this need by conceiving x and b (and often ancillary model parameters) asrandom variables that obey probability densities related by Bayes’ equation

p xjbð Þ ¼ p bjxð Þppr xð Þp bð Þ (27)

In Eq. 27, p(x|b) is the posterior density of x conditional on b, p(b|x) is thelikelihood of b occurring for a hypothetical x, ppr(x) is the probability of x basedon prior information, and p(b) is the evidence, which scales the right-hand side ofEq. 27 in order to satisfy the Law of Total Probabilities. (The terms “prior” and“posterior” denote “before” and “after” the instant at which the informationcontained in b is incorporated into the estimate of x.) The distribution width ofp(b|x) reflects the uncertainty in the measurement data in a parameter estimationproblem, or the “looseness” of the design tolerance in an inverse design problem,while that of p(x|b) corresponds to the uncertainty in the inferred parameters orthe range of acceptable design parameters. The remainder of this discussionfocuses on parameter estimation, with the understanding that it can be extended toinverse design.


In the context of parameter estimation, Bayes’ equation reflects the fact that themeasurement data is not deterministic, but is better described by a distributioncentered on a mean value. If measurement errors are caused by a sequence ofunrelated random events, the measurement data often obeys a normal distribution,and the probability of observing a single measurement bj conditional on a hypothet-ical set of measurement parameters is

p bjx� � ¼ 1ffiffiffiffiffiffiffiffiffiffi

2πσ2j

q exp � bj � aj xð Þ� 22σ2j

( )(28)

where σj is the distribution width. (Often bj is the average of a set of measurements,in which case σj is the standard deviation of the mean.) If the measurement noise isuncorrelated with the noise contaminating the other measurements and their distri-bution widths are equal, i.e. the noise is homoskedastic, then the joint probability ofobserving a set of measurements in b is given by

p bjxð Þ ¼ ∏m

j¼1

p bjx� � ¼ 1

2πσ2ð Þm=2exp � b� a xð Þk k2

2σ2

( )(29)

In the absence of prior information, Eq. 27 shows p(x|b) / p(b|x) and the mostlikely x conditional on the data in b is found by maximizing Eq. 29, which, subject tothe assumption of independent and identically-distributed measurement noise, alsominimizes ||b � a(x)||2. This is the maximum likelihood estimate, xMLE = argmaxx[p(b|x)].

The ill-posedness of inverse problems gives ||b � a(x)||2 a flat, shallow topogra-phy, which corresponds to a wide p(x|b). For 2D problems, this can be visualizedby plotting the contours of the joint probability density p(x|b), which appear similarto those in Figs. 3 and 4. It is often convenient to derive univariate probabilitydensities for each xj by marginalizing out the remaining n�1 variables from the jointprobability

p xjb� � ¼ ð

x1

ðx2

. . .

ðxj�1

ðxjþ1

. . .

ðxn

p xjbð Þdx1dx2 . . . dxj�1dxjþ1 . . . dxn (30)

These univariate distributions can then plotted or summarized by a set of cred-ibility intervals that contain a specified probability, e.g., xj,90% � [a, b]. Using thesetechniques, the impact of the ill-posedness on the recovered variables can be seenexplicitly, as shown in Fig. 7.

Quantifying and visualizing the ill-posedness of an inverse problem in thecontext of probability is an important advantage offered by Bayesian inferenceover deterministic inverse techniques. Arguably, the main advantage of the Bayesianmethodology, however, is that it presents a mathematically robust and explicit way to

20 K.J. Daun

incorporate information into the inference procedure and directly addresses theinformation deficit underlying the ill-posedness of the governing equations. This isdone through the prior, ppr(x), which conditions the likelihood and “steepens” thecontours of the posterior density, p(x|b) / p(b|x)ppr(x). When prior information isconsidered, the most probable value for x is the maximum a posteriori estimate,xMAP = arg maxx[p(b|x)�ppr(x)], but again, a key advantage of Bayesian inferenceis that the joint posterior probability density p(x|b) or its marginalized univariatedensities, p(xj|b), provide a direct indication of the uncertainty in x, which isessential in inverse analysis.

Bayesian priors are sometimes derived from the results of previous experiments,or they can be defined heuristically to promote a known solution attribute. If theelements of x are known to be spatially-smooth, for example, then

ppr xð Þ ¼ ppr, smooth xð Þ / exp � 1

2σ2prLxk k

!(31)

where L is a 2D version of Eq. 10, i.e., a discrete approximation of the ∇ operator,and σpr

2 is a heuristically-defined variance that quantifies the analyst’s “belief” in theprior information. If the measurement model is linear, i.e., Ax = b, and the elementsof b are corrupted with normally-distributed error having a variance of σ2, it can beshown that

0.4

0.3

0.2

0.1

0

5

5

0

0 0 0.05 0.1

1.5

0.15

5

0

0

1

0

05−5

−50.2 0.2 0.4

0.5

p x1

px2 px2

p x1

X2

X1 X1

X2

0.6 0.8−5

−5

Fig. 7 Joint probability densities corresponding to an ill-conditioned measurement equationAx = b, along with marginalized probability densities and 90% credibility intervals. The plot onthe left assumes no prior information, hence log[p(x|b)] = log[p(b|x)]. The plot on the right adds asmoothness prior, ppr(x) = exp.(λ2||Lx||2), which “steepens” the contours of log[p(x|b)] = log[p(b|x)ppr(x)] and narrows the credibility intervals. Note that the prior also shifts xMAP away from xMLE


xMAP ¼ arg maxx p bjxð Þppr xð Þ� ¼ arg minx �ln p bjxð Þppr xð Þ� � ¼ arg minx

A

σ=σprL

� �x� b

0

� ��2 (32)

which is the same as Tikhonov regularization, Eq. 12, where λ = σ/σpr expresses thereliability of the prior information relative to the measurement data.

3.5 Does Least-Squares Minimization Constitute InverseAnalysis?

Many of the examples of inverse analysis of radiative systems recover x by mini-mizing ||a(x) � b||2 using an NLP or a metaheuristic algorithm like the onesdescribed in Sects. 3.2 and 3.3. Does this actually constitute inverse analysis? Recallthat inverse analysis problems are distinct from ordinary inference problems becausethey are mathematically ill-posed, and consequently the art of inverse analysis lies inaugmenting the information contained in the model equations with “prior” informa-tion. If a problem can be solved by minimizing the residual without adding infor-mation, then it is not ill-posed and does not constitute an inverse problem.

The shallow topographies typical of “true” nonlinear inverse problems,cf. Fig. 4, present numerical challenges associated with ill-conditioned or singularHessians. In this context, it has been argued that nonlinear conjugate gradientand Levenberg–Marquardt algorithms constitute inverse techniques, since theyhave features for dealing with an ill-conditioned Hessian or JTJ in the vicinityof x* = arg minx||Ax � b||2: in the case of CG, semiconvergence is ex-ploited by terminating iterations before a local minimum is reached, while inLevenberg–Marquardt, the regularization parameter λ stabilizes the calculation ofpk. In these cases, one may tenuously argue that prior information is introduced,albeit in a very subtle way, since these schemes indirectly promote a small solutionnorm. Rarely, however, do studies establish that these techniques function as inverseanalysis algorithms. If CG iterations can be carried out to convergence and arriveat a sensible value of x, then the underlying problem is not ill-posed; simply usingCG to minimize a least-squares function does not, in itself, constitute inverseanalysis. Likewise, in some cases, Gauss–Newton iterations do not diverge in thelast few iterations because of ill-posedness, but rather diverge immediately becausethe quadratic model that forms the basis for the search direction is invalid far awayfrom x*. In this scenario, LM does not act like an inverse analysis technique.

Often, analysts turn to metaheuristics to avoid the numerical issues associatedwith an ill-conditioned Hessian or JTJ, since these algorithms are “derivative-free.”Like CG and LM iteration, metaheuristics do not directly introduce any additionalinformation to the problem, and do not actually avoid the ill-posedness of the inverseproblem. Consequently, the ill-posedness will manifest as a large set of candidatesolutions identified by the metaheuristic algorithm. Metaheuristics can be useful

22 K.J. Daun

in a design context, since these solutions represent feasible design alternatives. Theyare less useful for parameter estimation, however, since the set of recovered solutionscannot be used to infer statistical properties of x.

This question is further clarified in the context of Bayesian inference. Theill-posedness of the inverse problem is realized by a broad likelihood densityin p(b|x), due to the properties of the model equation, a(x) = b. The priorinformation introduced through ppr(x) alters the topography of the posterior proba-bility density, p(x|b), and narrows the credibility intervals. On the other hand, usinga CG/LM/metaheuristic to find xMLE by maximizing p(b|x) does nothing to addressthe ill-posedness of the problem.

In summary, when solving inverse problems, the analyst must consider the originof the ill-posedness, and what additional information can be imposed to mitigate theill-posedness.

4 Radiant Enclosure Design Problems

As noted in the introduction, inverse problems in radiative transfer can broadly becategorized as either inverse design or parameter estimation problems. This sectionbegins by considering inverse design problems, specifically those involving radiantenclosures; these are appealing because they have a simple physical interpretation ofthe mathematical ill-posedness, as well as important practical applications.

In conventional, or “forward” radiant enclosure analysis, the enclosure geometryis prescribed and one boundary condition, either temperature or heat flux, is imposedon each surface. The goal is then to recover the complementary set of unknownboundary conditions. Such a scenario is shown schematically in Fig. 8a. Under thesecircumstances, the problem can be written as a matrix equation Ax = b, whereb contains the known boundary conditions, the elements of A are derived from viewfactors, and x contains the unknown boundary conditions, or an intermediate vectorof radiosities that can be postprocessed for the unknown boundary conditions.Provided that one boundary condition is specified over each surface, A is alwaysfull-rank (Baranoski et al. 2001) and consequently has a unique and stable solution.This can be conceived as the well-posed forward problem.

In industrial settings, however, the objective is usually to find the enclosureconfiguration that achieves a specified heat flux and temperature distribution overa particular surface, e.g., a product being heat-treated. This surface is called thedesign surface. In problems involving transient heating, this scenario occurs whenboth the temperature and the heating rate are specified over a product. The goal of theinverse design problem is to satisfy the design surface conditions by altering the heatflux or temperature distribution over the other enclosure surfaces, changing theenclosure geometry, or arranging the location of heat sources within a participatingmedium. In these cases, shown schematically in Fig. 8b–d, the governing equationsare mathematically ill-posed, and one of the inverse analysis techniques described inSect. 3 must be employed to solve them.


4.1 Linear Problems

The simplest variation of the radiant enclosure design problems is the boundarycondition design problem, shown in Fig. 8b. In the case of an enclosure of gray-diffuse surfaces enveloping a transparent medium, the radiosity distribution over theenclosure surfaces is governed, in part, by a Fredholm integral equation of the first-kind having a form

J uð Þ ¼ g uð Þðba

J u0ð Þk u, u0ð Þdu0 (33)

where J(u) is the radiosity distribution, g(u) depends on the emissivity of the surface,and k(u,u0) is the view factor between two infinitesimal elements at u and u0 dividedby du0.

This inverse design problem can then be solved in one of two ways (Daun andHowell 2005): In the direct formulation, both the temperature and the heat flux arespecified over the design surface, and an ill-conditioned matrix equation Ax = b is

Fig. 8 Types of radiant enclosure design problems: (a) the forward problem, in which oneboundary condition is specified over each surface and the objective is to infer the remaining surface;(b) the inverse boundary condition design problem, in which both conditions are specified over thedesign surface; (c) the inverse source term problem, in which the goal is to infer the emissive poweror heat source distribution in a participating medium; and (d) the geometry design problem. In eachcase, the goal is to produce the desired conditions over the design surface. In the implicitformulations, one boundary condition is specified over the design surface, while the remainingboundary condition is used to specify an objective function

24 K.J. Daun

formed by discretizing the Fredholm IFK. The unknown boundary condition is thensolved using one of the linear regularization schemes in Sect. 3.1. Harutunian et al.(1995) solved the inverse boundary condition design problem in its direct form,using a modified version of TSVD to regularize the problem (Hansen et al. 1992).Candidate designs were visualized by systematically changing the regularizationparameter, p. Subsequent work by França et al. (2003), Ertürk et al. (2002b), Leducet al. (2004), among others, used TSVD, CG, and Tikhonov regularization to solvethe inverse boundary condition design problem in direct form. (In these examples,CG is used as a linear regularization technique as described in Sect. 3.1, as opposedto an NLP method in Sect. 3.2.) In each of these cases, the regularization parameter isvaried systematically to obtain a set of design alternatives. Ertürk et al. (2008) usedCG regularization to determine the heater settings in an axisymmetric rapid thermalprocessing chamber; the feasibility of this solution was then established on anexperimental test-rig. A major drawback of this approach is that it does not allowfor bound constraints, however, which limits the usefulness of these results.

Under certain circumstances, the inverse source term problem, shown in Fig. 8c,can also be written as a linear inverse problem, Ax = b, where x typically containsthe emissive power of volumetric elements within a participating medium andb specifies the designed boundary conditions over the design surface. Kudo et al.(1996) and França et al. (1999) used the direct formulation to estimate the temper-ature and heat source distribution within a participating medium required to producethe desired distribution over a design surface. Exchange factors between the volumeand surface elements were calculated using Monte Carlo, and the inverse problemwas solved using TSVD.

In the indirect formulation, the inverse design problem is solved as an optimiza-tion problem. Conceptually, all optimization problems consist of two distinct steps:First, the engineering design problem is transformed into minimization problem bydefining an objective function, F(x), which quantifies the “goodness” of a particulardesign, along with a vector of design constraints c(x) � 0 that define the problemdomain. In the context of inverse boundary condition design problems, one bound-ary condition is specified over each enclosure surface, including one of the designsurface conditions (say, the temperature, TDS). The remaining design surface bound-ary condition (say, the heat flux, qDS) is used to define a least-squares objectivefunction

F xð Þ ¼ 1

2f xð Þk k2 ¼ 1

2qDS xð Þ � q

targetDS

�� 2 (34)

The goal, then, is to solve

x ¼ arg minxF xð Þ such that c xð Þ � 0 (35)

where x* specifies the optimal design configuration, e.g., the distribution of theunknown boundary condition over the remaining surfaces. The second step is to


solve the constrained minimization problem using NLP (Sect. 3.2) or with a meta-heuristics algorithm (Sect. 3.3), which involves repeated solution of the forwardproblem. The choice of minimization algorithm depends on: (i) the number and typeof variables, (ii) the number and type of constraints, (iii) the topography of F(x) (e.g.,smooth and convex), and (iv) the computational effort required to evaluate F(x).

While the indirect formulation does not require solving the ill-conditionedmatrix equation directly, this does not, in itself, avoid the ill-posedness of theunderlying problem. While the ill-posedness manifests through the ill-conditionedA matrix in the direct formulation, and is addressed through regularization, in thecase of the indirect formulation, it is reflected by the shallow topography of F(x), andis often reduced by limiting the degrees of freedom involved in the optimizationproblem. In many cases, this must be done for ease of deployment, e.g., use a limitednumber of isothermal panel heaters as opposed to one heater surface over which thetemperature varies continuously. By specifying constraints, the designer introducesinformation into the inverse problem, which addresses the information deficit thatcauses the ill-posedness. In many cases, however, the problem remains sufficientlyill-posed such that a multitude of solutions may exist that minimize or “almost”minimize F(x).

Fedorov et al. (1998) solved for the optimal panel heater temperatures needed toproduce a desired temperature and heat flux distribution over a design surfacemoving on a conveyor through an industrial oven. The problem was solved in steadystate, using an advection term to account for the change in sensible energy as the loadmoves through the furnace. The least-squares objective function was minimizedusing the Levenberg–Marquardt method with bound constraints on the heater tem-peratures. Daun et al. (2003a, 2005) solved an inverse boundary condition designproblem using both the direct (TSVD, Tikhonov) and indirect (Newton’s methodwith a nonnegativity constraint) formulations, and highlighted the relationshipbetween these two approaches. In this class of problem, the gradient vector andHessian matrix can be readily calculated through direct differentiation of thegoverning equations. Specifically, if x contains the “design parameters” (e.g., theheat flux distribution over the heater surfaces), it is possible to compute qDS = A�1xbecause A is well-conditioned. The first- and second-order sensitivities needed tocalculate the gradient and Hessian can then be found by direct differentiation,qDS0 = A�1x0, etc. The solutions obtained through optimization were generallyconsidered superior since it was possible to apply bound constraints.

Metaheuristic schemes have also been applied to solve the inverse boundarycondition design problem, including simulated annealing (Porter et al. 2006), tabusearch (Porter et al. 2006), genetic algorithms (Safavinejad et al. 2009; Chopadeet al. 2012), and particle swarm optimization (Lee and Kim 2015). A key advantageof metaheuristics is that they allow visualization of different candidate solutions thatall satisfy conditions on the design surface. A drawback, however, is that they aremore computationally-intensive compared to NLP, because metaheuristics do notuse information about the local objective function curvature to generate a new designconfiguration. Algorithm performance also depends on the choice of heuristicparameters, which is mainly based on the analyst’s experience and trial and error.

26 K.J. Daun

Most importantly, many of these studies fail to consider that the inverse boundarycondition estimation problem is linear, and, in the absence of bound constraints, F(x)is convex. Consequently, it may be questionable whether metaheuristics is really the“right tool for the job” for many instances of the inverse boundary condition designproblem.

4.2 Nonlinear Problems

In many scenarios, it is impossible to reduce the problem into linear form, Ax = b,e.g., radiant enclosure problems involving transient or multimode heat transfer. Insome cases, it is possible to linearize the equations and solve a sequence of linearinverse problems Ax = b(x) in the direct formulation. França et al. (2001) used thisapproach to optimize the heater settings in an inverse boundary condition designproblem for an enclosure containing a conducting and participating medium. Theheater settings were found by repeatedly solving Axk = b(xk�1) using TSVD.Likewise, Ertürk et al. (2002a) used a similar scheme to solve for the transient heatersettings of a batch furnace needed to uniformly heat a design surface according to aspecified temperature history. At each time step, a linear inverse problem wasdefined by specifying the design surface temperature as well as the heat flux requiredto increase the temperature at the prescribed rate. The heater settings at each timestep were then found through CG regularization. More recently, Mossi et al. (2008)used iterative TSVD regularization to solve the inverse boundary condition designproblem for a radiant enclosure containing a turbulent, nonparticipating medium.

Daun et al. (2006b) carried out a comparative study in which the goal was also todetermine the transient heater settings in a batch furnace. In addition to CG regular-ization, this study also considered Tikhonov and TSVD regularization, as well assimulated annealing in the indirect formulation, i.e., find the optimal heater settingsat each time step by minimizing Eq. 34 subject to a nonnegativity constraint, whereqDS

target is the net radiant heat flux needed to increase the sensible energy of thedesign surface. These schemes were compared to an “all at once”minimization of anobjective function defined as the residual between the desired temperature historyand the one realized using particular heater settings. In order to obtain temporally-smooth heater inputs, these were parameterized as B-splines with the design param-eters as control points, and minimization was carried out using the quasi-Newtonmethod. This reduced-order parameterization constitutes prior information (i.e., theheater settings should change smoothly with time) which reduces the ill-posednessof the inverse problem.

The literature is replete with studies that use the indirect formulation to solvenonlinear radiant enclosure design problems. Hosseini Sarvari et al. (2003), forexample, solved an inverse boundary condition problem for an enclosure containinga conducting and radiating medium. In this work, the heat flux distribution over theheater surface was found by minimizing Eq. 34 using the conjugate gradient method,with sensitivities obtained through direct differentiation of the radiative transferequation. Kim and Baek (2007) solved a similar boundary condition design problem


using the Levenberg–Marquardt method, while Kowsary and Pooladvand (2007)compared the CG and quasi-Newton methods for solving this problem. As discussedin Sect. 3.2, it is important to distinguish between CG as a linear regularizationscheme and as an NLP scheme for well-posed optimization problems; in the lattercase, the CG iterations converge to x* = arg minx[F(x)].

It is also important to note that since NLP algorithms do not influence thetopography of F(x), the choice of solution scheme should not influence the solution.While the NLP algorithms take different paths, they all arrive at the same destination,x*, as shown in Fig. 4. Unfortunately, many authors, reviewers, and journal editorsfail to appreciate this important point, and consequently a large number of studieshave been published that are simply permutations of different NLP algorithms andRTE solution schemes. (Due to the shallow topography of F(x), the solution may bestrongly sensitive to the stopping criterion, typically ||∇F(x*)|| < ε, or other numer-ical artifacts that may be introduced in the solution procedure, but these are distinctfrom the NLP method.) In contrast, Kowsary and Pooladvand (2007) made a hybridobjective function by adding a first-order Tikhonov functional, equivalent toλ2||Lx||2, to the objective function defined in Eq. 34 to reduce the oscillations inthe recovered heat flux distributions over the heater surfaces. In this case, informa-tion is being added (an oscillatory solution is undesired) through the Tikhonovfunctional, which alters the topography of F(x). The solution x* = arg minx F(x)should not depend on the minimization algorithm, since the minimization algorithmdoes not introduce any information into the analysis.

This observation can also be extended to metaheuristics schemes used to solveenclosure design problems (e.g., Kim and Baek (2004) and Pourshaghaghy et al.(2006)), but with a caveat. Conceptually and ideally, the metaheuristics algorithmshould identify a single global minimizer of Eq. 35. An advantage of metaheuristicsis that they also often identify a set of alternate solutions that are nearly optimal,however, which is useful for nonconvex objective functions.

Enclosure geometric optimization problems, shown schematically in Fig. 8d, arealso nonlinear. In the case of diffuse-walled enclosures, the radiosity problemreduces to a nonlinear IFK,

J uð Þ ¼ g uð Þðba

J u0ð Þk u, u0, xð Þdu0 (36)

where x specifies the enclosure geometry. Because the problem is nonlinear, it isalways solved in the indirect formulation. Daun et al. (2003c), Hosseini Sarvari(2007), and Farahmand et al. (2012) parameterized the enclosure geometry usingB-splines, and optimized the location of the control points that produced a uniformirradiation of the design surface. Daun et al. (2003c) used Newton’s method withobjective function sensitivities calculated from an analytical differentiation of thekernel, k(u, u0, x), while Hosseini Sarvari (2007) and Farahmand et al. (2012) both

28 K.J. Daun

used metaheuristics algorithms, which are derivative-free. In these examples, theinverse problem is regularized by the choice of a low-order parameterization, whichsteepens the topography of F(x) around x*. In contrast, Tan and Liu (2009) used ameshless method to represent the geometry involving a large number of degrees offreedom. They then augmented a least-squares objective function, Eq. 34, with azeroth-order Tikhonov-type functional that promoted a small solution norm, andminimized the composite objective function using CG.

While the above examples focus on diffuse-walled enclosures, practical engi-neering problems often feature partially- or fully-specular surfaces, since the geom-etry of these surfaces have a much stronger influence on the radiation within theenclosure compared to diffuse surfaces. While geometric optics treatments (Winston1991) admit analytical solutions in a limited number of cases, e.g., various conicsections, more generally the enclosure problem must be solved using a Monte Carlo(MC) type ray-tracing algorithm. Monte Carlo techniques are extremely flexible:they can solve complicated problems, such as enclosures that contain obstructions aswell as spectrally- and directionally-dependent surface properties, with relative easecompared to deterministic schemes. In the context of optimization, there are twocomplicating factors: (i) analytical solutions to the objective function sensitivities arenot available; (ii) the MC-derived objective function is a statistical estimate of thedeterministic (but unknown) objective function, i.e., F xð Þ ¼ ~F xð Þ þ er, where er is arandom error. Consequently, Daun et al. (2003b) employed a stochastic program-ming technique called the Kiefer–Wolfowitz algorithm (Kiefer and Wolfowitz1952), a type of steepest descent that balances the truncation error in the forward-difference estimate of the gradient, which decreases with the finite differenceinterval, with the stochastic error caused by er, which increases with interval size.For computational expediency, the sample variance in ~F xð Þ is initially large, andgradually drops as xk approaches x* by increasing the number of samples (photonbundles) according to a power law. Daun et al. (2003b) demonstrated this schemeon the 2D enclosure problem shown in Fig. 9, as well as a 2D imaging furnacecontaining a faceted reflector surface. Subsequent work by Marston et al. (2012)used a quasi-Monte Carlo scheme to reduce the variance in the objective functionestimate, thereby requiring fewer bundles and greatly accelerating convergence time.Rukolaine (2015) presents a semianalytical approach for optimizing the geometryof enclosures containing specular and diffuse-specular surfaces. Sensitivities arecalculated using adjoint analysis, and optimization is carried out with conjugategradient minimization.

Nanotechnology represents an emerging new frontier for the inverse geometrydesign problem. Hajimirza et al. (2012), for example, optimized the geometry ofperiodic nanostructured amorphous silicon solar cells textured with rectangularmetallic nano-patterns. This was done by finding the geometric parameters (a-Silayer thickness, metal nano-pattern layout) that maximized the solar absorptionenhancement factor, calculated through a finite difference time domain (FDTD)solution of Maxwell’s equations. They compared the performance of nonlinearprogramming (quasi-Newton) and metaheuristics (simulated annealing, tabu search).Under certain conditions, the objective function is nonconvex, and consequently the


metahuristics schemes provide a superior solution compared to the quasi-Newtonalgorithm, which becomes trapped in a local minimum.

4.3 Case Study: Inverse Design

To demonstrate the explicit and implicit solution methodologies, consider theinverse boundary condition design problem shown in Fig. 10, inspired by therapid thermal processing chamber testbed examined by Ertürk et al. (2008). Theobjective is to identify the heater setting configuration that produces the desiredtemperature and heat flux over the design surface. (The problem can be scaled so thatall quantities are dimensionless.) All enclosure surfaces are black, and all surfacesother than the design surface and heater surfaces are cold. With these simplifications,the problem reduces to identifying the emissive power distribution over the heatersurface, Eb(r2), that produces the required (uniform) irradiation over each location ofthe design surface, G1 = 1. These quantities are related by the Fredholm IFK

G1 ¼ðR2

0

Eb r2� �

k r1, r2ð Þdr2 (37)

The kernel is defined by

Fig. 9 Geometric optimization of a radiant enclosure, Fig. 8d, using Monte Carlo ray-tracing andKiefer–Wolfowitz optimization (Adapted from Daun et al. (2003b))

30 K.J. Daun

k r1, r2ð Þ ¼ dFdr1�dr2

dr2¼

2r2r21

L

r1

� �2 L

r1

� �2

þ r2r1

� �2

þ 1

" #

Lr1

� �2þ r2

r1

� �2þ 1

� �2� 4 r2

r1

� �2( )3=2(38)

where dFdr1�dr2 is the view factor between an infinitesimal ring element on thedesign surface and an infinitesimal ring element on the heater surface.

In the explicit methodology, the IFK is transformed into a matrix equationAx= bby discretizing the domains of r1 and r2, and writing

bi ¼ G1 �Xnj¼1

Eb r2j� � ðr2jþΔr2=2

r2j�Δr2=2

k ri, rð Þdr ¼

Xnj¼1

xjAij (39)

for every discrete r1 value. In this example, it is assumed that both surfaces arediscretized into 50 elements. As one would expect, A is ill-conditioned, and Fig. 11shows that the singular values of A decay continuously over several orders ofmagnitude. Mathematically, the ill-conditioning is due to the smoothing or blendingaction of k(r1, r2) during integration, as discussed in Sect. 2.2. Physically, thissmoothing is due to the fact that any given location over the design surface isirradiated by the entire heater surface, so emission from one ring element is blendedwith emission from all the other elements. Consequently, the irradiation at a givenlocation on the design surface is insensitive to small variations in the radiosity overthe heater surface.

Since the irradiation of the design surface is a linear function of the radiosity/emissive power distribution over the heater surface, Eb(r2) can be solved using anyof the linear regularization techniques described in Sect. 3.1. Figure 12 shows the

Fig. 10 Example inverse boundary condition design problem, inspired by a rapid thermal pro-cessing chamber


results obtained from truncated singular value decomposition, with various values ofregularization parameter p. (The corresponding singular values are indicated inFig. 11, and the solutions are constructed using the summation terms up to andincluding k = n � p.) Solutions obtained using very few summation terms arespatially-smooth, as shown in Fig. 12a, but do not produce the desired irradiationover the design surface. The design surface irradiation approaches the desiredcondition as regularization is decreased and more summation terms are retained,but the solution becomes increasingly oscillatory, and would be difficult to deploy onan actual furnace. It should be noted that TSVD and other regularization techniquesonly produce a subset of the candidate design solutions that satisfy Ax = b within areasonable tolerance. The domain of candidate solutions is contained within ahyperellipsoid centered on xexact analogous to Fig. 3a for the 2D case.

In the implicit formulation, a candidate x is specified that represents Eb(r2) overthe heater surface, and the resulting irradiation over the design surface is used todefine an objective function

F xð Þ ¼ 1

2G

target1 �G1 xð Þ�� 2

2(40)

In the simplest scheme, each element of x corresponds to the emissive power ofan annular ring on the heater surface, as was the case in the explicit formulation, soG1(x) = Ax and minimizing F(x) is the same as solving the linear least-squaresminimization problem x* = arg minx(||Ax � b||2

2). As discussed in Sect. 3.2;however, this objective function is problematic to minimize because JTJ = ATA isill-conditioned. While one could adopt Levenberg–Marquardt technique to carryout the minimization, a major disadvantage of this approach is that there is no clearlink between regularizing the search direction in Eq. 25 and promoting desiredsolution attributes. A better approach is to add a regularizing function to Eq. 40,following Kowsary and Pooladvand (2007).

Fig. 11 The singular valuesof the A matrix decaycontinuously over severalorders of magnitude, due tothe smoothing action of thekernel in Eq. 38. Labeledsingular values correspond tothe solutions shown in Fig. 12

32 K.J. Daun

Alternatively, one could reduce the degrees of freedom by discretizing the heatersurface into four isothermal rings, so that

bi ¼ Gi ¼X4j¼1

Aijxj, Aij ¼ Fdi�j ¼ðr2j,max

r2j,min

k ri, rð Þdr (41)

where [r2j,min, r2j,max] are the minimum and maximum radii of the jth heatingelement, and the (m � 4) matrix equation is overdetermined. This reduces theill-posedness of this inverse design problem considerably, and also makes thesolution easier to implement from a design perspective. The unconstrained minimi-zation problem can then be solved by any NLP algorithm, or equivalently throughthe normal equations, x* = (ATA)�1ATb. Figure 13a shows the optimal heatersettings, while the corresponding design surface irradiation is plotted in Fig. 13c.While the overdetermined problem has a unique solution, the problem remainsill-posed since Cond(∇2F(x*)) = Cond(ATA) = O (104); the last two singularvalues are very small, and consequently, a family of candidate solutions could be

Fig. 12 Emissive power distribution over the heater surface, and corresponding design surfaceirradiation, found using truncated singular value decomposition. As the regularization parameter pincreases, the emissive power distribution becomes smoother, but the residual between the desiredand realized design surface irradiation grows. (Singular values corresponding to each solution areshown in Fig. 11)


constructed by carrying out an SVD on ATA and using the last two columns vectorsof V as an orthonormal basis, following Eq. 6.

A key advantage of the implicit formulation is that it is possible to imposeconstraints on the design variables, which ensure that the optimal solution can beimplemented in an industrial setting. Accordingly, Eq. 40 is minimized with non-negativity constraints on the design parameters, using a trust-region reflectivealgorithm (Moré and Sorensen 1983). The optimal heater settings are plotted inFig. 13b, while Fig. 13c shows that the irradiation over the design surface is onlyslightly further away from the target irradiation compared to the unconstrained case.

It is important to keep in mind, however, that if one were to implement thissolution in an industrial setting, the design surface irradiation would bedifferent from the target distribution due to the assumptions made to facilitate theenclosure analysis (i.e., diffuse-gray surfaces). In reality, the simplifying assump-tions needed to facilitate analysis coupled with the inherent uncertainty in theradiative properties of surfaces and participating media will degrade the quality ofthe optimized solution (Erturk et al. 2008; Amiri et al. 2013).

5 Parameter Estimation Problems

While the objective in inverse radiant enclosure design problems is to infer theenclosure configuration that produces the desired conditions over the design surface,in parameter estimation problems the goal is to infer some property or attribute from

Fig. 13 Solution of the inverse boundary condition design problem through (a) unconstrained and(b) constrained minimization of the objective function defined by Eq. 40. The corresponding designsurface irradiation is shown in (c) for both cases. The irradiation realized through the unconstrainedoptimization is slightly closer to the target irradiation compared to the constrained optimization

34 K.J. Daun

indirect experimental measurements. Like inverse design problems, parameter esti-mation problems can be categorized as either being linear or nonlinear.

5.1 Linear Problems

Arguably, the most prominent linear parameter estimation problem in thermalradiation is tomography with negligible scattering. Under these circumstances, theradiative transfer equation simplifies to

IL, λ ¼ I0, λexp

ðL0

�κλ uð Þdu24

35, (42)

where I0,λ and IL,λ are the incident and exit intensities, and κλ(u) is the spectralabsorption coefficient at a location u along the optical path, which scales with thelocal species concentration of interest. Equation 42 can be rewritten as

bi ¼ ln I0, λ=IL, λ� � ¼ ðL

0

κλ uð Þdu ¼Xnj¼1

Aijxj (43)

The objective of axisymmetric deconvolution is to infer the radial distribution ofκλ(r), from projected measurements, b(y), made through the tomography domain. Aclassic example is the inference of soot volume fraction concentration within anaxisymmetric laminar diffusion flame through line-of-sight-attenuation measure-ments made with a laser or some other collimated light source. It can be shownthat κλ(r) and b( y) are related through a Volterra integral equation of the first kindcalled Abel’s integral equation,

b yð Þ ¼ 2

ðRy

κλ rð Þrffiffiffiffiffiffiffiffiffiffiffiffiffiffir2 � y2

p dr (44)

Unlike most IFKs, Abel’s equation has an analytical solution,

κλ rð Þ ¼ 1

π

ðRy

db=dyffiffiffiffiffiffiffiffiffiffiffiffiffiffiy2 � r2

p dy (45)


but it cannot be used to analyze experimental data because db/dy is not available, andfinite difference approximations of the derivative amplify the measurement noise inb( y) to an unacceptable level. Instead, a more common route is to discretize theproblem domain into annular elements, as shown in Fig. 14a, and approximate theintegral in Eq. 44 as a summation assuming that κλ(r) is uniform over each annularelement. Writing this equation for each projection produces a lower-triangular matrixequation, Ax= b, which is ill-conditioned due to the smoothing action of the kernel.Consequently, solving for x by back substitution, a process called “onion peeling,”amplifies small amounts of measurement noise in b = bexact + δb into largeperturbations in the recovered field variable, x = xexact + δx. The condition numberof A increases with the spatial resolution of the measurement data, which exacer-bates the error amplification. One approach is to use Abel three-point inversion(Dasch 1992), which assumes that the projected data is locally cubic at eachmeasurement point. Hall and Bonczyk (1990) inferred both the soot volume fractionand soot temperature from a luminous laminar diffusion flame with negligiblescattering by solving a sequence of linear equations AIxI = bI, and AII(xI)xII = bII.In the first measurement, bI is obtained by measuring the intensity with and withoutback illumination, AI is the onion peeling matrix, and xI represents the soot volumefraction. A second matrix equation is derived that relates the emission measurementto the local emissive power of each annulus, and the elements of AII are definedusing the soot volume fraction of xI. Both deconvolutions are carried out using

Fig. 14 1D (axisymmetric) tomography problem, in which the goal is to infer a field variable fromintensity measurements made through the tomography field. (a) The governing Volterra IFK isdiscretized by dividing the tomography field into annular elements. (b) The resulting lower-triangular matrix equation, Ax = b, is ill-conditioned, so small perturbations in the measurementdata are amplified into large variations in the recovered field variable. (x, b = solid lines, xexact,bexact = dashed lines)

36 K.J. Daun

a Fourier-transform based algorithm that suppresses the high-frequency solutioncomponents associated with measurement noise; the fact that these high-frequencycomponents should be absent from the solution constitutes prior information.Mengüç and Dutta (1994) also used a Fourier-transform based algorithm to recon-struct the radial distribution of the extinction coefficient in an axisymmetric flame;while the majority of CST experiments presume negligible scattering, this techniqueinvolves inverting an integral equation that relates the scattering coefficient to theangular distribution of scattered light measured at the periphery of the domain.Again, the prior knowledge is that the scattering coefficient distribution is spa-tially-smooth.

First-order Tikhonov regularization is effective at stabilizing the deconvolu-tion (Daun et al. 2006a; Åkesson and Daun 2008); the measurement informationcontained in Ax = b is augmented with prior information that the radial distributionof the field variable, κλ(r), is smooth due to spatial diffusion. Tikhonov regularizationhas been widely adopted by other combustion researchers, and is particularlyeffective in scenarios in which the projected data has high spatial-refinement (Kashifet al. 2012).

The above technique has been extended to tomography problems in two- andthree-dimensions. Most often, the tomography field is discretized into pixels orvoxels, and the unknown concentration is assumed to be uniform within eachvoxel. If the axial and angular spacing of projection measurements is uniform anddense, the 2D- or 3D-tomography problem has similar mathematical properties asthe 1D axisymmetric problem. For example, Floyd and Kempf (2011) obtained a 3Dreconstruction of CH* concentration within a matrix burner based on 2D chemilu-minescence images using the algebraic reconstruction technique (ART), an iterativelinear regularization scheme that stabilizes the inverse problem by suppressing ortruncating high-frequency solution components.

More often, however, the number of voxels, n, far exceeds the number of opticalpaths,m, due to the cost and complexity of the apparatus or the limited optical accessafforded by the tomography field. For example, Wright et al. (2010) reconstructedthe fuel concentration within a 2D cross section of an internal combustion enginecylinder: the cross section was subdivided into 1844 pixels, and was transected by32 fiber-based diode lasers. In the rank-deficient scenario, the species concentrationis governed by a matrix equation Ax = b, where A�ℜ(m�n), and Rank(A) � m dueto the rank nullity theorem. In this case, the solution will have a nontrivial nullspacecomponent as described in Eq. 6. Figure 15 shows an example problem in which thenullspace component is inferred from the exact solution, xn = x � xr; in practice,however, constructing xn from the orthogonal basis formed by the last n-m columnvectors of V depends wholly on specifying specific prior information. More thor-ough discussions of the mathematical aspects of rank-deficient chemical speciestomography are provided by McCann et al. (2015) and Daun et al. (2016).


5.2 Nonlinear Problems

More often the equations that relate the measurements and the unknown parametersare nonlinear, which is usually the case when extinction, emission, and/orin-scattering terms are important in the radiative transfer equation. Under theseconditions, the unknown parameters, x, are recovered by minimizing a least-squaresfunction

F xð Þ ¼ 1

2bmeas � bmod xð Þ�� 2 (46)

where bmeas and bmod are vectors of the measured and modeled data, respectively.Ill-posedness manifests in the shallow topography of the objective function sur-rounding x*, which means that a large number of solutions exist that can explain themeasurement data in the context of measurement error, cf. Figs. 3a and 4a. Equiv-alently, the shallow curvature sensitizes the location of x* to small amount ofmeasurement noise that contaminates bmeas.

Many studies have been reported in the literature in which the objective is to inferthe properties of a participating medium from intensity measurements made at theperiphery of the medium, or temperature measurements made at locations withinthe medium, via the radiative transfer equation. A subset of these studies is describedbelow, to give the reader an overview of the research carried out in this area. Themajority of these studies consider simulated experiments in which x is specified, themodel equations are used to generate measurement data which is then contaminatedwith noise, b= bmod(x) + δb, (forward problem), and a least-squares minimization iscarried out to recover the parameters (inverse problem). A smaller number of studiesuse these simulated experiments to prototype true experiments; this is an important

Fig. 15 The solution to a rank-deficient chemical species tomography problem has componentsbelonging to the rank and nullspace of A. The rank component can be found from the measurementequations, but the nullspace component relies entirely on prior information specified by the analyst.In this case, the set of {Cj} is inferred from knowledge of xexact, but any set of {Cj} would satisfyAx = b

38 K.J. Daun

distinction, since very few studies consider the propagation of model error during theinversion procedure, which can be considerable in view of the standard simplifyingassumptions made when solving the radiative transfer equations as well as theuncertainty in ancillary model parameters that are not the focus of the inference.

The objective of many experiments is to infer the extinction coefficient, scatteringalbedo, and scattering phase function parameters of a participating media based on exitintensities. McCormick (1992) and Özşik and Orlande (2000) summarize the earlywork in this area. Ho and Özşik (1988) recovered the optical thickness and scatteringalbedo within an enclosure from measured exit intensities at various angles; this wasdone by minimizing Eq. 46 using the Levenberg–Marquardt algorithm, cf. Sect. 3.2.They also calculated confidence intervals; if the data in b are contaminated withindependent and identically-distributed measurement noise having a variance of σ2,so that Cov(b) = σ2I, it can be shown that Cov(x) = σ2[J(x*)TJ(x*)]�1 (Aster et al.2013), cf. Fig. 3a. Silva Neto and Özşik (1995) extended this treatment to recoverphase function parameters, assuming a modified Henyey–Greenstein phase function.As mentioned in Sect. 2.5, LM minimization presents an expedient way to find x*, butit does not affect J(x*)TJ(x*) and thus does not alter the covariance estimate. Thishighlights the fact that LM is not a true inverse method, since it does not supplementthe information in the measurement equations with additional information.

Hendricks and Howell (1996) inferred the absorption, scattering, and phasefunction parameters reticulated porous ceramics based on spectral hemisphericaltransmittance and reflectance measurements; least-squares minimization was carriedout using the Levenberg–Marquardt method. They compared the suitability of twocandidate dual-parameter phase functions based on the relative magnitudes of theirresidual norms.

Randrianalisoa et al. (2006) recovered the extinction coefficient, the scatteringalbedo, and the scattering phase function of fused quartz containing closed cells frombidirectional transmissivities measured over a range of exit directions. Least-squaresminimization was carried out using Gauss–Newton method, cf. Sect. 3.2. Numericaldifficulties caused by the ill-conditioning of JTJ were surmounted by computing aninitial guess for the extinction measurement from transmittances measured in thenormal direction. This reduced the number of degrees of freedoms in the multi-parameter least-squares minimization, and thus improved the conditioning of JTJ.

Deiveegan et al. (2006) compared LM, Bayesian inference, genetic algorithms,and an artificial neural network (ANN) to recover the absorption and scatteringcoefficients of participating medium contained between infinite parallel plate, as wellas the surface emissivities of the bounding plates based on temperatures and heatfluxes measured at the boundary. Their results highlight a major advantage ofthe Bayesian approach, in that it quantifies the uncertainties in the recoveredparameters, which is rarely mentioned in other studies. They also found the meta-heuristics (GA and ANN) schemes to be more robust and less sensitive to conver-gence problems associated with the difficult function topography. Lee et al. (2008)compared the performance of particle swarm optimization and repulsive particleswarm optimization on a similar problem.


Ren et al. (2015) retrieved temperature and concentration of a homogeneous gaslayer at high temperature from spectral transmittance measurements made witha Fourier transform infrared (FTIR) spectrometer. The modeled data in Eq. 46,bmod(x),was generated using a spectral line database and then convolved by theFTIR instrument response data; the temperature and concentration were thenretrieved through LM minimization. The recovered parameters were within 5% ofindependently-estimated values in most cases, although different temperatures andconcentrations were recovered using different spectral databases even though thesesolutions reproduced the measurement data with a small residual. This highlights theimportance of considering the impact of model error (i.e., some of the high-temperature spectral line databases are thought to be incorrect), which may besmall in magnitude but have a significant impact on the recovered parameters dueto the ill-conditioning of the underlying problem. In a subsequent study, Ren andModest (2016) used LM regularization to determine the average concentrationand temperature profile across a planar layer of CO2 using spectral emittancemeasurements. In order to infer these parameters, the least-squares function had tobe augmented with a first-order Tikhonov regularization functional, which promotesa spatially-smooth temperature profile. This is an example of how supplementingthe measurement data with additional prior information alters the topography of theobjective function (i.e., reduces ||[J(x*)TJ(x*)]�1||) and consequently reduces thecovariance in the recovered parameters.

In the context of nanoscale radiative heat transfer, Charnigo et al. (2012) pre-sented a technique for inferring the size and agglomeration of nanoparticles depos-ited on a metallic film. The approach is based on generating polarations with theair/metal interface by evanescent waves, which are then scattered by the nano-particles. Bayesian inference is used to derive marginalized probability densitiesand credibility intervals. They demonstrated this technique using numerically-simulated scattering data generated with the T-matrix method.

Diffuse optical tomography is a particularly important subset of parameter esti-mation problems. In these experiments, visible or near infrared light sources (usuallydiode lasers) emit collimated light into a highly scattering medium, and the objectiveis to infer the spatial distribution of the scattering and absorption coefficients basedon the response of sensors located around the periphery of the tomography domain.A classic example is the detection of cancerous tumors, which have absorption andscattering coefficients different from healthy tissue. Recent work in this area hasbeen summarized by Haisch (2012) and Charette et al. (2008). This type of tomog-raphy is distinct from the linear tomography problems discussed in Sect. 5.1 due tothe predominance of scattering, which obfuscates the relationship between thetransmitters and receivers. This is an example of “soft field” tomography, as opposedto “hard field” tomography in which there is an unambiguous relationship betweenpairs of transmitters and receivers. The scattering between the transmitter andreceiver acts as a blending function in the same way as the kernel in a linear IFK;consequently, the nonlinear measurement equations are ill-conditioned and requireregularization. Nonlinear conjugate gradient minimization, cf. Sect. 3.2, is particu-larly suitable, since it is computationally-efficient and exploits the semiconvergence

40 K.J. Daun

property of ill-posed problems. The first few steps fill in the low-frequency solutioncomponents, while iterations are terminated before x* is reached. This limits themaximum obtainable spatial resolution (i.e., the “sharpness” of the reconstruction),but it also suppresses measurement noise. Diffuse optical tomography is sometimescarried out in the frequency domain, using pulsed light sources. The model equationis derived from the time-dependent radiative transfer equation, which accountsfor the finite rate of photon propagation through the media. The additional informa-tion contained in the time-resolved measurements reduces the ill-posedness of theunderlying inverse problem.

5.3 Case Study: Parameter Estimation

To better understand the issues involved in inverse parameter estimation, considerthe problem of inferring the morphology of aerosolized soot aggregates frommultiangle elastic light scattering data. Soot consists of nanospheres called primaryparticles, which assemble into fractal-like aggregates through primary particle col-lisions within the aerosol. A sample transmission electron micrograph of a sootaggregate is shown in Fig. 16a. The aggregate structure is often described by thefractal relationship

Np ¼ kf 2Rg=dp� �Df (47)

where Rg is the radius of gyration (loosely, the “size”) of the soot aggregate, Np is thenumber of primary particles per aggregate, dp is the primary particle diameter, and kfand Df are fractal parameters.

One way to characterize the size distribution of aerosolized soot aggregates isthough multiangle elastic light scattering (MAELS), in which a collimated lightsource (usually a laser) is shone through a soot-laden aerosol, and the scattered lightis measured at various scattering angles, often using a goniometer as shown inFig. 16b. It can be shown that the scattering phase function of soot aggregates is aunique function of Rg, Df, and kf (Sorensen 2001). In the hypothetical case where anaerosol consists of a single size class of soot aggregates, these parameters can beinferred unambiguously from angular scattering data. More often, however, a soot-laden aerosol contains aggregates that obey a size distribution p(Rg), or equivalentlyp(Np), and the observed light scattering at angle θ are given by

b θð Þ ¼ C

ð10

k θ,Rg

� �p Rg

� �dRg (48)

where C is a coefficient that depends on the particle volume fraction, opticalcollection efficiency, excitation beam intensity, among other parameters, and thekernel k(θ, Rg) is derived from light-scattering theory. The kernel is also a function ofother “nuisance” parameters like kf, Df, and dp, which are often presumed known


and/or may not be the focus of the inference procedure. The smoothing action of thekernel makes inverting Eq. 48 extremely ill-posed, because the scattering crosssection of soot aggregates increases geometrically with respect to Rg. Consequently,the scattering contributions from many smaller aggregates are overwhelmed by thosefrom a few large aggregates, and conversely, the observed angular scattering data isinsensitive to variations of p(Rg) for small Rg.

If p(Rg) is the unknown and C and k(θ,Rg) are considered known, Eq. 48 isa linear Fredholm IFK. Writing this equation for m multiple discrete measure-ment angles, and expressing the continuous function p(θ) as a discrete linearparameterization,

p Rg

� � �Xnj¼1

xjβj Rg

� �(49)

transforms Eq. 48 into a matrix equation, Ax = b, where A is ill-conditioned. Acommon choice is to discretize Rg into n uniformly-spaced segments between 0 andsome maximum Rg and then use basis functions βj(Rg) that are delta functions equalto unity over a range [Rg,j,min, Rg,j,max] and are otherwise zero. This is equivalent torepresenting p(Rg) using strips, as shown in Fig. 17. When written this way, x can berecovered using classical linear regularization schemes such as first-order Tikhonov,which presume a degree of smoothness among the elements of x (Burr et al. 2011;di Stasio et al. 2006).

There are two important drawbacks with this approach: First, linear regularizationby itself only provides a single candidate distribution, and ignores the existence ofother candidate distributions, which may be very different from each other. Second,because the even with regularization, the inverse problem remains extremelyill-posed (Burr et al. 2011).

One way to reduce the ill-posedness of the problem is to use a lower-orderparametrization of p(Rg). For example, the competing agglomeration and fracturing

Fig. 16 (a) A TEM micrograph shows a soot aggregate, consisting of Np nanospheres calledprimary particles arranged in a fractal-like pattern. (b) A schematic of a multiangle elastic lightscattering experiment used to infer the size distribution of aggregates in a soot-laden aerosol

42 K.J. Daun

processes in diffusion-limited aerosol transport often lead to a self-preserving sizedistribution (Sorensen 2001), which can be modeled as a lognormal distribution

p Rg, x� � ¼ 1

Rg

ffiffiffiffiffi2π

plogσg

exp � log Rg

� �� log μg� �� 2

2 logσg� �2

" #, x ¼ μg, σg

� T(50)

where the distribution parameters μg and σg are inferred in the inverse problem. Thisconstitutes incorporation of prior information, i.e., that the size distribution shouldbe lognormal, in order to mitigate the ill-posedness of the inverse problem. Thisparameterization transforms the linear IFK, Eq. 48, into a nonlinear IFK having thegeneral form of Eq. 7 since b(θ) is a nonlinear function of the unknowns, μg and σg.

Huber et al. (2016) used how Bayesian analysis to estimate posterior probabilitydensities for μg and σg from highly-resolved angular scattering data. They firstderived the likelihood function using a detailed analysis of the measurement noise,

p bjxð Þ ¼ 1

2πð Þm=2 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiDet Γbð Þp exp � b� a xð Þ½ �TΓ�1

b b� a xð Þ½ �2

" #(51)

where Γb is the measurement noise covariance matrix and a(x) is found by carryingout the integration in Eq. 48 for a lognormal p(Rg, x) with x = [μg, σg]

T. Measure-ment noise consisted of photonic shot noise and electronic noise, which affect eachmeasurement angle independently, and process noise due to variations in laserfluence and particle concentrations between measurements, which contributes toa nontrivial covariance between the measurement angles. An additional unknowncalibration error is masked with 2% white noise, with the understanding that thisprovides a conservative estimate of the credibility bounds.

Fig. 17 The Fredholm IFK deconvolution problem can be rewritten in a matrix equation byparameterizing p(Rg) using strips. The resulting A matrix is ill-conditioned, and has singular valuesthat resemble those in Fig. 2


An “artificial” 2D problem is generated by treating the nuisance parameters (C,Df, kf, and dp in Eqs. 47 and 48) as deterministic, and using uninformative priors forμg and σg, i.e., ppr(μg)= ppr(σg)= 1, so p(x|b)= p(b|x) and xMAP = xMLE. Figure 18shows a contour plot of the probability density of p(x|b), marginalized probabilitydensities for x1 = μg and x2 = σg, and 90% credibility intervals. Marginalization isdone in two ways: a Markov-Chain Monte Carlo sampling procedure, which usesMetropolis–Hastings criterion to generate samples that become ergodic to p(x|b);and an approximating normal distribution is constructed from the Jacobian,x ~ Ν(xMAP, Γx), Γx = J(xMAP)Γb

�1 J(xMAP).It is straightforward to extend the analysis to consider uncertainty in the nuisance

parameters, which provides a more realistic reflection of the uncertainty in theinferred quantities of interest. Increasing the number of degrees of freedom alsoincreases the ill-posedness of the problem, so it is often necessary to incorporateadditional information through a Bayesian prior. The prior should be defined ina way that incorporates all available “testable” prior information (roughly speaking,information that can be empirically confirmed) while avoiding subjective informa-tion that could unduly bias the inferred parameters. The Principle of MaximumEntropy (Jaynes 1957; von Toussaint 2011) can be used to maximize the informationentropy of the prior distribution subject to constraints related to the testable infor-mation. If point estimates and corresponding uncertainties are available, the infor-mation entropy is maximized by a normal distribution centered on the estimate witha standard deviation equal to the uncertainty. Gaussian priors are also convenient,since they can readily be combined with the likelihood function following theexample in Sect. 3.4.

Figure 19 shows an example in which uncertainties in C and Df are considered byadding them to the unknown vector x = [μg, σg, Df, C]

T. The posterior probabilitydensities obtained using an uninformative prior are wide, due to the ill-posedness ofthe inference problem. The maximum likelihood estimate deviates from the exactsolution, denoted by vertical red lines, due to random noise contaminating themeasurements. Specifying a Gaussian prior for Df centered at 1.7 with a standarddeviation of 0.05 dramatically narrows the posterior probability density. The priorand marginalized posteriors for Df are almost indistinguishable, highlighting that theestimate for Df relies almost entirely on the prior information.

While the likelihood function derived by Huber et al. (2016) considers measure-ment noise and an unknown calibration error, their study excludes uncertaintyassociated with the scattering model used to derive the kernel. Most researchersderive k(θ, Rg) from Rayleigh–Debye–Gans fractal aggregate (RDF-FA) theory,which is computationally-efficient and analytically-tractable, but can differ consid-erably from scattering predictions derived from more accurate but costly numericaltechniques. (An error of 5–15% is typical, depending on the aggregate size, fractalstructure, and scattering angle, e.g., Berg and Sorensen (2013).) Model error must beincluded in order to accurately assess the uncertainty in the estimated quantities ofinterest. Huber et al. (2016) highlight this by comparing the estimated size param-eters found from experimental light scattering data using kernels derived from threevariants of RDG-FA theory. The size distribution parameters recovered from

44 K.J. Daun

Fig. 18 Using a lognormalparameterization for p(Rg)reduces the ill-posedness ofthe inference problem andmakes it problem nonlinear.Contours correspond toposterior probability densitiesp(x|b) = p(b|x) for ppr = 1.Marginalization is carried outusing Markov-Chain MonteCarlo and with a Gaussianapproximation. Shadedregions of the marginalizedprobability densities denote90% credibility intervals(Huber et al. 2016)

Fig. 19 Uncertainties in the nuisance parameters, C and Df, can be incorporated by adding them tothe unknown vector x, but results in a broad posterior probability density (gray curves). Specifyinga Gaussian prior on Df (green curve) narrows the distributions (blue curves). The similarity betweenthe posterior density and prior for Df shows that the measurement equations contain very littleinformation about this parameter relative to the prior (Huber et al. 2016)


experimental data differ considerably, even though simulated datasets generated bysubstituting the same size distribution into the three measurement equations arenearly indistinguishable.

6 Conclusions and Outlook

Inverse problems are ubiquitous in radiative transfer: the goal of inverse design is todetermine the configuration of a thermal system that produces a desired engineeringoutcome, while in parameter estimation quantities of interest are inferred fromindirect experimental measurements. Both types of problems are mathematicallyill-posed due to an information deficit. In the case of inverse design problems,multiple solutions may exist that satisfy the design specifications, or the designproblem may not have a solution at all. Parameter estimation problems are oftenill-posed because multiple candidate parameter sets exist that, when substituted intothe measurement equations, produce simulated data that is nearly indistinguishablefrom the measurements, particularly in the presence of noise.

A diverse suite of techniques have been applied to inverse problems arising inevery subarea of radiative transfer. Each of these techniques addresses the informa-tion deficient that makes the inverse problem ill-posed, by injecting additional priorinformation into the analysis. There are many ways to do this: using a lower-orderparameterization (e.g., reducing the number of heaters in a radiant enclosure designproblem); linear regularization schemes that promote smooth solutions bysuppressing error amplification by small singular values; and Bayesian priors,which explicate the role of prior information in the inference procedure.

The inverse analysis technique should be chosen based on the mathematicalproperties of the ill-posed problem (linear, nonlinear, convex, etc.), the prior infor-mation available, the number and type of variables, and the cost of evaluatingthe forward problem. While many schemes produce a single solution, it is oftenimportant to consider the existence of other candidate solutions, since these representalternative design options, or reflect uncertainty in parameter estimation problems. Ifthe inverse problem is linear, the solution space can be explored by varying theregularization parameter or examining the orthonormal basis formed from a singularvalue decomposition of the coefficient matrix. A similar procedure can be done usingthe Jacobian matrix for nonlinear inverse problems, while most metaheuristictechniques inherently generate a family of candidate solutions. In the context ofBayesian inference, the existence of multiple solutions is reflected in the posteriorprobability density, and univariate marginalized densities and credibility intervals.

A review of the current literature shows inverse analysis to be a very dynamicarea within radiative transfer. The emergence of nanoscale radiation heat transfer, inparticular, has resulted in inverse design problems involving engineered nano-structures for energy conversion and diagnostics, as well as parameter estimationproblems focused on characterizing the attributes of aerosolized nanoparticles basedon absorption, emission, and scattering measurements. Inverse problems arising inmore traditional areas of radiative transfer remain important, e.g., for understanding

46 K.J. Daun

climate change, developing cleaner, more efficient energy conversion technologies,and new manufacturing processes. Advancements in optics and electronics, e.g.,hyperspectral imaging and new light sources, also lay a foundation for noveldiagnostics at the macro-, micro-, and nanoscales to help further scientific under-standing and solve important engineering problems.

New inverse analysis algorithms are being developed to realize the full potentialof this technology. These algorithms exploit improvements in computational poweras well as improved solution techniques for the radiative transfer equation, whichallow for multiple objective function evaluations needed to carry out design optimi-zation and the high-order integrations required to calculate the evidence and mar-ginalize probability densities for Bayesian inference. Researchers in thermalradiation also stand to benefit from emerging techniques in inverse analysis. In thecase of parameter estimation, for example, new design-of-experiment techniques arebeing developed that maximize the information content of the measurement data,thereby reducing the underlying ill-posedness of the underlying inference problem.Likewise, new statistical tools help guide the development of physical models basedon experimental data, in order to prevent over-tuning in the presence of measurementnoise and model parameter uncertainty. In summary, inverse analysis will remain animportant area of thermal radiation research for years to come.

7 Cross-References

▶Monte Carlo Methods for Radiative Transfer▶Near-Field Radiative Transfer▶Optical and Radiative Properties of Surfaces▶Radiative Properties of Gases▶Radiative Properties of Particles▶Radiative Transfer Equation and Solutions▶Radiative Transfer in Combustion Systems

References

Åkesson EO, Daun KJ (2008) Parameter selection methods for axisymmetric flame tomographythrough Tikhonov regularization. Appl Opt 47:407–416

Amiri H, Mansouri SH, Coelho PJ (2013) Inverse optimal design of radiant enclosures withparticipating media: a parametric study. Heat Transf Eng 34:288–302

Aster CR, Borchers B, Thurber CH (2013) Parameter estimation and inverse problems, 2nd edn.Academic Press, San Diego

Baranoski G, Bramley R, Rokne JG (2001) Examining the spectrum of radiative transfer systems.Int Comm Heat Mass Transf 28:519–525

Beck JV (1968) Surface heat flux determination using an integral method. Nucl Eng Des 7:170–178Berg MJ, Sorensen CM (2013) Internal fields of soot fractal aggregates. J Opt Soc Am A

30:1947–1955Bertsekas DP (1999) Nonlinear programming, 2nd edn. Athena Scientific, Belmont


http://link.springer.com/Monte Carlo Methods for Radiative Transfer

http://link.springer.com/Near-Field Radiative Transfer

http://link.springer.com/Optical and Radiative Properties of Surfaces

http://link.springer.com/Radiative Properties of Gases

http://link.springer.com/Radiative Properties of Particles

http://link.springer.com/Radiative Transfer Equation and Solutions

http://link.springer.com/Radiative Transfer in Combustion Systems

Bohren CF, Huffman DR (1983) Absorption and scattering of light by small particles. Wiley,New York

Burr DW, Daun KJ, Link O, Thomson KA (2011) Determination of the soot aggregate sizedistribution from elastic light scattering through Bayesian inference. J Quant Spectrosc RadiatTransf 112:1099–1107

Charette A, Boulanger J, Kim HK (2008) An overview on recent radiation transport algorithmdevelopment for optical tomography imaging. J Quant Spectrosc Radiat Transf 109:2743–2766

Charnigo R et al (2012) Credible intervals for nanoparticle characteristics. J Quant Spectrosc RadiatTransf 113:182–193

Chopade RP, Mishra SC, Mahanta P, Maruyama S (2012) Estimation of power of heaters in aradiant furnace for uniform thermal conditions on 3-D irregular shaped objects. Int J Heat MassTransf 55:4340–4351

Cormack AM (1973) Reconstructions of densities from their projections with applications inradiological physics. Phys Med Bio 18:195–207

Dasch CJ (1992) One-dimensional tomography: a comparison of Abel, onion-peeling, and filteredbackprojection methods. Appl Opt 31:1146–1152

Daun KJ, Howell JR (2005) Inverse design methods for radiative transfer systems. J QuantSpectrosc Radiat Transf 93:43–60

Daun KJ, Howell JR, Morton DP (2003a) Design of radiant enclosures using inverse and non-linearprogramming techniques. Inverse Prob Eng 11:541–560

Daun KJ, Howell JR, Morton DP (2003b) Geometric optimization of radiant enclosures containingspecular surfaces. J. Heat Transf. 125:845–851

Daun KJ, Howell JR, Morton DP (2003c) Geometric optimization of radiant enclosures throughnonlinear programming. Numer Heat Transf, Part B 43:203–219

Daun KJ, Thomson KA, Liu F, Smallwood GJ (2006a) Deconvolution of axisymmetric flameproperties using Tikhonov regularization. Appl Opt 45:4638–4646

Daun KJ et al (2006b) Comparison of methods for inverse Design of Radiant Enclosures. ASME JHeat Transf 128:269–282

Daun KJ, Grauer SJ, Hadwin PJ (2016) Chemical species tomography of turbulent flows: discreteill-posed and rank deficient problems and the use of prior information. J Quant Spectrosc RadiatTransf 172:58–74

Deiveegan M, Balaj C, Venkateshan SP (2006) Comparison of various methods for simultaneousretrieval of surface emissivities and gas properties in gray participating media. ASME J HeatTransf 128:829–837

Ertürk H, Ezekoye OA, Howell JR (2002a) Comparison of three regularized solution techniques ina three-dimensional inverse radiation problem. J Quant Spectrosc Radiat Transf 73:307–316

Ertürk H, Ezekoye OA, Howell JR (2002b) The application of an inverse formulation in the designof boundary conditions for transient radiating enclosures. ASME J Heat Transf 124:1095–1102

Ertürk H, Gamba MEOA, Howell JR (2008) Validation of inverse boundary condition design in athermometry test bed. J Quant Spectrosc Radiat Transf 109:317–326

Farahmand A, Payan S, Hosseini Sarvari S (2012) Geometric optimization of radiative enclosuresusing PSO algorithm. Int J Therm Sci 60:61–69

Fedorov AG, Lee KH, Viskanta R (1998) Inverse optimal design of the radiant heating in materialsprocessing and manufacturing. J Mater Eng Perform 7:719–726

Floyd J, Kempf AM (2011) Computed tomography of Chemiluminescence (CTC): high resolutionand instantaneous 3-D measurements of a matrix burner. Proc Combust Inst 11:751–758

França F, Ezekoye O & Howell J 1999 Inverse determination of heat source distribution inradiative systems with participating media. In: 33rd national heat transfer conferenceNHTC’99, Albuquerque, NM, USA, 15–17 August 1999

França FHR, Ezekoye OA, Howell JR (2001) Inverse boundary design combining radiation andconvection heat transfer. ASME J Heat Transf 123:884–891

França FHR, Howell JR, Ezekoye OA, Morales JC (2003) Inverse design of thermal systems withdominant radiative transfer. Adv Heat Transf 36:1–110

48 K.J. Daun

Gill PE, Murray W, Wright MH (1986) Practical optimization. Academic Press, San DiegoGlover F (1986) Future paths for integer programming and links to artificial intelligence. Comput

Oper Res 5:533–549Golub GH, Heath MT, Wahba G (1979) Ceneralized cross-validation as a method for choosing a

good ridge parameter. Technometrics 21:215–223Hadamard J (1923) Lectures on Cauchy's problem in linear differential equations. Yale University

Press, New HavenHaisch C (2012) Optical tomography. Annu Rev Anal Chem 5:57–77Hajimirza S, El Hitti G, Heltzel A, Howell J (2012) Using inverse analysis to find optimum nano-

scale radiative surface patterns to enhance solar cell performance. Int J Therm Sci 62:93–102Hall RJ, Bonczyk PA (1990) Sooting flame thermometry using emission/absorption tomography.

Appl Opt 29:4590–4598Hansen PC (1999) Rank-deficient and discrete ill-posed problems: numerical aspects of linear

inversion. SIAM, PhiladelphiaHansen PC, O'Leary DP (1993) The use of the L-curve in the regularization of discrete ill-posed

problems. J Sci Comput 14:1487–1503Hansen PC, Sekii S, Shibahashi H (1992) The modified truncated SVD method for regularization in

general form. SIAM J Sci Stat Comput 13:1142–1150Harutunian V, Morales JC, Howell JR (1995) Radiation exchange within an enclosure of diffuse-

gray surfaces: the inverse problem. In: National heat transfer conference, Portland OR, 1995Hendricks TJ, Howell JR (1996) Absorption/scattering coefficients and scattering phase functions

in reticulated porous ceramics. ASME J Heat Transf 118:79–87Ho C-H, Özşik MN (1988) Inverse radiation problems in inhomogeneous media. J Quant Spectrosc

Radiat Transf 40:533–560Hosseini Sarvari SM (2007) Optimal geometry design of radiative enclosures using the genetic

algorithm. Numer Heat Transf, Part A 52:127–143Hosseini Sarvari S, Howell J, Mansouri S (2003) Inverse boundary design conduction-radiation

problem in irregular two-dimensional domains. Numer Heat Transf, Part B 44:209–224Howell JR, Mengüç MP, Siegel R (2016) Thermal radiation heat transfer, 6th edn. CRC Press,

Boca RatonHuber FJT, Will S, Daun KJ (2016) Sizing aerosolized fractal nanoparticle aggregates through

Bayesian analysis of wide-angle light scattering (WALS) data. J Quant Spectrosc Radiat Transf184:27–39

Jaynes ET (1957) Information theory and statistical mechanics. Phys Rev 106:620–630Jones MR (1999) Inverse analysis of radiative transfer systems. ASME J Heat Transf 121:481–484Kaipio J, Somersalo E (2005) Statistical and computational inverse problems. Springer, BerlinKashif M et al (2012) Soot volume fraction fields in unsteady axis-symmetric flames by continuous

laser extinction technique. Opt Express 20:28742–28751Kennedy J, Eberhart R (1995) Particle swarm optimization. Perth WA (ed) IEEE international

conference on neural networks, pp. 1942–1948Kiefer J, Wolfowitz J (1952) Stochastic estimation of the maximum of a regression function. Ann

Math Stat 23:462–466Kim KW, Baek SW (2004) Inverse surface radiation analysis in an axisymmetric cylinderical

enclosure using a hybrid genetic algorithm. Numer Heat Transf, Part A 46:367–381Kim KW, Baek SW (2007) Inverse radiation–conduction design problem in a participating con-

centric cylindrical medium. Int J Heat Mass Transf 50:2828–2837Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science

220:671–680Kowsary F, Pooladvand KPA (2007) Regularized variable metric method versus the conjugate

gradient method in solution of radiative boundary design problem. J Quant Spectrosc RadiatTransf 108:277–294

Kudo K et al (1996) Solution of the inverse radiative load problem using the singular valuedecomposition technique. JSME Int J, Ser B 39:808–814


Leduc G, Monchoux F, Thellier F (2004) Inverse radiative design in human thermal environment.Int J Heat Mass Transf 47:3291–3300

Lee KH, Kim KW (2015) Performance comparison of particle swarm optimization and geneticalgorithm for inverse surface radiation problem. Int J Heat Mass Transf 88:330–337

Lee KH, Baek SW, Kim KW (2008) Inverse radiation analysis using repulsive particle swarmoptimization algorithm. Int J Heat Mass Transf 51:2772–2783

Marston AJ, Daun KJ, Collins MR (2012) Geometric optimization of radiant enclosures containingspecularly-reflecting surfaces through quasi-Monte Carlo simulation. Numer Heat Transf,Part A 59:81–97

McCann H, Wright P, Daun K (2015) Chemical species tomography. In: Industrial Tomography:Systems and applications. Woodhead Publishing, Sawston, pp 135–174

McCormick NJ (1992) Inverse radiative transfer problems: a review. Nucl Sci Eng 112:185–198Mengüç PM, Dutta P (1994) Scattering tomography and its application to sooting diffusion flames.

ASME J Heat Transf 144:144–151Metropolis N et al (1953) Equations of state calculations by fast computing machines. J Chem Phys

21:1087–1092Mitchell M (1996) An introduction to genetic algorithms. MIT Press, CambridgeModest MF (2013) Radiative heat transfer, 3rd edn. Academic Press, San DiegoMoré J, Sorensen D (1983) Computing a trust region step. SIAM J Sci Stat Comput 3:553–572Morozov VA (1968) On the discrepancy principle for solving operator equations by the method of

regularization. Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki 8:295–309Mossi AC, Vielmo HA, França FHR, Howell JR (2008) Inverse design involving combined

radiative and turbulent convective heat transfer. Int. J. Heat Mass Trans. 51:3217–3226Özşik MN, Orlande HRB (2000) Inverse heat transfer: fundamentals and applications. CRC Press,

Boca RatonPorter JM, Larsen ME, Wesley BJ, Howell JR (2006) Metaheuristic optimization of a discrete Array

of radiant heaters. ASME J Heat Transf 128:1031–1040Pourshaghaghy A, Pooladvand K, Kowsary F, Karimi-Zand K (2006) An inverse radiation bound-

ary design problem for an enclosure filled with an emitting, absorbing, and scattering media.Int Commun Heat Mass Transf 33:381–390

Randrianalisoa J, Baillis D, Pilon L (2006) Improved inverse method for radiative characteristics ofclosed-cell absorbing porous media. AIAA J Thermo Heat Transf 20:871–883

Ren T, Modest MF (2016) Temperature profile inversion from carbon-dioxide spectral intensitiesthrough Tikhonov regularization. AIAA J Thermo Heat Transf 30:211–218

Ren T, Modest MF, Fateev A, Clausen S (2015) An inverse radiation model for optical determina-tion of temperature and species concentration: development and validation. J Quant SpectroscRadiat Transf 151:198–209

Rukolaine SA (2015) Shape optimization of radiant enclosures with specular-diffuse surfaces bymeans of a random search and gradient minimization. J Quant Spectrosc Radiat Transf151:174–191

Safavinejad A, Mansouri SH, Sakurai A, Maruyama S (2009) Optimal number and location ofheaters in 2-D radiant enclosures composed of specular and diffuse surfaces using micro-geneticalgorithm. Appl Therm Eng 29:1075–1085

Santoro RJ, Semerjian HJ, Emmerman P, Goulard R (1981) Optical tomography for flow fielddiagnostics. Int J Heat Mass Transf 24:1139–1150

Silva Neto AJ, Özşik MN (1995) An inverse problem of simultaneous estimation of radiation phasefunction, albedo and optical thickness. J Quant Spectrosc Radiat Transf 53:397–409

Sorensen C (2001) Light scattering from fractal aggregates: a review. Aerosol Sci Technol35:648–687

di Stasio S et al (2006) Synchrotron SAXS 〈in situ〉identification of three different size modes forsoot nanoparticles in a diffusion flame. Carbon 44:1267–1279

Tan JY, Liu LH (2009) Inverse geometry design of radiating enclosure filled with participatingmedia using a meshless method. Numer Heat Transf, Part A 56:132–152

50 K.J. Daun

Tikhonov AN, Arsenin VY (1977) Solution of ill-posed problems. Winston and Sons,Washington, DC

von Toussaint U (2011) Bayesian inference in physics. Rev Mod Phys 83:943–999Wing GM, Zhart JD (1991) A primer on integral equations of the first kind: the problem of

deconvolution and unfolding. SIAM, PhiladelphiaWinston R (1991) Nonimaging optics. Sci Am 264:76–81Wright P et al (2010) High speed chemical species tomography in a multi-cylinder automotive

engine. Chem Eng J 158:2–10


inverse problems in radiative transfer … · inverse problems are ubiquitous in all areas of...

Documents