computing in complex systems
DESCRIPTION
Center for Engineering Science Advanced Research. OAK RIDGE NATIONAL LABORATORY U. S. DEPARTMENT OF ENERGY. Computing in Complex Systems. J. Barhen Computing and Computational Sciences Directorate. - PowerPoint PPT PresentationTRANSCRIPT
Computing in Complex Systems
J. Barhen
Computing and Computational Sciences Directorate
Research Alliance for MinoritiesFall Workshop
ORNL Research Office BuildingDecember 2, 2003
Cen
ter
for
En
gin
eeri
ng
Sci
ence
Ad
van
ced
R
esea
rch
OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY
Advanced Computing Activities at CESARIn 1983 DOE established CESAR at ORNL. Its purpose was to conduct fundamental theoretical, experimental, and computational research in intelligent systems.
Over the past decade, the Center has experienced tremendous growth. Today, its primary activities are in support of DOD and the Intelligence Community. Typical examples include:
missile defense: BMC3, war games, HALO-2 project, multi-sensor fusion
sensitivity and uncertainty analysis of large computational models
laser array synchronization (directed energy weapons)
complex systems: neural networks, global optimization, chaos
quantum optics applied to cryptography
mobile cooperating robots, multi-sensor and computer networks
nanoscale science (friction at the nanoscale, interferometric nanolithography)
CESAR sponsors include: MDA, DARPA, Army, OSD/JTO, NRO, ONR, NASA, NSA, ARDA, DOE/SC, NSF, DOE/FE, and private industry.
Within the CCS Directorate, revolutionary computing technologies (optical, quantum, nanoscale, neuromorphic) are an essential focus of CESAR’s research portfolio.
The Global Optimization ProblemIllustrative Example of Computing in Complex Systems
Nonlinear Optimization problems arise in every field of scientific, technologic,
economic, or social interest. Typically, The objective function (the function to be optimized) is multimodal, i.e., it
possesses many local minima in the parameter region of interest In most cases it is desired to find the local minimum at which the function takes its
lowest value, i.e., the global minimum
The design of algorithms that can reach and distinguish between local and global minima is known as the global optimization problem.
Examples abound:► Computer Science: design of VLSI circuits, load balancing, …► Biology: protein folding► Geophysics: determination of unknown geologic parameters from surface measurements► Physics: elasticity, hydrodynamics, …► Industrial technology: optimal control, design, production flow, …► Economics: transportation, cartels, …
Problem Formulation
Definitions
► x is a vector of state variables or parameters► f is referred to as the objective function
Goal
Find the values fG and xG such that
► is the domain of interest over which one seeks the global minimum. It is assumed to be compact and connected.
► without loss of generality, we will take as the hyper parallelepiped
Let be the function to be op():timized.nf→ xRR
()min() | GGfff==∈xxx}D{
()() | ; 1,..., LUjjjjxxxxjn=≤≤={}D
Local vs Global Minima
**We assume to be a with a finite numbe r of discontinuities. l every (ower semicont) of in satisfiesinuous functionlocal mini the conditions mu () m fff=∂∂xxxxxxuD§ *220 (( )0 ) Tnf==∂∀∈≥∂xxyyxyxu§R
***lim inf()() except at a finite number of points where We further assume that the satisfies th e local minimum criglobal teria, minimumand that it doesfff→≥=xxxx occur on the boundarynot .ofD
Why is Global Optimization so Difficult? Illustration of Practical Challenges
Complex Landscapes • we need to find global
minimum of functions of many variables
• Typical problem size is (102 – 105) variables
Difficulty
• number of local minima grows exponentially with the number of variables
• local and global minima have the same signature, namely zero gradient
Schubert function: This function arises in signal processing applications. It is used as one of the SIAM benchmarks for
Global Optimization. Even its two dimensional instantiation exhibits a complex landscape.
Leading Edge Global Optimization Methods
The Center for Engineering Science Advanced Research (CESAR) at the Oak RidgeNational Laboratory (ORNL) has been developing, demonstrating, and documentingin the open literature leading edge global optimization (GO) algorithms.
What is the Approach?• three complementary methods address GO challenge
• exploit different aspects of problem but can be used in synergistic fashion
What are the Options? TRUST: fastest published algorithm for searching complex landscapes via tunneling NOGA: performs nonlinear optimization while incorporating uncertainties from model and
from external information (sensors, …) EO: exploits the availability of information typically available to the user but never exploited
by conventional optimization tools
Goal: Further develop, adapt, and demonstrate these methods on relevant DOE, DOD, and NASA applications where major impact is expected.
Leading Edge Global Optimization MethodsTRUST
What is TRUST ?• a new, extremely powerful global optimization paradigm developed at CESAR / ORNL
How does it work ? three innovative concepts subenergy tunneling: a nonlinear transformation that creates a virtual landscape where all
function values greater than the last found minimum are suppressed non-Lipschitzian “terminal” repellers: enable escape from local minima by “pushing” the
solution flow under the virtual landscape stochastic Pijavskyi cones: eliminate unproductive regions by using information on the
Lipschitz constant of the objective function acquired during the optimization process iterative decomposition & recombination of large scale problems
How does it perform ?• unprecedented speed and accuracy: overall efficiency up to 3 orders of magnitude higher than
best publicly available competitors for SIAM benchmarks• successfully tested on large-scale seismic imaging problem• outstanding performance led to article in Science (1997), to R&D 100 award (1998), and to a
patent in 2001.
TRUSTTerminal Repeller Unconstrained Subenergy Tunneling
**** find such that To attack this problem, define a : where ()min()|(,)(,)(,)1:l1 (,)ogggsubrepsubxfxfxxExxExxExxExxe=∈=+=+Goal{}Dvirtual objective function*[()()]**4/3**and Here, fixed value of , which can be a local minimum or an initial s(,)()[ ()() tate Heaviside step fu][]nctiofxfxarepExxxxHfxfxxxH−−+3=−ρ−−4=•=n
TRUSTComputational Approach
* search for in terms of the flow of a differential equation constructed from the virtual objective functionSp(,) : ecifically,re gxExxxx∂=−∂ Basic Idea&**1/3*[()()]*sults in Each equilibrium state of this equation will be a local minimizer()1()[ ()() ]1 of ,hence, a local or global min(,)ifxfxafxxxxHfxfxxeExx−−+∂=−+ρ−−∂+&()mizer of . fx
Uniqueness of TRUST
Virtual objective function E( x, x* ) is a superposition of two contributing terms ► Esub (x, x*): subenergy tunneling
► Erep (x, x*): repelling from latest found local minimum Its effect is to transform the current local minimum of f(x) into a global maximum,
while preserving any lower laying local minima
-15
-10
-5
0
5
10
15
20
25
30
-2.25 -1.5 -0.75 0 0.75 1.5 2.25 3
Current local minimum We seek global minimum of blue function
Subenergy tunneling transformation is applied to shifted (green) function
Motion on virtual surface
Effective tunneling
Gradient descent applied to f(x) and initialized at x*+ can not escape from the basin of attraction of x*
Gradient descent applied to E( x, x* ) and initialized at x*+ always escapes it.
TRUST has a global descent
property. Erep Esub
Key Advantage of TRUST
Leading Edge Global Optimization Methods
Benchmark BR CA GP RA SH H3
Method
SDE 2700 10822 5439 241215 3416
GA / SA 430 460 5917
IA 1354 326 7424
Levy TUN 1469 12160
Tabu 492 486 540 727 508
TRUST 55 31 103 59 72 58
Comparison of TRUST performance to leading publicly available competitors for SIAM benchmarks• data correspond to number of function evaluations needed to reach global minimum• symbol indicates that no solution was found for method under consideration• benchmark functions: BR (Branin), CA (camelback), GP (Goldstein-Price), RA (Rastrigin), SH (Shubert), H3 (Hartman)• methods: SDE (stochastic differential equations), GA/SA (genetic algorithms and simulated annealing), IA (interval arithmetic), Levy TUN (conventional Levy tunneling), Tabu (Tabu search)
Leading Edge Global Optimization Methods
NOGA► The explicit incorporation of uncertainties into the optimization process is essential for the design of robust
mission architectures and systems ► NOGA = method for Nonlinear Optimization and Generalized Adjustments ► explicitly computes the uncertainties in model predicted results in terms of uncertainties in intrinsic model
parameters and inputs► determine best-estimates of model parameters and reduces uncertainties by consistently incorporating
external information ► NOGA methodology is based on the concepts and tools of sensitivity and uncertainty analysis. It
performs a non-linear optimization of a constrained Lagrange function that uses the inverse of a generalized total covariance matrix as natural metric
EO EO = Ensemble Optimization Builds on systematic study on the role that additional information may have in significantly reducing the
complexity of the GOP while in most practical problems additional information is readily available either at no cost at all or at
rather low cost, present optimization algorithms cannot take advantage of it to increase the efficiency of the search.
to overcome this shortcoming, we have developed EO, a radically new class of optimization algorithms that can readily fold in additional information and - as a result – dramatically increase their efficiency
Leading Edge Global Optimization MethodsSelected References
TRUST Barhen, J., V. Protopopescu and D. Reister, “TRUST: A Deterministic Algorithm for Global Optimization”,
Science, 276, 1094-1097 (1997). Reister, D., E. Oblow, J. Barhen, and J. DuBose, “Global Optimization to Maximize Stack Energy”,
Geophysics, 66(1), 320-326 (2001).
NOGA Barhen, J. and D. Reister, “Uncertainty Analysis based on Sensitivities Generated using Automated
Differentiation”, Lecture Notes in Computer Science, 2668, 70-77, Springer (2003). Barhen, J., V. Protopopescu, and D. Reister, “Consistent Uncertainty Reduction in Modeling nonlinear
Systems”, SIAM Journal of Scientific Computing (in press, 2003).
EO Protopopescu, V. and J. Barhen, "Solving a Class of Continuous Global Optimization Problems using
Quantum Algorithms", Physics Letters, A 296, 9-14 (2002). Protopopescu, V., C. d’Helon, and J. Barhen, “Constant-time Solution to the Global Optimization Problem
using Bruschweiler’s Ensemble Search Algorithm, Jour. Phys., A 36(24), L399-L407 (2003).
Frontiers in Computing
Three decades ago, fast computational units were only present in vector super-computers.
Twenty years ago, the first message-passing machines (Ncube, Intel) were introduced.
Today, the availability of fast, low-cost chips, has revolutionized the way calculations are performed in various fields, from personal workstation to tera-scale machines.
An innovative approach to high performance, massively parallel computing remains a key factor for progress in science and national defense applications.
In contrast to conventional approaches, one must develop computational paradigms that exploit, from the onset (1) the concept of massive parallelism and (2) the physics of the implementation device.
Ten to twenty years from now, asynchronous, optical, nanoelectronic, biologically inspired, and quantum technologies have the potential of further revolutionizing computational science and engineering by
offering unprecedented computational power for a wide class of demanding applications enabling the implementation of novel paradigms