myopic policies for budgeted optimization with constrained experiments
Post on 15-Feb-2016
50 Views
Preview:
DESCRIPTION
TRANSCRIPT
1
Myopic Policies for Budgeted Optimization
with Constrained Experiments
Javad Azimi, Xiaoli Fern, Alan Fern
Oregon State University AAAI, July 2010
2
Motivation: Electricity Production in a Microbial Fuel Cell (MFC)
AnodeCathode
bact
eria
Oxidation products
(CO2)
Fuel (organic matter)
e-
e-
O2
H2OH+
This is how an MFC works
SEM image of bacteria sp. on Ni nanoparticle enhanced carbon fibers.
Nano-structure of anode significantly impact the electricity production.
We should optimize anode nano-structure to maximize power.
3
Experiment Selection• Experiments are costly and there is a fixed budget.• How to select the best sequence of experiments.
Current Experiments Scientist selects Experiment
Run Experiment
Bayesian Optimization (BO)• Since Running experiment is very expensive we use BO.
• Select one experiment to run at a time based on results of previous experiments.Current Experiments Gaussian Process Surface Select Single Experiment
Run Experiment 4
5
Bayesian Optimization (BO)• BO assumes that we can ask for specific experiment.• This is unreasonable assumption in many applications.
– In Fuel Cell it takes many trials to create a nano-structure with specific requested properties.
– Costly to fulfill.Space of Experiments
Average Circularity
Ave
rage
d A
rea
6
Constrained Experiments• It is less costly to fulfill a request that specifies ranges for the nanostructure
properties
• E.g. run an experiment with Averaged Area in range r1 and Average Circularity in range r2
• We will call such requests “constrained experiments”Space of Experiments
Average Circularity
Ave
rage
d A
rea
Constrained Experiment 1• large ranges • low cost• high uncertainty about which experiment will be run
Constrained Experiment 2• small ranges• high cost• low uncertainty about which experiment will be run
7
BO for Constrained Experiment
• Given a fixed budget, select the best constrained experiments.
Run Experiment
Current Experiments Gaussian Process Surface Select Single ExperimentSelect Constrained Experiment
8
Constrained Experiment• We generalized BO heuristics to constrained
experiments.
• Two challenges:– How to compute heuristics for constrained experiment?– How to take experimental cost into account?(which has
been ignored by most of the approaches in BO).
9
Standard BO Heuristics• Standard heuristics are statistics of the posterior p(y|x,D) where D is our
current observation.
• Maximum Upper bound Interval (MUI)– Select point with highest 95% upper confidence bound– Purely explorative approach.
• Maximum Probability of Improvement (MPI)– It computes the probability that the output is more than (1+m) times of the best current
observation , m>0. – Explorative and Exploitative.
• Maximum Expected of Improvement (MEI)– Similar to MPI but parameter free– It simply computes the expected amount of improvement after sampling at any point.
10
Generalizing BO for Constrained Experiment
• Having the posterior distribution of p(y|x,D) and px(.|D) we can calculate the posterior of the output of each constrained experiment which has a closed form solution.
• Therefore we can compute standard BO heuristics for constrained experiments.– There are closed form solution for these heuristics.
Input spaceDiscretization
Level
11
Budgeted Constrained Experiments• We are limited with Budget B.• Unfortunately heuristics will typically select the smallest and most
costly constrained experiments which is not a good use of budget.
• How can we consider the cost of each constrained experiment in making the decision?– Cost Normalized Policy (CN)– Constraint Minimum Cost Policy(CMC)
-Low uncertainty
-High uncertainty
-Better heuristic value
-Lower heuristic value
-Expensive -Cheap
12
Cost Normalized Policy• It selects the constrained experiment achieving
the highest expected improvement per unit cost.
• We report this approach for MEI policy only.
13
Constraint Minimum Cost Policy (CMC)• Motivation:
1. Approximately maximizes the heuristic value.2. Has expected improvement at least as great as spending
the same amount of budget on random experiments.• Example:
Very expensive: 10 random experiments likely to be better
Selected Constrained experiment
Poor heuristic value: not select due to 1st
condition
Cost=4 random Cost=10 random Cost=5 random
14
Experimental Results(Setup)
• Gaussian process is used as our model with squared exponential kernel.
• Cost function is defined as:
– There is a constant cost for running any constrained experiment plus an additional cost depending on the size of the experiment.
– The value of slope dictates how fast the cost increases as the size of a constrained experiment decreases.
Space of Experiments
r1
r2
15
Experimental Results(Real Applications)
• Two Real data sets:– Fuel Cell:• Fuel Cell electricity generation
– Hydrogene:• Biosolar hydrogen production
16
Experimental Results(Benchmarks Functions)
• 3 popular benchmarks used in BO literature.
Rosenbrock Discontinuous Cosines
17
Overall Performance• The normalized regret of each framework is shown which is calculated
as y*- ymax over Random performance for budget 15 where y* is the highest possible output.
Average regret of each approach over all frameworks.
Different Budget(1)
18
Random
Cosines
Fuel CellReal
Rosenbrock
CMC-MUI
Different Budget(2)
19
CN-MEI
Cosines
Fuel CellReal
Rosenbrock
Different Budget(3)
20
CMC-MPI(0.2)
Cosines
Fuel CellReal
Rosenbrock
21
Different Budget(4)
CMC-MEICosines
Fuel CellReal
Rosenbrock
22
Conclusion• We introduced a new constrained experiment
framework which asks for hyper-rectangle rather than exact point.
• We extended model-free BO heuristics to our frame work.
• We introduced two approaches to optimize our budgeted framework.
• CMC-MEI is working better than other approaches.
23
Thanks for attendance
Question?
top related