lecture 09 value of information - college of …lecture outline 1 introduction to value of...

Introduction to Value of InformationExpected Value of Perfect Information

Expected Value of Imperfect InformationApplication of Value of Information in Engineering Design

Lecture 09Value of Information

Jitesh H. Panchal

ME 597: Decision Making for Engineering Systems Design

Design Engineering Lab @ Purdue (DELP)School of Mechanical Engineering

Purdue University, West Lafayette, INhttp://engineering.purdue.edu/delp

October 07, 2014c©Jitesh H. Panchal Lecture 09 1 / 24

http://engineering.purdue.edu/delp



Lecture Outline

1 Introduction to Value of Information

2 Expected Value of Perfect Information

3 Expected Value of Imperfect Information

4 Application of Value of Information in Engineering Design

Clemen, R. T. (1996). Making Hard Decisions: An Introduction to Decision Analysis. Belmont, CA, Wadsworth Publishing Company.Chapter 12.

c©Jitesh H. Panchal Lecture 09 2 / 24



Illustrative Example - Investing in the Stock Market

Investor has three choices:

High-risk stock

Low-risk stock

Savings account

Stocks have a brokerage feeof $200

Payoff on stocks depends onwhether the market goes up,remains same, or goes down.

Savings account

Up (0.5)

Same (0.3)

Down (0.2)

High-Risk

Stock

Low-Risk

Stock

Up (0.5)

Same (0.3)

Down (0.2)

1500

100

-1000

1000

200

-100

500

Figure: 12.1 on Page 436 (Clemen)




Basic Questions

What does it mean for an expert to provide perfect information?

How does probability relate to the idea of information?

What is an appropriate basis on which to evaluate information in adecision situation?




The notion of “Perfect Information”

An expert’s information is said to be perfect if it is always correct.

1 When state S will occur, the expert always says so. For example,P(Expert says “Market Up” | Market really Does Go Up) = 1

Since the probability must add to one,P(Expert says “Market will stay the same or fail” | Market really Does

Go Up) = 0

2 Also, the expert must never say that the state S will occur if any otherstate (S̄) will occur.

P(Expert says “Market will go up” | Market really will stay the Sameor Fail) = 0




Flipping the Probabilities Using Bayes’ Theorem

P(Market UP|Exp says UP) =P(Exp says UP|Market UP)P(Market UP)

P(Exp says UP)

The denominator can be written as the sum of:

P(Exp says UP) =

P(Exp says UP|Market UP)P(Market UP)

+

P(Exp says UP|Market NOT UP)P(Market NOT UP)

We can check that

P(Market UP|Exp says UP) = 1




The Expected Value of Information – Key Concepts (1)

If the decision maker will make the same decision regardless of what the newinformation is, then the information has no value!

1 In the example, the best decision without the information is to invest.2 If the expert says that the market will go up, then the decision still

remains the same → no value.3 If the expert says that market will go down, then the decision would

change. So, information has value.4 This is ex-post value!




The Expected Value of Information – Key Concepts (2)

Need to define the value of information before hiring the expert (i.e., beforegetting the information).

1 Worst case scenario: Decision remains the same even after hiring theexpert ⇒ Zero value of information.

2 Better scenarios: Expected value increases ⇒ Positive value ofinformation

3 Best case scenario: Perfect Information (resolving all uncertainty; Experttells us exactly what will happen) ⇒ Maximum value of information (i.e.,Expected Value of Perfect Information)




Expected Value of Perfect Information

An expert (clairvoyant) who could reveal exactly what the market would do.

Note: The uncertainty in the problem is resolved before the investmentdecision is made. Once the decision is made, the market must still go thoughits performance. It is simply that the investor knows exactly what theperformance will be.





Market

up (0.5)

Up (0.5)

Same (0.3)

Down (0.2)

High-Risk

Stock

Low-Risk

Stock

Up (0.5)

Same (0.3)

Down (0.2)

1500

100

-1000

1000

200

-100

500

(EMV = 540)

(EMV = 580)

High Risk Stock

Low Risk Stock

Savings Account

1500

1000

500

100

200

500

-1000

-100

500

Savings account

Market down

(0.2)

Market flat (0.3)

Consult

Clairvoyant

(EMV=1000)

High Risk Stock

Low Risk Stock

Savings Account

High Risk Stock

Low Risk Stock

Savings Account

Figure: 12.3 on Page 440 (Clemen)c©Jitesh H. Panchal Lecture 09 11 / 24




Ex-ante nature: The decision maker has not yet consulted the clairvoyant. Heis considering whether or not to consult!

There is 50% chance that the clairvoyant would say that the market will go up.




Expected Value of “Imperfect” Information

Suppose that the expert’s track record shows that if the market actually willrise, he says:

“Up” 80% of the time

“Flat” 10% of the time

“Down” 10% of the time

True Market StateEconomist’s Prediction Up Flat Down“Up” 0.80 0.15 0.20“Flat” 0.10 0.70 0.20“Down” 0.10 0.15 0.60

1.00 1.00 1.00




Decision Tree for the Investment Example

Economist says

“Market up” (?)

High-Risk Stock (EMV = 580)

Low-Risk Stock (EMV = 540)

High Risk Stock

Low Risk Stock

Savings Account

Savings account (EMV = 500)

Economist’s

forecast

Economist says

“Market flat” (?)

Economist says

“Market Down” (?) High Risk Stock

Low Risk Stock

Savings Account

High Risk Stock

Low Risk Stock

Savings Account

500

Down (?)

Up(?)Same(?)

1000

200

-100

Market Activity

Down (?)

Up(?)Same(?)

1500

100

-1000

Down (?)

Up(?)Same(?)

1500

100

-1000

Down (?)

Up(?)Same(?)

1000

200

-100

Down (?)

Up(?)Same(?)

1500

100

-1000

Down (?)

Up(?)Same(?)

1000

200

-100

500

500

Consult

Economist





Flipping the Probability Tree

“Market up” (0.80)

Market up

(0.5)

Market

Flat (0.3)

Market

Down

(0.2)

Actual Market

Performance

Economist’s

Forecast

“Market Flat” (0.10)

“Market Down” (0.10)







Market up (?)

“Market

up” (?)

“Market

Flat” (?)

“Market

Down” (?)

Economist’s

Forecast

Actual Market

Performance

Market Flat (?)

Market Down (?)

Market up (?)

Market Flat (?)

Market Down (?)

Market up (?)

Market Flat (?)

Market Down (?)





Posterior Probabilities For Market Trends

Posterior Probability forEconomist’s Prediction Market Up Market Flat Market Down“Up” 0.8247 0.0928 0.0825 1.0000“Flat” 0.1667 0.7000 0.1333 1.0000“Down” 0.2325 0.2093 0.5581 1.0000




Completed Decision Tree

Economist says

“Market up”

(0.485)

High-Risk Stock (EMV = 580)

Low-Risk Stock (EMV = 540)

High Risk Stock

Low Risk Stock

Savings Account

Savings account (EMV = 500)

Economist’s

forecast

Economist says

“Market flat”

(0.300)

Economist says

“Market Down”

(0.215)

High Risk Stock

Low Risk Stock

Savings Account

High Risk Stock

Low Risk Stock

Savings Account

500

Down (0.0825)

Up(0.8247)Same(0.0928)

1000

200

-100

Down (0.0825)

Up(0.8247)Same(0.0928)

1500

100

-1000

Down (0.1333)

Up(0.1667)Same(0.7000)

1500

100

-1000

Down (0.1333)

Up (0.1667)Same(0.7000)

1000

200

-100

Down (0.5581)

Up(0.2325)Same(0.2093)

1500

100

-1000

Down (0.5581)

Up(0.2325)Same(0.2093)

1000

200

-100

500

500

Consult

Economist

(EMV=822)

EMV=1164

EMV=835

EMV=187

EMV=293

EMV=-188

EMV=219





Application of Value of Information in Engineering Design

Design is a decision making process.

Design process = Network of decisions.




Is Further Refinement Necessary? A Conceptual Example

How much refinement of a simulation

model is appropriate for design?

R

R LR L

T

Material Alternatives

Decision

Material 1

Simulation model predicts

material strength

Objective: Maximize

the pressure that the

vessel can withstand

Selected

Material

Alternative

Material 2

Should this model be refined further?

• Improved accuracy does not imply

improved usefulness in design !!!

• The appropriateness depends on

the overall design problem

(constraints, objectives, and

variable bounds)

Inferences

Material 1

Material 2Predicted Strength

Predicted Strength

Strength (MPa)

[150 175] MPa

[190 225] MPa




Model Refinement Using Value-of-Information




Application of Value of Information in Engineering Design

All models are wrong, some models are useful! [George Box]

Designers care about usefulness of a model. So, the question is:

How much refinement of a simulation model is appropriate for design?




Model Refinement Using Value-of-Information

Value of Information = Improvement in the outcome of a design decision dueto added information

Decision

Design

Alternatives

(e.g., options for

particle sizes)

a1a2a3…

an

Simulation Results for

System Behavior

(y)

(y, ai)

Designers’

Preferences

a0

+ Additional Information

from Model Refinement (d)

ad(yr, ad)Selected

Alternative

Build

& Test

Achieved

Utility

(yr, a0)




References

1 Clemen, R. T. (1996). Making Hard Decisions: An Introduction toDecision Analysis. Belmont, CA, Wadsworth Publishing Company.Chapter 12.

2 Panchal, J. H., Paredis, C. J. J., Allen, J. K., and Mistree, F., 2008, “AValue-of-Information Based Approach to Simulation Model Refinement,”Engineering Optimization, Vol. 40, No. 3, pp. 223 - 251.

3 Panchal, J.H., Paredis, C. J. J., Allen, J. K., and Mistree, F., 2009,“Managing Design-Process Complexity: A Value-of-Information BasedApproach for Scale and Decision Decoupling,” ASME Journal ofComputing and Information Sciences in Engineering, Vol. 9, No. 2,(021005)1-12.


lecture 09 value of information - college of …lecture outline 1 introduction to value of...

Documents