Bootstrap Confidence Intervals for Reservoir

Model Selection Techniques

Céline Scheidt and Jef Caers

SCRF Affiliate Meeting– April 30, 2009

Uncertainty in reservoir modeling is represented through a possibly large set of reservoir models◦ Generated by varying several input parameters

High CPU demand for flow simulations requires the use of model selection techniques ◦ Evaluate uncertainty on a subset of models

Model selection techniques select a subset of representative realizations which should preserve the statistics of the entire set of realizations◦ Eg.: Ranking, Distance-Kernel Method (DKM)

Model Selection Techniques

2SCRF Affiliate Meeting – 04/30/09

If we select N realizations, perform flow simulation, and quantify uncertainty:◦ How do we know if the results are accurate?◦ Can we be confident with the results?◦ Should we do more simulations?

We use of bootstrap methodology to evaluate the accuracy of the uncertainty quantification ◦ Applicable to standard ranking or new distance-kernel

method (DKM)

Goal

3SCRF Affiliate Meeting – 04/30/09

Distance Kernel Method (DKM)

Distance Matrix D1 2 3 4

1 d11 d12 d13 d14

2 d21 d22 d23 d24

3 d31 d32 d33 d34

4 d41 d42 d43 d44

Model 1 Model 2

Model 3 Model 4

d12

d13 d24

d34

d32

d14

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

F

2D projection of Feature Space

-5 -4 -3 -2 -1 0 1 2 3 4

x 104

-3000

-2500

-2000

-1500

-1000

-500

0

500

1000

1500

2000

All realizationsSelected realizations

2D projection of Metric Space

Apply Clustering in F

P10,P50,P90 model selection

-5 -4 -3 -2 -1 0 1 2 3 4

x 104

-3000

-2500

-2000

-1500

-1000

-500

0

500

1000

1500

2000

M

2D projection of Metric Space

MDS

Kernels j

j-1

Pre-image

4SCRF Affiliate Meeting – 04/30/09

SCRF, 2008SPE Journal, 2009

Generate a proxy response for each L realizations (ranking measure)◦ Should be strongly correlated to the actual response

Select N realizations for flow simulations◦ Traditionally, N=3◦ Realizations equally spaced according to the ranking measure

Estimation of the distribution of the response using the N simulations◦ Compute P10, P50 and P90 statistics

Traditional Ranking Techinque

5SCRF Affiliate Meeting – 04/30/09

Review: Parametric Bootstrap – Simple Example

)ˆ,ˆ(ˆ NF

],...,[ **1

* bL

bb xxX

)ˆ̂,ˆ̂(ˆ̂ *** bbb

],...,[ 1 LxxX

2 3 4 5 6 7 80

50

100

150

200

250

300

Bootstrap Mean

Freq

uenc

y

0 5 10 15 20 250

50

100

150

200

250

300

Bootstrap Variance

Freq

uenc

y

SCRF Affiliate Meeting – 04/30/09

),( NF

)ˆ,ˆ(ˆ B bootstrap estimates of the mean and variance

6

?

1st estimate

2nd estimate

b = 1,..,B

b*ˆ̂b*ˆ̂

: Proxy response (ranking measure) Eg. Streamline simulations

: True response Eg. Eclipse simulations

: Selected realizations by model selection

: estimate of P10, P50 and P90 values From ranking or DKM & flow simulation (1st estimate)

: bootstrap estimate of P10, P50 and P90 values

From ranking or DKM & parametric distribution (2nd estimate)No additional flow simulations

Notations

7

Lyy ,,1

Nxx ,,1

**1 ,, Nxx

***

905010ˆ,ˆ,ˆ PPP xxx

bP

bP

bP xxx ***

905010

ˆ̂,ˆ̂,ˆ̂

SCRF Affiliate Meeting – 04/30/09

Proposed Bootstrap Methodology

Model selection

+ flow simulation

**1 ,, Nxx

**1 ,, Nyy

bN

b xx **1 ,,

bN

b yy **1 ,,

Proxy Values

Lyy ,,1

)()(1 ,, b

Lb yy

)()(1 ,, b

Lb xx b

PbP

bP xxx ***

905010

ˆ̂,ˆ̂,ˆ̂

***

905010ˆ,ˆ,ˆ PPP xxx

Application to model selection technique

8

Model selection+ response evaluation

Parametric Bootstrap

Estimation of distribution

Generation of B samples from

NF̂

b = 1,…,B

NF̂

SCRF Affiliate Meeting – 04/30/094 4.5 5 5.5 6 6.5

0

50

100

150

200

250

Bootstrap P50Fr

eque

ncy

Distribution of the target and proxy responses:

Proposed bootstrap technique applied for several correlation scenarios between target and proxy responses◦ Scenarios for:

rxy = 1, 0.9, 0.8, 0.7, 0.6,0.5

Illustration: Bivariate Gaussian Distribution

),(~),( biNYX

r = 0.9

9

yyxy

xyxx

SCRF Affiliate Meeting – 04/30/09

L = 100, r = 0.9 Selection of 15 realizations using DKM Number of bootstrap samples: B = 1000

Histograms of the Estimated Quantiles

1 2 3 4 50

50

100

150

200

250

300

Bootstrap P10

Freq

uenc

y

4 4.5 5 5.5 6 6.50

50

100

150

200

250

Bootstrap P50

Freq

uenc

y

5 6 7 8 9 100

50

100

150

200

250

300

Bootstrap P90

Freq

uenc

y

10

Bootstrap estimated P90Bootstrap estimated P50Bootstrap estimated P10

SCRF Affiliate Meeting – 04/30/09

Estimated P10 Estimated P50 Estimated P90

For each of the B samples, a dimensionless error is defined to evaluate the accuracy of the estimated quantiles:

Definition of the Error on the Bootstrap Estimated Quantiles

11

90

9090

50

5050

10

1010

ˆ

ˆˆ̂

ˆ

ˆˆ̂

ˆ

ˆˆ̂

3

1***

*

P

PbP

P

PbP

P

PbPb

x

xx

x

xx

x

xxerror

Error on bootstrap estimated quantiles:

5 10 15 200

0.05

0.1

0.15

0.2

0.25

0.3

= 1.0

# of function evaluation

Err

or o

n qu

antil

e es

timat

ion

Bivariate Gaussian distribution Confidence Intervals for different correlations scenarios

5 10 15 200

0.05

0.1

0.15

0.2

0.25

0.3

= 0.9

# of function evaluation

Err

or o

n qu

antil

e es

timat

ion

5 10 15 200

0.05

0.1

0.15

0.2

0.25

0.3

= 0.7

# of function evaluation

Err

or o

n qu

antil

e es

timat

ion

02

46

81

0

0 2 4 6 8

10

# of function evaluation

legend

Quantiles estim

ation

KK

MR

ankingR

ando

m

0 2 4 6 8 10

0

2

4

6

8

10

# of function evaluation

lege

nd

Quantiles estimation

KKMRankingRandom

12

5 10 15 200

0.05

0.1

0.15

0.2

0.25

0.3

= 0.8

# of function evaluation

Err

or o

n qu

antil

e es

timat

ion

5 10 15 200

0.05

0.1

0.15

0.2

0.25

0.3

= 0.6

# of function evaluation

Err

or o

n qu

antil

e es

timat

ion

5 10 15 200

0.05

0.1

0.15

0.2

0.25

0.3

= 0.5

# of function evaluationE

rror

on

quan

tile

estim

atio

n

r = 1.0 r = 0.9 r = 0.8

r = 0.5r = 0.6r = 0.7

SCRF Affiliate Meeting – 04/30/09

WCA is a deepwater turbidite offshore reservoir located in a slope valley

Dimensions of the reservoir model◦ 78 x 59 x 116 gridblocks◦ 100,000 active gridblocks

28 wells◦ 20 production wells (red)◦ 8 injection wells (blue)

West Coast African Reservoir (WCA) Courtesy of Chevron

1 mile0.5 m

ile

800 feet13

SCRF Affiliate Meeting – 04/30/09

4 depositional facies◦ Facies 1: Shale (55% of the reservoir)◦ Facies 2: Poor quality sand #1 (debris flows or levees)◦ Facies 3: Poor quality sand #2 (debris flows or levees)◦ Facies 4: Good quality channels (28 %)

West Coast African Reservoir

Porosity for each facies determined by SGS conditioned to well data

Vshale for each facies modeled by SGS correlated to porosity

Permeability calculated analytically from Vshale

14SCRF Affiliate Meeting – 04/30/09

Uncertainty exists for:◦ Depositional environment

Modeled using 12 training images (TI) & snesim◦ Facies proportions

Modeled with 3 different probability cubes Probability cubes come from seismic

2 realizations were generated for each combination of TI and facies probability cube◦ 72 possible realizations of the WCA reservoir

Uncertainty in Reservoir Description

15SCRF Affiliate Meeting – 04/30/09

True response X:◦ Cumulative oil production after 1200 days of production

(evaluated by full flow simulation)

Proxy response Y:◦ Cumulative oil production after 1215 days of production

(evaluated by fast streamline simulation)

Correlation coefficient: (r X,Y) = 0.92

Definition of the Responses

16SCRF Affiliate Meeting – 04/30/09

Parametric bootstrap requires an assumption of the bivariate distribution function ( )◦ Not known a priori in real case (contrary to previous

example)

Use of a smoothing technique to obtain the distribution of the N selected bivariate samples

Definition of Distribution Function

NF̂

NF̂

17SCRF Affiliate Meeting – 04/30/09

True Response (Eclipse)

Pro

xy R

espo

nse

(Str

eam

lines

)

5.5 6 6.5 7 7.5 8

x 104

4.5

5

5.5

6

x 104

True and proxy responses on the N Selected points

Niyx ii ,,1,, **

NF̂

True Response (Chears)

Pro

xy R

esp

on

se (

Str

ea

mlin

es)

5.5 6 6.5 7 7.5 8

x 104

4.5

5

5.5

6

x 104

True Response (Eclipse)

Pro

xy R

espo

nse

(Str

eam

lines

)

5.5 6 6.5 7 7.5 8

x 104

4.5

5

5.5

6

x 104

Sampling to generate new bivariate bootstrap datasets

NF̂

Lyy ,,1

Proxy measure (Streamline)

Flow simulations (Chears)on N selected realizations

)()(1 ,, b

Lb yy

)()(1 ,, b

Lb xx b

Nb xx **

1 ,,bN

b yy **1 ,,

1st Model Selection

to select N real.

**1 ,, Nxx

**1 ,, Nyy

2nd Model Selection

to select N real.

bP

bP

bP xxx ***

905010

ˆ̂,ˆ̂,ˆ̂

Generation of Bootstrap Samples

SCRF Affiliate Meeting – 04/30/09

**1 ,, Nxx

Bivariate response

True Response (Eclipse)

Pro

xy R

espo

nse

(Str

eam

lines

)

5.5 6 6.5 7 7.5 8

x 104

4.5

5

5.5

6

x 104

Smoothing on N selected realizations

NF̂

B times

True Response (Chears)

Pro

xy R

esp

on

se (

Str

ea

mlin

es)

5.5 6 6.5 7 7.5 8

x 104

4.5

5

5.5

6

x 104

True Response (Chears)

Pro

xy R

esp

on

se (

Str

ea

mlin

es)

5.5 6 6.5 7 7.5 8

x 104

4.5

5

5.5

6

x 104

Distance (for DKM only)◦ Difference in proxy response for every pair of

realizations

Comparison between 3 model selection methods:◦ DKM, ranking and random selection

Selection of N realizations: N = 3,5,8,10,15,20◦ The set of selected realizations are different for each N

Number of new bootstrap data sets generated: B = 1000

Application of Bootstrap to WCA

19SCRF Affiliate Meeting – 04/30/09

Bootstrap Estimated P10, P50 and P90 Quantiles

5 10 15 206.6

6.8

7

7.2

7.4

7.6

7.8

8

8.2x 10

4 Quantiles estimation

# of function evaluation

P90

02

46

81

0

0 2 4 6 8

10

# of function evaluation

legend

Quantiles estim

ation

KK

MR

ankingR

ando

m0 2 4 6 8 10

0

2

4

6

8

10

# of function evaluation

lege

nd

Quantiles estimation

KKMRankingRandom

5 10 15 206

6.2

6.4

6.6

6.8

7

7.2

7.4

7.6x 10

4 Quantiles estimation

# of function evaluation

P50

5 10 15 205.5

6

6.5

7

7.5x 10

4 Quantiles estimation

# of function evaluation

P10

20SCRF Affiliate Meeting – 04/30/09

SCRF Affiliate Meeting – 04/30/09

Error on Bootstrap Quantiles Estimations

0 0.02 0.04 0.06 0.08 0.1 0.120

50

100

150

200

250

300

Response Value

Fre

quen

cyN = 5

DKMRankingRandom

0 0.02 0.04 0.06 0.08 0.1 0.120

50

100

150

200

250

300

Response Value

Fre

quen

cy

N = 10

DKMRankingRandom

0 0.02 0.04 0.06 0.08 0.1 0.120

50

100

150

200

250

300

Response Value

Fre

quen

cy

N = 20

DKMRankingRandom

0 0.02 0.04 0.06 0.08 0.1 0.120

50

100

150

200

250

300

Response Value

Fre

quen

cy

N = 15

DKMRankingRandom

21

5 simulations 10 simulations

15 simulations 20 simulations

Error on Bootstrap Quantiles Estimations

5 10 15 200

0.02

0.04

0.06

0.08

0.1Quantiles estimation

# of function evaluation

Err

or o

n qu

antil

e es

timat

ion

02

46

81

0

0 2 4 6 8

10

# of function evaluation

legend

Quantiles estim

ation

KK

MR

ankingR

ando

m

0 2 4 6 8 10

0

2

4

6

8

10

# of function evaluation

lege

nd

Quantiles estimation

KKMRankingRandom

22

N = 8 or 10 simulations should be sufficient to obtain an accurate uncertainty quantification Previous work (SCRF 2008) showed that with 7 simulations, uncertainty

quantification on cumulative oil production was very accurate

SCRF Affiliate Meeting – 04/30/09

Comparison of the Results to the “Truth”

23

N = 3 N = 83 simulations 8 simulations

Conclusion

We have established a workflow to construct confidence intervals for quantile estimations

Workflow uses any model selection technique and parametric bootstrap procedure

DKM provides more robust results and outperforms ranking

The magnitude of the confidence intervals can show if more simulations are required for a better uncertainty quantification◦ Does not suggest how many more, only if sufficiently accurate

24SCRF Affiliate Meeting – 04/30/09