Chapter 13 Time series models with covariates and acase study of polio

Objectives

1 Discuss covariates in POMP models as a generalization of regressionwith ARMA errors

2 Demonstrate the use of covariates in pomp to add demographic data(birth rates and total population) and seasonality to anepidemiological model

3 Present a case study developing and fitting a POMP model withcovariates

1 58

Covariates in time series analysis

Suppose our time series of primary interest is y1N

A covariate time series is an additional time series z1N which is usedto help explain y1N

When we talk about covariates it is often implicit that we think ofz1N as a measure of an external forcing to the system producingy1N This means that the process generating the data z1N affectsthe process generating y1N but not vice versa

For example the weather might affect human health but humanhealth has negligible effect on weather weather is an external forcingto human health processes

When we make an assumption of external forcing we should try tomake it explicit

In regression analysis we usually condition on covariatesEquivalently we model them as fixed numbers rather than modelingthem as the outcome of random variables

2 58

When the process leading to z1N is not external to the systemgenerating it we must be alert to the possibility of reversecausation and confounding variables

Issues involved in inferring causation from fitting statistical models areessentially the same whether the model is linear and Gaussian or not

Any experience you have with causal interpretation of linearregression coefficients also applies to POMP models

3 58

Covariates in linear time series analysis

The main tool we have seen previously for investigating dependenceon covariates is regression with ARMA errors

This tool can also be used to identify lag relationships where yndepends on znminusL

Another way to investigate associations at different lags is bycomputing the sample correlation between yn and znminusL forn isin L+ 1 N and plotting this against L

This is called the cross-correlation function and can be computedwith the R function ccf

An example of the use of the cross-correlation function in a midtermproject The Association between Recent Cholera Epidemics andRainfall in Haiti (httpsionidesgithubio531w16midterm_projectproject3midterm_projecthtml)

4 58

When the process leading to z1N is not external to the systemgenerating it we must be alert to the possibility of reversecausation and confounding variables

Issues involved in inferring causation from fitting statistical models areessentially the same whether the model is linear and Gaussian or not

Any experience you have with causal interpretation of linearregression coefficients also applies to POMP models

3 58

Covariates in linear time series analysis

The main tool we have seen previously for investigating dependenceon covariates is regression with ARMA errors

This tool can also be used to identify lag relationships where yndepends on znminusL

Another way to investigate associations at different lags is bycomputing the sample correlation between yn and znminusL forn isin L+ 1 N and plotting this against L

This is called the cross-correlation function and can be computedwith the R function ccf

An example of the use of the cross-correlation function in a midtermproject The Association between Recent Cholera Epidemics andRainfall in Haiti (httpsionidesgithubio531w16midterm_projectproject3midterm_projecthtml)

4 58

Covariates in linear time series analysis

The main tool we have seen previously for investigating dependenceon covariates is regression with ARMA errors

This tool can also be used to identify lag relationships where yndepends on znminusL

Another way to investigate associations at different lags is bycomputing the sample correlation between yn and znminusL forn isin L+ 1 N and plotting this against L

This is called the cross-correlation function and can be computedwith the R function ccf

An example of the use of the cross-correlation function in a midtermproject The Association between Recent Cholera Epidemics andRainfall in Haiti (httpsionidesgithubio531w16midterm_projectproject3midterm_projecthtml)

4 58

Covariates in nonlinear POMP models

The general POMP modeling framework allows essentially arbitrarymodeling of covariates

Scientific considerations may suggest sensible ways to model therelationship

In an epidemiological model for malaria rainfall might affect thenumber of mosquitoes (and hence the disease transmission rate) butnot the duration of infection

In an economic model geopolitical shocks to the oil supply mighthave direct influence on energy prices secondary direct effects oninflation and investment and indirect consequences forunemployment

In a hydrology model precipitation is a covariate explaining river flowbut the exact nature of the relationship is a question of interest

5 58

Including covariates in the general POMP framework

Recall that a POMP model is specified by defining the following

fX0(x0 θ)fXn|Xnminus1

(xn |xnminus1 θ)

fYn|Xn(yn |xn θ)

for n = 1 N

The possibility of a general dependence on n includes the possibilitythat there is some covariate time series z0N such that

fX0(x0 θ) = fX0(x0 θ z0)fXn|Xnminus1

(xn |xnminus1 θ) = fXn|Xnminus1(xn |xnminus1 θ zn)

fYn|Xn(yn |xn θ) = fYn|Xn

(yn |xn θ zn)

for n = 1 N

6 58

Seasonality in a POMP model

One specific choice of covariates is to construct z0N so that itfluctuates periodically once per year This allows seasonality enterthe POMP model in whatever way is appropriate for the system underinvestigation

All that remains is to hypothesize what is a reasonable way to includecovariates for your system and to fit the resulting model

Now we can evaluate and maximize the log likelihood we canconstruct AIC or likelihood ratio tests to see if the covariate helpsdescribe the data

This also lets us compare alternative ways the covariates might enterthe process model andor the measurement model

7 58

Covariates in the pomp package

pomp provides facilities for including covariates in a pomp object

Named covariate time series entered via the covar argument to pomp

are automatically defined within Csnippets used for the rprocessdprocess rmeasure dmeasure and rinit arguments

We see this in practice in the following epidemiological model whichhas population census birth data and seasonality as covariates

8 58

Including covariates in the general POMP framework

Recall that a POMP model is specified by defining the following

fX0(x0 θ)fXn|Xnminus1

(xn |xnminus1 θ)

fYn|Xn(yn |xn θ)

for n = 1 N

The possibility of a general dependence on n includes the possibilitythat there is some covariate time series z0N such that

fX0(x0 θ) = fX0(x0 θ z0)fXn|Xnminus1

(xn |xnminus1 θ) = fXn|Xnminus1(xn |xnminus1 θ zn)

fYn|Xn(yn |xn θ) = fYn|Xn

(yn |xn θ zn)

for n = 1 N

6 58

Seasonality in a POMP model

One specific choice of covariates is to construct z0N so that itfluctuates periodically once per year This allows seasonality enterthe POMP model in whatever way is appropriate for the system underinvestigation

All that remains is to hypothesize what is a reasonable way to includecovariates for your system and to fit the resulting model

Now we can evaluate and maximize the log likelihood we canconstruct AIC or likelihood ratio tests to see if the covariate helpsdescribe the data

This also lets us compare alternative ways the covariates might enterthe process model andor the measurement model

7 58

Covariates in the pomp package

pomp provides facilities for including covariates in a pomp object

Named covariate time series entered via the covar argument to pomp

are automatically defined within Csnippets used for the rprocessdprocess rmeasure dmeasure and rinit arguments

We see this in practice in the following epidemiological model whichhas population census birth data and seasonality as covariates

8 58

Seasonality in a POMP model

One specific choice of covariates is to construct z0N so that itfluctuates periodically once per year This allows seasonality enterthe POMP model in whatever way is appropriate for the system underinvestigation

All that remains is to hypothesize what is a reasonable way to includecovariates for your system and to fit the resulting model

Now we can evaluate and maximize the log likelihood we canconstruct AIC or likelihood ratio tests to see if the covariate helpsdescribe the data

This also lets us compare alternative ways the covariates might enterthe process model andor the measurement model

7 58

Covariates in the pomp package

pomp provides facilities for including covariates in a pomp object

Named covariate time series entered via the covar argument to pomp

are automatically defined within Csnippets used for the rprocessdprocess rmeasure dmeasure and rinit arguments

We see this in practice in the following epidemiological model whichhas population census birth data and seasonality as covariates

8 58

Covariates in the pomp package

pomp provides facilities for including covariates in a pomp object

Named covariate time series entered via the covar argument to pomp

are automatically defined within Csnippets used for the rprocessdprocess rmeasure dmeasure and rinit arguments

We see this in practice in the following epidemiological model whichhas population census birth data and seasonality as covariates

8 58

Case study polio in Wisconsin

The massive global polio eradication initiative (GPEI) has broughtpolio from a major global disease to the brink of extinction

Finishing this task is proving hard and improved understanding polioecology might assist

Martinez-Bakker et al (2015) investigated this using extensive statelevel pre-vaccination era data in USA

We will follow the approach of Martinez-Bakker et al (2015) for onestate (Wisconsin) In the context of their model we can quantifyseasonality of transmission the role of the birth rate in explaining thetransmission dynamics and the persistence mechanism of polio

Martinez-Bakker et al (2015) carrried out this analysis for all 48contigous states and District of Columbia and their data and codeare publicly available The data we study in polio_wisconsincsvconsist of cases the monthly reported polio cases births themonthly recorded births pop the annual census time date in years

9 58

polio_data lt- readcsv(polio_wisconsincsvcomment=)

head(polio_data4)

time cases births pop

1 1931083 7 4698 2990000

2 1931167 0 4354 2990000

3 1931250 7 4836 2990000

4 1931333 3 4468 2990000

10 58

11 58

We use the compartment model of Martinez-Bakker et al (2015)

Compartments representing susceptible babies in each of sixone-month birth cohorts (SB

1 SB6 ) susceptible older individuals

(SO) infected babies (IB) infected older individuals (IO) andrecovered with lifelong immunity (R)

The state vector of the disease transmission model consists ofnumbers of individuals in each compartment at each time

X(t) =(SB1 (t) SB

6 (t) IB(t) IO(t) R(t))

Babies under six months are modeled as fully protected fromsymptomatic poliomyelitis

Older infections lead to reported cases (usually paralysis) at a rate ρ

The flows through the compartments are graphically represented onthe following slide (Figure 1A of Martinez-Bakker et al (2015))

12 58

SBk suscepbible babies k monthsIB infected babiesSO susceptible older peopleIO infected older people

13 58

Setting up the model

Duration of infection is comparable to the one-month reportingaggregation so a discrete time model may be appropriateMartinez-Bakker et al (2015) fitted monthly reported cases May1932 through January 1953 so we set tn = 1932 + (4 + n)12 and

Xn = X(tn) =(SB1n S

B6n I

Bn I

On Rn

)

The mean force of infection in units of yrminus1 is modeled as

λn =

(βnIOn + IBnPn

+ ψ

)where Pn is census population interpolated to time tn and seasonalityof transmission is modeled as

βn = exp

Ksumk=1

bkξk(tn)

with ξk(t) k = 1 K being a periodic B-spline basis withK = 6Pn and ξk(tn) are covariate time series 14 58

The force of infection has a stochastic perturbation

λn = λnεn

where εn is a Gamma random variable with mean 1 and varianceσ2env + σ2dem

λn These two terms capture variation on the

environmental and demographic scales respectively Allcompartments suffer a mortality rate set at δ = 160yrminus1

Within each month all susceptible individuals are modeled as havingexposure to constant competing hazards of mortality and polioinfection The chance of remaining in the susceptible populationwhen exposed to these hazards for one month is therefore

pn = expminus (δ + λn)12

with the chance of polio infection being

qn = (1minus pn)λn

(λn + δ)

15 58

We employ a continuous population model with no demographicstochasticity Writing Bn for births in month n we obtain thedynamic model of Martinez-Bakker et al (2015)

SB1n+1 = Bn+1

SBkn+1 = pnS

Bkminus1n for k = 2 6

SOn+1 = pn(SO

n + SB6n)

IBn+1 = qnsum6

k=1 SBkn

IOn+1 = qnSOn

16 58

The measurement model

The model for the reported observations conditional on the state is adiscretized normal distribution truncated at zero with bothenvironmental and Poisson-scale contributions to the variance

Yn = maxround(Zn) 0 Zn sim normal(ρIOn

(τIOn

)2+ ρIOn

)

17 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

11 58

We use the compartment model of Martinez-Bakker et al (2015)

Compartments representing susceptible babies in each of sixone-month birth cohorts (SB

1 SB6 ) susceptible older individuals

(SO) infected babies (IB) infected older individuals (IO) andrecovered with lifelong immunity (R)

The state vector of the disease transmission model consists ofnumbers of individuals in each compartment at each time

X(t) =(SB1 (t) SB

6 (t) IB(t) IO(t) R(t))

Babies under six months are modeled as fully protected fromsymptomatic poliomyelitis

Older infections lead to reported cases (usually paralysis) at a rate ρ

The flows through the compartments are graphically represented onthe following slide (Figure 1A of Martinez-Bakker et al (2015))

12 58

SBk suscepbible babies k monthsIB infected babiesSO susceptible older peopleIO infected older people

13 58

Setting up the model

Duration of infection is comparable to the one-month reportingaggregation so a discrete time model may be appropriateMartinez-Bakker et al (2015) fitted monthly reported cases May1932 through January 1953 so we set tn = 1932 + (4 + n)12 and

Xn = X(tn) =(SB1n S

B6n I

Bn I

On Rn

)

The mean force of infection in units of yrminus1 is modeled as

λn =

(βnIOn + IBnPn

+ ψ

)where Pn is census population interpolated to time tn and seasonalityof transmission is modeled as

βn = exp

Ksumk=1

bkξk(tn)

with ξk(t) k = 1 K being a periodic B-spline basis withK = 6Pn and ξk(tn) are covariate time series 14 58

The force of infection has a stochastic perturbation

λn = λnεn

where εn is a Gamma random variable with mean 1 and varianceσ2env + σ2dem

λn These two terms capture variation on the

environmental and demographic scales respectively Allcompartments suffer a mortality rate set at δ = 160yrminus1

Within each month all susceptible individuals are modeled as havingexposure to constant competing hazards of mortality and polioinfection The chance of remaining in the susceptible populationwhen exposed to these hazards for one month is therefore

pn = expminus (δ + λn)12

with the chance of polio infection being

qn = (1minus pn)λn

(λn + δ)

15 58

We employ a continuous population model with no demographicstochasticity Writing Bn for births in month n we obtain thedynamic model of Martinez-Bakker et al (2015)

SB1n+1 = Bn+1

SBkn+1 = pnS

Bkminus1n for k = 2 6

SOn+1 = pn(SO

n + SB6n)

IBn+1 = qnsum6

k=1 SBkn

IOn+1 = qnSOn

16 58

The measurement model

The model for the reported observations conditional on the state is adiscretized normal distribution truncated at zero with bothenvironmental and Poisson-scale contributions to the variance

Yn = maxround(Zn) 0 Zn sim normal(ρIOn

(τIOn

)2+ ρIOn

)

17 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

We use the compartment model of Martinez-Bakker et al (2015)

Compartments representing susceptible babies in each of sixone-month birth cohorts (SB

1 SB6 ) susceptible older individuals

(SO) infected babies (IB) infected older individuals (IO) andrecovered with lifelong immunity (R)

The state vector of the disease transmission model consists ofnumbers of individuals in each compartment at each time

X(t) =(SB1 (t) SB

6 (t) IB(t) IO(t) R(t))

Babies under six months are modeled as fully protected fromsymptomatic poliomyelitis

Older infections lead to reported cases (usually paralysis) at a rate ρ

The flows through the compartments are graphically represented onthe following slide (Figure 1A of Martinez-Bakker et al (2015))

12 58

SBk suscepbible babies k monthsIB infected babiesSO susceptible older peopleIO infected older people

13 58

Setting up the model

Duration of infection is comparable to the one-month reportingaggregation so a discrete time model may be appropriateMartinez-Bakker et al (2015) fitted monthly reported cases May1932 through January 1953 so we set tn = 1932 + (4 + n)12 and

Xn = X(tn) =(SB1n S

B6n I

Bn I

On Rn

)

The mean force of infection in units of yrminus1 is modeled as

λn =

(βnIOn + IBnPn

+ ψ

)where Pn is census population interpolated to time tn and seasonalityof transmission is modeled as

βn = exp

Ksumk=1

bkξk(tn)

with ξk(t) k = 1 K being a periodic B-spline basis withK = 6Pn and ξk(tn) are covariate time series 14 58

The force of infection has a stochastic perturbation

λn = λnεn

where εn is a Gamma random variable with mean 1 and varianceσ2env + σ2dem

λn These two terms capture variation on the

environmental and demographic scales respectively Allcompartments suffer a mortality rate set at δ = 160yrminus1

Within each month all susceptible individuals are modeled as havingexposure to constant competing hazards of mortality and polioinfection The chance of remaining in the susceptible populationwhen exposed to these hazards for one month is therefore

pn = expminus (δ + λn)12

with the chance of polio infection being

qn = (1minus pn)λn

(λn + δ)

15 58

We employ a continuous population model with no demographicstochasticity Writing Bn for births in month n we obtain thedynamic model of Martinez-Bakker et al (2015)

SB1n+1 = Bn+1

SBkn+1 = pnS

Bkminus1n for k = 2 6

SOn+1 = pn(SO

n + SB6n)

IBn+1 = qnsum6

k=1 SBkn

IOn+1 = qnSOn

16 58

The measurement model

The model for the reported observations conditional on the state is adiscretized normal distribution truncated at zero with bothenvironmental and Poisson-scale contributions to the variance

Yn = maxround(Zn) 0 Zn sim normal(ρIOn

(τIOn

)2+ ρIOn

)

17 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

SBk suscepbible babies k monthsIB infected babiesSO susceptible older peopleIO infected older people

13 58

Setting up the model

Duration of infection is comparable to the one-month reportingaggregation so a discrete time model may be appropriateMartinez-Bakker et al (2015) fitted monthly reported cases May1932 through January 1953 so we set tn = 1932 + (4 + n)12 and

Xn = X(tn) =(SB1n S

B6n I

Bn I

On Rn

)

The mean force of infection in units of yrminus1 is modeled as

λn =

(βnIOn + IBnPn

+ ψ

)where Pn is census population interpolated to time tn and seasonalityof transmission is modeled as

βn = exp

Ksumk=1

bkξk(tn)

with ξk(t) k = 1 K being a periodic B-spline basis withK = 6Pn and ξk(tn) are covariate time series 14 58

The force of infection has a stochastic perturbation

λn = λnεn

where εn is a Gamma random variable with mean 1 and varianceσ2env + σ2dem

λn These two terms capture variation on the

environmental and demographic scales respectively Allcompartments suffer a mortality rate set at δ = 160yrminus1

Within each month all susceptible individuals are modeled as havingexposure to constant competing hazards of mortality and polioinfection The chance of remaining in the susceptible populationwhen exposed to these hazards for one month is therefore

pn = expminus (δ + λn)12

with the chance of polio infection being

qn = (1minus pn)λn

(λn + δ)

15 58

We employ a continuous population model with no demographicstochasticity Writing Bn for births in month n we obtain thedynamic model of Martinez-Bakker et al (2015)

SB1n+1 = Bn+1

SBkn+1 = pnS

Bkminus1n for k = 2 6

SOn+1 = pn(SO

n + SB6n)

IBn+1 = qnsum6

k=1 SBkn

IOn+1 = qnSOn

16 58

The measurement model

The model for the reported observations conditional on the state is adiscretized normal distribution truncated at zero with bothenvironmental and Poisson-scale contributions to the variance

Yn = maxround(Zn) 0 Zn sim normal(ρIOn

(τIOn

)2+ ρIOn

)

17 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

Setting up the model

Duration of infection is comparable to the one-month reportingaggregation so a discrete time model may be appropriateMartinez-Bakker et al (2015) fitted monthly reported cases May1932 through January 1953 so we set tn = 1932 + (4 + n)12 and

Xn = X(tn) =(SB1n S

B6n I

Bn I

On Rn

)

The mean force of infection in units of yrminus1 is modeled as

λn =

(βnIOn + IBnPn

+ ψ

)where Pn is census population interpolated to time tn and seasonalityof transmission is modeled as

βn = exp

Ksumk=1

bkξk(tn)

with ξk(t) k = 1 K being a periodic B-spline basis withK = 6Pn and ξk(tn) are covariate time series 14 58

The force of infection has a stochastic perturbation

λn = λnεn

where εn is a Gamma random variable with mean 1 and varianceσ2env + σ2dem

λn These two terms capture variation on the

environmental and demographic scales respectively Allcompartments suffer a mortality rate set at δ = 160yrminus1

Within each month all susceptible individuals are modeled as havingexposure to constant competing hazards of mortality and polioinfection The chance of remaining in the susceptible populationwhen exposed to these hazards for one month is therefore

pn = expminus (δ + λn)12

with the chance of polio infection being

qn = (1minus pn)λn

(λn + δ)

15 58

We employ a continuous population model with no demographicstochasticity Writing Bn for births in month n we obtain thedynamic model of Martinez-Bakker et al (2015)

SB1n+1 = Bn+1

SBkn+1 = pnS

Bkminus1n for k = 2 6

SOn+1 = pn(SO

n + SB6n)

IBn+1 = qnsum6

k=1 SBkn

IOn+1 = qnSOn

16 58

The measurement model

The model for the reported observations conditional on the state is adiscretized normal distribution truncated at zero with bothenvironmental and Poisson-scale contributions to the variance

Yn = maxround(Zn) 0 Zn sim normal(ρIOn

(τIOn

)2+ ρIOn

)

17 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

The force of infection has a stochastic perturbation

λn = λnεn

where εn is a Gamma random variable with mean 1 and varianceσ2env + σ2dem

λn These two terms capture variation on the

environmental and demographic scales respectively Allcompartments suffer a mortality rate set at δ = 160yrminus1

Within each month all susceptible individuals are modeled as havingexposure to constant competing hazards of mortality and polioinfection The chance of remaining in the susceptible populationwhen exposed to these hazards for one month is therefore

pn = expminus (δ + λn)12

with the chance of polio infection being

qn = (1minus pn)λn

(λn + δ)

15 58

We employ a continuous population model with no demographicstochasticity Writing Bn for births in month n we obtain thedynamic model of Martinez-Bakker et al (2015)

SB1n+1 = Bn+1

SBkn+1 = pnS

Bkminus1n for k = 2 6

SOn+1 = pn(SO

n + SB6n)

IBn+1 = qnsum6

k=1 SBkn

IOn+1 = qnSOn

16 58

The measurement model

The model for the reported observations conditional on the state is adiscretized normal distribution truncated at zero with bothenvironmental and Poisson-scale contributions to the variance

Yn = maxround(Zn) 0 Zn sim normal(ρIOn

(τIOn

)2+ ρIOn

)

17 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

We employ a continuous population model with no demographicstochasticity Writing Bn for births in month n we obtain thedynamic model of Martinez-Bakker et al (2015)

SB1n+1 = Bn+1

SBkn+1 = pnS

Bkminus1n for k = 2 6

SOn+1 = pn(SO

n + SB6n)

IBn+1 = qnsum6

k=1 SBkn

IOn+1 = qnSOn

16 58

The measurement model

The model for the reported observations conditional on the state is adiscretized normal distribution truncated at zero with bothenvironmental and Poisson-scale contributions to the variance

Yn = maxround(Zn) 0 Zn sim normal(ρIOn

(τIOn

)2+ ρIOn

)

17 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

The measurement model

The model for the reported observations conditional on the state is adiscretized normal distribution truncated at zero with bothenvironmental and Poisson-scale contributions to the variance

Yn = maxround(Zn) 0 Zn sim normal(ρIOn

(τIOn

)2+ ρIOn

)

17 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

Initial conditions

Additional parameters are used to specify initial state values at timet0 = 1932 + 412We will suppose there are parameters

(SB10 S

B60 I

B0 I

O0 S

O0

)that

specify the population in each compartment at time t0 via

SB10 = SB

10 SB60 = SB

60 IB0 = P0IB0 SO

0 = P0SO0 IO0 = P0I

O0

Following Martinez-Bakker et al (2015) we make an approximationfor the initial conditions of ignoring infant infections at time t0 Thuswe set IB0 = 0 and use monthly births in the preceding months(ignoring infant mortality) to fix SB

k0 = B1minusk for k = 1 6 The

estimated initial conditions are then defined by the two parameters IO0and SO

0 since the initial recovered population R0 is specified bysubtraction of all the other compartments from the total initialpopulation P0It is convenient to parameterize the estimated initial states asfractions of the population whereas the initial states fixed at birthsare parameterized directly as a count

18 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

Building a pomp object for the polio model

We code the state and observation variables and the choice of t0 as

polio_statenames lt- c(SB1SB2SB3SB4SB5SB6

IBSOIO)

polio_obsnames lt- cases

polio_t0 lt- 1932+412

We do not explictly code R since it is defined implicitly as the totalpopulation minus the sum of the other compartments Due to lifelongimmunity individuals in R play no role in the dynamics Evenoccasional negative values of R (due to a discrepancy between thecensus and the mortality model) would not be a fatal flaw

19 58

Setting up the covariate table

time gives the time at which the covariates are defined

P is a smoothed interpolation of the annual census

B is monthly births

xi1xi6 is a periodic B-spline basis

polio_K lt- 6

polio_covar lt- covariate_table(

t=polio_data$time

B=polio_data$births

P=predict(smoothspline(x=19311954

y=polio_data$pop[seq(122412by=12)]))$y

periodicbsplinebasis(tnbasis=polio_K

degree=3period=1names=xid)

times=t

)

20 58

Regular parameters and initial value parameters

The parameters b1 bK ψ ρ τ σdem σenv in the model above areregular parameters (RPs) coded as

polio_rp_names lt- c(b1b2b3b4b5b6

psirhotausigma_demsigma_env)

The initial value parameters (IVPs) IO0 and SO0 are coded for each

state named by adding _0 to the state name

polio_ivp_names lt- c(SO_0IO_0)

polio_paramnames lt- c(polio_rp_namespolio_ivp_names)

21 58

Fixed parameters (FPs)

Two quantities in the dynamic model specification δ = 160yrminus1 andK = 6 are not estimated

Six other initial value quantities SB10 S

B60 are treated as fixed

Fixed quantities could be coded as constants using the globals

argument of pomp but here we pass them as fixed parameters (FPs)

polio_fp_names lt- c(deltaK

SB1_0SB2_0SB3_0SB4_0SB5_0SB6_0)

polio_paramnames lt- c(polio_rp_names

polio_ivp_namespolio_fp_names)

covar_index_t0 lt- which(abs(polio_covartimes-polio_t0)lt001)

polio_initial_births lt- polio_covartable[Bcovar_index_t0-05]

names(polio_initial_births) lt- c(SB1_0SB2_0

SB3_0SB4_0SB5_0SB6_0)

polio_fixed_params lt- c(delta=160K=polio_K

polio_initial_births)

22 58

A starting value for the parameters

We have to start somewhere for our search in parameter space

The following parameter vector is based on informal modelexploration and prior research

polio_params_guess lt- c(b1=3b2=0b3=15b4=6b5=5b6=3

psi=0002rho=001tau=0001sigma_dem=004sigma_env=05

SO_0=012IO_0=0001polio_fixed_params)

23 58

polio_rprocess lt- Csnippet(

double beta = exp(dot_product( (int) K ampxi1 ampb1))

double lambda = (beta (IO+IB) P + psi)

double var_epsilon = pow(sigma_dem2) lambda +

pow(sigma_env2)

lambda = (var_epsilon lt 10e-6) 1

rgamma(1var_epsilonvar_epsilon)

double p = exp(- (delta+lambda)12)

double q = (1-p)lambda(delta+lambda)

SB1 = B

SB2= SB1p

SB3=SB2p

SB4=SB3p

SB5=SB4p

SB6=SB5p

SO= (SB6+SO)p

IB=(SB1+SB2+SB3+SB4+SB5+SB6)q

IO=SOq

)

24 58

polio_dmeasure lt- Csnippet(

double tol = 10e-25

double mean_cases = rhoIO

double sd_cases = sqrt(pow(tauIO2) + mean_cases)

if(cases gt 00)lik = pnorm(cases+05mean_casessd_cases10)

- pnorm(cases-05mean_casessd_cases10) + tol

elselik = pnorm(cases+05mean_casessd_cases10) + tol

if (give_log) lik = log(lik)

)

polio_rmeasure lt- Csnippet(

cases = rnorm(rhoIO sqrt( pow(tauIO2) + rhoIO ) )

if (cases gt 00) cases = nearbyint(cases)

else cases = 00

)

25 58

The map from the initial value parameters to the initial value of the statesat time t0 is coded by the rinit function

polio_rinit lt- Csnippet(

SB1 = SB1_0

SB2 = SB2_0

SB3 = SB3_0

SB4 = SB4_0

SB5 = SB5_0

SB6 = SB6_0

IB = 0

IO = IO_0 P

SO = SO_0 P

)

26 58

polio_rprocess lt- Csnippet(

double beta = exp(dot_product( (int) K ampxi1 ampb1))

double lambda = (beta (IO+IB) P + psi)

double var_epsilon = pow(sigma_dem2) lambda +

pow(sigma_env2)

lambda = (var_epsilon lt 10e-6) 1

rgamma(1var_epsilonvar_epsilon)

double p = exp(- (delta+lambda)12)

double q = (1-p)lambda(delta+lambda)

SB1 = B

SB2= SB1p

SB3=SB2p

SB4=SB3p

SB5=SB4p

SB6=SB5p

SO= (SB6+SO)p

IB=(SB1+SB2+SB3+SB4+SB5+SB6)q

IO=SOq

)

24 58

polio_dmeasure lt- Csnippet(

double tol = 10e-25

double mean_cases = rhoIO

double sd_cases = sqrt(pow(tauIO2) + mean_cases)

if(cases gt 00)lik = pnorm(cases+05mean_casessd_cases10)

- pnorm(cases-05mean_casessd_cases10) + tol

elselik = pnorm(cases+05mean_casessd_cases10) + tol

if (give_log) lik = log(lik)

)

polio_rmeasure lt- Csnippet(

cases = rnorm(rhoIO sqrt( pow(tauIO2) + rhoIO ) )

if (cases gt 00) cases = nearbyint(cases)

else cases = 00

)

25 58

The map from the initial value parameters to the initial value of the statesat time t0 is coded by the rinit function

polio_rinit lt- Csnippet(

SB1 = SB1_0

SB2 = SB2_0

SB3 = SB3_0

SB4 = SB4_0

SB5 = SB5_0

SB6 = SB6_0

IB = 0

IO = IO_0 P

SO = SO_0 P

)

26 58

polio_dmeasure lt- Csnippet(

double tol = 10e-25

double mean_cases = rhoIO

double sd_cases = sqrt(pow(tauIO2) + mean_cases)

if(cases gt 00)lik = pnorm(cases+05mean_casessd_cases10)

- pnorm(cases-05mean_casessd_cases10) + tol

elselik = pnorm(cases+05mean_casessd_cases10) + tol

if (give_log) lik = log(lik)

)

polio_rmeasure lt- Csnippet(

cases = rnorm(rhoIO sqrt( pow(tauIO2) + rhoIO ) )

if (cases gt 00) cases = nearbyint(cases)

else cases = 00

)

25 58

The map from the initial value parameters to the initial value of the statesat time t0 is coded by the rinit function

polio_rinit lt- Csnippet(

SB1 = SB1_0

SB2 = SB2_0

SB3 = SB3_0

SB4 = SB4_0

SB5 = SB5_0

SB6 = SB6_0

IB = 0

IO = IO_0 P

SO = SO_0 P

)

26 58

The map from the initial value parameters to the initial value of the statesat time t0 is coded by the rinit function

polio_rinit lt- Csnippet(

SB1 = SB1_0

SB2 = SB2_0

SB3 = SB3_0

SB4 = SB4_0

SB5 = SB5_0

SB6 = SB6_0

IB = 0

IO = IO_0 P

SO = SO_0 P

)

26 58

Parameter transformations

For parameter estimation it is helpful to have transformations thatmap each parameter into the whole real line and which putuncertainty close to a unit scale

polio_partrans lt- parameter_trans(

log=c(psirhotausigma_demsigma_env)

logit=c(SO_0IO_0)

)

Since the seasonal splines are exponentiated the beta parameters arealready defined on the real line with unit scale uncertainty

27 58

We now put these pieces together into a pomp object

polio lt- pomp(

data=subset(polio_data

(time gt polio_t0 + 001) amp (time lt 1953+112+001)

select=c(casestime))

times=time

t0=polio_t0

params=polio_params_guess

rprocess = euler(stepfun = polio_rprocess deltat=112)

rmeasure= polio_rmeasure

dmeasure = polio_dmeasure

covar=polio_covar

obsnames = polio_obsnames

statenames = polio_statenames

paramnames = polio_paramnames

rinit=polio_rinit

partrans=polio_partrans

)

plot(polio)

28 58

Setting run levels to control computation time

run_level=1 will set all the algorithmic parameters to the firstcolumn of values in the following code for debugging

Here Np is the number of particles and Nmif is the number ofiterations of the optimization procedure carried

run_level=2 uses Np=1000 and Nmif=100 enough effort in thiscase to gives reasonably stable results for a moderate amount ofcomputational time

Larger values give more refined computations implemented here byrun_level=3 which was run on a computing node

run_level=3

polio_Np lt- switch(run_level100 1e3 5e3)

polio_Nmif lt- switch(run_level 10 100 200)

polio_Nreps_eval lt- switch(run_level 2 10 20)

polio_Nreps_local lt- switch(run_level 10 20 40)

polio_Nreps_global lt-switch(run_level 10 20 100)

polio_Nsim lt- switch(run_level 50 100 500)

29 58

Setting run levels to control computation time

run_level=1 will set all the algorithmic parameters to the firstcolumn of values in the following code for debugging

Here Np is the number of particles and Nmif is the number ofiterations of the optimization procedure carried

run_level=2 uses Np=1000 and Nmif=100 enough effort in thiscase to gives reasonably stable results for a moderate amount ofcomputational time

Larger values give more refined computations implemented here byrun_level=3 which was run on a computing node

run_level=3

polio_Np lt- switch(run_level100 1e3 5e3)

polio_Nmif lt- switch(run_level 10 100 200)

polio_Nreps_eval lt- switch(run_level 2 10 20)

polio_Nreps_local lt- switch(run_level 10 20 40)

polio_Nreps_global lt-switch(run_level 10 20 100)

polio_Nsim lt- switch(run_level 50 100 500)

29 58

Comments on setting algorithmic parameters

run_level is a facility that is convenient for when you are editing thesource code It plays no fundamental role in the final results If youare not editing the source code or using the code as a template fordeveloping your own analysis it has no functionWhen you edit a document with different run_level options youcan debug your code by editing run_level=1 Then you can getpreliminary assessment of whether your results are sensible withrun_level=2 and get finalized results with reduced Monte Carloerror by editing run_level=3In practice you probably want run_level=1 to run in minutesrun_level=2 to run in tens of minutes and run_level=3 to run inhoursYou can increase or decrease the numbers of particles or the numberof mif2 iterations or the number of global searches carried out tomake sure this procedure is practical on your machineAppropriate values of the algorithmic parameters for each run-levelare context dependent

30 58

Question 131 Choosing algorithmic parametersDiscuss how you choose the algorithmic parameters for each run levelwhen building a new likelihood-based data analysis using pfilter() andmif2() in pomp

31 58

Likelihood evaluation at the starting parameter estimate

require(doParallel)

registerDoParallel()

stew(file=sprintf(pf1-drdarun_level)t1 lt- systemtime(

pf1 lt- foreach(i=120packages=pomp) dopar pfilter(

polioNp=polio_Np)

)

seed=493536993kind=LEcuyer)L1 lt- logmeanexp(sapply(pf1logLik)se=TRUE)

In 25 seconds we obtain a log likelihood estimate of -81682 with aMonte standard error of 018

32 58

Simulation to investigate local persistence

The scientific purpose of fitting a model typically involves analyzingproperties of the fitted model often investigated using simulation

Following Martinez-Bakker et al (2015) we are interested in howoften months with no reported cases (Yn = 0) correspond to monthswithout any local asymptomatic cases defined for our continuousstate model as IBn + IOn lt 12

For Wisconsin using our model at the estimated MLE we simulate inparallel as follows

stew(sprintf(persistence-drdarun_level)t_sim lt- systemtime(

sim lt- foreach(i=1polio_Nsim

packages=pomp) dopar simulate(polio)

)

seed=493536993kind=LEcuyer)

33 58

no_cases_data lt- sum(obs(polio)==0)

no_cases_sim lt- sum(sapply(simobs)==0)length(sim)

fadeout1_sim lt- sum(sapply(simfunction(po)states(po)[IB]

+states(po)[IO]lt1))length(sim)

fadeout100_sim lt- sum(sapply(simfunction(po)states(po)[IB]

+states(po)[IO]lt100))length(sim)

imports_sim lt- coef(polio)[psi]mean(sapply(sim

function(po) mean(states(po)[SO]+states(po)[SB1]

+states(po)[SB2]+states(po)[SB3]+states(po)[SB4]

+states(po)[SB5]+states(po)[SB6])))12

For the data there were 26 months with no reported cases similar to themean of 513 for simulations from the fitted model Months with noasyptomatic infections for the simulations were rare on average 08months per simulation Months with fewer than 100 infections averaged638 per simulation which in the context of a reporting rate of 001 canexplain the absences of case reports The mean monthly infections due toimportations modeled by ψ is 1175 This does not give muchopportunity for local elimination of poliovirus A profile over ψ would showhow sensitive this conclusion is to values of ψ consistent with the data

34 58

It is also good practice to look at simulations from the fitted model

mle_simulation lt- simulate(polioseed=127)

plot(mle_simulation)

35 58

We see from this simulation that the fitted model can generate reporthistories that look qualitatively similar to the data However there arethings to notice in the reconstructed latent states Specifically thepool of older susceptibles SO(t) is mostly increasing The reducedcase burden in the data in the time interval 1932ndash1945 is explained bya large initial recovered (R) population which implies much higherlevels of polio before 1932 There were large epidemics of polio in theUSA early in the 20th century so this is not implausible

A liklihood profile over the parameter SO0 could help to clarify to what

extent this is a critical feature of how the model explains the data

36 58

Local likelihood maximization

Letrsquos see if we can improve on the previous MLE We use the IF2algorithm We set a constant random walk standard deviation foreach of the regular parameters and a larger constant for each of theinitial value parameters

polio_rwsd_rp lt- 002

polio_rwsd_ivp lt- 02

polio_coolingfraction50 lt- 05

polio_rwsd lt- rwsd(

b1=polio_rwsd_rp b2=polio_rwsd_rp

b3=polio_rwsd_rp b4=polio_rwsd_rp

b5=polio_rwsd_rp b6=polio_rwsd_rp

psi=polio_rwsd_rp rho=polio_rwsd_rp

tau=polio_rwsd_rp sigma_dem=polio_rwsd_rp

sigma_env=polio_rwsd_rp

IO_0=ivp(polio_rwsd_ivp) SO_0=ivp(polio_rwsd_ivp)

)

37 58

stew(sprintf(mif-drdarun_level)t2 lt- systemtime(m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=c) dopar mif2(polio

Np=polio_Np

Nmif=polio_Nmif

coolingfraction50=polio_coolingfraction50

rwsd=polio_rwsd)

lik_m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_evallogLik(

pfilter(polioparams=coef(m2[[i]])Np=polio_Np)))

se=TRUE)

)seed=318817883kind=LEcuyer)

38 58

r2 lt- dataframe(logLik=lik_m2[1]logLik_se=lik_m2[2]

t(sapply(m2coef)))

if (run_levelgt1)

writetable(r2file=polio_paramscsvappend=TRUE

colnames=FALSErownames=FALSE)

summary(r2$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-7967 -7956 -7952 -7953 -7950 -7945

This investigation took 236 minutes

These repeated stochastic maximizations can also show us thegeometry of the likelihood surface in a neighborhood of this pointestimate

39 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

Simulation to investigate local persistence

The scientific purpose of fitting a model typically involves analyzingproperties of the fitted model often investigated using simulation

Following Martinez-Bakker et al (2015) we are interested in howoften months with no reported cases (Yn = 0) correspond to monthswithout any local asymptomatic cases defined for our continuousstate model as IBn + IOn lt 12

For Wisconsin using our model at the estimated MLE we simulate inparallel as follows

stew(sprintf(persistence-drdarun_level)t_sim lt- systemtime(

sim lt- foreach(i=1polio_Nsim

packages=pomp) dopar simulate(polio)

)

seed=493536993kind=LEcuyer)

33 58

no_cases_data lt- sum(obs(polio)==0)

no_cases_sim lt- sum(sapply(simobs)==0)length(sim)

fadeout1_sim lt- sum(sapply(simfunction(po)states(po)[IB]

+states(po)[IO]lt1))length(sim)

fadeout100_sim lt- sum(sapply(simfunction(po)states(po)[IB]

+states(po)[IO]lt100))length(sim)

imports_sim lt- coef(polio)[psi]mean(sapply(sim

function(po) mean(states(po)[SO]+states(po)[SB1]

+states(po)[SB2]+states(po)[SB3]+states(po)[SB4]

+states(po)[SB5]+states(po)[SB6])))12

For the data there were 26 months with no reported cases similar to themean of 513 for simulations from the fitted model Months with noasyptomatic infections for the simulations were rare on average 08months per simulation Months with fewer than 100 infections averaged638 per simulation which in the context of a reporting rate of 001 canexplain the absences of case reports The mean monthly infections due toimportations modeled by ψ is 1175 This does not give muchopportunity for local elimination of poliovirus A profile over ψ would showhow sensitive this conclusion is to values of ψ consistent with the data

34 58

It is also good practice to look at simulations from the fitted model

mle_simulation lt- simulate(polioseed=127)

plot(mle_simulation)

35 58

We see from this simulation that the fitted model can generate reporthistories that look qualitatively similar to the data However there arethings to notice in the reconstructed latent states Specifically thepool of older susceptibles SO(t) is mostly increasing The reducedcase burden in the data in the time interval 1932ndash1945 is explained bya large initial recovered (R) population which implies much higherlevels of polio before 1932 There were large epidemics of polio in theUSA early in the 20th century so this is not implausible

A liklihood profile over the parameter SO0 could help to clarify to what

extent this is a critical feature of how the model explains the data

36 58

Local likelihood maximization

Letrsquos see if we can improve on the previous MLE We use the IF2algorithm We set a constant random walk standard deviation foreach of the regular parameters and a larger constant for each of theinitial value parameters

polio_rwsd_rp lt- 002

polio_rwsd_ivp lt- 02

polio_coolingfraction50 lt- 05

polio_rwsd lt- rwsd(

b1=polio_rwsd_rp b2=polio_rwsd_rp

b3=polio_rwsd_rp b4=polio_rwsd_rp

b5=polio_rwsd_rp b6=polio_rwsd_rp

psi=polio_rwsd_rp rho=polio_rwsd_rp

tau=polio_rwsd_rp sigma_dem=polio_rwsd_rp

sigma_env=polio_rwsd_rp

IO_0=ivp(polio_rwsd_ivp) SO_0=ivp(polio_rwsd_ivp)

)

37 58

stew(sprintf(mif-drdarun_level)t2 lt- systemtime(m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=c) dopar mif2(polio

Np=polio_Np

Nmif=polio_Nmif

coolingfraction50=polio_coolingfraction50

rwsd=polio_rwsd)

lik_m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_evallogLik(

pfilter(polioparams=coef(m2[[i]])Np=polio_Np)))

se=TRUE)

)seed=318817883kind=LEcuyer)

38 58

r2 lt- dataframe(logLik=lik_m2[1]logLik_se=lik_m2[2]

t(sapply(m2coef)))

if (run_levelgt1)

writetable(r2file=polio_paramscsvappend=TRUE

colnames=FALSErownames=FALSE)

summary(r2$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-7967 -7956 -7952 -7953 -7950 -7945

This investigation took 236 minutes

These repeated stochastic maximizations can also show us thegeometry of the likelihood surface in a neighborhood of this pointestimate

39 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

no_cases_data lt- sum(obs(polio)==0)

no_cases_sim lt- sum(sapply(simobs)==0)length(sim)

fadeout1_sim lt- sum(sapply(simfunction(po)states(po)[IB]

+states(po)[IO]lt1))length(sim)

fadeout100_sim lt- sum(sapply(simfunction(po)states(po)[IB]

+states(po)[IO]lt100))length(sim)

imports_sim lt- coef(polio)[psi]mean(sapply(sim

function(po) mean(states(po)[SO]+states(po)[SB1]

+states(po)[SB2]+states(po)[SB3]+states(po)[SB4]

+states(po)[SB5]+states(po)[SB6])))12

For the data there were 26 months with no reported cases similar to themean of 513 for simulations from the fitted model Months with noasyptomatic infections for the simulations were rare on average 08months per simulation Months with fewer than 100 infections averaged638 per simulation which in the context of a reporting rate of 001 canexplain the absences of case reports The mean monthly infections due toimportations modeled by ψ is 1175 This does not give muchopportunity for local elimination of poliovirus A profile over ψ would showhow sensitive this conclusion is to values of ψ consistent with the data

34 58

It is also good practice to look at simulations from the fitted model

mle_simulation lt- simulate(polioseed=127)

plot(mle_simulation)

35 58

We see from this simulation that the fitted model can generate reporthistories that look qualitatively similar to the data However there arethings to notice in the reconstructed latent states Specifically thepool of older susceptibles SO(t) is mostly increasing The reducedcase burden in the data in the time interval 1932ndash1945 is explained bya large initial recovered (R) population which implies much higherlevels of polio before 1932 There were large epidemics of polio in theUSA early in the 20th century so this is not implausible

A liklihood profile over the parameter SO0 could help to clarify to what

extent this is a critical feature of how the model explains the data

36 58

Local likelihood maximization

Letrsquos see if we can improve on the previous MLE We use the IF2algorithm We set a constant random walk standard deviation foreach of the regular parameters and a larger constant for each of theinitial value parameters

polio_rwsd_rp lt- 002

polio_rwsd_ivp lt- 02

polio_coolingfraction50 lt- 05

polio_rwsd lt- rwsd(

b1=polio_rwsd_rp b2=polio_rwsd_rp

b3=polio_rwsd_rp b4=polio_rwsd_rp

b5=polio_rwsd_rp b6=polio_rwsd_rp

psi=polio_rwsd_rp rho=polio_rwsd_rp

tau=polio_rwsd_rp sigma_dem=polio_rwsd_rp

sigma_env=polio_rwsd_rp

IO_0=ivp(polio_rwsd_ivp) SO_0=ivp(polio_rwsd_ivp)

)

37 58

stew(sprintf(mif-drdarun_level)t2 lt- systemtime(m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=c) dopar mif2(polio

Np=polio_Np

Nmif=polio_Nmif

coolingfraction50=polio_coolingfraction50

rwsd=polio_rwsd)

lik_m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_evallogLik(

pfilter(polioparams=coef(m2[[i]])Np=polio_Np)))

se=TRUE)

)seed=318817883kind=LEcuyer)

38 58

r2 lt- dataframe(logLik=lik_m2[1]logLik_se=lik_m2[2]

t(sapply(m2coef)))

if (run_levelgt1)

writetable(r2file=polio_paramscsvappend=TRUE

colnames=FALSErownames=FALSE)

summary(r2$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-7967 -7956 -7952 -7953 -7950 -7945

This investigation took 236 minutes

These repeated stochastic maximizations can also show us thegeometry of the likelihood surface in a neighborhood of this pointestimate

39 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

It is also good practice to look at simulations from the fitted model

mle_simulation lt- simulate(polioseed=127)

plot(mle_simulation)

35 58

We see from this simulation that the fitted model can generate reporthistories that look qualitatively similar to the data However there arethings to notice in the reconstructed latent states Specifically thepool of older susceptibles SO(t) is mostly increasing The reducedcase burden in the data in the time interval 1932ndash1945 is explained bya large initial recovered (R) population which implies much higherlevels of polio before 1932 There were large epidemics of polio in theUSA early in the 20th century so this is not implausible

A liklihood profile over the parameter SO0 could help to clarify to what

extent this is a critical feature of how the model explains the data

36 58

Local likelihood maximization

Letrsquos see if we can improve on the previous MLE We use the IF2algorithm We set a constant random walk standard deviation foreach of the regular parameters and a larger constant for each of theinitial value parameters

polio_rwsd_rp lt- 002

polio_rwsd_ivp lt- 02

polio_coolingfraction50 lt- 05

polio_rwsd lt- rwsd(

b1=polio_rwsd_rp b2=polio_rwsd_rp

b3=polio_rwsd_rp b4=polio_rwsd_rp

b5=polio_rwsd_rp b6=polio_rwsd_rp

psi=polio_rwsd_rp rho=polio_rwsd_rp

tau=polio_rwsd_rp sigma_dem=polio_rwsd_rp

sigma_env=polio_rwsd_rp

IO_0=ivp(polio_rwsd_ivp) SO_0=ivp(polio_rwsd_ivp)

)

37 58

stew(sprintf(mif-drdarun_level)t2 lt- systemtime(m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=c) dopar mif2(polio

Np=polio_Np

Nmif=polio_Nmif

coolingfraction50=polio_coolingfraction50

rwsd=polio_rwsd)

lik_m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_evallogLik(

pfilter(polioparams=coef(m2[[i]])Np=polio_Np)))

se=TRUE)

)seed=318817883kind=LEcuyer)

38 58

r2 lt- dataframe(logLik=lik_m2[1]logLik_se=lik_m2[2]

t(sapply(m2coef)))

if (run_levelgt1)

writetable(r2file=polio_paramscsvappend=TRUE

colnames=FALSErownames=FALSE)

summary(r2$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-7967 -7956 -7952 -7953 -7950 -7945

This investigation took 236 minutes

These repeated stochastic maximizations can also show us thegeometry of the likelihood surface in a neighborhood of this pointestimate

39 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

We see from this simulation that the fitted model can generate reporthistories that look qualitatively similar to the data However there arethings to notice in the reconstructed latent states Specifically thepool of older susceptibles SO(t) is mostly increasing The reducedcase burden in the data in the time interval 1932ndash1945 is explained bya large initial recovered (R) population which implies much higherlevels of polio before 1932 There were large epidemics of polio in theUSA early in the 20th century so this is not implausible

A liklihood profile over the parameter SO0 could help to clarify to what

extent this is a critical feature of how the model explains the data

36 58

Local likelihood maximization

Letrsquos see if we can improve on the previous MLE We use the IF2algorithm We set a constant random walk standard deviation foreach of the regular parameters and a larger constant for each of theinitial value parameters

polio_rwsd_rp lt- 002

polio_rwsd_ivp lt- 02

polio_coolingfraction50 lt- 05

polio_rwsd lt- rwsd(

b1=polio_rwsd_rp b2=polio_rwsd_rp

b3=polio_rwsd_rp b4=polio_rwsd_rp

b5=polio_rwsd_rp b6=polio_rwsd_rp

psi=polio_rwsd_rp rho=polio_rwsd_rp

tau=polio_rwsd_rp sigma_dem=polio_rwsd_rp

sigma_env=polio_rwsd_rp

IO_0=ivp(polio_rwsd_ivp) SO_0=ivp(polio_rwsd_ivp)

)

37 58

stew(sprintf(mif-drdarun_level)t2 lt- systemtime(m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=c) dopar mif2(polio

Np=polio_Np

Nmif=polio_Nmif

coolingfraction50=polio_coolingfraction50

rwsd=polio_rwsd)

lik_m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_evallogLik(

pfilter(polioparams=coef(m2[[i]])Np=polio_Np)))

se=TRUE)

)seed=318817883kind=LEcuyer)

38 58

r2 lt- dataframe(logLik=lik_m2[1]logLik_se=lik_m2[2]

t(sapply(m2coef)))

if (run_levelgt1)

writetable(r2file=polio_paramscsvappend=TRUE

colnames=FALSErownames=FALSE)

summary(r2$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-7967 -7956 -7952 -7953 -7950 -7945

This investigation took 236 minutes

These repeated stochastic maximizations can also show us thegeometry of the likelihood surface in a neighborhood of this pointestimate

39 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

Local likelihood maximization

Letrsquos see if we can improve on the previous MLE We use the IF2algorithm We set a constant random walk standard deviation foreach of the regular parameters and a larger constant for each of theinitial value parameters

polio_rwsd_rp lt- 002

polio_rwsd_ivp lt- 02

polio_coolingfraction50 lt- 05

polio_rwsd lt- rwsd(

b1=polio_rwsd_rp b2=polio_rwsd_rp

b3=polio_rwsd_rp b4=polio_rwsd_rp

b5=polio_rwsd_rp b6=polio_rwsd_rp

psi=polio_rwsd_rp rho=polio_rwsd_rp

tau=polio_rwsd_rp sigma_dem=polio_rwsd_rp

sigma_env=polio_rwsd_rp

IO_0=ivp(polio_rwsd_ivp) SO_0=ivp(polio_rwsd_ivp)

)

37 58

stew(sprintf(mif-drdarun_level)t2 lt- systemtime(m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=c) dopar mif2(polio

Np=polio_Np

Nmif=polio_Nmif

coolingfraction50=polio_coolingfraction50

rwsd=polio_rwsd)

lik_m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_evallogLik(

pfilter(polioparams=coef(m2[[i]])Np=polio_Np)))

se=TRUE)

)seed=318817883kind=LEcuyer)

38 58

r2 lt- dataframe(logLik=lik_m2[1]logLik_se=lik_m2[2]

t(sapply(m2coef)))

if (run_levelgt1)

writetable(r2file=polio_paramscsvappend=TRUE

colnames=FALSErownames=FALSE)

summary(r2$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-7967 -7956 -7952 -7953 -7950 -7945

This investigation took 236 minutes

These repeated stochastic maximizations can also show us thegeometry of the likelihood surface in a neighborhood of this pointestimate

39 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

stew(sprintf(mif-drdarun_level)t2 lt- systemtime(m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=c) dopar mif2(polio

Np=polio_Np

Nmif=polio_Nmif

coolingfraction50=polio_coolingfraction50

rwsd=polio_rwsd)

lik_m2 lt- foreach(i=1polio_Nreps_local

packages=pomp combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_evallogLik(

pfilter(polioparams=coef(m2[[i]])Np=polio_Np)))

se=TRUE)

)seed=318817883kind=LEcuyer)

38 58

r2 lt- dataframe(logLik=lik_m2[1]logLik_se=lik_m2[2]

t(sapply(m2coef)))

if (run_levelgt1)

writetable(r2file=polio_paramscsvappend=TRUE

colnames=FALSErownames=FALSE)

summary(r2$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-7967 -7956 -7952 -7953 -7950 -7945

This investigation took 236 minutes

These repeated stochastic maximizations can also show us thegeometry of the likelihood surface in a neighborhood of this pointestimate

39 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

r2 lt- dataframe(logLik=lik_m2[1]logLik_se=lik_m2[2]

t(sapply(m2coef)))

if (run_levelgt1)

writetable(r2file=polio_paramscsvappend=TRUE

colnames=FALSErownames=FALSE)

summary(r2$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-7967 -7956 -7952 -7953 -7950 -7945

This investigation took 236 minutes

These repeated stochastic maximizations can also show us thegeometry of the likelihood surface in a neighborhood of this pointestimate

39 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r2logLikgtmax(logLik)-20))

40 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

We see strong tradeoffs between ψ ρ and σdem By itself in theabsence of other assumptions the pathogen immigration rate ψ isfairly weakly identified However the reporting rate ρ is essentiallythe fraction of poliovirus infections leading to acute flaccid paralysiswhich is known to be around 1 This plot suggests that fixing anassumed value of ρ might lead to much more precise inference on ψthe rate of pathogen immigration presumably being important forunderstanding disease persistence These hypotheses could beinvestigated more formally by construction of profile likelihood plotsand likelihood ratio tests

41 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

Comparison of our implementation with Martinez-Bakkeret al (2015)

This setup has minor differences in notation model construction andcode compared to Martinez-Bakker et al (2015)

The maximized likelihood reported for these data by Martinez-Bakkeret al (2015) was -79434 with Monte Carlo evaluation error of 018

This is similar to the value -79449 found by this search

The differences between the two implementations do not substantiallyimprove or decrease the fit of our model compared to Martinez-Bakkeret al (2015) demonstrating reproducibility of both results

42 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

Global likelihood maximization

Practical parameter estimation involves trying many starting values for theparameters One can specify a large box in parameter space that containsall parameter vectors which seem remotely sensible If an estimationmethod gives stable conclusions with starting values drawn randomly fromthis box this gives some confidence that an adequate global search hasbeen carried outFor our polio model a box containing reasonable parameter values mightbe

polio_box lt- rbind(

b1=c(-28) b2=c(-28)

b3=c(-28) b4=c(-28)

b5=c(-28) b6=c(-28)

psi=c(001) rho=c(001) tau=c(001)

sigma_dem=c(005) sigma_env=c(01)

SO_0=c(01) IO_0=c(0001)

)

43 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

We then carry out a search identical to the local one except for thestarting parameter values This can be succinctly coded by calling mif2 onthe previously constructed object m2[[1]] with a reset starting value

stew(file=sprintf(box_eval-drdarun_level)t3 lt- systemtime(m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=c) dopar mif2(

m2[[1]]

params=c(apply(polio_box1function(x)runif(1x[1]x[2]))

polio_fixed_params)

)

lik_m3 lt- foreach(i=1polio_Nreps_globalpackages=pomp

combine=rbind) dopar logmeanexp(

replicate(polio_Nreps_eval

logLik(pfilter(polio

params=coef(m3[[i]])Np=polio_Np)))

se=TRUE)

)seed=290860873kind=LEcuyer)

44 58

r3 lt- dataframe(logLik=lik_m3[1]logLik_se=lik_m3[2]t(sapply(m3coef)))

if(run_levelgt1) writetable(r3file=polio_paramscsvappend=TRUEcolnames=FALSErownames=FALSE)

summary(r3$logLikdigits=5)

Min 1st Qu Median Mean 3rd Qu Max

-8564 -7967 -7957 -7976 -7951 -7945

Evaluation of the best result of this search gives a likelihood of -7945with a standard error of 01 We see that optimization attempts fromdiverse remote starting points can approach our MLE but do notexceed it This gives us some reasonable confidence in our MLE

Plotting these diverse parameter estimates can help to give a feel forthe global geometry of the likelihood surface

45 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r3logLikgtmax(logLik)-20))

46 58

Benchmark likelihoods for non-mechanistic models

To understand these global searches many of which may correspondto parameter values having no meaningful scientific interpretation itis helpful to put the log likelihoods in the context of somenon-mechanistic benchmarks

The most basic statistical model for data is independent identicallydistributed (IID) Picking a negative binomial model

nb_lik lt- function(theta) -sum(dnbinom(asvector(obs(polio))

size=exp(theta[1])prob=exp(theta[2])log=TRUE))

nb_mle lt- optim(c(0-5)nb_lik)

-nb_mle$value

[1] -1036227

We see that a model with likelihood below -10362 is unreasonableThis explains a cutoff around this value in the global searches inthese cases the model is finding essentially IID explanations for thedata

47 58

ARMA models as benchmarks

Linear Gaussian auto-regressive moving-average (ARMA) modelsprovide non-mechansitic fits to the data including flexible dependencerelationships

We fit to log(ylowastn + 1) and correct the likelihood back to the scaleappropriate for the untransformed data

log_y lt- log(asvector(obs(polio))+1)

arma_fit lt- arima(log_yorder=c(202)

seasonal=list(order=c(101)period=12))

arma_fit$loglik-sum(log_y)

[1] -8220827

This 7-parameter model which knows nothing of susceptibledepletion attains a likelihood of -8221

Although the goal of mechanistic modeling here is not to beatnon-mechanstic models it is comforting that wersquore competitive withthem

48 58

Mining previous investigations of the likelihood

polio_params lt- readtable(polio_paramscsvrownames=NULL

header=TRUE)

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(polio_paramslogLikgtmax(logLik)-20))

49 58

Here we see that the most successful searches have always led tomodels with reporting rate around 1-2 This impression can bereinforced by looking at results from the global searches

plot(logLik~rhodata=subset(r3logLikgtmax(r3$logLik)-10)log=x)

Reporting rates close to 1 provide a small but clear advantage(several units of log likelihood) in explaining the data For thesereporting rates depletion of susceptibles can help to explain thedynamics

50 58

Question 132 Parameter estimation using randomized startingvalues Comment on the computations above for parameter estimationusing randomized starting values Propose and try out at least onemodification of the procedure How could one make a formal statementquantifying the error of the optimization procedure

51 58

Question 133 Demography and discrete time It can be surprisinglyhard to include birth death immigration emmigration and aging into adisease model in satisfactory ways Consider the strengths and weaknessesof the analysis presented For example how does it compare to acontinuous-time model In an imperfect world it is nice to check theextent to which the conclusions are insensitive to alternative modelingdecisions If you have some ideas to change the treatmentof demography(or an other aspect of the model) you could have a go at coding it up tosee if it makes a difference

52 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(r3logLikgtmax(logLik)-20))

46 58

Benchmark likelihoods for non-mechanistic models

To understand these global searches many of which may correspondto parameter values having no meaningful scientific interpretation itis helpful to put the log likelihoods in the context of somenon-mechanistic benchmarks

The most basic statistical model for data is independent identicallydistributed (IID) Picking a negative binomial model

nb_lik lt- function(theta) -sum(dnbinom(asvector(obs(polio))

size=exp(theta[1])prob=exp(theta[2])log=TRUE))

nb_mle lt- optim(c(0-5)nb_lik)

-nb_mle$value

[1] -1036227

We see that a model with likelihood below -10362 is unreasonableThis explains a cutoff around this value in the global searches inthese cases the model is finding essentially IID explanations for thedata

47 58

ARMA models as benchmarks

Linear Gaussian auto-regressive moving-average (ARMA) modelsprovide non-mechansitic fits to the data including flexible dependencerelationships

We fit to log(ylowastn + 1) and correct the likelihood back to the scaleappropriate for the untransformed data

log_y lt- log(asvector(obs(polio))+1)

arma_fit lt- arima(log_yorder=c(202)

seasonal=list(order=c(101)period=12))

arma_fit$loglik-sum(log_y)

[1] -8220827

This 7-parameter model which knows nothing of susceptibledepletion attains a likelihood of -8221

Although the goal of mechanistic modeling here is not to beatnon-mechanstic models it is comforting that wersquore competitive withthem

48 58

Mining previous investigations of the likelihood

polio_params lt- readtable(polio_paramscsvrownames=NULL

header=TRUE)

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(polio_paramslogLikgtmax(logLik)-20))

49 58

Here we see that the most successful searches have always led tomodels with reporting rate around 1-2 This impression can bereinforced by looking at results from the global searches

plot(logLik~rhodata=subset(r3logLikgtmax(r3$logLik)-10)log=x)

Reporting rates close to 1 provide a small but clear advantage(several units of log likelihood) in explaining the data For thesereporting rates depletion of susceptibles can help to explain thedynamics

50 58

Question 132 Parameter estimation using randomized startingvalues Comment on the computations above for parameter estimationusing randomized starting values Propose and try out at least onemodification of the procedure How could one make a formal statementquantifying the error of the optimization procedure

51 58

Question 133 Demography and discrete time It can be surprisinglyhard to include birth death immigration emmigration and aging into adisease model in satisfactory ways Consider the strengths and weaknessesof the analysis presented For example how does it compare to acontinuous-time model In an imperfect world it is nice to check theextent to which the conclusions are insensitive to alternative modelingdecisions If you have some ideas to change the treatmentof demography(or an other aspect of the model) you could have a go at coding it up tosee if it makes a difference

52 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

Benchmark likelihoods for non-mechanistic models

To understand these global searches many of which may correspondto parameter values having no meaningful scientific interpretation itis helpful to put the log likelihoods in the context of somenon-mechanistic benchmarks

The most basic statistical model for data is independent identicallydistributed (IID) Picking a negative binomial model

nb_lik lt- function(theta) -sum(dnbinom(asvector(obs(polio))

size=exp(theta[1])prob=exp(theta[2])log=TRUE))

nb_mle lt- optim(c(0-5)nb_lik)

-nb_mle$value

[1] -1036227

We see that a model with likelihood below -10362 is unreasonableThis explains a cutoff around this value in the global searches inthese cases the model is finding essentially IID explanations for thedata

47 58

ARMA models as benchmarks

Linear Gaussian auto-regressive moving-average (ARMA) modelsprovide non-mechansitic fits to the data including flexible dependencerelationships

We fit to log(ylowastn + 1) and correct the likelihood back to the scaleappropriate for the untransformed data

log_y lt- log(asvector(obs(polio))+1)

arma_fit lt- arima(log_yorder=c(202)

seasonal=list(order=c(101)period=12))

arma_fit$loglik-sum(log_y)

[1] -8220827

This 7-parameter model which knows nothing of susceptibledepletion attains a likelihood of -8221

Although the goal of mechanistic modeling here is not to beatnon-mechanstic models it is comforting that wersquore competitive withthem

48 58

Mining previous investigations of the likelihood

polio_params lt- readtable(polio_paramscsvrownames=NULL

header=TRUE)

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(polio_paramslogLikgtmax(logLik)-20))

49 58

Here we see that the most successful searches have always led tomodels with reporting rate around 1-2 This impression can bereinforced by looking at results from the global searches

plot(logLik~rhodata=subset(r3logLikgtmax(r3$logLik)-10)log=x)

Reporting rates close to 1 provide a small but clear advantage(several units of log likelihood) in explaining the data For thesereporting rates depletion of susceptibles can help to explain thedynamics

50 58

Question 132 Parameter estimation using randomized startingvalues Comment on the computations above for parameter estimationusing randomized starting values Propose and try out at least onemodification of the procedure How could one make a formal statementquantifying the error of the optimization procedure

51 58

Question 133 Demography and discrete time It can be surprisinglyhard to include birth death immigration emmigration and aging into adisease model in satisfactory ways Consider the strengths and weaknessesof the analysis presented For example how does it compare to acontinuous-time model In an imperfect world it is nice to check theextent to which the conclusions are insensitive to alternative modelingdecisions If you have some ideas to change the treatmentof demography(or an other aspect of the model) you could have a go at coding it up tosee if it makes a difference

52 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

ARMA models as benchmarks

Linear Gaussian auto-regressive moving-average (ARMA) modelsprovide non-mechansitic fits to the data including flexible dependencerelationships

We fit to log(ylowastn + 1) and correct the likelihood back to the scaleappropriate for the untransformed data

log_y lt- log(asvector(obs(polio))+1)

arma_fit lt- arima(log_yorder=c(202)

seasonal=list(order=c(101)period=12))

arma_fit$loglik-sum(log_y)

[1] -8220827

This 7-parameter model which knows nothing of susceptibledepletion attains a likelihood of -8221

Although the goal of mechanistic modeling here is not to beatnon-mechanstic models it is comforting that wersquore competitive withthem

48 58

Mining previous investigations of the likelihood

polio_params lt- readtable(polio_paramscsvrownames=NULL

header=TRUE)

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(polio_paramslogLikgtmax(logLik)-20))

49 58

Here we see that the most successful searches have always led tomodels with reporting rate around 1-2 This impression can bereinforced by looking at results from the global searches

plot(logLik~rhodata=subset(r3logLikgtmax(r3$logLik)-10)log=x)

Reporting rates close to 1 provide a small but clear advantage(several units of log likelihood) in explaining the data For thesereporting rates depletion of susceptibles can help to explain thedynamics

50 58

Question 132 Parameter estimation using randomized startingvalues Comment on the computations above for parameter estimationusing randomized starting values Propose and try out at least onemodification of the procedure How could one make a formal statementquantifying the error of the optimization procedure

51 58

Question 133 Demography and discrete time It can be surprisinglyhard to include birth death immigration emmigration and aging into adisease model in satisfactory ways Consider the strengths and weaknessesof the analysis presented For example how does it compare to acontinuous-time model In an imperfect world it is nice to check theextent to which the conclusions are insensitive to alternative modelingdecisions If you have some ideas to change the treatmentof demography(or an other aspect of the model) you could have a go at coding it up tosee if it makes a difference

52 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

Mining previous investigations of the likelihood

polio_params lt- readtable(polio_paramscsvrownames=NULL

header=TRUE)

pairs(~logLik+psi+rho+tau+sigma_dem+sigma_env

data=subset(polio_paramslogLikgtmax(logLik)-20))

49 58

Here we see that the most successful searches have always led tomodels with reporting rate around 1-2 This impression can bereinforced by looking at results from the global searches

plot(logLik~rhodata=subset(r3logLikgtmax(r3$logLik)-10)log=x)

Reporting rates close to 1 provide a small but clear advantage(several units of log likelihood) in explaining the data For thesereporting rates depletion of susceptibles can help to explain thedynamics

50 58

Question 132 Parameter estimation using randomized startingvalues Comment on the computations above for parameter estimationusing randomized starting values Propose and try out at least onemodification of the procedure How could one make a formal statementquantifying the error of the optimization procedure

51 58

Question 133 Demography and discrete time It can be surprisinglyhard to include birth death immigration emmigration and aging into adisease model in satisfactory ways Consider the strengths and weaknessesof the analysis presented For example how does it compare to acontinuous-time model In an imperfect world it is nice to check theextent to which the conclusions are insensitive to alternative modelingdecisions If you have some ideas to change the treatmentof demography(or an other aspect of the model) you could have a go at coding it up tosee if it makes a difference

52 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

Here we see that the most successful searches have always led tomodels with reporting rate around 1-2 This impression can bereinforced by looking at results from the global searches

plot(logLik~rhodata=subset(r3logLikgtmax(r3$logLik)-10)log=x)

Reporting rates close to 1 provide a small but clear advantage(several units of log likelihood) in explaining the data For thesereporting rates depletion of susceptibles can help to explain thedynamics

50 58

Question 132 Parameter estimation using randomized startingvalues Comment on the computations above for parameter estimationusing randomized starting values Propose and try out at least onemodification of the procedure How could one make a formal statementquantifying the error of the optimization procedure

51 58

Question 133 Demography and discrete time It can be surprisinglyhard to include birth death immigration emmigration and aging into adisease model in satisfactory ways Consider the strengths and weaknessesof the analysis presented For example how does it compare to acontinuous-time model In an imperfect world it is nice to check theextent to which the conclusions are insensitive to alternative modelingdecisions If you have some ideas to change the treatmentof demography(or an other aspect of the model) you could have a go at coding it up tosee if it makes a difference

52 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

Question 132 Parameter estimation using randomized startingvalues Comment on the computations above for parameter estimationusing randomized starting values Propose and try out at least onemodification of the procedure How could one make a formal statementquantifying the error of the optimization procedure

51 58

Question 133 Demography and discrete time It can be surprisinglyhard to include birth death immigration emmigration and aging into adisease model in satisfactory ways Consider the strengths and weaknessesof the analysis presented For example how does it compare to acontinuous-time model In an imperfect world it is nice to check theextent to which the conclusions are insensitive to alternative modelingdecisions If you have some ideas to change the treatmentof demography(or an other aspect of the model) you could have a go at coding it up tosee if it makes a difference

52 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

Question 133 Demography and discrete time It can be surprisinglyhard to include birth death immigration emmigration and aging into adisease model in satisfactory ways Consider the strengths and weaknessesof the analysis presented For example how does it compare to acontinuous-time model In an imperfect world it is nice to check theextent to which the conclusions are insensitive to alternative modelingdecisions If you have some ideas to change the treatmentof demography(or an other aspect of the model) you could have a go at coding it up tosee if it makes a difference

52 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

Question 134 Diagnosing filtering and maximization convergenceAre there outliers in the data (ie observations that do not fit well withour model) Are we using unnecessarily large amounts of computer timeto get our results Are there indications that we would should run ourcomputations for longer Or maybe with different choices of algorithmicsettings In particular coolingfraction50 gives the fraction by whichthe random walk standard deviation is decreased (rdquocooledrdquo) in 50iterations If coolingfraction50 is too small the search will rdquofreezerdquotoo soon evidenced by flat parallel lines in the convergence diagnostics Ifcoolingfraction50 is too large the researcher may run of of timepatience or computing budget (or all three) before the parametertrajectories approach an MLE

53 58

plot(m3[r3$logLikgtmax(r3$logLik)-10])

54 58

The likelihood is particularly important to keep in mind If parameterestimates are numerically unstable that could be a consequence of aweakly identified parameter subspace

The presence of some weakly identified combinations of parameters isnot fundamentally a scientific flaw rather our scientific inquiry looksto investigate which questions can and cannot be answered in thecontext of a set of data and modeling assumptions

As long as the search is demonstrably approaching the maximumlikelihood region we should not necessarily be worried about thestability of parameter values (at least from the point of diagnosingsuccessful maximization)

So we zoom in on the likelihood convergence

55 58

loglik_convergence lt- docall(cbind

traces(m3[r3$logLikgtmax(r3$logLik)-10]loglik))

matplot(loglik_convergencetype=llty=1

ylim=max(loglik_convergencenarm=T)+c(-100))

56 58

Acknowledgments and License

Produced with R version 362 and pomp version 23

These notes build on previous versions ationidesgithubio531w16 and ionidesgithubio531w18

Those notes draw on material developed for a short course onSimulation-based Inference for Epidemiological Dynamics(httpkingaagithubiosbied) by Aaron King and EdwardIonides taught at the University of Washington Summer Institute inStatistics and Modeling in Infectious Diseases from 2015 through2019

Licensed under the Creative Commons attribution-noncommerciallicense httpcreativecommonsorglicensesby-nc30Please share and remix noncommercially mentioning its origin

57 58

The likelihood is particularly important to keep in mind If parameterestimates are numerically unstable that could be a consequence of aweakly identified parameter subspace

The presence of some weakly identified combinations of parameters isnot fundamentally a scientific flaw rather our scientific inquiry looksto investigate which questions can and cannot be answered in thecontext of a set of data and modeling assumptions

As long as the search is demonstrably approaching the maximumlikelihood region we should not necessarily be worried about thestability of parameter values (at least from the point of diagnosingsuccessful maximization)

So we zoom in on the likelihood convergence

55 58

loglik_convergence lt- docall(cbind

traces(m3[r3$logLikgtmax(r3$logLik)-10]loglik))

matplot(loglik_convergencetype=llty=1

ylim=max(loglik_convergencenarm=T)+c(-100))

56 58

Acknowledgments and License

Produced with R version 362 and pomp version 23

These notes build on previous versions ationidesgithubio531w16 and ionidesgithubio531w18

Those notes draw on material developed for a short course onSimulation-based Inference for Epidemiological Dynamics(httpkingaagithubiosbied) by Aaron King and EdwardIonides taught at the University of Washington Summer Institute inStatistics and Modeling in Infectious Diseases from 2015 through2019

Licensed under the Creative Commons attribution-noncommerciallicense httpcreativecommonsorglicensesby-nc30Please share and remix noncommercially mentioning its origin

57 58

loglik_convergence lt- docall(cbind

traces(m3[r3$logLikgtmax(r3$logLik)-10]loglik))

matplot(loglik_convergencetype=llty=1

ylim=max(loglik_convergencenarm=T)+c(-100))

56 58

Acknowledgments and License

Produced with R version 362 and pomp version 23

These notes build on previous versions ationidesgithubio531w16 and ionidesgithubio531w18

Those notes draw on material developed for a short course onSimulation-based Inference for Epidemiological Dynamics(httpkingaagithubiosbied) by Aaron King and EdwardIonides taught at the University of Washington Summer Institute inStatistics and Modeling in Infectious Diseases from 2015 through2019

Licensed under the Creative Commons attribution-noncommerciallicense httpcreativecommonsorglicensesby-nc30Please share and remix noncommercially mentioning its origin

57 58

Acknowledgments and License

Produced with R version 362 and pomp version 23

These notes build on previous versions ationidesgithubio531w16 and ionidesgithubio531w18

Those notes draw on material developed for a short course onSimulation-based Inference for Epidemiological Dynamics(httpkingaagithubiosbied) by Aaron King and EdwardIonides taught at the University of Washington Summer Institute inStatistics and Modeling in Infectious Diseases from 2015 through2019

Licensed under the Creative Commons attribution-noncommerciallicense httpcreativecommonsorglicensesby-nc30Please share and remix noncommercially mentioning its origin

57 58

References I

Martinez-Bakker M King A A and Rohani P (2015) Unraveling thetransmission ecology of polio PLoS Biology 13(6) e1002172

58 58

• References