malariaaa

7/28/2019 malariaaa

1/14

DOI: 10.1542/peds.2011-0043; originally published online August 22, 2011;2011;128;e666Pediatrics

Elizabeth M. Uleryk and Patricia C. Parkin

Jonathon L. Maguire, Dina M. Kulik, Andreas Laupacis, Nathan Kuppermann,Clinical Prediction Rules for Children: A Systematic Review

http://pediatrics.aappublications.org/content/128/3/e666.full.html

located on the World Wide Web at:The online version of this article, along with updated information and services, is

of Pediatrics. All rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.Boulevard, Elk Grove Village, Illinois, 60007. Copyright 2011 by the American Academypublished, and trademarked by the American Academy of Pediatrics, 141 Northwest Point

publication, it has been published continuously since 1948. PEDIATRICS is owned,PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly

at Indonesia:AAP Sponsored on May 30, 2013pediatrics.aappublications.orgDownloaded from
http://pediatrics.aappublications.org/content/128/3/e666.full.htmlhttp://pediatrics.aappublications.org/content/128/3/e666.full.htmlhttp://pediatrics.aappublications.org/content/128/3/e666.full.htmlhttp://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/content/128/3/e666.full.html

7/28/2019 malariaaa

2/14

Clinical Prediction Rules for Children: A Systematic

Review

abstractCONTEXT: The degree to which clinical prediction rules (CPRs) for chil-

dren meet published standards is unclear.

OBJECTIVE: To systematically review the quality, performance, and val-

idation of published CPRs for children, compare them with adult CPRs,

and suggest pediatric-specific changes to CPR methodology.

METHODS: Medline was searched from 1950 to 2011. Studies were

selected if they included the development of a CPR involving children

younger than 18 years. Two investigators assessed study quality, rule

performance, and rule validation as methodologic standards.

RESULTS: Of 7298 titles and abstracts assessed, 137 eligible studies

were identified. They describe the development of 101 CPRs addressing

36 pediatric conditions. Quality standards met in fewer than half of the

studies were blind assessment of predictors (47%), reproducibility of

predictors (18%), blind assessment of outcomes (42%), adequate

follow-up of outcomes (36%), adequate power (43%), adequate report-

ing of results (49%), and 95% confidence intervals reported (36%). For

rule performance, 48% had a sensitivity greater than 0.95, and 43% had

a negative likelihood ratio less than 0.1. For rule validation, 76% had no

validation, 17% had narrow validation, 8% had broad validation, and

none had impact analysis performed. Compared with CPRs for adult

health conditions, quality and rule validation seem to be lower.

CONCLUSIONS: Many CPRs have been derived for children, but few

have been validated. Relative to adult CPRs, several quality indicators

demonstrated weaknesses. Existing performance standards may

prove elusive for CPRs that involve children. CPRs for children that are

more assistive and less directive and include patients values and pref-

erences in decision-making may be helpful. Pediatrics 2011;128:

e666e677

AUTHORS: Jonathon L. Maguire, MSc, MD, FRCPC,a,b,c,d,e,f

Dina M. Kulik, MD,g Andreas Laupacis, MD, MSc,

FRCPC,b,e,h Nathan Kuppermann, MD, MPH,i Elizabeth M.

Uleryk, BA, MLS,j and Patricia C. Parkin, MD, FRCPCc,d,e,f

aDepartment of Pediatrics andbKeenan Research Centre, Li Ka

Shing Knowledge Institute, St Michaels Hospital, Toronto,

Ontario, Canada;cDivision of Pediatric Medicine and the

Pediatric Outcomes Research Team, gDivision of Pediatric

Emergency Medicine, and jHospital Library, Hospital for Sick

Children, Toronto, Ontario, Canada;dChild Health Evaluative

Sciences, Hospital for Sick Children Research Institute, Toronto,

Ontario, Canada; Departments ofeHealth Policy Management

and Evaluation, fPediatrics, andhMedicine, University of Toronto,

Toronto, Ontario, Canada; and iDepartments of Emergency

Medicine and Pediatrics, University of California, Davis School ofMedicine, Davis, California

KEY WORDS

review, clinical prediction rule, child, preschool child,

adolescent, decision trees, predictive value of tests, models,

multivariate analysis

ABBREVIATIONS

CPRclinical prediction rule

LRnegative likelihood ratio

CIconfidence interval

EBMWGEvidence-Based Medicine Working Group

Drs Maguire and Parkin provided the study concept and design,

drafted the manuscript, performed statistical analysis, and

supervised the study; Ms Uleryk developed and performed the

electronic literature search; Drs Maguire and Kulik acquired the

data; Drs Maguire, Kulik, Laupacis, Kuppermann, and Parkin

performed analysis and interpretation of data; Drs Laupacis,

Kuppermann, Kulik, and Parkin critically revised the manuscript

for important intellectual content; and Dr Parkin provided

administrative, technical, and material support. Dr Maguire had

full access to all of the data in the study and takes responsibility

for the integrity of the data and the accuracy of the data

analysis.

www.pediatrics.org/cgi/doi/10.1542/peds.2011-0043

doi:10.1542/peds.2011-0043

Accepted for publication May 27, 2011

Address correspondence to Jonathon L. Maguire, MSc, MD,

FRCPC, Pediatric Ambulatory Clinic, St Michaels Hospital, 61

Queen St East, 2nd Floor, Toronto, Ontario, Canada M5C 2T2.

E-mail: [email protected]

PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).

Copyright 2011 by the American Academy of Pediatrics

FINANCIAL DISCLOSURE: The authors have indicated they have

no financial relationships relevant to this article to disclose.

e666 MAGUIRE et alat Indonesia:AAP Sponsored on May 30, 2013pediatrics.aappublications.orgDownloaded from
http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/

7/28/2019 malariaaa

3/14

Clinical prediction rules (CPRs) have

been defined as clinical decision-

making tools that use 3 or more vari-

ables from history, physical examina-

tion, or simple tests to provide the

probability of an outcome or suggest a

diagnostic or therapeutic course of ac-tion for an individual patient.13 They

are potentially powerful tools for re-

ducing uncertainty and improving ac-

curacy in medical decision-making by

standardizing the collection and inter-

pretation of clinical data.2 In some in-

stances, they may minimize the use of

potentially harmful or costly diagnos-

tic testing. CPRs differ from decision

analyses, which quantify the value of

specified outcomes and use data fromthe literature to formulate health care

policy; decision-support tools, which

are designed to prevent errors when

implementing decisions that have al-

ready been made; and practice guide-

lines, which reflect a summation of the

existing data on a particular topic and

represent a consensus of expert opin-

ion to address several patient care is-

sues within a particular syndrome.4

To meet their objectives and be rou-tinely incorporated into patient care,

CPRs must be rigorously developed

and meet the performance expecta-

tions of clinicians who use them.1,5

Methodologic standards for the devel-

opment of CPRs have been described

previously.1,3,4,6,7 These standards in-

clude several steps in rule develop-

ment: creating the rule (derivation);

testing the rule (validation); translat-

ing the results of the validated ruleinto practice (knowledge translation);

and assessing the impact of the rule

on physician behavior and clinical out-

comes (impact analysis).

The authors of 3 reviews have exam-

ined the methodologic quality of CPRs

largely for adult health conditions and

suggested methods for improving

their quality.1,35 To our knowledge, a

review focused on CPRs for child

health conditions has not yet been un-

dertaken. We hypothesized that meth-

odologic standards may need to be

modified to meet the needs of child

health practitioners, researchers, chil-

dren, and parents because (1) severe

outcomes for many pediatric condi-tions are uncommon, (2) uncertainty is

inherent in communicating with and

examining young children, and (3) par-

ents and physicians place a particu-

larly high value on not missing impor-

tant diagnoses in children.811

The objectives of this study were to sys-

tematically identify existing studies of

the derivation, validation, or impact of

CPRs for health conditions of childhood;

to evaluate their methodologic quality,performance, and rule validation by us-

ing current methodologic standards; to

comparechild CPRs with adult CPRs; and

to suggest potential modifications to

those standards for CPRs developed for

health conditions of childhood.

METHODS

Search Strategy

We searched Medline and the Evidence-

Based Medicine Reviewsup to April 2011.

Because there is no medical subject

heading that specifies CPRs, an elec-

tronic search strategy was developed by

an expert librarian (Ms Uleryk) on the

basis of a previously validated strategy

that was modified to exclude studies of

only adults (see Appendix).12,13 In addi-

tion, thereference lists of identified CPRs

were searched manually. There was no

restriction on language.

Inclusion Criteria

Only prospective or retrospective stud-

ies that derived, validated, or assessed

the impact of CPRs were included. A

CPR was defined as a clinical decision-

making tool that1,2,5:

includes 3 or more predictive vari-

ables obtained from the history,

physical examination, or simple di-

agnostic tests;

provides the probability of an out-

come or suggests a diagnostic or

therapeutic course of action for an

individual patient; and

is not a decision analysis, decision

support tool, or practice guideline.

Only studies that involved children

(from term birth to 18 years of age)

were included. Studies that involved

both adults and children were in-

cluded if a separate analysis was per-

formed for children. Studies that re-

quired the use of artificial neural

networks were not included.

Selection of Studies

Two reviewers (Drs Maguire and Kulik)independently assessed the inclusion

of potentially relevant articles by using

a 2-step process. First, the title and ab-

stract from each article identified by

the electronic search were assessed

for inclusion. Second, when publica-

tions were identified as potentially rel-

evant according to title and abstract or

when uncertainty existed, the publica-

tions were reviewed manually. When

there was discrepancy between the 2reviewers, the studies were discussed

and decisionswere made by DrsMagu-

ire and Kulik by consensus. Blinding of

journal, institution, and author was not

performed.

Assessment of Methodologic

Quality

The quality of included studies was as-

sessed by using 17 items from pub-

lished guidelines for use in the devel-opment of CPRs.15 Each item was

recorded as present (score of 1) or ab-

sent (score of 0), and the maximum

total score was 17 (Table 1). Differ-

ences in opinion between reviewers

(Drs Maguire and Kulik) were resolved

by consensus. Given the importance of

rule validation, hierarchy of rule vali-

dation was assessed separately (see

next section).

REVIEW ARTICLES

PEDIATRICS Volume 128, Number 3, September 2011 e667at Indonesia:AAP Sponsored on May 30, 2013pediatrics.aappublications.orgDownloaded from
http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/

7/28/2019 malariaaa

4/14

Assessment of Rule Performance

We evaluated CPR performance by us-

ing sensitivity and negative likelihood

ratio (LR), which have been used by

others for evaluating CPRs.1,2,5,13,14 Sen-

sitivity and LR are also independent

of disease prevalence, which makes

them useful measures for comparing

CPRs from different populations with

different outcomes.13,14 Data were ex-

tracted from each publication to con-

struct a 2 2 table to calculate sensi-

tivity, specificity, and LR and their

95% confidence intervals (CIs).1,13 The

following 4 rule-performance cutoffs

were used to identify high-performing

CPRs: sensitivity 0.95; lower limit of

sensitivity 95% CI 0.95; LR 0.1;

and upper limit of the LR 95% CI

0.1.1,2,5,13,1517 A sensitivity cutoff of

0.95 was chosen because it has been

argued that few physicians would tol-

erate missing 5% of outcomes,5,6,17

and an LR of0.1 was chosen be-

cause it is generally considered indic-

ative of a clinically useful test.13,16

Assessment of Hierarchy of Rule

Validation

Rule validation for CPRs that met inclu-

sion criteria was assessed according

to the hierarchy of evidence for CPRs

published by the Evidence-Based Med-

icine Working Group (EBMWG).2 In this

hierarchy, prediction rules that have

been derived but not validated are the

lowest level of evidence (level 4), rules

that have been prospectively validated

in only 1 sample are level 3, rules that

have been broadly validated in multi-

ple settings are level 2, and rules that

have had impact analysis performed

and demonstrated a change in clini-

cian behavior with beneficial conse-

quences are level 1.

Comparison With CPRs for Adult

Health Conditions

To compare CPRs for childhood health

conditions with those for adults, meth-

odologic quality indicators and hierar-

chy of rule validation abstracted

through this review were compared

with those abstracted from adult CPRs

published between 1981 and 2003 in 3

published reviews.1,3,4

Data ExtractionTwo reviewers (Drs Maguire and Kulik)

independently used a standardized

data-collection form to record meth-

odologic quality indicators, perfor-

mance, and hierarchy of rule valida-

tion for each study. Discrepancies

between the reviewers were dis-

cussed and resolved by consensus.

Summary statistics were calculated by

using SAS 9.0 (SAS Institute, Inc Cary,

NC).

RESULTS

Study Selection

The electronic search strategy identi-

fied 7298 citations, which were

screened by title and abstract to yield

392 potentially relevant studies. Re-

view of the full text of these studies

revealed 137 studies that fulfilled all

inclusion criteria (Fig 1).11,18152

TABLE 1 Assessment of Methodologic Quality

Quality Item Reports That Met Quality Item, % (n)

Childrens Studies Adult Studiesa

All

(19822011) (N 137)

Recent

(20052011) (N 61)

Level 2 Studies

(19822011) (N 24)

Wasson et al3

(19811984) (N 33)

Laupacis et al1

(19911994) (N 29)

Prospective 72 (98) 67 (41) 100 (23) b b

Study site well described 52 (71) 55 (33) 71 (17) 94 (32) 66 (19)Population well described 81 (11) 80 (49) 92 (22) 76 (25) 79 (23)

Rule applied to all patients at risk 55 (76) 43 (26) 75 (18) b b

Predictive variables

Definition 53 (72) 57 (35) 54 (13) 97 (32) 59 (17)

Blind assessment 47 (65) 39 (24) 62 (15) 27 (3) 79 (23)

Reproducible 18 (24) 18 (11) 25 (6) b 3 (1)

Outcome variable

Definition 84 (115) 79 (48) 96 (23) 85 (28) 83 (24)

Blind assessment 42 (58) 28 (17) 46 (11) 25 (3) 41 (12)

Adequate follow-up 36 (49) 39 (24) 42 (10) b b

Sensibility

Clinically sensible 99 (135) 98 (60) 100 (24) b 97 (28)

Easy to use 69 (94) 64 (39) 79 (19) b 41 (12)

Course of action 57 (78) 59 (36) 79 (19) b b

Statistical analysis

Mathematical technique reported 89 (122) 89 (54) 96 (23) 82 (23) 100 (29)

Adequate power 43 (59) 50 (30) 46 (11) b

Adequate reporting of results 49 (67) 57 (35) 63 (15) b 100 (29)

95% CIs reported on rule properties 36 (49) 51 (31) 46 (11) b b

a Reilly and Evans4 did not publish a detailed assessment of CPR methodologic quality.b Data not available.


7/28/2019 malariaaa

5/14

Characteristics of Included

Studies

Included studies were published be-

tween 1982 and 2011 (Fig 2) and in-

volved a total of 285 404 children. No

studies were identified before 1982.

The reports on these studies de-

scribed the development of 101 unique

CPRs that address 36 pediatric condi-

tions (Table 2). The most common con-

ditions were occult serious bacterial

infection in infants, streptococcal

pharyngitis, bacterial meningitis, ap-

pendicitis, intracranial injury, extrem-

ity fractures, and malaria. The articles

were published in a wide variety of

journals including 4 general medical, 6

general pediatric, 6 emergency medi-

cine, 6 surgical, 2 family medicine, and

22 subspecialty journals. The median

number of children enrolled for the

derivation of each CPR was 324 (range:

37140 661). The median number of

predictors assessed for possible inclu-

sion was 10 (range: 3 84), and the me-

dian number of predictors included inthe rules was 5 (range: 317). The me-

dian prevalence of the outcome being

predicted by the rule was 0.25 (range:

0.006 [intracranial injury] to 0.75

[appendicitis]).

Assessment of Methodologic

Quality

Studies met between 2 and 16 of the 17

quality items (Fig 3 and Table 1). The

quality item that was met most fre-

quently was the clinical sensibility of

the rule (99%). Quality items met in

fewer than half of the studies included

reproducibility of predictor variable

assessment (18%), adequate follow-up

to assess outcomes (36%), CIs re-ported on rule properties (36%), ade-

quate blinding of outcomes (42%), suf-

ficient study power (43%), adequate

blinding of predictor variables (47%),

and adequate reporting of results

(49%). Of the 8 studies that met 16 of

the 17 quality items,* 3 were level 2

studies (validated in multiple set-

tings), and the remainder were level 4

studies (derived but not validated).

Quality was higher for level 2 studiescompared with level 3 and 4 studies

(median: 12 vs 10 vs 9 quality items,

respectively; Pfor trend .01).

Assessment of Rule Performance

The median sensitivity of the rules was

0.97 (range: 0.171.0), and the median

lower 95% confidence limit was 0.85

(range: 0.10 0.99). The median LR

was 0.1 (range: 0.0010.89), and the

median upper 95% confidence limit

was 0.4 (range: 0.015.0).

Sixty studies (48%) had a sensitivity of

0.95, and 13 of them (11%) had a

lower 95% confidence limit of0.95.

Fifty studies (43%) had an LR of0.1,

and 7 of them (6%) had an upper 95%

confidence limit of 0.1. Forty-four

studies (35%) had both sensitivities of

95% and an LR of0.1. Three stud-

ies (3%) had both a lower 95% confi-

dence limit for sensitivities of0.95

and an upper 95% CI for LR of0.1.

Assessment of Hierarchy of Rule

Validation

Using the hierarchy described by the

EBMWG2to assess the degree of valida-

tion for the 101 CPRs, 76 rules (76%)

were derived but not validated (level 4

evidence), 17 (17%) were prospec-

*Refs 11, 59, 60, 96, 98, 121, 124, and 134.

Titles and abstracts retrieved from electronic and

bibliographical searches (N= 7298)

Titles and abstracts that were not relevant, excluded

(n = 6906)

Full-text articles excluded (n = 255)

Prognostic score (n = 108)

7/28/2019 malariaaa

6/14

tively validated in 1 sample (level 3 ev-

idence), 8 (8%) were validated in more

than 1 setting (level 2 evidence), and

no rule had undergone impactanalysis

(see Table 3 and Fig 4). Of the 8 level 2

rules, one was prospectively validated

in 6 settings,39,66,73,84,98,124 one wasprospectively validated in 5 set-

tings,29,30,43,52,65 3 were prospective-

ly validated in 4 settings, and 3

were prospectively validated in 2

settings.28,37,52,87,102,124

Comparison With CPRs for Adult

Health Conditions

Compared with CPRs for adult health

conditions, the methodologic quality

(Table 1) and hierarchy of rule valida-tion (see Table 3 and Fig 4) qualitatively

seem to be lower for CPRs for child

health conditions.

DISCUSSION

We performed a systematic review to

identify published CPRs for health con-

ditions of childhood. One hundred

thirty-seven study reports that de-

scribed the development of 101 unique

CPRs that addressed 36 childhood con-

ditions were identified. The most inten-

sively investigated conditions were

acute infections and trauma, which

are attractive candidates for improv-

ing patient care with a CPR because

they are common, have the potential

for poor outcomes, are prone to con-

siderable clinical diagnostic uncer-

tainty, and frequently lead to diagnos-

tic testing or treatment that may be

unnecessary or harmful.

To evaluate the current state of CPRs

for child health conditions, we de-

scribed their methodologic quality,

performance, and hierarchy of rule

validation by using previously de-

scribed guidelines for CPRs. The most

important quality deficiencies that af-

fected the majority of studies were

inadequate blinding of predictor vari-

Refs 34, 54, 77, 8183, 87, 95, 103, 107, and 121.

TABLE 2 Clinical Conditions for Which CPRs Have Been Developed for Children

Outcome Population of Children No. of Studies

(N 137)

Occult serious bacterial infection Febrile infants 21

Febrile neutropenia 4

Streptococcal pharyngitis Sore throat 13

Bacterial meningitis Children at risk of meningitis 12

Appendicitis Abdominal pain 11Intracranial injury Head trauma 11

Extremity fracture Blunt ankle injury 11

Malaria Fever in malaria-endemic region 6

Chest radiograph infiltrate Suspected pneumonia 4

Septic joint Irritable joint 4

Vesicoureteric reflux Urinary tract infection 3

Intra-abdominal injury Blunt abdominal trauma 3

Lyme meningitis Meningitis 2

Urinary tract infection Young girls with fever 2

Normal chest radiograph Respiratory syncytial virus infection 2

Influenza Influenza-like illness 3

Safe discharge from the

emergency department

Bronchiolitis 2

Dehydration Vomiting or diarrhea 2

Uneventful course Idiopathic thrombocytopenia 1

Pathologic diagnosis Back pain 1

Pneumocystis pneumonia HIV infection and pneumonia 1

Persistent disease Graves disease 1

Undervaccination Emergency department patients 1

False-positive blood culture Children in the emergency department

with blood culture taken

1

Emergency operative management Trauma 1

Intrathoracic injury Blunt torso trauma 1

Cervical spine injury Trauma 1

Difficult i ntravenous ac ce ss Children w ho re quire an intravenous line 1

Cervical infection Adolescents who require pelvic exam 1

Active rickets Third-world children with leg deformity 1

Tumor lysis syndrome Leukemia 1

Cervical infection Suspected pelvic inflammatory disease 1HIV infection Suspected HIV infection 1

Pulmonary embolism Suspected pulmonary embolism 1

Tuberculosis Suspected tuberculosis 1

Pyloric stenosis Suspected pyloric stenosis 1

Esophageal varices Chronic liver disease 1

FIGURE 3Number of CPRs for health conditions of childhood that included each number of quality items.


7/28/2019 malariaaa

7/14

ables and outcomes, limited assess-

ment of the reproducibility of predic-

tor variables, inadequate follow-up to

assess outcomes, insufficient study

power, inadequate reporting of results

such as sensitivity and specificity, and

lack of 95% CI reporting on ruleproperties.

The methodologic quality measures

used in the current study have been

used previously to evaluate CPRs1,35,17

and are quite similar to validated

items used for evaluating the quality of

diagnostic tests.153155 Furthermore,

there is considerable evidence that

CPRs that fail to meet these methodo-

logic standards are likely to be biased.

Deficiencies in blinding may result in

an overestimation of diagnostic per-

formance.16,153,155 Assessment of the in-

terobserver reliability of predictor

variables is necessary to determine if

a rule will perform similarly when

used by other physicians.1,5 Insufficient

power for statistical modeling caused

by an inadequate number of outcomes

for a given number of predictor vari-

ables increases the possibility of spu-

rious results through random ef-

fects.14,156,157 Incomplete follow-up for

important outcomes can result in

missed outcomes that can lead to an

overestimation of diagnostic perfor-

mance,5 which is particularly impor-

tant for studies in which the reference

standard forassessingoutcomes,such as

cranial computed-tomographic scanning

for pediatric minor head injury, is not

or cannot be applied to all patients,

and clinical follow-up is used to cap-ture outcomes.17 Lastly, inadequate re-

porting of CPR results makes it impos-

sible for clinicians to know if the

performance of a rule is adequate to

meet their needs.1,3 We find it concern-

ing that 51% of the studies did not re-

port sensitivity and specificity. These

quality deficiencies should be im-

proved when developing CPRs for child

health conditions in the future.

Thirty-five percent of identified studies

had both a CPRsensitivity of0.95 and

anLR of0.1, but only 3% of the stud-

ies had 95% confidence in both of

these performance indicators. Al-

though CPRs that are almost perfect

may be desirable, it may not be attain-

able for pediatric clinical scenarios in

which the history and physical exami-

nation are prone to interobserver vari-

ability (particularly common with very

young children), the number of chil-

dren with the disorder of interest is

small, and the outcomes are rare. It

may simply be impossible to recruit

the tens of thousands of patients re-

quired to develop and then prospec-

tively validate CPRs to this degree

of precision for many child health con-

ditions. Furthermore, attempts to

achieve 95% sensitivity may come at a

cost of overexposing children to harm-

ful effects of diagnostic testing such

as computed-tomographic scanning,

which is associated with measurable

lifetime risk of lethal malignancy.158,159

Therefore, the challenges of attaining

ideal rule performance must be recon-

ciled with the realities of pediatric

practice and the expectations of physi-

cians and parents. Seeking 95%

sensitivity for CPRs for child health

conditions maybe an elusive and coun-

terproductive goal, especially when

the sensitivity of a less-than-perfect

CPR is superior to a clinicians judg-

ment alone. As the field of evidence-

based medicine moves toward involv-

ing patients in decision-making,160,161

there may be opportunities to increase

the interface between 2 areas of re-

search: CPRs (aimed at assisting clini-

cians) and decision aids (aimed at as-

sisting patients/parents in making

choices that fit with their values and

preferences).162 We propose that, mov-

ing forward, CPRs for childhood health

conditions be considered aids to clini-

cal decision-making and not rigid

rules. We suggest that CPRs for child

TABLE 3 Hierarchy of Rule Validation

Hierarchy of Rule

Validation

Rules, % (n)

Child Rules, Present

Study (19822011)

(N 101)a

Adult Rules

Wasson et al3

(19811984)a

(N 33)

Laupacis et al1

(19911994)a

(N 29)

Reilly and Evans4

(20002003)a

(N 41)

Level 4: derivation 76 (76) 56 (20) 47 (15) 24 (10)Level 3: narrow validation 17 (17) 28 (10) 13 (4) 24 (10)

Level 2: broad validation 8 (8) 11 (4) 34 (11) 39 (16)

Level 1: impact analysis 0 (0) 6 (2) 6 (2) 12 (5)

a Year of publication of CPRs included in each review.

FIGURE 4Percentage of CPRs for children and adults that met each level of the EBMWG hierarchy.

REVIEW ARTICLES

http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/

7/28/2019 malariaaa

8/14

health conditions aim to empower clini-

cians with data to augment clinical judg-

ment and incorporate the values and

preferences of parents and children in

the decision-making process.11

CPRs for childhood health conditions

seem to lag behind their adult counter-

parts in terms of the EBMWG hierarchy

of rule validation.2 However, close in-

spection of similar rules for children

and adults reveals that there are

unique challenges to undertaking the

derivation, validation, and impact anal-

ysis of CPRs for children. Two exam-

ples are illustrative. In the first exam-

ple, investigators studied CPRs for

ankle injuries in both children and

adults.39,73,98,163 Although the authors of

these studies reported similar mecha-

nisms, outcome rates, number of

study sites, sample sizes, and rule-

performance characteristics, the

studies in children took 2 to 3 times as

long to complete as those for adults. In

the second example, investigators

studied CPRs for traumatic brain in-

jury in both children and adults.11,164 Al-

though the authors of those studies

reported similar rule-performancecharacteristics, the study in children

demonstrated 10-fold fewer outcomes,

and required twice the number of

study sites and study subjects as the

study in adults.

Of the 101 rules assessed by using the

EBMWG hierarchy, only 8 rules for

health conditions of childhood would

be considered rules that can be used

in various settings with confidence in

their accuracy (level 2 evidence) be-

cause of prospective validation in

broad or multiple settings. The re-

mainder would be considered rules

that clinicians may use with caution

(level 3 evidence) because of valida-

tion in only 1 narrow prospective sam-

ple or rules that require further eval-

uation (level 4 evidence) because they

are not validated or validated only in

split samples, large retrospective da-

tabases, or by statistical techniques.

However, close inspection of the 70

rules that the EBMWG has categorized

as level 4 reveals a spectrum of valida-

tion methods, some of which may pro-

vide a level of evidence higher than

level 4. Examples of these methods in-clude (1) statistical techniques such

as cross-validation, bootstrapping,

classification, and regression-tree

techniques,25,36,38,60 (2) split-set valida-

tion using a percentage of the data to

derive the rule and the remainder to

validate it,88 and (3) prospective valida-

tion using an extension of the deriva-

tion cohort in which the derivation set

is closed and the rule is derived after

enrolling an a priori number of out-comes followed by prospective enroll-

ment of a validation cohort using the

full set of predictor variables and clini-

cians blinded to the derived rule.11

Each of these techniques provides

progressively superior validation

but may all be considered level 4 ev-

idence, equivalent to rules with no

validation, according to the EBMWG

hierarchy. Given the challenges in-

volved with achieving the samplesizes needed to provide level 1 and 2

CPR validation for child health condi-

tions, we suggest that these efficient

approaches to validation be differen-

tiated in future modifications of the

evidence-based medicine validation

hierarchy for CPRs.

It is important to acknowledge the lim-

itations of this systematic review.

First, our electronic search strategy

may not have identified all CPRs forchildren. However, examination of the

reference lists of all identified

prediction-rule publications failed to

reveal any additional studies. Second,

the proposed quality and performance

metrics we used treated each item

with equal weight, and certain compo-

nents may be more important than

others. Third, although most CPRs

maximized sensitivity at the expense of

specificity, this was not true for all

rules. For the few that prioritized spec-

ificity over sensitivity, our sensitivity

and LR performance benchmarks

are not appropriate. Finally, although

the items we used to assess methodo-

logic quality and performance havebeen well described in the literature,

they have not been rigorously devel-

oped or validated.15,13

CONCLUSIONS

High-performing, rigorously devel-

oped, and well-validated CPRs have the

potential for improving child health

outcomes and limiting resource use

but are uncommonly developed and

rarely used in pediatric practice. Wehave identified several important is-

sues related to the quality, perfor-

mance, and validation of CPRs for

childhood health conditions that are

barriers to their development and im-

plementation. We have made several

recommendations including modifying

existing methodologic standards to in-

clude more efficient approaches to val-

idation and considering CPRs for chil-

dren to be more assistive and less

directive. We also suggest that pediat-

ric CPRs attempt to incorporate pa-

tients and parents values and prefer-

ences in the decision-making process,

especially when the performance of

high-quality rules is less than ideal but

considerably better than a clinicians

judgment alone. We hope that this re-

view will assist developers and users

of pediatric CPRs in overcoming these

barriers and increase the use of CPRs

in pediatric practice.

APPENDIX: ELECTRONIC SEARCH

STRATEGY

With the assistance of Ms Uleryk (di-

rector of the Hospital for Sick Children

Library), a comprehensive literature

search was run by using the OVID

search platform in Medline and the

Evidence-Based Medicine Reviews

from the beginning of the database un-


7/28/2019 malariaaa

9/14

til April 2011. The following terms were

searched by using specific database

indexing and text-word equivalents to

identify articles for review: (models,

statistical/ or Monte Carlo method/ or

probability/ or regression analysis/ or

multivariate analysis/ or predict*.mp.)and (Decision Trees/ or predictive

value of tests/ or ((decision: or pre-

dict:) adj5 (rule: or model: or algo-

rithm: or aid or score:)).ti,ab.) and

(limit to age groups birth to 18 years of

age or pediatrics/) and (cohort stud-

ies/ or longitudinal studies/ or

follow-up studies/ or prospective stud-

ies/ or prognosis/ or disease-free sur-

vival/ or treatment outcome/ or treat-

ment failure/ or disease progression/

or morbidity/ or incidence/ or preva-

lence/ or mortality/ or cause of death/

or fatal outcome/ or hospital mortali-ty/ or infant mortality/ or maternal

mortality/ or survival rate/ or survival

analysis/ or disease-free survival/ or

natural history.tw. or evaluation stud-

ies.pt. or evaluation studies as topic/

or validation studies.pt. or validation

studies as topic/ sensitivity and spec-

ificity/ or predictive value of tests/ or

ROC curve/ or diagnostic errors/ or

false negative reactions/ or false posi-

tive reactions/ or observer variation/

or likelihood functions/ or (likelihood

or likelihood ratio:).tw.

ACKNOWLEDGMENTS

The Pediatric Outcomes Research

Team is supported by a grant from the

Hospital for Sick Children Foundation.

Dr Maguire was supported by a Cana-

dian Institutes of Health Research

fellowship.

REFERENCES

1. Laupacis A, Sekar N, Stiell IG. Clinical pre-

diction rules: a review and suggested

modifications of methodological stan-

dards. JAMA. 1997;277(6):488 494

2. McGinn TG, Guyatt GH, Wyer PC, Naylor CD,

Stiell IG, Richardson WS. Users guides to

the medical literature: XXIIhow to use

articles about clinical decision rules.

Evidence-Based Medicine Working Group.

JAMA. 2000;284(1):79 84

3. Wasson JH, Sox HC, Neff RK, Goldman L.

Clinical prediction rules: applications and

methodological standards. N Engl J Med.

1985;313(13):793799

4. Reilly BM, Evans AT. Translating clinical re-

search into clinical practice: impact of us-

ing prediction rules to make decisions.

Ann Intern Med. 2006;144(3):201209

5. Stiell IG, WellsGA. Methodologic standards

for the development of clinical decision

rules in emergency medicine. Ann Emerg

Med. 1999;33(4):437447

6. Stiell IG, Lesiuk H, Wells GA, et al; Canadian

CT Head and C-Spine Study Group. The Ca-

nadian CT Head Rule Study for patients

with minor head injury: rationale, objec-

tives, and methodologyfor phase I (deriva-

tion). AnnEmerg Med. 2001;38(2):160169

7. Stiell IG, Lesiuk H, Wells GA, et al; Canadian

CT Head and C-Spine Study Group. Cana-

dian CT Head Rule Study for patients with

minor head injury: methodology for phase

II (validation and economic analysis). Ann

Emerg Med. 2001;38(3):317322

8. Bennett JE, Sumner W 2nd, Downs SM,

JaffeDM. Parents utilities for outcomes of

occult bacteremia. Arch Pediatr Adolesc

Med. 2000;154(1):4348

9. Apkon M.Decisionanalysis andclinicalun-

certainty. Curr Opin Pediatr. 2003;15(3):

272277

10. Brennan CA, Somerset M, Granier SK, Fa-

hey TP, Heyderman RS. Management of di-

agnostic uncertainty in children with pos-

sible meningitis: a qualitative study. Br J

Gen Pract. 2003;53(493):626 631

11. Kuppermann N, Holmes JF, Dayan PS, et al;

Pediatric Emergency Care Applied Re-

search Network (PECARN).Identificationof

children at very low risk of clinically-

important brain injuries after head

trauma: a prospective cohort study. Lan-

cet. 2009;374(9696):1160 1170

12. Ingui BJ, Rogers MA. Searching for clinical

prediction rules in Medline. J Am Med In-

form Assoc. 2001;8(4):391397

13. Hess EP, Thiruganasambandamoorthy V,

Wells GA, et al. Diagnostic accuracy of clin-

ical prediction rules to exclude acute cor-

onary syndrome in the emergency depart-

ment setting: a systematic review. CJEM.

2008;10(4):373382

14. Concato J, Feinstein AR, Holford TR. The

riskof determining riskwith multivariable

models. Ann Intern Med. 1993;118(3):

201210

15. Stiell IG, McKnight RD, Greenberg GH, et al.

Implementation of the Ottawa ankle rules.

JAMA. 1994;271(11):827 832

16. Jaeschke R, Guyatt G, Lijmer J. Diagnostic

tests. In: Guyatt G, Rennie D, Meade MO,

Cook DJ, eds. Users Guides to the Medical

Literature. Chicago, IL: AMA Press; 2002:

121140

17. Maguire JL, Boutis K, Uleryk EM, Laupacis A,

Parkin PC. Should a head-injured child receive

a head CT scan?A systematicreview of clinical

prediction rules. Pediatrics. 2009;124(1). Avail-

able at: www.pediatrics.org/cgi/content/full/

124/1/e145

18. Haydel MJ, Shembekar AD. Prediction of

intracranial injury in children aged five

years and older with loss of conscious-

ness after minor head injury due to non-

trivial mechanisms. Ann Emerg Med. 2003;

42(4):507514

19. Friedman JN, Goldman RD, Srivastava R,

Parkin PC. Development of a clinical dehy-

dration scale forusein children between1

and 36 months of age. J Pediatr. 2004;

145(2):201207

20. Da Dalt L, Marchi AG, Laudizi L, et al. Pre-

dictors of intracranial injuries in children

after blunt head trauma. Eur J Pediatr.

2006;165(3):142148

21. Chandramohan D, Carneiro I, Kavishwar A,

Brugha R, Desai V, Greenwood B. A clinical

algorithm for the diagnosis of malaria: re-

sults of an evaluation in an area of low

endemicity. Trop MedInt Health. 2001;6(7):

505510

22. Attia M, Zaoutis T, Eppes S, Klein J, Meier F.

Multivariate predictive models for group A

beta-hemolytic streptococcal pharyngitis

in children. Acad Emerg Med. 1999;6(1):

813

23. Attia MW, Zaoutis T, Klein JD, Meier FA. Per-

formance of a predictive model for strep-

toco ccal phary ngit is in chil dren [pub-

lished correction appears in Arch Pediatr

Adolesc Med. 2001;155(10):1179]. Arch Pe-diatr Adolesc Med. 2001;155(6):687 691

24. Ammann RA, Hirt A, Luthy AR, Aebi C. Iden-

tification of children presenting with fever

in chemotherapy-induced neutropenia at

low risk for severe bacterial infection.

Med Pediatr Oncol. 2003;41(5):436 443

25. Atabaki S, Steill IG, Bazarian JJ, et al. A

clinical decision rulefor cranial computed

tom ogr aph y in min or ped iat ric hea d

trauma. Arch Pediatr Adolesc Med. 2008;

162(5):439 445

26. Avery RA, Frank G, Glutting JJ, et al. Predic-

REVIEW ARTICLES

http://www.pediatrics.org/cgi/content/full/124/1/e145http://www.pediatrics.org/cgi/content/full/124/1/e145http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/http://www.pediatrics.org/cgi/content/full/124/1/e145http://www.pediatrics.org/cgi/content/full/124/1/e145

7/28/2019 malariaaa

10/14

tion of Lyme meningitis in children from a

Lyme diseaseendemic region: a logistic-

regression model using history, physical,

and laboratory findings. Pediatrics. 2006;

117(1). Available at: www.pediatrics.org/

cgi/content/full/117/1/e1

27. Bachur RG, Harper MB. Predictive model

for serious bacterial infections among in-fants younger than 3 months of age. Pedi-

atrics. 2001;108(2):311316

28. Baker MD, Bell LM, Avner JR. Outpatient

management without antibiotics of fever

in selected infants. N Engl J Med. 1993;

329(20):14371441

29. Baskin MN, ORourke EJ, Fleisher GR. Out-

patient treatment of febrile infants 28 to

89 days of age with intramuscular admin-

istration of ceftriaxone. J Pediatr. 1992;

120(1):2227

30. Berger RM, Berger MY, van Steensel-Moll

HA, Dzoljic-Danilovic G, Derksen-Lubsen G.A predictive model to estimate the risk of

serious bacterial infections in febrile in-

fants. Eur J Pediatr. 1996;155(6):468 473

31. Bleeker SE, Moons KG, Derksen-Lubsen G,

Grobbee DE, Moll HA. Predicting serious

bacterial infection in young children with

fever without apparent source. Acta Pae-

diatr. 2001;90(11):12261232

32. Bleeker SE, Moll HA, Steyerberg EW, et al.

External validation is necessary in predic-

tion research: a clinical example. J Clin

Epidemiol. 2003;56(9):826 832

33. Bleeker SE, Derksen-LubsenG, GrobbeeDE,

et al. Validating and updating a prediction

rule for serious bacterial infection in pa-

tients with fever without source. Acta Pae-

diatr. 2007;96(1):100104

34. Bond GR, Tully SB, Chan LS, Bradley RL. Use

of the MANTRELS score in childhood

appendicitis: a prospective study of 187

children with abdominal pain. Ann Emerg

Med. 1990;19(9):10141018

35. Bonsu BK, Harper MB. Differentiating

acute bacterial meningitis from acute vi-

ral meningitis among children with cere-

brospinal fluid pleocytosis: a multivari-

able regression model. Pediatr Infect Dis

J. 2004;23(6):511517

36. Bonsu BK, Ortega HW, Marcon MJ, Harper

MB. A decision rule for predicting bacte-

rial meningitis in children with cerebro-

spinalfluidpleocytosiswhenGram stain is

negative or unavailable. Acad Emerg Med.

2008;15(5):437444

37. Boutis K, Komar L, Jaramillo D, et al. Sen-

sitivity of a clinical examination to predict

need for radiography in children with an-

kle injuries: a prospective study. Lancet.

2001;358(9299):2118 2121

38. ChavanetP, SchallerC, Levy C, et al.Perfor-

mance of a predictive rule to distinguish

bacterial and viral meningitis. J Infect.

2007;54(4):328336

39. Clark KD, Tanner S. Evaluation of the Ot-

tawa ankl e rule s in chi ldre n. Pediatr

Emerg Care. 2003;19(2):7378

40. Cotton BA, Beckert BW, Smith MK, Burd RS.The utility of clinical and laboratory data

for predicting intraabdominal injury

among children. J Trauma. 2004;56(5):

10681074; discussion 10741065

41. Crain EF,BulasD, BijurPE, Goldman HS.Is a

chest radiograph necessary in the evalua-

tion of every febrile infant less than 8

weeks of age? Pediatrics. 1991;88(4):

821 824

42. Dado G, Anania G, Baccarani U, et al. Appli-

cation of a clinical score for the diagnosis

of acute appendicitis in childhood: a retro-

spective analysis of 197 patients. J PediatrSurg. 2000;35(9):13201322

43. Dagan R, Sofer S, Phillip M, Shachak E. Am-

bulatory care of febrile infants younger

than 2 months of age classified as being at

low risk for having serious bacterial infec-

tions. J Pediatr. 1988;112(3):355360

44. Dayan PS, Vitale M, Langsam DJ, et al. Der-

ivation of clinical prediction rules to iden-

tify children with fractures after twisting

injuries of the ankle. Acad Emerg Med.

2004;11(7):736743

45. Dubos F, Lamotte B, Bibi-Triki F, et al. Clin-

ical decision rules to distinguish between

bacterial and aseptic meningitis. Arch Dis

Child. 2006;91(8):647 650

46. Dunning J, Daly JP, Lomas JP, et al. Deriva-

tion of the childrens head injury algo-

rithm for the prediction of important clin-

ical events decision rule for head injury in

children. Arch Dis Child. 2006;91(11):

885 891

47. Edslev PW, Rosthoj S, Treutiger I, et al. A

clinical score predicting a brief and un-

eventful course of newly diagnosed idio-

pathic thrombocytopenic purpura in chil-

dren. Br J Haematol. 2007;138(4):513516

48. Fatti GL, Zar HJ, Swingler GH, Fatti GL, Zar

HJ, Swingler GH. Clinical indicators of

Pneumocystis jirovecipneumonia (PCP) in

South African children infected with the

human immunodeficiency virus. Int J In-

fect Dis. 2006;10(4):282285

49. Feldman DS, Straight JJ, Badra MI, Mo-

haideen A, Madan SS. Evaluation of an al-

gorithmic approach to pediatric back

pain. J Pediatr Orthop. 2006;26(3):353357

50. Fine AM, Nigrovic LE, Reis BY, Cook EF,

Mandl KD. Linking surveillance to action:

incorporation of real-time regional data

into a medical decision rule. J Am Med In-

form Assoc. 2007;14(2):206211

51. SteinhoffMC, Walker CF, Rimoin AW, Hamza

HS. A clinical decision rule for manage-

ment of streptococcal pharyngitis in low-

resource settings. Acta Paediatr. 2005;

94(8):1038 1042

52. Garra G, Cunningham SJ, Crain EF. Reap-praisal of criteria used to predict serious

bacterialillness in febrile infants less than

8 weeks of age. Acad Emerg Med. 2005;

12(10):921925

53. Glaser NS, Styne DM; Organization of Pedi-

atric Endocrinologists of Northern Califor-

nia Collaborative Graves Disease Study

Group. Predicting the likelihood of remis-

sion in children with Graves disease: a

prospective, multicenter study. Pediatrics.

2008;121(3). Available at: www.pediatrics.

org/cgi/content/full/121/3/e481

54. Goldman RD, Carter S, Stephens D, AntoonR, Mounstephen W, Langer JC. Prospective

validation of the pediatric appendicitis

score. J Pediatr. 2008;153(2):278282

55. Gorelick MH, Shaw KN, Murphy KO. Validity

and reliability of clinical signs in the diag-

nosis of dehydration in children. Pediat-

rics. 1997;99(5). Available at: www.

pediatrics.org/cgi/content/full/99/5/e6

56. Gorelick MH, Shaw KN. Clinical decision

rule to identify febrile young girls at risk

for urinary tract infection. Arch Pediatr

Adolesc Med. 2000;154(4):386390

57. Gorelick MH, Hoberman A, Kearney D, Wald

E, Shaw KN. Validation of a decision rule

identifying febrile young girls at high risk

for urinary tract infection. Pediatr Emerg

Care. 2003;19(3):162164

58. Greenes DS, Schutzman SA. Clinical signif-

icance of scalp abnormalities in asymp-

tom ati c hea d-i nju red inf ant s. Pediatr

Emerg Care. 2001;17(2):88 92

59. Holmes JF, Sokolove PE, Brant WE, Kupper-

mann N. A clinical decision rule for identi-

fying children with thoracic injuries after

blunt torso trauma. Ann Emerg Med. 2002;

39(5):492 499

60. Holmes JF, Sokolove PE, Brant WE, et al.

Identification of children with intra-

abdominal injuries after blunt trauma.

Ann Emerg Med. 2002;39(5):500509

61. Lynch T, Platt R, Gouin S, Larson C, Pat-

enaude Y. Can we predict which children

with clinically suspected pneumonia will

have the presence of focal infiltrates on

chest radiographs? Pediatrics. 2004;113(3

pt 1). Available at: www.pediatrics.org/


62. Humiston SG, Rodewald LE, Szilagyi PG, et

al. Decision rules for predicting vaccina-

http://www.pediatrics.org/cgi/content/full/117/1/e1http://www.pediatrics.org/cgi/content/full/117/1/e1http://www.pediatrics.org/cgi/content/full/121/3/e481http://www.pediatrics.org/cgi/content/full/121/3/e481http://www.pediatrics.org/cgi/content/full/99/5/e6http://www.pediatrics.org/cgi/content/full/99/5/e6http://www.pediatrics.org/cgi/content/full/113/3/e186http://www.pediatrics.org/cgi/content/full/113/3/e186http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://www.pediatrics.org/cgi/content/full/113/3/e186http://www.pediatrics.org/cgi/content/full/113/3/e186http://www.pediatrics.org/cgi/content/full/99/5/e6http://www.pediatrics.org/cgi/content/full/99/5/e6http://www.pediatrics.org/cgi/content/full/121/3/e481http://www.pediatrics.org/cgi/content/full/121/3/e481http://www.pediatrics.org/cgi/content/full/117/1/e1http://www.pediatrics.org/cgi/content/full/117/1/e1

7/28/2019 malariaaa

11/14

tion status of preschool-age emergency

department patients. J Pediatr. 1993;

123(6):887 892

63. Isaacman DJ, Shults J, Gross TK, Davis PH,

Harper M. Predictors of bacteremia in fe-

brile children 3 to 36 months of age. Pedi-

atrics. 2000;106(5):977982

64. Jaffe DM, Binns H, Radkowski MA. Develop-ing a clinical algorithm for early manage-

ment of cervical spine injury in child

trauma victims. Ann Emerg Med. 1987;

16(3):270276

65. Jaskiewicz JA, McCarthy CA, Richardson

AC, et al; Febrile Infant Collaborative Study

Group. Febrile infants at low risk for seri-

ous bacterial infection: an appraisal of the

Rochester criteria and implications for

management. Pediatrics. 1994;94(3):

390396

66. Karpas A, Hennes H, Walsh-Kelly CM. Utili-

zation of the Ottawa ankle rules by nursesin a pediatric emergency department.

Acad Emerg Med. 2002;9(2):130133

67. Kharbanda AB, Taylor GA, Fishman SJ,

Bachur RG. A clinical decision rule to iden-

tify children at low risk for appendicitis.

Pediatrics. 2005;116(3):709716

68. Kneyber MC,Moons KG,de Groot R,Moll HA.

Predictors of a normal chest x-rayin respi-

ratory syncytial virus infection. Pediatr

Pulmonol. 2001;31(4):277283

69. Kneyber MCJ, Moons KGM, de Groot R, Moll

HA. Prediction of duration of hospitaliza-

tion in respiratory syncytial virus infec-

ti on . Pediatr Pulmonol. 2002;33(6):

453 457

70. Kocher MS, Zurakowski D, Kasser JR. Dif-

ferentiating between septic arthritis and

transient synovitis of the hip in children:

an evidence-based clinical prediction al-

gorithm. J Bone Joint Surg Am. 1999;

81(12):16621670

71. Kocher MS, Mandiga R, Zurakowski D,

Barnewolt C, Kasser JR. Validation of a

clinical prediction rule for the differentia-

tion between septic arthritis and transient

synovitis of the hip in children. J Bone

Joint Surg Am. 2004;86-A(8):16291635

72. Leroy S, Marc E, Adamsbaum C, Gendrel D,

Breart G, Chalumeau M. Prediction of vesi-

coureteral reflux after a first febrile uri-

nary tract infection in children: validation

of a clinical decision rule. Arch Dis Child.

2006;91(3):241244

73. Libetta C, Burke D, Brennan P, Yassa J. Val-

idation of the Ottawa ankle rules in chil-

dren. J Accid Emerg Med. 1999;16(5):

342344

74. Lintula H, Pesonen E, Kokki H, Vanamo K,

Eskelinen M. A diagnostic score for chil-

drenwith suspected appendicitis. Langen-

becks Arch Surg. 2005;390(2):164170

75. Luhmann SJ, Jones A, Schootman M, et al.

Differentiation between septic arthritis

and transient synovitis of the hip in chil-

dren with clinical prediction algorithms. J

Bone Joint Surg Am. 2004;86-A(5):956962

76. Biss TT, Brandao LR, Kahr WHA, Chan AKC,Williams S. Clinical probability score and

D-dimer estimation lack utility in the diag-

nosis of childhood pulmonary embolism. J

Thromb Haemost. 2009;7(10):16331638

77. Macklin CP, Radcliffe GS, Merei JM,

Stringer MD. A prospective evaluation of

the modified Alvarado score for acute ap-

pendicitisin children. Ann R CollSurg Engl.

1997;79(3):203205

78. Mansbach JM, Clark S, Christopher NC, et

al. Prospective multicenter study of

bronchiolitis: predicting safe discharges

from the emergency department. Pediat-

rics. 2008;121(4):680 688

79. McCarthy PL, Sharpe MR, Spiesel SZ, et al.

Observation scales to identify serious ill-

ness in febrile children. Pediatrics. 1982;

70(5):802809

80. McConnochie KM, Roghmann KJ, Paster-

nack J, Monroe DJ, Monaco LP. Prediction

rules for selective radiographic assess-

ment of extremity injuries in children and

adolescents. Pediatrics. 1990;86(1):4557

81. McIsaac WJ, Goel V, To T, Low DE. The valid-

ity of a sore throat score in family prac-

tice. CMAJ. 2000;163(7):811815

82. McIsaac WJ, Kellner JD, Aufricht P, et al.

Empirical validation of guidelines for the

management of pharyngitis in children

and adults [published correction appears

in JAMA. 2005;294(21):2700]. JAMA. 2004;

291(13):15871595

83. McIsaac WJ, White D, Tannenbaum D, Low

DE. A clinical score to reduce unnecessary

antibiotic use in patients with sore throat.

CMAJ. 1998;158(1):75 83

84. McBride KL. Validation of the Ottawa ankle

rules: experience at a communityhospital.

Can Fam Physician. 1997;43(PubMed did

not provide):459 465

85. Meehan WP 3rd, Bachur RG. Predictors of

cerebrospinal fluid pleocytosis in febrile

infants aged 0 to 90 days. Pediatr Emerg

Care. 2008;24(5):287293

86. Moreno L, Krishnan JA, Duran P, Ferrero F.

Development and validation of a clinical

prediction rule to distinguish bacterial

from viral pneumonia in children [pub-

lished correction appears in Pediatr Pul-

monol. 2006;41(5):494]. Pediatr Pulmonol.

2006;41(4):331337

87. Fischer Walker CL, Rimoin AW, Hamza HS,

Steinhoff MC. Comparison of clinical pre-

diction rulesfor management of pharyngi-

tis in settings with limited resources. J Pe-

diatr. 2006;149(1):64 71

88. Nigrovic LE, Kuppermann N, Malley R. De-

velopment and validation of a multivari-

ablepredictive model to distinguishbacte-

rial from aseptic meningitis in children inthe postHaemophilus influenzaeera. Pe-

diatrics. 2002;110(4):712719

89. Nigrovic LE, Kuppermann N, Macias CG, et

al. Clinical prediction rule for identifying

children with cerebrospinalfluid pleocyto-

sis at very low risk of bacterial meningitis.

JAMA. 2007;297(1):52 60

90. Oman JA, Cooper RJ, Holmes JF, et al;

NEXUS II Investigators. Performance of a

decision rule to predict need for com-

puted tomography among children with

blunt head trauma. Pediatrics. 2006;



91. Oostenbrink R, van der Heijden AJ, Moons

KG, Moll HA. Prediction of vesico-ureteric

reflux in childhood urinary tract infection:

a multivariate approach. Acta Paediatr.

2000;89(7):806 810

92. Oostenbrink R, Moons KG, Donders AR,

Grobbee DE, Moll HA. Prediction of bacte-

rial meningitis in children with meningeal

signs: reduction of lumbar punctures.

Acta Paediatr. 2001;90(6):611 617

93. Oostenbrink R, Moons KG, Twijnstra MJ,

Grobbee DE, Moll HA. Children with menin-

geal signs: predicting who needs empiric

antibiotic treatment. Arch Pediatr Adolesc

Med. 2002;156(12):11891194

94. Oostenbrink R, Moons KG, Derksen-Lubsen

AG, Grobbee DE, Moll HA. A diagnostic deci-

sion rule for management of children with

meningeal signs. Eur J Epidemiol. 2004;

19(2):109116

95. Owen TD, Williams H, Stiff G, Jenkinson LR,

Rees BI.Evaluationof theAlvarado scorein

acute appendicitis. J R Soc Med. 1992;

85(2):8788

96. Palchak MJ, Holmes JF, Vance CW, et al. A

decision rule for identifying children atlow risk for brain injuries after blunt head

tra uma . Ann Emerg Med. 2003;42(4):

492506

97. Pantell RH, Newman TB, Bernzweig J, et al.

Management and outcomes of care of fe-

ver in early infancy. JAMA. 2004;291(10):

12031212

98. Plint AC, Bulloch B, Osmond MH, et al. Vali-

dation of the Ottawa ankle rules in chil-

dren with ankle injuries. Acad Emerg Med.

1999;6(10):10051009

99. Redd SC, Kazembe PN, Luby SP, et al. Clini-

REVIEW ARTICLES

http://www.pediatrics.org/cgi/content/full/117/2/e238http://www.pediatrics.org/cgi/content/full/117/2/e238http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/http://www.pediatrics.org/cgi/content/full/117/2/e238http://www.pediatrics.org/cgi/content/full/117/2/e238

7/28/2019 malariaaa

12/14

cal algorithm for treatment of Plasmo-

dium falciparummalaria in children. Lan-

cet. 1996;347(8996):223227

100. Reed JL, Mahabee-Gittens EM, Huppert JS.

A decision rule to identify adolescent fe-

males with cervical infections. J Womens

Health (Larchmt). 2007;16(2):272280

101. Rietveld E, Vergouwe Y, Steyerberg EW, etal. Hospitalizationfor respiratory syncytial

virus infection in young children: develop-

ment of a clinical prediction rule. Pediatr

Infect Dis J. 2006;25(3):201207

102. Rimoin AW, Hamza HS, Vince A, et al. Evalu-

ation of the WHO clinical decision rule for

streptococcal pharyngitis. Arch Dis Child.

2005;90(10):10661070

103. Samuel M. Pediatric appendicitis score. J

Pediatr Surg. 2002;37(6):877881

104. Santolaya ME, Alvarez AM, Becker A, et al.

Prospective, multicenter evaluation of risk

factors associated with invasive bacterial

infection in children with cancer, neutro-

penia, and fever. J Clin Oncol. 2001;19(14):

34153421

105. Santolaya ME, Alvarez AM, Aviles CL, et al.

Prospective evaluation of a model of pre-

diction of invasive bacterial infection risk

among children with cancer, fever, and

neutropenia. Clin Infect Dis. 2002;35(6):

678683

106. Sard B, Bailey MC, Vinci R. An analysis of

pediatric blood cultures in the postpneu-

mococcal conjugate vaccine era in a com-

munity hospital emergency department.Pediatr Emerg Care. 2006;22(5):295300

107. Schneider C, Kharbanda A, Bachur R. Eval-

uating appendicitis scoring systems using

a prospective pediatric cohort. Ann Emerg

Med. 2007;49(6):778784

108. Silva JMP, Diniz JSS, Lima EM, Vergara RM,

Oliveira EA. Predictivefactors of resolution

of primary vesico-ureteric reflux: a multi-

variate analysis. BJU Int. 2006;97(5):

10631068

109. Smeesters PR, Campos D Jr, Van Melderen

L, de Aguiar E, Vanderpas J, Vergison A.

Pharyngitis in low-resources settings: apragmatic clinical approach to reduce un-

necessary antibiotic use. Pediatrics. 2006;



110. Steele R, Green SM, Gill M, Coba V, Oh B.

Clinical decision rules for secondary

trauma triage: predictors of emergency

operative management. Ann Emerg Med.

2006;47(2):135

111. Steinhoff MC, Abd el Khalek MK, Khallaf N,

et al. Effectiveness of clinical guidelines

for the presumptive treatment of strepto-

coccal pharyngitis in Egyptian children.

Lancet. 1997;350(9082):918 921

112. Sun BC, Hoffman JR, Mower WR. Evaluation

of a modified prediction instrument to

identify significant pediatric intracranial

injury after blunt head trauma. Ann Emerg

Med. 2007;49(3):325332, 332.e321

113. Thacher TD, Fischer PR, Pettifor JM. Theusefulness of clinical features to identify

active rickets. Ann Trop Paediatr. 2002;

22(3):229237

114. Van den Bruel A, Aertgeerts B, Bruyninckx

R, Aerts M, Buntinx F. Signs and symptoms

for diagnosis of serious infections in

children: a prospective study in primary

care. B r J G e n P r a c t . 2007;57(540):

538546

115. van den Broek WT, van der Ende ED, Bijnen

AB, Breslau PJ, Gouma DJ. Which children

could benefit from additional diagnostic

tools in case of suspected appendicitis? JPediatr Surg. 2004;39(4):570574

116. WebsterAP, GoodacreS, Walker D,BurkeD.

How do clinical features help identify pae-

diatric patients with fractures following

blunt wrist trauma? Emerg Med J. 2006;

23(5):354357

117. Young Infants Clinical Signs Study Group.

Clinical signs that predict severe illness in

children under age 2 months: a multicen-

tre study. Lancet. 2008;371(9607):135142

118. Yen K, Riegert A, Gorelick MH. Derivation of

the DIVA score: a clinical prediction rule

for the identification of children with diffi-cult intravenous access. Pediatr Emerg

Care. 2008;24(3):143147

119. Ayoola OO, Adeyemo AA, Osinusi K. Predic-

tors of bacteraemia among febrile infants

in Ibadan, Nigeria. J Health Popul Nutr.

2002;20(3):223229

120. Baqui AH, Arifeen SE, Rosen HE, et al.

Community-based validation of assess-

ment of newborn illnessesby trained com-

munity health workers in Sylhet district of

Bangladesh. Trop Med Int Health. 2009;

14(12):1448 1456

121. Bhatt M, Joseph L, Ducharme FM, Dough-erty G, McGillivray D. Prospective valida-

tion of the pediatric appendicitis score in a

Canadian pediatric emergency depart-

ment. Acad Emerg Med. 2009;16(7):

591596

122. Bonadio WA, Hagen E, Rucka J, Shallow K,

Stommel P, Smith D. Efficacy of a protocol

to distinguish risk of serious bacterial in-

fection in the outpatient evaluation of fe-

brile young infants. Clin Pediatr (Phila).

1993;32(7):401404

123. Friedman MJ, Attia MW. Clinical predictors

of influenza in children. Arch Pediatr Ado-

lesc Med. 2004;158(4):391394

124. Gravel J, Hedrei P, Grimard G, Gouin S. Pro-

spective validation and head-to-head com-

parison of 3 ankle rules in a pediatric pop-

ulation. Ann Emerg Med. 2009;54(4):

534 540.e531

125. Holmes JF, Mao A, Awasthi S, McGahan JP,Wisner DH, Kuppermann N. Validation of a

prediction rule for the identification of

children with intra-abdominal injuries af-

ter blunt torso trauma. Ann Emerg Med.

2009;54(4):528533

126. Horwood C, Liebeschuetz S, Blaauw D, Cas-

sol S, Qazi S. Diagnosis of paediatric HIV

infection in a primary health care setting

with a clinical algorithm. Bull World Health

Org. 2003;81(12):858 866

127. Kuppermann N, Fleisher GR, Jaffe DM. Pre-

dictors of occult pneumococcal bactere-

mia in young febrile children. Ann EmergMed. 1998;31(6):679687

128. Lacour AG, Zamora SA, Gervaix A. A score

identifying serious bacterial infections in

children with fever without source. Pedi-

atr Infect Dis J. 2008;27(7):654 656

129. Muhe L, Oljira B, Degefu H, Enquesellassie

F, Weber MW. Clinical algorithm for ma-

laria during low and high transmission

seasons. Arch Dis Child. 1999;81(3):

216220

130. Mwangi TW, Mohammed M, Dayo H, Snow

RW, Marsh K. Clinical algorithms for ma-

lariadiagnosis lackutility among people ofdifferent age groups. Trop Med Int Health.

2005;10(6):530536

131. Nandi S, Kumar R, Ray P, Vohra H, Ganguly

NK. Clinical score card for diagnosis of

group A streptococcal sorethroat. IndianJ

Pediatr. 2002;69(6):471 475

132. Narayan S, Mahadevan S, Serane VT. Keith

Edwards score for diagnosis of tuberculo-

sis. Indian J Pediatr. 2003;70(6):467 469

133. Olaleye BO, Williams LA, DAlessandro U, et

al. Clinical predictors of malaria in Gam-

bian children with fever or a history of fe-

ver. Trans R Soc Trop Med Hyg. 1998;92(3):300304

134. Osmond MH, Klassen TP, Wells GA, et al;

Pediatric Emergency Research Canada

(PERC) Head Injury Study Group. CATCH: a

clinical decision rule for the use of com-

puted tomography in children with minor

head injury. CMAJ. 182(4):341348

135. van der Schouw YT, van der Velden MT,

Hitge-Boetes C, Verbeek AL, Ruijs SH. Diag-

nosis of hypertrophic pyloric stenosis:

value of sonography when used in con-

junction with clinical findings and labora-

http://www.pediatrics.org/cgi/content/full/118/6/e1607http://www.pediatrics.org/cgi/content/full/118/6/e1607http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://www.pediatrics.org/cgi/content/full/118/6/e1607http://www.pediatrics.org/cgi/content/full/118/6/e1607

7/28/2019 malariaaa

13/14

tory data. AJR Am J Roentgenol. 1994;

163(4):905909

136. Slap GB, Forke CM, Cnaan A, et al. Recogni-

tion of tubo-ovarian abscess in adoles-

cents with pelvic inflammatory disease. J

Adolesc Health. 1996;18(6):397 403

137. Truong TH, Beyene J, Hitzler J, et al. Fea-

tures at present ation predict child renwith acute lymphoblastic leukemia at low

risk for tumor lysis syndrome. Cancer.

2007;110(8):18321839

138. Tuerlinckx D, Bodart E, Jamart J, Glupczyn-

ski Y. Prediction of Lyme meningitis based

on a logistic regression model using clini-

cal and cerebrospinal fluid analysis: a Eu-

ropean study. Pediatr Infect Dis J. 2009;

28(5):394397

139. Viccellio P, Simon H, Pressman BD, et al;

NEXUS Group. A prospective multicenter

study of cervical spine injury in children.

Pediatrics. 2001;108(2). Available at: www.

pediatrics.org/cgi/content/full/108/2/e20

140. Wald ER,GreenMD, Schwartz B, Barbadora

K. A streptococcal scorecard revisited. Pe-

diatr Emerg Care. 1998;14(2):109111

141. Yacoub S, Mohammed MJ, Ramsan M, Al-

bonico M. Clinical predictors of malaria

and other febrile illnesses in children un-

der five on Pemba Island, Tanzania. Trop

Doct. 2005;35(2):78 81

142. Yilmaz HL, Yildizdas RD, Alparslan N, Ozcan

K, Yaman A, Kibar F. Screening tools for

bacteraemia in a selected populationof fe-

brile children. Ann Acad Med Singapore.

2008;37(3):192199

143. Bailey B, Gravel J, Goldman RD, Friedman

JN, Parkin PC. External validation of the

clinical dehydration scale for children

with acute gastroenteritis. Acad Emerg

Med. 2010;17(6):583588

144. Joachim L, Campos D Jr, Smeesters PR.

Pragmatic scoring system for pharyngitis

in low-resource settings. Pediatrics. 2010;



145. Gana JC,TurnerD, Roberts EA,LingSC. Der-

ivation of a clinical prediction rule for the

noninvasive diagnosis of varices in chil-

dren. J Pediatr Gastroenterol Nutr. 2010;

50(2):188193

146. Bilkis MD, Gorgal N, Carbone M, et al. Vali-

dation and development of a clinical pre-

diction rule in clinically suspectedcommunity-acquired pneumonia. Pediatr

Emerg Care. 2010;26(6):399 405

147. Anwar M, Ahmad A, Ahmad F, Mazhar A.

Modified Kenneth Jones criteria for diag-

nosing tuberculous meningitis in children.

J Coll Physicians Surg Pak. 2010;20(4):

258261

148. Bin SS, Schutzman SA, Greenes DS. Valida-

tion of a clinical score to predict skull frac-

tu re in hea d-i nj ure d in fan ts . Pediatr

Emerg Care. 2010;26(9):633 639

149. Dubos F, Korczowski B, Aygun DA, et al. Dis-

tinguishing between bacterial and aseptic

meningitis in children: European compari-

son of two clinical decision rules. Arch Dis

Child. 2010;95(12):963967

150. Prasarnphanich T. The accuracy of clinical

diagnosis of influenza in Thai children. J

Pediatr Infect Dis. 2010;5(2):155159

151. Houben ML,BontL, WilbrinkB, etal. Clinical

prediction rule for RSV bronchiolitis in

healthy newborns:prognostic birthcohort

study. Pediatrics. 2011;127(1):35 41

152. Sultan J, Hughes PJ. Septic arthritis or

transient synovitis of the hip in children:

the value of clinical prediction algorithms.

J B o n e J o i n t S u r g B r . 2 0 1 0 ;9 2 ( 9 ):12891293

153. Whiting P, Rutjes AW, Reitsma JB, Bossuyt

PM, Kleijnen J. The development of

QUADAS: a tool for the quality assessment

of studies of diagnostic accuracy included

in systematic reviews. BMC Med Res Meth-

odol. 2003;3:25

154. Whiting PF, Weswood ME, Rutjes AW, Re-

itsmaJB, Bossuyt PN, Kleijnen J. Evaluation

of QUADAS, a tool for the quality assess-

ment of diagnostic accuracy studies. BMC

Med Res Methodol. 2006;6:9

155. Bossuyt PM, Reitsma JB, Bruns DE, et al.

Toward complete and accurate reporting

of studies of diagnostic accuracy: the

STARD initiative. Acad Radiol. 2003;10(6):

664669

156. Moss M, Wellman DA, Cotsonis GA. An ap-

praisal of multivariable logistic models in

the pulmonary and critical care literature.

Chest. 2003;123(3):923928

157. Ciampi A, Chang CH, Hogg S, McKinney S.

Recursive partitioning: a versatile method

for exploratory data analysis in biostatis-

tics. In: MacNeill I, Umphrey G, eds. Time

Series and Econometric Modelling/

Biostatistics. Boston, MA: D. Reidel Publish-

ing Co; 1987:2350

158. Brenner DJ, Hall EJ. Computed tomogra-

phy: an increasing source of radiation ex-

posure. N Engl J Med. 2007;357(22):2277

2284

159. Smith-Bindman R. Is computed tomogra-

phy safe? N Engl J Med. 2010;363(1):1 4

160. GuyattG, Cook D, Haynes B. Evidencebased

medicine has come a long way. BMJ. 2004;

329(7473):990991

161. Kon A. The shared decision-making contin-

uum. JAMA. 2010;304(8):903

162. OConnor AM, Legare F, Stacey D. Risk com-

munication in practice: the contribution of

decision aids. BM J. 2003;327(7417):

736740163. Stiell IG, Greenberg GH, McKnight RD, Nair

RC, McDowell I, Worthington JR. A study to

develop clinical decision rules for the use

of radiography in acute ankle injuries. Ann

Emerg Med. 1992;21(4):384390

164. Stiell IG, Wells GA, Vandemheen K, et al. The

Canadian CT head rule for patients with

minor head injury. Lancet. 2001;357(9266):

13911396

REVIEW ARTICLES

http://www.pediatrics.org/cgi/content/full/108/2/e20http://www.pediatrics.org/cgi/content/full/108/2/e20http://www.pediatrics.org/cgi/content/full/126/3/e608http://www.pediatrics.org/cgi/content/full/126/3/e608http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://pediatrics.aappublications.org/http://localhost/var/www/apps/conversion/tmp/scratch_3/pediatrics.aappublications.org/http://www.pediatrics.org/cgi/content/full/126/3/e608http://www.pediatrics.org/cgi/content/full/126/3/e608http://www.pediatrics.org/cgi/content/full/108/2/e20http://www.pediatrics.org/cgi/content/full/108/2/e20

7/28/2019 malariaaa

14/14

DOI: 10.1542/peds.2011-0043

; originally published online August 22, 2011;2011;128;e666PediatricsElizabeth M. Uleryk and Patricia C. Parkin

Jonathon L. Maguire, Dina M. Kulik, Andreas Laupacis, Nathan Kuppermann,Clinical Prediction Rules for Children: A Systematic Review

ServicesUpdated Information &

tmlhttp://pediatrics.aappublications.org/content/128/3/e666.full.hincluding high resolution figures, can be found at:

References

tml#ref-list-1http://pediatrics.aappublications.org/content/128/3/e666.full.hfree at:This article cites 152 articles, 30 of which can be accessed

Citations

tml#related-urlshttp://pediatrics.aappublications.org/content/128/3/e666.full.hThis article has been cited by 5 HighWire-hosted articles:

Subspecialty Collections

cticehttp://pediatrics.aappublications.org/cgi/collection/office_praOffice Practicethe following collection(s):This article, along with others on similar topics, appears in

Permissions & Licensing

mlhttp://pediatrics.aappublications.org/site/misc/Permissions.xhttables) or in its entirety can be found online at:Information about reproducing this article in parts (figures,

Reprintshttp://pediatrics.aappublications.org/site/misc/reprints.xhtml

Information about ordering reprints can be found online:

rights reserved. Print ISSN: 0031-4005. Online ISSN: 1098-4275.Grove Village, Illinois, 60007. Copyright 2011 by the American Academy of Pediatrics. Alland trademarked by the American Academy of Pediatrics, 141 Northwest Point Boulevard, Elkpublication, it has been published continuously since 1948. PEDIATRICS is owned, published,PEDIATRICS is the official journal of the American Academy of Pediatrics. A monthly
http://pediatrics.aappublications.org/content/128/3/e666.full.htmlhttp://pediatrics.aappublications.org/content/128/3/e666.full.htmlhttp://pediatrics.aappublications.org/content/128/3/e666.full.htmlhttp://pediatrics.aappublications.org/content/128/3/e666.full.html#ref-list-1http://pediatrics.aappublications.org/content/128/3/e666.full.html#ref-list-1http://pediatrics.aappublications.org/content/128/3/e666.full.html#ref-list-1http://pediatrics.aappublications.org/content/128/3/e666.full.html#related-urlshttp://pediatrics.aappublications.org/content/128/3/e666.full.html#related-urlshttp://pediatrics.aappublications.org/cgi/collection/office_practicehttp://pediatrics.aappublications.org/cgi/collection/office_practicehttp://pediatrics.aappublications.org/site/misc/Permissions.xhtmlhttp://pediatrics.aappublications.org/site/misc/Permissions.xhtmlhttp://pediatrics.aappublications.org/site/misc/Permissions.xhtmlhttp://pediatrics.aappublications.org/site/misc/reprints.xhtmlhttp://pediatrics.aappublications.org/site/misc/reprints.xhtmlhttp://pediatrics.aappublications.org/site/misc/reprints.xhtmlhttp://pediatrics.aappublications.org/http://pediatrics.aappublications.org/site/misc/reprints.xhtmlhttp://pediatrics.aappublications.org/site/misc/Permissions.xhtmlhttp://pediatrics.aappublications.org/cgi/collection/office_practicehttp://pediatrics.aappublications.org/content/128/3/e666.full.html#related-urlshttp://pediatrics.aappublications.org/content/128/3/e666.full.html#ref-list-1http://pediatrics.aappublications.org/content/128/3/e666.full.html

malariaaa

Documents