emergency department capacity planning: a recurrent neural...

Research ArticleEmergency Department Capacity Planning A Recurrent NeuralNetwork and Simulation Approach

Serkan Nas and Melik Koyuncu

Department of Industrial Engineering Ccedilukurova University Sarıccedilam 01330 Turkey

Correspondence should be addressed to Melik Koyuncu mkoyuncucuedutr

Received 16 April 2019 Accepted 28 October 2019 Published 15 November 2019

Academic Editor Anna Tsantili-Kakoulidou

Copyright copy 2019 Serkan Nas and Melik Koyuncu +is is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in anymedium provided the original work isproperly cited

Emergency departments (EDs) play a vital role in the whole healthcare system as they are the first point of care in hospitals for urgent andcritically ill patients +erefore effective management of hospitalrsquos ED is crucial in improving the quality of the healthcare service +eeffectiveness depends on how efficiently the hospital resources are used particularly under budget constraints Simulationmodeling is oneof the best methods to optimize resources and needs inputs such as patientsrsquo arrival time patientrsquos length of stay (LOS) and the route ofpatients in the ED +is study develops a simulation model to determine the optimum number of beds in an ED by minimizing thepatientsrsquo LOS+e hospital data are analyzed and patientsrsquo LOS and the route of patients in the ED are determined To determine patientsrsquoarrival times the features associated with patientsrsquo arrivals at ED are identifiedMean arrival rate is used as a feature in addition to climaticand temporal variables+e exhaustive feature-selectionmethod has been used to determine the best subset of the features and the meanarrival rate is determined as one of the most significant features +is study is executed using the one-year ED arrival data together withfive-year (43824 study hours) ED arrival data to improve the accuracy of predictions Furthermore ten different machine learning (ML)algorithms are used utilizing the same best subset of these features After a tenfold cross-validation experiment based on mean absolutepercentage error (MAPE) the stateful long short-term memory (LSTM) model performed better than other models with an accuracy of47 followed by the decision tree and random forestmethods Using the simulationmethod the LOS has beenminimized by 7 and thenumber of beds at the ED has been optimized

1 Introduction

Health is the primary asset for human beings and the healthneeds of a population in a country are typically met throughthe healthcare system In the healthcare service network theprimary healthcare units are hospitals in which emergencydepartments (ED) play a critical role At a hospitalrsquos ED everyarriving patient has to be examined which causes high op-erating costs and results in budget shortages Moreover anyserious error in the practice of an ED may be life-threatening[1] +us the effectiveness of EDs is crucial in increasing thequality of health services particularly in developing countries+e effectiveness of EDs depends heavily upon the allocationof hospitalrsquos critical resources such as space and budget

Our observation on healthcare systems is that there is nostrong correlation between the health expenditure and theeffectiveness of the healthcare systems +is implies that

optimizing the resource allocation has a critical role inincreasing the quality of healthcare systems particularly indeveloping countries with limited resource capacity In theimprovement of the resource planning and the managementof hospitals various operational researchmanagementmethods are employed including mathematical pro-gramming queuing theory and simulation +ese methodsare more needed when EDs face overcrowding and highuncertainty Obviously the fact that patients cannotmake anappointment for ED service causes uncertainty +e un-certainty of patients arriving at EDs causes overcrowdingthereby resulting in more waiting time and less healthcareservice quality One should note that EDs face uncertaintynot only in terms of patient arrivals but also in terms of thetreatment times

Simulation is one of the most effective methods used formodeling the systems with a high degree of uncertainty

HindawiComputational and Mathematical Methods in MedicineVolume 2019 Article ID 4359719 13 pageshttpsdoiorg10115520194359719

+erefore simulation is an important method that con-tributes to the operational tactical and strategic decision-making process about EDs [2] +e success of the simu-lation model depends on its resemblance to the corre-sponding real system as much as possible and to achievethis the ED simulation model needs an accurate estimationof patient arrivals and the length of stay (LOS) +ereforethe prediction of patient arrivals has a vital role in modelingthe system In addition the most significant cause of EDcrowding is the inability to manage inpatient bed avail-ability thereby making it critical for hospitals to managepatient flow [3] Human resource allocation is also adjustedaccording to the bed demand to treat patients in the EDwith the aim of providing universal access to healthcare toall the patients therein [4] Predicting patientsrsquo arrivalsrates and consequently the bed demand at EDsmay lead toeffective use of hospital resources and reduce the over-crowding and waiting time of the patients at ED [5] +ereare different approaches to predicting patient arrival ratesMachine learning as a subset of artificial intelligence (AI) isone of the best approaches in the prediction of targetvariables such as patient arrivals AI technologies are likelyto impact many aspects of emergency medicine in the nearfuture [6] Apart from the traditional mathematical models(linear or nonlinear) the artificial neural network (ANN)approach (which is a set of algorithms used in ML) canhandle complicated problems associated with a largenumber of datasets further the implemented ANN is userfriendly for the hospital staff as a nonsite predictivetool [7]

Published literature associated with modeling the visitsof patient to EDs in general use two types of methods +efirst method analyzes the correlations that are often linearbetween patient arrivals and a number of independentvariables such as calendar or weather variables the secondmethod predicts future values from the past values using theassumption that patient visits obey a time series [8] In theliterature the features are generally temporal weather ordemographic variables Furthermore the literature studiesgenerally agree on the point that temporal variables are moreimportant than weather variables for forecasting the patientarrival rates at EDs All these studies used any subset of thesevariables +e patient arrival problems are generally relateddaily [7 9ndash14] weekly [4] or monthly [4 15] however a fewstudies focus on hourly arrival rates which include highvariation [16ndash18] +e forecasting accuracy worsened as theforecast time intervals became smaller the best MAPE rate is2 for a month 11 for a day 38 for four-hour periodand 50 for an hour [18] +ese results of forecasting ac-curacy are very close to those in the other studies in theliterature with the variation mainly due to the used datasetand prediction algorithms +e studies were performedusing different types of patient data such as one-year data[4 5 9ndash11 16 17] 2- to 6-year data [7 12ndash15 17] 10-yeardata [19] and the daily number of patient arrivals generallyused over a period ranging from 1- to 10-year data (usually 3years) [8] Different techniques used for the prediction ofpatient arrival rate are the ANN [7 9 11 14] the autore-gressive integrated moving average (ARIMA) [4 8 12ndash14]

the linear regression (LR) [14 18] the exponentialsmoothing [14 15 18] the logistic regression [5 10 11 19]the decision tree (DT) [1 10] the gradient boosted machines(GBM) [10] the Poisson regression model [16] the randomforest (RF) the AdaBoost (AB) the support vector machine(SVM) [5] and the LSTM model [20]

In this study nine different ML techniques and thestateful LSTM model have been used to predict patientarrival rates +e LSTM model is a type of recurrent neuralnetwork (RNN) which is a class of the ANN ANNs are usedin many ED studies such as optimum resource planning [1]modeling a ward [7] predicting patientsrsquo arrival rate [20]diagnosing illness [21 22] and predicting patient LOS atEDs [23] +is study is different from the LSTM model [20]by predicting hourly patientsrsquo arrival not only forecasting thedaily patientsrsquo arrival

It is worthy to point out that the employed algorithmmust be assisted by feeding it with appropriate featuresDue to the fact that wrongly selected features can decreasethe effectiveness of the prediction feature selection is acritical part of ML and plays a significant role in creating aneffective model for ML studies Intending to achieve thisthis study employs two main methods (1) a wrappermethod that evaluates the contribution of features by usingthe ML algorithm and (2) a filter method that ranks thevariables according to the heuristics based on generalcharacteristics of the data and not testing the subset of thefeatures by any ML model to determine whether a featureincreases the accuracy of the prediction Because of itsexhaustive nature the wrapper method has been consid-ered to enhance the modelrsquos performance with betterfeature subsets When we compare the filter method withthe wrapper methods the latter is slower [24 25] +ewrapper method that uses correlation coefficients will havefaster processing ability and makes a better subset of thefeatures [26] Another variant of the wrapper method is theexhaustive feature-selection method which finds the bestof the features by minimizing the loss criteria of the chosenalgorithm However this variant is not suitable for a largenumber of features but if computing time can be handledthis is the best method An exhaustive feature-selectionmethod has been used as this method tests all the com-binations of the features and finds the best subset of thefeatures

+ere are different types of computer simulationmethods such as system dynamics (SD) discrete-eventsimulation (DES) and agent-based simulation (ABS)models DES models and SD models have used modelingunder different situations Comparing the DES model withthe ABS model regarding which one is better in the simu-lation of the real-world problem remains controversial [27]Modeling an ED by using the DES models enables us toanalyze the system and how the process changes influencepatient flow and resource availability [28] In this study theDES model has been chosen suitable enough for tacklingcomplex systems such as EDs By changing the number ofbeds the reduction in patientsrsquo waiting times has beensought using our developed DES model +e procedure ofour developed simulation method is depicted in Figure 1

2 Computational and Mathematical Methods in Medicine

We have observed some potential gaps in the literatureand have tried to contribute First the main contribution ofthis study is to use ML algorithms to generate input pa-rameters to a DES model which is then used to optimizeresource allocation in an ED In particular we use ML al-gorithms to predict patient arrivals and use simulationoptimization to determine the capacity of the ED (in termsof the number of beds) that leads to the desired patient LOS+is is in contrast to using a theoretical (eg Poisson) arrivalprocess or a deterministic arrival pattern whichmay lead to asignificant inaccuracy in prediction and inefficiency in re-source allocation Second this study predicts the patientsrsquoarrival rate and is one of the rare studies that use the LSTMmodelndasha type of RNNndashin doing so +ird this studyidentifies the best subset of the features by using the ex-haustive feature-selection method which has not been usedin the previous studies in prediction of patient arrival ratesMoreover few applications are found in hourly patientarrival problems and the studies used therein [16ndash18]usually use the data for one year however this study usesreal data hourly one yearrsquos data daily data and also fiveyearsrsquo hourly data to check if the additional data improve theaccuracy of the prediction which yields positive results

2 Methods

+is study is designed in four sections to contribute to therelevant literature First the system has been analyzed andthe overall system structure has been described for thesimulation model Second all the required data have beencollected and the best subset is found by using the ex-haustive feature-selection method +ird an appropriateRNNmodel is chosen and then the parameters of the modelare selected following the stateful LSTM model which is atype of RNN model has been constructed +e structure ofthe stateful LSTM model is presented in Figure 2 Also theother 9ML algorithms are determined and their models havebeen developed by using Python programming Patientsrsquoarrival rates have been estimated by using 10 ML techniquesincluding the stateful LSTM models All the strategiescomparisons and adjustments have been executed at thisstage to predict the most accurate hourly and daily patientarrival rates Fourth the simulation model has beendesigned according to the system description after whichthe verification and validation of the system have beenperformed By changing the number of beds the minimi-zation in the patientsrsquo LOS and the optimization in thenumber of beds have been sought Hourly patient arrivalrates which are the outcomes of ML algorithms have been

used as the simulation input variables +e duration oftreatments and the hospital services have been taken fromthe observations determined by the hospital managementsubjective discretion has been used +e patient route in EDacts as the third input extracted from the analysis of hospitaldata +e details of each section are as follows

21 OutcomeMeasures Hourly Patient Arrival andOptimumNumber of Beds +e first outcome for the analysis is thetotal number of hourly ED arrivals for patients Some studiescategorize the patients and drop some of them according totheir constraints Because this input (ie the total number ofhourly ED arrivals) is used for the simulation to allocate bedresources categorizing and then eliminating may give in-adequate results Hence all the patient arrivals have beenincluded in this study +e second outcome is the optimumnumber of beds obtained by using our developed simulationmodel

22 Section One System Description +e private CeyhanCcedilınar Hospital is the first and only private hospital in theCeyhan district of Adana in Turkey+e ED therein operates247 and receives 75 patients per day on average +e X-rayfacility and the laboratory are very close to the ED +e EDprocess starts with the arrival of patients Patients may arriveat the ED by their own or by ambulance Except the patientsarriving by ambulance all the patients must visit the re-ceptionist to register their personal information and thenthe triage nurse checks the acuity of the patient on the basisof their triage system According to the triage system of thishospital the patients are categorized into three categoriesred yellow or green +e red class is for the patients thatneed the most urgent treatment the yellow for normalacuity and the green for the lowest acuity Following thetriage process the patients are treated by the emergencyphysician If needed the emergency physician refers a pa-tient to the laboratory and the X-ray facility After furtherexamination the patient should go to the patient stabili-zation room and if necessary the emergency physician canrefer the patient to an attending physician +e attendingphysician further may refer the patient again to the labo-ratory or the X-ray facility After the examination the at-tending physician can refer the patients to hospitaladmission or the stabilization room and then discharge thepatients from the ED Anyhow patients arriving by am-bulance can be excused from the registration process at thetriage room and they directly go to the emergency

Real system System descriptionof real system

(i)

(ii)

Key performanceindicator identifiedData collection

Verification andvalidation of the model

Process changes ordifferent scenariosPerform simulationCompare and improve

experiments Simulation results

Figure 1 Flowchart of the typical DES model

Computational and Mathematical Methods in Medicine 3

physician Except this part the process is the same as thatwith the other patients +e flow of the patients is dem-onstrated in Figure 3

23 Section Two Data Collection and Feature Selection

231 Data Collection +is study focuses on the number ofpatient arrivals which span across a total of 8760 hours inthe one-year study period (January 1 2017ndashDecember 312017) because it aims to predict the hourly arrival rate to theED with consideration that the analysis of daily data ratherthan hourly would lead to loss of significant informationregarding variant patient arrivals We classified the arrivaldate for each study hour on the basis of the calendar yearseason weekday weekend and hour of the day Addi-tionally we have identified all the holidays whether officialor religious and assigned additional information to thehourly data whether it is a holiday or a normal day Wedefined the seasons as winter (December 1 2017ndashJanuary 12017 and January 1 2017ndashFebruary 28 2017) spring (March1 2017ndashMay 31 2017) summer (June 1 2017ndashAugust 312017) and fall (September 1 2017ndashNovember 30 2017)However we collected the data of the weather every threehours (ie 3 am 6 am and 9 am) and not every hour So forthe hours during which we could not reach the weather datawe assigned the data to two hours after the time on which wehad reached the data For example if the data were collectedat 3 am then we assigned the data collected at 3 am to theweather data at 4 am and 5 am We performed the sameprocedure for the other four years ie 2016 2015 2014 and2013 Also for the year 2017 we prepared the daily arrivalrates by using the same features except the hour of the dayand the mean arrival rate

232 Feature Selection We needed the future selectionprocess to reduce overfitting and training time and also toimprove accuracy We determined the features on the basis ofthe previously conducted studies [1 4 5 8 10ndash12 14 16 18 19]and the data availability After the determination of the featureswe performed the exhaustive feature-selection method to

eliminate any doubt about the features subset because themethod selects the optimum subset by minimizing the lossfunction with the help of any ML techniques and tests all thecombination of the features In this study for evaluating thecontribution of features we applied the RF regression methodand for loss function we employed the mean absolute error(MAE) method which is described in Section 242 Also wecalculated the correlation values as shown in Figure 4 anddetermined that the day of the hour has a moderate positiverelationship with the number of patients and also this table willhelp us in comparing with the correlated variables and thevariables of best subset of the features

24 Section2ree Machine Learning Algorithms and StatefulLSTM Method

241 Machine Learning Algorithms ML a subset of arti-ficial intelligence focuses on learning for themselves +eprediction task is executed using ML algorithms which aredefined as learning a target function (f ) that best maps inputvariables (X) to an output variable (Y) Y f(X) A neuralnetwork which is an ML algorithm computes systems onthe basis of the neural structure of the brain Unlike thehuman brain the integrated circuits are two-dimensionaldevices consisting of layers +e layers can be at least threefirst for input the last for output and the hidden layersbetween the input and output layers to compute and solvethe real problem Neurons convert weighted input to itsoutput that passes to other neurons by using the activationfunction +e supervised learning which is a task of in-terfering a function from labeled training data is one of thebest ways to predict patient arrival rates +e best featuresand the best ML algorithm are vital to predicting unknowntest sets with high accuracy +e trial-and-error approachcould help us choose the best learning algorithm Conse-quently we tried different algorithmsrsquo performances andselected the winner for our problem Except the statefulLSTM method nine ML algorithms that were applied to thetraining data to build the models are as follows DT SVMGBM AB stochastic gradient regression (SG) K-nearestneighbors (KNN) RF multilayer perceptron (MLP) and LR

x2

xn

x0

x1

LSTM

LSTM

LSTM

LSTM

h1

h2

hn

h0LSTM

LSTM

LSTM

LSTM

Fully connected layer Output layerInput layer

y

Figure 2 Structure of the stateful LSTM model


242 Constructing the Architecture of a Type of RNN StatefulLSTM Forecast Accuracy Measures RNNs are an advancedtype of ANNs and unlike the traditional neural networksRNNs can learn to use the past information RNNs arepopular models which have shown a great promise in manytasks+ey do not assume that inputs are independent of oneanother and use sequential information Whenever there is asequence of data and the temporal dynamics that connectsthe data are more important than the spatial content of eachframe one needs to use RNNs RNNs connect to the pastdata to the present one thereby allowing the information to

persist if the past information is the close to the point whereit is required otherwise RNNs cannot manage connectingthe information However LSTM networks solve thisconnection problem as they are capable of learning long-term dependencies For these reasons in this study theLSTMmodel which is a very special kind of RNNs has beenused and is found out to be much better than the standardversion+e correlation coefficient between the features andthe number of patients was calculated and is presented inFigure 4 which shows that the hour of the day is highlycorrelated with the number of patients +is correlation

Ambulancepatient arrival

Normal patientarrival

ED admissiondesk

Triage room

Red zone

Attendingphysicianneeded

Yellow zone Green zone

ED physicianexamination

Bed assignment

Attendingphysician

Lab and X-rayneeded

Attendingphysician

examination

Lab and X-ray

Lab and X-rayneeded

Patient fromattendingphysician

Need foradmission

Admission tohospital

Patient stabilization

Bed release anddismiss



Y N

Y

Y

Y

N

Y

N

N

N

Y N

Figure 3 Flow of patients at the emergency department


shows us that arrival rate is sequential dependent Becausethe arrival rate to EDs is sequential dependent we designedan RNN which is a model type of ANN Also the type ofANN used is significant for ensuring the accuracy of themodel +e accuracy of ANN models depends upon thechosen parameters the number of hidden layers learningrate etc By adjusting these parameters the model eitherperforms better or not We chose the activation function theleaky ReLU As an optimizer we used the Adam optimi-zation because hourly patient arrival data include a lot ofvariation and noise and these data are nonstationary dataWe tested different numbers of hidden layers and thenadjusted the model to 300 hidden layers Afterward wetuned the learning rate to 00005 and used 80 of the datafor the training process and the remaining data for vali-dation Further we scaled all the attributes by using min-max scaler to ensure better model performance We used theloss function MAPE and MAE to improve the modelrsquosaccuracy because MAPE is scale-independent which isfrequently recommended as the primary measure of forecastaccuracy particularly for comparing forecast models acrossdatasets with different scales [29] But if any actual data inthe dataset are close to zero then MAPE is undefined or theactual data must not be included in MAPE calculation [30]In addition considering MAPE for resource allocation

guides the model in a right way For a series of predictedvalues (p1 p2 pn) and the corresponding series of actualvalues (x1 x2 xn) we have the following equations

MAPE 100

n1113944

n

i1

abs xi minus pi( 1113857

xi

MAE 1n

1113944

n

i1abs xi minus pi( 1113857

(1)

We tried to find the best parameters for our model usingthe trial-and-error method but finding the best parametersfor any project still remains an open question

25 Section Four Simulation Model We developed a dis-crete-event simulation model to analyze the EDrsquos processWe use the Arena which is one of the most popular softwarefor DES Arena is a true Microsoft Windows operatingsystem application so most features and operations existedin it are familiar to the end users Arena allows to integrationwith Microsoft Technologies such as importing MicrosoftVisio flowcharts as well as reading the data from or sendingto the outputs to Excel [31] Figure 5 shows a screenshot ofthe user interface for part of our developed model

100Days of the week

Weekday

Hour of the day

Weekend

Season

Temperature

Wind

Rain

Humidity

Holiday

Number of patients

Day

s of t

he w

eek

Wee

kday

Hou

r of t

he d

ay

Wee

kend

Seas

on

Tem

pera

ture

Win

d

Rain

Hum

idity

Hol

iday

Num

ber o

f pat

ient

s

ndash079 079000 ndash000 ndash004 ndash002 002 008 014001

ndash079 100 ndash100ndash000 000 002 003 ndash000 ndash011 ndash01708

04

00

ndash04

ndash08

001

000 ndash000 000100 ndash000 021 004 ndash039 002 059027

079 ndash100 100000 ndash000 ndash002 ndash003 000 011 017ndash001

ndash000 000 ndash000ndash000 100 ndash002 003 ndash003 004 001008

001 001 ndash001027 008 011 ndash011 ndash026 004 014100

ndash004 002 ndash002021 ndash002 100 003 ndash037 003 014011

ndash002 003 ndash003004 003 003 100 021 000 000ndash011

002 ndash000 000ndash039 ndash003 ndash037 021 100 002 ndash022ndash026

008 ndash011 011002 004 003 000 002 100 016004

014 ndash017 017059 001 014 000 ndash022 016 100014

Figure 4 Correlation matrix of the features and the number of patients


In order to get reliable results from simulation modelsaccurate data are essential +erefore we predicted the mostimportant data hourly patient arrival rates by 10 ML al-gorithms Further we used the most accurate result as aninput for our developed simulation model +e other re-quired data are the treatment time of each doctor and thepercentage number of all the patients for each route followedby patients in the ED We made observations and took theadvice of the hospital management to determine thetreatment and hospital service time We determined thepercentage number of each route by using the process of dataanalysis +e analysis revealed that 70 of the patientscoming to ED are sent to the laboratory and X-ray sectionsand nearly half of the patients coming to ED are sent to theattendant physician +e emergency physician sent 22 ofthe patients to a pediatrician 17 to an otorhinolaryn-gologist 13 to an internal medicine specialist 11 to anorthopedics and traumatology specialist and the rest to tendifferent attending physicians After the treatment all of thepatients are sent to the stabilization room +e treatmentservice times for each section at the ED are presented inTable 1

251 Verification and Validation Simulation model shouldbe as similar as possible to the actual system To check thatthe model is created correctly the hospital managementperformed verification +e coordinator of the hospitalmanagement owns the responsibility to confirm that thesimulation model is correctly implemented and that themodel is a good representation of the conceptual model

252 Determining the Number of ReplicationReplicating the simulation model more than once makes thesimulation results more reliable and determines the width ofthe confidence interval To determine the best replicationnumber the fixed sample size method or the sequentialmethod is generally used In this study we use the sequentialmethod to determine the replication number +is methodleads to a sequence of steps [32] Davis et al also used the

sequential method to determine the optimal replicationnumber [33]

X(n) sample meanδ(n a) confidence interval half lengths2(n) sample varianceprop significance leveltiminus 11minus α2 t table valuec0prime δ(n a) timinus 11minus α2

(s2(n))n

1113968

+e steps of the evaluation procedure are as follows+e starting number of the replication must be chosen as

n0 ge 2 and n n0

(1) Compute X (n) and δ(n a) from X1 X2 Xn(2) If δ(n a)X(n) le cprime use X (n) as the point estimate

for μ and stop At this point I(α c) [X(n) minus δ(n a) X(n) + δ(n a)] is an approximate 100(1 minus α)

percent confidence interval for μ with the desiredprecision

(3) Otherwise replace n by n+ 1 make an additionalreplication of the simulation and return to step one

We used the method mentioned above and found thatthe number of replication must be 5

253 Determining the Warm-Up Period +e simulationmust be set to warm-up period to make the simulation reachsteady state to eliminate initial bias and achieve reliableresults Kadı et al [34] used Welch method for determiningthe warm-up period [34] To identify the warm-up periodwe also used Welch graphical method According to theWelch graphical method first we ran the model with theone-time unit of the replication length and then recorded theobserved average values of the chosen performance criteriaSecond we determined the time period (w) and add to it thepast w days and the following w days of this observed valueand then we take the average of these values and plot themon graph We repeated this method by increasing one-timeunit at every simulation runWhen the trend of the observed

Figure 5 User interface from Arena software


values came to end and began to smooth we determined thatpoint as the warm-up period of the simulationmodel [32] Inthe model proposed by us we determine the time period (w)as 1 and set the warm-up period to 29 hours as shown inFigure 6 To make bias near zero in our simulation resultswe ran the model for 7500 hours and with five replicationsfor the 10 different data groups

Figure 7 shows the forecast accuracy (based on MAPE)of Poisson hourly arrival rates only hourly arrival rates andthe prediction of hourly arrival rates by the stateful LSTMand the other 9 ML algorithms Figure 7 shows that oursimulation model must use the outputs of the stateful LSTMto be a more accurate representation of an actual system+ecomparison of the actual number of arrival rates to ED andthe number of arrival rates which are predicted by LSTMmodel is shown in Figure 8 Unfortunately the Poissonarrival rates have less accuracy (78) than that of the meanarrival rates (59) However most of the reviewed studiesused the fact that the patientsrsquo arrival rate to an ED is anonhomogeneous Poisson process with rate λ(t) [35 36]

3 Results

31 Statistics of Patient Arrival Every hospital has somespecial characteristics At this hospital 60 of the patientsarrive at the ED between 5 pm and 12 pm the most crowdedtime being between 8 pm and 11 pm and the least crowdedtime between 4 am and 6 am Further 36 of the patientscome at weekends and the rest of them come at weekdays Inthe winter and summer the number of patient increaseshowever the number of patient decreases by 13 in autumncompared to that in winter On official holidays the hourlypatient arrival rate to ED is 545 however on normal daysthe hourly patient arrival rate is 331 On nearly everyweekday the patient arrival rate is the same daily patientarrival to the ED constitutes 13 of the patients coming tothe ED on weekends the daily patient arrival to the EDconstitutes 19 of patients coming to the ED for each day

32 Feature Optimization Our target is to find the mostaccurate predictions and then use these predictions as aninput to our simulation model to minimize patientsrsquo waitingtime and optimize the number of beds in EDs To find themost accurate predictions we needed to determine the bestsubset of the features We calculated the correlation

coefficients between all the variables as shown in Figure 4+e most correlated variables with the number of patientsare the mean arrival rate hour of the day humidity andweekday However the combination of the features can givevery different results Hence to find the optimum subset ofthe features we used the exhaustive feature-selectionmethod and determined the best subset with mean arrivalrate weekday season and holiday as features +is bestsubset is determined for both the 5-year hourly arrival ratedata and the one-year hourly arrival rate data For dailyarrival rate data we determined the optimal subset of

Table 1 Treatmentservice times in each ED section

Process Distribution parameter Source ReferenceTriage room UNIF (2 4) Nurse Hospital dataED physician examination TRIA (5 8 15) ED physician Hospital dataLab and X-ray test TRIA (15 30 45) Laboratorian Hospital dataAttending physician examination TRIA (5 10 15) Attending physician Expert viewPatient stabilization TRIA (30 60 120) Bed Expert viewPatient stabilization TRIA (30 60 120) Attending physician Expert viewPediatrician TRIA (15 20 40) Attending physician Expert viewOtorhinolaryngologist TRIA (5 8 20) Attending physician Expert viewInternal medicine specialist TRIA (15 20 30) Attending physician Expert viewOrthopedics and traumatology specialist TRIA (5 10 20) Attending physician Expert view

20 40 60 80 100 120 140 160 180 200Number of replications

0

20

40

60

80

100

120

140

LOS

Figure 6 Moving average with w 1 for length of stay ED system

00000

SVM

Poiss

on LR SG AB

MA

R

KNN GB RF GB

DT

MLP

LSTM

010000200003000040000500006000

MA

PE

07000080000900010000

Figure 7 Comparison of the forecast accuracy between the LSTMPoisson mean arrival rate and the other 9 ML algorithms based onMAPE values


features such as temperature humidity days of the weekseason holiday and weekend

33 Model Performance We used the ML algorithms topredict the patientsrsquo arrival rates First we predicted the dailyarrival rates as shown in Table 2

As shown in Table 2 the stateful LSTM method has thelowest MAPE mean and thus gives the best result HoweverEDs have a very dynamic structure and the predictions ofarrival rate will be used as an input for the simulation so thehourly arrival rate is predicted with the 2017 year data whichcomprises 8760 hourly patient arrival rates +e predictionof hourly arrivals also offered the possibility of higher ad-ministrative response than that while predicting ordinaryarrival rates during a several-hour period Table 3 presentsthe performance of ML according to the MAPE of thepredictions and the RFmodel gives us the best performance

We aimed at increasing the accuracy of the prediction soour expectation is that the additional data could potentiallyincrease the accuracy [10] So we took the data between theyears 2013 and 2018 from the hospital management systemand adjusted them for use by ML algorithms By comparingRF mean value from Table 3 with LSTM mean value fromTable 4 we calculated the increase in the accuracy of theprediction as 59

So we used the output of the predictions calculated bythe stateful LSTMmodel by using 5-year hourly arrival dataas an input for our developed simulation model to minimizeLOS A lot of factors affect the LOS but in this study wefocused on the number of beds and its impacts on the LOSWhile other resource remains the same we changed thenumber of beds and analyzed the LOS by performing thesimulation Figure 9 depicts the impact on LOS by changingthe number of beds and Figure 10 depicts the impact on bedqueues by changing the number of beds +is impact as-sessment is expected to be helpful for hospital managementsfor their tactical decision-making process

4 Discussion

In general empirical knowledge or correlation analyses wereperformed to select the variables that contribute more to theaccuracy of prediction [1] In this study in contrast to usingthe correlation analysis or other feature-selection tech-niques the exhaustive feature-selection method was used tofind the best subset of features For example humidity andtemperature were observed to be correlated with the numberof patients whereas holiday and season have less correlationas presented in Figure 4 In contrast for the hourly arrivalrate data the best subset of the features is as follows meanarrival rates weekday holiday and season However fordaily arrival rate data the best subset of the features istemperature humidity days of the week season holidayand weekday+is shows us that the best subset of the featuredepends on the data structure rather than the correlations ofthe features with the target variable +us except exhaustivefeature selection the correlation analyses or any othermethod may lead astray during implementing feature-se-lection methods Each feature can be significant for pre-diction accuracy but the interactions of features may worsenthe accuracy Feature optimization also gives us a chance tostudy with a small subset of features +e optimization re-sults in fast computing time and makes the data easilyreachable +e simplicity of our model is an excellent ad-vantage and also allows us to predict the patient arrivalswithout using any predicted value for hourly arrival ratedata For example if we need weather features then we willalso need the prediction of weather variables of the year 2019to predict the patient arrival rates of the year 2019 As aresult we do not need to estimate any feature to predict thepatient arrival In several studies that predict hourly arrivalrates the hour of the day has been the most importantfeature [4] Until mean arrival rate feature is added the dayof the hour was a very important feature for us because afteradding the feature of mean arrival rate to the feature subsetthe exhaustive feature-selection method was observed to

14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0

0 25 50 75 100Hourly arrival

125 150 175 200

Patient_countPredicted_randomforest

Figure 8 Comparison of the actual number of hourly patient arrivals and the prediction number of hourly patient arrivals to ED


eliminate the day of the hour from the best subset of featuresand select the mean arrival rate for the best subset of featureAlso by using the feature of mean arrival rate the modelupdated itself because mean arrival rate continuously up-dates itself as the time passes +e reviewed studies arguethat temporal variables are more influential to patient ar-rivals than holiday and weather [1 4 37] Our results in-dicate that weather variables do not affect the predictions ofhourly patient arrivals +e reason of this can be themoderate climatic conditions in this region +is can bechecked by performing this study in different regions with

severe climatic conditions In several studies a highernumber of patient arrivals at ED were observed to be onMondays [1 2 4] On weekends the patient arrivals slightlyincreased [2 14] Our data analysis showed that the patientarrivals are nearly the same in any weekday but at theweekend the patient arrivals increase nearly by 25 +esignificance of features was observed to be heavily dependenton the data structure

In this study for the same ED the daily arrivals arepredicted hourly arrival rates are predicted both for one yearand for five years Using the five year dataset is one of the

Table 2 Summary statistics of MAPE of 10 ML techniques across the 10-fold cross-validation 1-year daily arrival data

ML LSTM LR DT GBM SG AB KNN SVM MLP RFMean 1679 2083 2610 2019 2104 2224 2054 2157 2181 2188SD 649 627 733 626 606 660 668 675 737 639Min 1178 1418 1428 1337 1451 1375 1394 1438 1405 1320Max 3505 3047 3654 2969 3050 3105 3213 3231 3231 3063SD standard deviation

Table 3 Summary statistics of MAPE of 10 ML techniques across the 10-fold cross-validation 1-year hourly arrival data




6 8 10 12 14Number of beds

178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)

Figure 9 Impact of changing the number of beds on the totallength of stay

6 8 10 12 14Number of bed

30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)

Figure 10 Impact of changing the number of beds on the length ofbed queue


strengths of our model because this is one of the largestdatasets among the studies that build an admission pre-diction model Because multiple years of data would offer anadvantage of more accurate predictions there will be achance to estimate discrepancies such as seasonal variationand to obtain more accurate values of mean arrival rate Formore complicated values the LSTM model was observed toperform better than the other ML algorithms Also hourlyprediction showed us that the ED faced several hours ofcrowding +e EDrsquos decision-makers should focus on thesehours and resources should be planned by taking intoconsideration the overcrowding hours

+e RNN had several parameters using trial and errordifferent number of hidden neurons learning rate anddifferent activation functions were tested By tuning theseparameters it is possible to improve the accuracy But it isimpossible to know whether the structure is optimal +estateful LSTMmodel was observed to perform better than inother studies presented in the literature (eg [18]) +ereason for this can be the architecture of the RNN model orthe specific dataset we use As demonstrated the RF wasobserved to perform the best for the one-year data but notfor the five-year data

In most studies the LOS at the ED was observed toincrease when hospitalrsquos bed occupancy exceeds 90 andstrong negative correlation was observed between the pa-tients who are waiting at the ED and the number of availablebeds (eg [17]) +e relationship with the number of bedsand overcrowding was easily demonstrated by the simula-tion models Further focus should be laid on the busiesthours LOS decreases as the number of beds increase from 7to 10 However increasing the number of beds beyond 10does not lead to further decrease After this point otherresources such as the staff must be adjusted to decrease theLOS As the number of beds increased the bed queue de-creased however after a certain threshold there is nofurther improvement in queue Hence we found the opti-mum number of beds in our case to be 10 +e solution wedevelop here has a potential to be used in practice because ofits computing time good performance and simplicity In thefuture the solution can be incorporated to the hospitalrsquosmanagement information system to adjust the resources ofEDs on a periodic basis

5 Conclusion

In this study the potential features that have associationwith the number of patient arrivals were identified +en byusing the exhaustive feature-selection method the bestsubset of the feature that gives the most accurate predictionswas determined Ten ML algorithms were developed topredict the patientsrsquo arrival rates +e stateful LSTM wasobserved to perform the best but the RF and the DTmodelsalso performed well +e predictions of patientsrsquo arrival rateswere used as an input parameter for the simulation modelthat we developed +e route of the patients at the ED wasextracted from the analysis of the hospitalrsquos data andtreatmentservice times were determined from our obser-vations with the help of the hospital senior management

doctors and nurses According to the results of our simu-lation several resources are identified as bottlenecks in theED In this study the main focus was on the number of bedsWe used our simulation model with different number ofbeds and the impact of the number of beds on the waitingtime of patients at the ED was observed

+e RNN method is a black box approach that cannotinterpret the relationship between input and output Toovercome this a feature-selection approach was combinedwith the RNN model However feature-selection methodssuch as other wrapper approaches filter methods or cor-relation analyses may lead astray in feature-selection ap-proaches +is is because the feature with the highestcorrelation with the target variable can worsen the accuracyof the prediction when used along with the combination ofother features +erefore if allowed by the computing timeconstraints a possible exhaustive feature-selection should beused to find the best subset for our prediction model +eimportance of the features for the prediction model dependson the data structure and the used ML model

Most of the studies in the literature used the fact that theprediction of patientsrsquo arrival rates to ED is a non-homogeneous Poisson process with a rate λ(t) However onthe basis of MAPE the mean arrival rate was observed togive more accurate results than those by nonhomogeneousPoisson processes

In this study it was easily observed that the performanceof the ML algorithms depends heavily on the data structureNo algorithm was observed to work the best for everyproblem and it was particularly relevant in the case ofsupervised learning (ie predictive modeling) For exampleone cannot say that neural networks are always better thanRF or vice versa +ere are many factors at play such as thesize and structure of the dataset As a result one should trydifferent algorithms for the problem while using test set ofdata to evaluate the performance and select the model thatperforms the best In light of these ten ML algorithms weretested in this study

+e predictions of the stateful LSTMmodel were used asinput to the simulation model that we developed Using thesimulation model it was possible to observe the effect of thenumber of beds on the total LOS Increasing the number ofbeds was observed to decrease the waiting time until acertain threshold After that threshold other resources mustbe optimized to further decrease the LOS Hospital managersoften find it difficult to plan and allocate resources efficientlyon the expected patient inflow from the ED +is studyshows that a decision tool can help to use resources effi-ciently reduce overcrowding and improve patient satis-faction +e main limitation of this study is that the data ofthe LOS at ED could not be reached

+e most important contribution of this study is usingML methods to generate input parameters (patient arrivalrates) for a simulation model that is then used for capacityplanning In further studies LOS can also be predicted byusing ML methods +ese predictions can be used as inputsto the simulation model leading to more reliable resultsWhile performing future studies the optimization should becarried out simultaneously considering all resources rather


than optimizing each resource independently +e approachof usingMLrsquos prediction outputs as the input parameters to asimulation model can be utilized in other healthcaremanagement problems that require efficient resource utili-zation and are characterized by complex input and outputrelations

Abbreviations

AB AdaBoostAI Artificial intelligenceABS Agent-based simulationANN Artificial neural networkARIMA Autoregressive integrated moving averageDES Discrete-event simulationDT Decision treeED Emergency departmentGBM Gradient boosted machinesKNN K-nearest neighborsLR Logistic regressionLOS Length of stayLSTM Long short-term memoryMAE Mean absolute percentage errorML Machine learningMLP Multilayer perceptronRF Random forestRNN Recurrent neural networkSD System dynamicSG Stochastic gradient regressionSVM Support vector machine

Data Availability

+e data used to support the findings of the study areavailable from the corresponding author upon request

Conflicts of Interest

+e authors declare no conflicts of interest

Acknowledgments

+e authors would like to thank private Ccedilınar Hospitalrsquosmanagement team for the help in simulation design and datacollection and for all their opinions +is research wasfunded by Ccedilukurova University (Adana Turkey) Co-ordination Unit for Scientific Research Projects (grant(project) number FDK-2018-11379)

References

[1] M Yousefi M Yousefi R P M Ferreira J H Kim andF S Fogliatto ldquoChaotic genetic algorithm and adaboostensemble metamodeling approach for optimum resourceplanning in emergency departmentsrdquo Artificial Intelligence inMedicine vol 84 pp 23ndash33 2018

[2] M Gul and A F Guneri ldquoA comprehensive review ofemergency department simulation applications for normaland disaster conditionsrdquo Computers amp Industrial Engineeringvol 83 pp 327ndash344 2015

[3] J S Olshaker and N K Rathlev ldquoEmergency departmentovercrowding and ambulance diversion the impact andpotential solutions of extended boarding of admitted patientsin the emergency departmentrdquo 2e Journal of EmergencyMedicine vol 30 no 3 pp 351ndash356 2006

[4] M Carvalho-Silva M T T Monteiro F d Sa-Soares andS Doria-Nobrega ldquoAssessment of forecasting models forpatients arrival at emergency departmentrdquo Operations Re-search for Health Care vol 18 pp 112ndash118 2018

[5] F R Lucini F S Fogliatto G J C da Silveira et al ldquoTextmining approach to predict hospital admissions using earlymedical records from the emergency departmentrdquo In-ternational Journal of Medical Informatics vol 100 pp 1ndash82017

[6] J Stewart P Sprivulis and G Dwivedi ldquoArtificial intelligenceand machine learning in emergency medicinerdquo EmergencyMedicine Australasia vol 30 no 6 pp 870ndash874 2018

[7] M L Schiavo B Prinari J A Gronski and A V Serio ldquoAnartificial neural network approach for modeling the wardatmosphere in a medical unitrdquo Mathematics and Computersin Simulation vol 116 pp 44ndash58 2015

[8] M Wargon B Guidet T D Hoang and G Hejblum ldquoAsystematic review of models for forecasting the number ofemergency department visitsrdquo Emergency Medicine Journalvol 26 no 6 pp 395ndash399 2009

[9] M Xu T C Wong and K S Chin ldquoModeling daily patientarrivals at emergency department and quantifying the relativeimportance of contributing variables using artificial neuralnetworkrdquo Decision Support Systems vol 54 no 3pp 1488ndash1498 2013

[10] B Graham R Bond M Quınn and M Mulvenna ldquoUsingdata mining to predict hospital admissions from the emer-gency departmentrdquo IEE Access vol 6 pp 10458ndash10469 2018

[11] D Golmohammadi ldquoPredicting hospital admissions to re-duce emergency department boardingrdquo International Journalof Production Economics vol 182 pp 535ndash544 2016

[12] G Abraham G B Byrnes and C A Bain ldquoShort-termforecasting of emergency inpatient flowrdquo IEEE Transactionson Information Technology in Biomedicine vol 13 no 3pp 380ndash388 2009

[13] Y Sun B H Heng Y T Seow and E Seow ldquoForecastingdaily attendances at an emergency department to aid resourceplanningrdquo BMC Emergency Medicine vol 9 no 1 2009

[14] S S Jones A+omas R S Evans S J Welch P J Haug andG L Snow ldquoForecasting daily patient volumes in theemergency departmentrdquo Academic Emergency Medicinevol 15 no 2 pp 159ndash170 2008

[15] J Bergs P Heerinckx and S Verelst ldquoKnowing what toexpect forecasting monthly emergency department visits atime-series analysisrdquo International Emergency Nursingvol 22 no 2 pp 112ndash115 2014

[16] M L McCarthy S L Zeger R Ding D Aronsky N R Hootand G D Kelen ldquo+e challenge of predicting demand foremergency department servicesrdquo Academic EmergencyMedicine vol 15 no 4 pp 337ndash346 2008

[17] N Kortbeek A Braaksma H F Smeenk P J M Bakker andR J Boucherie ldquoIntegral resource capacity planning for in-patient care services based on hourly bed census predictionsby hourrdquo Journal of the Operational Research Society vol 66no 7 pp 1061ndash1076 2014

[18] J Boyle M Jessup J Crilly et al ldquoPredicting emergencydepartment admissionsrdquo EmergencyMedicine Journal vol 29no 5 pp 358ndash365 2012


[19] C A Parker N Liu S X Wu Y Shen S S W Lam andM E H Ong ldquoPredicting hospital admission at the emer-gency department triage a novel predictionmodelrdquoAmericanJournal of Emergency Medicine vol 37 no 8 pp 1498ndash15042018

[20] M Yousefi M Yousefi M Fathi and F S Fogliatto ldquoPatientvisit forecasting in an emergency department using a deepneural network approachrdquo Kybernetes 2019

[21] W G Baxt F S Shofer F D Sites and J E Hollander ldquoAneural network aid for the early diagnosis of cardiac ischemiain patients presenting to the emergency department withchest painrdquo Annals of Emergency Medicine vol 40 no 6pp 575ndash583 2002

[22] R L Kennedy R F Harrison A M Burton et al ldquoAn ar-tificial neural network system for diagnosis of acute myo-cardial infarction (AMI) in the accident and emergencydepartment evaluation and comparison with serum myo-globin measurementsrdquo Computer Methods and Programs inBiomedicine vol 52 no 2 pp 93ndash103 1997

[23] M Gul and A F Guneri ldquoForecasting patient length of stay inan emergency department by artificial neural networksrdquoJournal of Aeronautics and Space Technologies vol 8 no 2pp 43ndash48 2015

[24] M A Hall and L A Smith ldquoFeature selection for machinelearning comparing a correlation-based filter approach to thewrapperrdquo in Proceedings of the Twelfth International FLAIRSConference Orlando FL USA May 1999

[25] A L Blum and P Langley ldquoSelection of relevant features andexamples in machine learningrdquo Artificial Intelligence vol 97no 1-2 pp 245ndash271 1997

[26] M Kabir M Islam and K Murase ldquoA new wrapper featureselection approach using neural networkrdquo Neurocomputingvol 73 no 16ndash18 pp 3273ndash3283 2010

[27] S Brailsford ldquoDiscrete-event simulation is alive and kickingrdquoJournal of Simulation vol 8 no 1 pp 1ndash8 2014

[28] A Komashie and A Mousavi ldquoModeling emergency de-partments using discrete event simulation techniquesrdquo inProceedings of the 2005 Winter Simulation ConferenceOrlando FL USA December 2005

[29] B L Bowerman R T OrsquoConnell and A B Koehler Fore-casting Time Series and Regression An Applied Approach+omson BrooksCole Belmont CA USA 4th edition 2005

[30] R J Hyndman and A B Koehler ldquoAnother look at measuresof forecast accuracyrdquo International Journal of Forecastingvol 22 no 4 pp 679ndash688 2006

[31] W D Kelton R P Sadowski and N Swets Simulation withArena McGrawHill International +e McGraw Hill Com-panies Inc 1221 Avenue of the Americas New York NYUSA Fifth edition 2010

[32] A M Law Simulation Modelling amp Analysis McGraw-HillNew york NY USA 4th edition 2004

[33] B H Davis C E McLaren A J Carcio et al ldquoDeterminationof optimal replicate number for validation of imprecisionusing fluorescence cell-based assays proposed practicalmethodrdquoCytometry Part B (Clinical Cytometry) vol 84 no 5pp 329ndash337 2013

[34] D Kadı Y Kuvvetli and S Ccedilolak ldquoPerformance analysis of auniversity hospital blood laboratory via discrete event sim-ulationrdquo Simulation vol 92 no 5 pp 473ndash484 2016

[35] M A Ahmed and T M Alkhamis ldquoSimulation optimizationfor an emergency department healthcare unit in KuwaitrdquoEuropean Journal of Operational Research vol 198 no 3pp 936ndash942 2009

[36] K Ghanes M Wargon O Jouini et al ldquoSimulation-basedoptimization of staffing levels in an emergency departmentrdquoSimulation vol 91 no 10 pp 942ndash953 2015

[37] R Calegari F S Fogliatto F R Lucini J Neyeloff S Ricardoand K B D Schaan ldquoForecasting daily volume and acuity ofpatients in the emergency departmentrdquo Computational andMathematical Methods in Medicine vol 2016 Article ID3863268 8 pages 2016


Stem Cells International

Hindawiwwwhindawicom Volume 2018


MEDIATORSINFLAMMATION

of

EndocrinologyInternational Journal of



Disease Markers


BioMed Research International

OncologyJournal of



Oxidative Medicine and Cellular Longevity


PPAR Research

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Immunology ResearchHindawiwwwhindawicom Volume 2018

Journal of

ObesityJournal of



Computational and Mathematical Methods in Medicine


Behavioural Neurology

OphthalmologyJournal of


Diabetes ResearchJournal of



Research and TreatmentAIDS


Gastroenterology Research and Practice


Parkinsonrsquos Disease

Evidence-Based Complementary andAlternative Medicine

Volume 2018Hindawiwwwhindawicom

Submit your manuscripts atwwwhindawicom

+erefore simulation is an important method that con-tributes to the operational tactical and strategic decision-making process about EDs [2] +e success of the simu-lation model depends on its resemblance to the corre-sponding real system as much as possible and to achievethis the ED simulation model needs an accurate estimationof patient arrivals and the length of stay (LOS) +ereforethe prediction of patient arrivals has a vital role in modelingthe system In addition the most significant cause of EDcrowding is the inability to manage inpatient bed avail-ability thereby making it critical for hospitals to managepatient flow [3] Human resource allocation is also adjustedaccording to the bed demand to treat patients in the EDwith the aim of providing universal access to healthcare toall the patients therein [4] Predicting patientsrsquo arrivalsrates and consequently the bed demand at EDsmay lead toeffective use of hospital resources and reduce the over-crowding and waiting time of the patients at ED [5] +ereare different approaches to predicting patient arrival ratesMachine learning as a subset of artificial intelligence (AI) isone of the best approaches in the prediction of targetvariables such as patient arrivals AI technologies are likelyto impact many aspects of emergency medicine in the nearfuture [6] Apart from the traditional mathematical models(linear or nonlinear) the artificial neural network (ANN)approach (which is a set of algorithms used in ML) canhandle complicated problems associated with a largenumber of datasets further the implemented ANN is userfriendly for the hospital staff as a nonsite predictivetool [7]

Published literature associated with modeling the visitsof patient to EDs in general use two types of methods +efirst method analyzes the correlations that are often linearbetween patient arrivals and a number of independentvariables such as calendar or weather variables the secondmethod predicts future values from the past values using theassumption that patient visits obey a time series [8] In theliterature the features are generally temporal weather ordemographic variables Furthermore the literature studiesgenerally agree on the point that temporal variables are moreimportant than weather variables for forecasting the patientarrival rates at EDs All these studies used any subset of thesevariables +e patient arrival problems are generally relateddaily [7 9ndash14] weekly [4] or monthly [4 15] however a fewstudies focus on hourly arrival rates which include highvariation [16ndash18] +e forecasting accuracy worsened as theforecast time intervals became smaller the best MAPE rate is2 for a month 11 for a day 38 for four-hour periodand 50 for an hour [18] +ese results of forecasting ac-curacy are very close to those in the other studies in theliterature with the variation mainly due to the used datasetand prediction algorithms +e studies were performedusing different types of patient data such as one-year data[4 5 9ndash11 16 17] 2- to 6-year data [7 12ndash15 17] 10-yeardata [19] and the daily number of patient arrivals generallyused over a period ranging from 1- to 10-year data (usually 3years) [8] Different techniques used for the prediction ofpatient arrival rate are the ANN [7 9 11 14] the autore-gressive integrated moving average (ARIMA) [4 8 12ndash14]

the linear regression (LR) [14 18] the exponentialsmoothing [14 15 18] the logistic regression [5 10 11 19]the decision tree (DT) [1 10] the gradient boosted machines(GBM) [10] the Poisson regression model [16] the randomforest (RF) the AdaBoost (AB) the support vector machine(SVM) [5] and the LSTM model [20]

In this study nine different ML techniques and thestateful LSTM model have been used to predict patientarrival rates +e LSTM model is a type of recurrent neuralnetwork (RNN) which is a class of the ANN ANNs are usedin many ED studies such as optimum resource planning [1]modeling a ward [7] predicting patientsrsquo arrival rate [20]diagnosing illness [21 22] and predicting patient LOS atEDs [23] +is study is different from the LSTM model [20]by predicting hourly patientsrsquo arrival not only forecasting thedaily patientsrsquo arrival

It is worthy to point out that the employed algorithmmust be assisted by feeding it with appropriate featuresDue to the fact that wrongly selected features can decreasethe effectiveness of the prediction feature selection is acritical part of ML and plays a significant role in creating aneffective model for ML studies Intending to achieve thisthis study employs two main methods (1) a wrappermethod that evaluates the contribution of features by usingthe ML algorithm and (2) a filter method that ranks thevariables according to the heuristics based on generalcharacteristics of the data and not testing the subset of thefeatures by any ML model to determine whether a featureincreases the accuracy of the prediction Because of itsexhaustive nature the wrapper method has been consid-ered to enhance the modelrsquos performance with betterfeature subsets When we compare the filter method withthe wrapper methods the latter is slower [24 25] +ewrapper method that uses correlation coefficients will havefaster processing ability and makes a better subset of thefeatures [26] Another variant of the wrapper method is theexhaustive feature-selection method which finds the bestof the features by minimizing the loss criteria of the chosenalgorithm However this variant is not suitable for a largenumber of features but if computing time can be handledthis is the best method An exhaustive feature-selectionmethod has been used as this method tests all the com-binations of the features and finds the best subset of thefeatures

+ere are different types of computer simulationmethods such as system dynamics (SD) discrete-eventsimulation (DES) and agent-based simulation (ABS)models DES models and SD models have used modelingunder different situations Comparing the DES model withthe ABS model regarding which one is better in the simu-lation of the real-world problem remains controversial [27]Modeling an ED by using the DES models enables us toanalyze the system and how the process changes influencepatient flow and resource availability [28] In this study theDES model has been chosen suitable enough for tacklingcomplex systems such as EDs By changing the number ofbeds the reduction in patientsrsquo waiting times has beensought using our developed DES model +e procedure ofour developed simulation method is depicted in Figure 1



2 Methods






(i)

(ii)














x2

xn

x0

x1

LSTM

LSTM

LSTM

LSTM

h1

h2

hn

h0LSTM

LSTM

LSTM

LSTM


y







ED admissiondesk

Triage room

Red zone




Bed assignment

Attendingphysician

Lab and X-rayneeded

Attendingphysician

examination

Lab and X-ray

Lab and X-rayneeded


Need foradmission






Y N

Y

Y

Y

N

Y

N

N

N

Y N





MAPE 100

n1113944

n

i1


xi

MAE 1n

1113944

n


(1)



100Days of the week

Weekday

Hour of the day

Weekend

Season

Temperature

Wind

Rain

Humidity

Holiday

Number of patients

Day

s of t

he w

eek

Wee

kday

Hou

r of t

he d

ay

Wee

kend

Seas

on

Tem

pera

ture

Win

d

Rain

Hum

idity

Hol

iday

Num

ber o

f pat

ient

s



04

00

ndash04

ndash08

001








008 ndash011 011002 004 003 000 002 100 016004

014 ndash017 017059 001 014 000 ndash022 016 100014








(s2(n))n

1113968


n0 ge 2 and n n0











3 Results







0

20

40

60

80

100

120

140

LOS


00000

SVM

Poiss

on LR SG AB

MA

R

KNN GB RF GB

DT

MLP

LSTM

010000200003000040000500006000

MA

PE

07000080000900010000








4 Discussion


14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0


125 150 175 200














178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of




















2 Methods






(i)

(ii)














x2

xn

x0

x1

LSTM

LSTM

LSTM

LSTM

h1

h2

hn

h0LSTM

LSTM

LSTM

LSTM


y







ED admissiondesk

Triage room

Red zone




Bed assignment

Attendingphysician

Lab and X-rayneeded

Attendingphysician

examination

Lab and X-ray

Lab and X-rayneeded


Need foradmission






Y N

Y

Y

Y

N

Y

N

N

N

Y N





MAPE 100

n1113944

n

i1


xi

MAE 1n

1113944

n


(1)



100Days of the week

Weekday

Hour of the day

Weekend

Season

Temperature

Wind

Rain

Humidity

Holiday

Number of patients

Day

s of t

he w

eek

Wee

kday

Hou

r of t

he d

ay

Wee

kend

Seas

on

Tem

pera

ture

Win

d

Rain

Hum

idity

Hol

iday

Num

ber o

f pat

ient

s



04

00

ndash04

ndash08

001








008 ndash011 011002 004 003 000 002 100 016004

014 ndash017 017059 001 014 000 ndash022 016 100014








(s2(n))n

1113968


n0 ge 2 and n n0











3 Results







0

20

40

60

80

100

120

140

LOS


00000

SVM

Poiss

on LR SG AB

MA

R

KNN GB RF GB

DT

MLP

LSTM

010000200003000040000500006000

MA

PE

07000080000900010000








4 Discussion


14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0


125 150 175 200














178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of


























x2

xn

x0

x1

LSTM

LSTM

LSTM

LSTM

h1

h2

hn

h0LSTM

LSTM

LSTM

LSTM


y







ED admissiondesk

Triage room

Red zone




Bed assignment

Attendingphysician

Lab and X-rayneeded

Attendingphysician

examination

Lab and X-ray

Lab and X-rayneeded


Need foradmission






Y N

Y

Y

Y

N

Y

N

N

N

Y N





MAPE 100

n1113944

n

i1


xi

MAE 1n

1113944

n


(1)



100Days of the week

Weekday

Hour of the day

Weekend

Season

Temperature

Wind

Rain

Humidity

Holiday

Number of patients

Day

s of t

he w

eek

Wee

kday

Hou

r of t

he d

ay

Wee

kend

Seas

on

Tem

pera

ture

Win

d

Rain

Hum

idity

Hol

iday

Num

ber o

f pat

ient

s



04

00

ndash04

ndash08

001








008 ndash011 011002 004 003 000 002 100 016004

014 ndash017 017059 001 014 000 ndash022 016 100014








(s2(n))n

1113968


n0 ge 2 and n n0











3 Results







0

20

40

60

80

100

120

140

LOS


00000

SVM

Poiss

on LR SG AB

MA

R

KNN GB RF GB

DT

MLP

LSTM

010000200003000040000500006000

MA

PE

07000080000900010000








4 Discussion


14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0


125 150 175 200














178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of























ED admissiondesk

Triage room

Red zone




Bed assignment

Attendingphysician

Lab and X-rayneeded

Attendingphysician

examination

Lab and X-ray

Lab and X-rayneeded


Need foradmission






Y N

Y

Y

Y

N

Y

N

N

N

Y N





MAPE 100

n1113944

n

i1


xi

MAE 1n

1113944

n


(1)



100Days of the week

Weekday

Hour of the day

Weekend

Season

Temperature

Wind

Rain

Humidity

Holiday

Number of patients

Day

s of t

he w

eek

Wee

kday

Hou

r of t

he d

ay

Wee

kend

Seas

on

Tem

pera

ture

Win

d

Rain

Hum

idity

Hol

iday

Num

ber o

f pat

ient

s



04

00

ndash04

ndash08

001








008 ndash011 011002 004 003 000 002 100 016004

014 ndash017 017059 001 014 000 ndash022 016 100014








(s2(n))n

1113968


n0 ge 2 and n n0











3 Results







0

20

40

60

80

100

120

140

LOS


00000

SVM

Poiss

on LR SG AB

MA

R

KNN GB RF GB

DT

MLP

LSTM

010000200003000040000500006000

MA

PE

07000080000900010000








4 Discussion


14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0


125 150 175 200














178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of





















MAPE 100

n1113944

n

i1


xi

MAE 1n

1113944

n


(1)



100Days of the week

Weekday

Hour of the day

Weekend

Season

Temperature

Wind

Rain

Humidity

Holiday

Number of patients

Day

s of t

he w

eek

Wee

kday

Hou

r of t

he d

ay

Wee

kend

Seas

on

Tem

pera

ture

Win

d

Rain

Hum

idity

Hol

iday

Num

ber o

f pat

ient

s



04

00

ndash04

ndash08

001








008 ndash011 011002 004 003 000 002 100 016004

014 ndash017 017059 001 014 000 ndash022 016 100014








(s2(n))n

1113968


n0 ge 2 and n n0











3 Results







0

20

40

60

80

100

120

140

LOS


00000

SVM

Poiss

on LR SG AB

MA

R

KNN GB RF GB

DT

MLP

LSTM

010000200003000040000500006000

MA

PE

07000080000900010000








4 Discussion


14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0


125 150 175 200














178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of
























(s2(n))n

1113968


n0 ge 2 and n n0











3 Results







0

20

40

60

80

100

120

140

LOS


00000

SVM

Poiss

on LR SG AB

MA

R

KNN GB RF GB

DT

MLP

LSTM

010000200003000040000500006000

MA

PE

07000080000900010000








4 Discussion


14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0


125 150 175 200














178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of





















3 Results







0

20

40

60

80

100

120

140

LOS


00000

SVM

Poiss

on LR SG AB

MA

R

KNN GB RF GB

DT

MLP

LSTM

010000200003000040000500006000

MA

PE

07000080000900010000








4 Discussion


14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0


125 150 175 200














178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of
























4 Discussion


14

12

10

8

Num

ber o

f pat

ient

s

6

4

2

0


125 150 175 200














178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of





























178

180

182

184

186

188

190

192

194

196

198

LOS

(min

utes

)



30

40

50

60

70

80

90

100

Bed

queu

e (m

inut

es)






5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of






















5 Conclusion










Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of




















Abbreviations


Data Availability




Acknowledgments


References












































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of











































of




Disease Markers



OncologyJournal of





PPAR Research



Volume 2018


Journal of

ObesityJournal of



















emergency department capacity planning: a recurrent neural...

Documents