lable at ScienceDirect

Applied Geography 45 (2013) 1e9

Contents lists avai

Applied Geography

journal homepage: www.elsevier .com/locate/apgeog

Spatially-explicit sensitivity analysis for land suitability evaluation

Erqi Xu a,b, Hongqi Zhang a,*

a Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, PR ChinabGraduate University of Chinese Academy of Sciences, Beijing 100049, PR China

Keywords:Land suitability evaluationMulti-criteria decision-makingOne-dimensional sensitivitySpatially explicit analysisEarth mover’s distance

* Corresponding author. Tel.: þ86 10 64889106.E-mail address: zhanghq@igsnrr.ac.cn (H. Zhang).

0143-6228/$ e see front matter � 2013 Elsevier Ltd.http://dx.doi.org/10.1016/j.apgeog.2013.08.005

a b s t r a c t

Land suitability evaluation (LSE) is an important step in land-use planning. Using multi-criteria decision-making (MCDM) techniques based on geographic information systems is a flexible and effectiveapproach for this evaluation process. Implementation of sensitivity analysis to validate and calibrate theMCDM can enhance the understanding of the LSE results and assist in making informed planning de-cisions. The main limitation of sensitivity analysis in MCDM applications is a lack of insight into thespatial dimensions. To address this issue, this paper presents a new framework that incorporates thespatial configuration information from sensitivity analysis for MCDM. The framework consists of a landsuitability evaluation and a spatially explicit sensitivity analysis. The sensitivity analysis couples spatialvisualization and summary indicators, which include a traditional metric (i.e., the mean of the absolutechange rate, MACR) and a novel spatially explicit metric (the Earth Mover’s Distance, EMD). The newlyreclaimed region of Yili in China was studied as the representative area. We assumed that the weightswere the only source of uncertainty and used a one-dimensional sensitivity analysis. This experimentindicated that the expert LSE results for wheat are robust but relatively sensitive in local areas to changesin the weights. Our results confirm that the MACR and EMD can effectively identify sensitive parametersbased on various sensitivity aspects. The EMD explores the new information from the spatial dimensions,which differs from traditional methods for sensitivity analysis. This approach provides a suitableframework based on a spatially explicit sensitivity analysis for the effective implementation of MCDM forrobust LSE results.

� 2013 Elsevier Ltd. All rights reserved.

Introduction

Agricultural production activities are the foundation of humansurvival and development. With the growth in the population andthe reduction of arable lands, ensuring effective use of arable landto meet the growing demand for food requires rational land usemanagement and planning. Land suitability evaluation (LSE) is animportant step in this planning. Because the Food and AgriculturalOrganization (FAO) recommended an approach for LSE based onclimatic, terrain, and soil properties data (FAO, 1976), multi-criteriadecision-making (MCDM) techniques have been widely applied tocombine information from different criteria for the LSE. The inter-est of researchers in integrating geographic information systems(GIS) with MCDM has grown steadily (Ceballos-Silva & López-Blanco, 2003; Hossain & Das, 2010; Kumar, Patel, Sarkar, &Dadhwal, 2013; Nisar Ahamed, Gopal Rao, & Murthy, 2000;Pereira & Duckstein, 1993; Tenerelli & Carver, 2012). However,

All rights reserved.

GIS-based MCDM is a multi-disciplinary and multi-step processthat can result in many sources of uncertainty (Burgman, 2005;Chen, Wood, Linstead, & Maltby, 2011; Wood, Beresford, Barnett,Copplestone, & Leah, 2009), including criteria selection, inputdata accuracy, standardization method, weight calculation, andaggregation method (Elaalem, Comber, & Fisher, 2011; Reshmidevi,Eldho, & Jana, 2009).

The uncertainties can be classified as aleatory or epistemic(Helton, 1993; Refsgaard, van der Sluijs, Højberg, & Vanrolleghem,2007). Particularly, the weight assigned to each criterion is one ofthe most sensitive parameters in MCDM and is a potential source ofconsiderable uncertainty (Larichev & Moshkovich, 1995). Forexample, the Analytical Hierarchy Process (AHP) (Saaty, 2008) isone of the most popular methods for calculating criteria weights inMCDM via an expert pair-wise comparison matrix (Hossain & Das,2010; Marinoni, 2004; Ohta et al., 2007; Vaidya & Kumar, 2006).Using their weights, the criteria can be subsequently aggregatedinto a single imprecise MCDM estimation point, which results inuncertainties with no confidence (Benke, Pelizaro, & Lowell, 2009).Meanwhile, multiple decision-makers are able to set differentweights and thus derive a variety of MCDM results for various

Fig. 1. Location of the newly reclaimed region Yili.

E. Xu, H. Zhang / Applied Geography 45 (2013) 1e92

policy targets (Al-Mashreki, Akhir, Rahim, Lihan, & Haider, 2011;Chen et al., 2011; Roura-Pascual, Krug, Richardson, & Hui, 2010).Therefore, the robustness of the LSE results should be evaluated foreffective implementation in land-use planning (Fuller, Gross, Duke-Sylvester, & Palmer, 2008; Ligmann-Zielinska & Jankowski, 2008).For this purpose, use of the uncertainty and sensitivity analysis ishelpful in the validation and calibration of MCDM (Delgado &Sendra, 2004; Merritt, Croke, & Jakeman, 2005; Zoras,Triantafyllou, & Hurley, 2007).

Until now, sensitivity analysis has received only minimal atten-tion in previous MCDM studies, although this situation is changing(Chen, Yu, & Khan, 2010; Delgado & Sendra, 2004; Ligmann-Zielinska, Jankowski, & Watkins, 2012; Lowell, Christy, Benke, &Day, 2011). It should be noted that the most critical shortcoming ofsensitivity analysis is a lack of insight into the spatial dimensions(Chen et al., 2010; Feick & Hall, 2004). This situation therefore re-quires spatial visualization techniques and spatially explicitmethods applied in the sensitivity analysis to create effective in-formation for the planning decision process, i.e., GIS techniques andsimulation algorithms (Ligmann-Zielinska & Jankowski, 2008;Mosadeghi, Warnken, Tomlinson, & Mirfenderesk, 2012; Pannell,1997). Spatial visualization can display the uncertainties of theevaluation results graphically based on the uncertainty of the inputparameters and enhance the experts’ and decision-makers’ under-standing of the possible risk in identification of parameter sensi-tivity in MCDM (Blaser, Sester, & Egenhofer, 2000; BojóRquez-Tapia,Cruz-Bello, & Luna-González, 2012; Chen et al., 2011; Hallisey, 2005;Vitek, Giardino, & Fitzgerald, 1996).

Few studies have attempted to develop a spatial sensitivityanalysis for MCDM. Feick and Hall (2004) presented a method forinvestigating the spatial dimension of the sensitivity of multi-criteria weights. Chen et al. (2010) presented a visualizedapproach for analyzing the dependency of MCDM output onweightchanges and identifying those criteria that are especially sensitiveto weight changes in a given spatial dimension. Chen et al. (2011)used an indicator-based method to visually explore the influenceof uncertainties on MCDM with the application of the CatchmentEvaluation Decision Support System in the Tamar catchment.Ligmann-Zielinska et al. (2012) employed a Monte Carlo simulationand output variance decomposition to represent output uncer-tainty in spatial form. Tenerelli and Carver (2012) set up a landcapability model for assessing the potential of perennial energycrops and performed an uncertainty analysis of the model with aspatial distribution. Ligmann-Zielinska and Jankowski (2012) pre-sented an approach for adjusting the criteria preferences based ondistance measures using the explicit consideration of a locationalstructure.

However, the aforementioned studies focused primarily onspatial visualization of the sensitivity analysis and used traditional

statistical methods to summarize the sensitivity results. Traditionalmethods for calculating the sensitivity indicators of outputs underuncertainty simulation, i.e., change percentage (Maguire,Goodchild, & Rhind, 1991), rank order (Benke, Steel, & Weiss,2011; Butler, Jia, & Dyer, 1997), standard deviation (Heumann,Walsh, & McDaniel, 2011; Lowell et al., 2011; Pelizaro, Benke, &Sposito, 2011) and correlation coefficient (Tenerelli & Carver,2012), consider the outputs of MCDM as discrete and indepen-dent elements and ignore the spatial configuration of the evalua-tion results. Evaluating spatially explicit LSE results in sensitivityanalysis requires insight into the spatial information of the sensi-tivity analysis. Fortunately, the Earth Mover’s Distance (EMD),which is a spatial metric used in image retrieval and histogramcomparison (Rubner, Tomasi, & Guibas, 2000), provides an oppor-tunity to consider the spatial dimension of sensitivity analysis.

The objective of this study is to present a new framework thatincorporates the spatial configuration information of sensitivityanalysis. We evaluated the LSE based on GIS-MCDM with weightscalculated using the AHP. The framework examined the sensitivityof different criteria with changes inweights via spatial visualizationof the uncertainty outputs and summary sensitivity indicatorsgenerated by traditional and spatially explicit methods.

Materials and methods

Study area

The newly reclaimed region located in the valley of Yili liesroughly between 80�2201400 and 83�305400E and 43�2203700 and44�802200N (Fig. 1) and is one of seven important land resourcedevelopment regions established by the Ministry of Land and Re-sources of the People’s Republic of China. Land resource develop-ment engineers aim to achieve a balance of arable lands andimprove the land productivity. The Yili River valley, with a bettermatch of soil and water resources, is a limited potential region forland resource development inWestern China. Therefore, this regionrequires effective land-use planning to both combat desertificationand improve the quality of newly cultivated lands.

The study area belongs to Yining, Chabuchaer AutonomousCounty, Huocheng County, andGongliu County in the administrativeregion. The region covers an area of ca. 5000 km2, with elevationsranging from 661 m to 1572 m and lies within the temperate con-tinental semi-arid climate zone with a mean annual temperature of8e9 �C, a mean annual precipitation of 200e500 mm, a meanannual evaporation of 1200e1900mm, andwater resources that arethe richest in Xinjiang. Land-use types primarily include grasslandand farmlandwith a partial distribution of sand and saline areas. Thesoil types primarily consist of sierozemwith a partial distribution ofkastanozem above an altitude of 850 m. Other soil types include

E. Xu, H. Zhang / Applied Geography 45 (2013) 1e9 3

marsh soil, meadow soil, aeolian sandy soil, and small amounts ofsaline-alkali soil, damp soil, and chernozem soil.

Methods and data description

The flowchart (Fig. 2) of GIS-MCDM shows a series of basic stepsfor implementing spatially explicit sensitivity analysis based onMCDM for LSE.We assumed that theweightswere the only source ofuncertainty and thus evaluated the sensitivity of the weights alone.Application of our framework based on a raster consists primarily oftwo stages: land suitability evaluation and sensitivity analysis.

Land suitability evaluationThe first stage includes a sequence of evaluated crop selection,

relevant criteria selection, criteria standardization, weight calcula-tions, and final aggregation of criteria into the LSE results. Becausewheat is the main food crop in Yili, it was chosen for suitabilityevaluation in our study. Seven criteria related to the wheat suit-ability evaluation were selected by the authors and experts in Yili,who are familiar with the land-use characteristics and agriculturaldevelopment of Yili. These criteria are soil texture (ST), soil depth(SD), soil organic matter (SOM), sand dune waviness (SDW), soilerosion (SE), water supply and drainage (WSD), and groundwaterdepth (GD). The detailed data sources of the seven criteria are listedin Table 1. Our study team excavated soil profiles and collected soilsamples in the study area (Shi, Yang, & Wang, 2009), and soil depthand soil organic matter maps were generated using Kriging inter-polation based on the soil sample properties in ArcGIS 9.3 (ESRI Inc.,USA). All data were processed and converted to pixels with 300-mresolution in ArcGIS. Each criterion was standardized to four suit-ability classes, i.e., high suitability, moderate suitability, marginalsuitability and unsuitability, based on the FAO system (FAO, 1976).The classification threshold values of the criteria and the standardscores for the corresponding classes given in Table 2 were obtainedfrom a literature survey and expert opinions. Next, the weights ofthe criteria were calculated in the AHP by constructing a pair-wisecomparison matrix (Saaty, 2008) (Table 3). The scale for the pair-wise comparison in our study ranges from 1 to 9, indicating equalimportance to extreme importance (Saaty, 2008). The final pair-wisecomparison matrix was determined using the mean integer of everyexpert’s criteria pair-wise comparison.

Fig. 2. Flowchart of the spatially explicit sensitivit

The standardized criteria scores were aggregated linearly byweight for the final land suitability evaluation result. The equationused for this is given by:

R ¼Xni¼1

wi � ci (1)

where R is the result of the LSE with a high R indicating highsuitability of the crop,wi is the weight of the i-th criterion from theAHPwith

Pni¼1 wi ¼ 1, ci is the standard score of the i-th criterion,

and n is the number of criteria.

One-at-a-time methodIn previous studies, two methods were used for simulating the

uncertainty of the criteria weights (Feick & Hall, 2004). Monte Carlosimulation is frequently used to generate random criteria weights.However, the subjective assumptions for the parameters of theprobability distributions and the normality of the distribution areoften subject to bias (Crosetto, Tarantola, & Saltelli, 2000). Incontrast, the One-At-a-Time (OAT) method investigates the sensi-tivities of one-dimensional weights by changing the relative in-fluence of each factor separately, without assumptions. However,this method ignores the interactions caused by modifying theweights of multiple factors simultaneously (Butler et al., 1997). TheOAT method estimates the effect on the evaluation results of vari-ation in a single input parameter while holding all other parame-ters fixed at their nominal values (Daniel, 1958; Rabitz, 1989;Saltelli, Chan, & Scott, 2000). In particular, the weights are deter-mined by experts instead of random selection and are constrainedwithin a certain range. Therefore, the OAT method was used forsensitivity analysis in the second stage of our framework.

The OAT method requires the setting of two parameters, i.e., therange and the step size of the particular weight changes. Weassigned a step size of �5% and a range of �100%. To ensure that allcriteria weights sum to one, the new adjusted weight used for thesensitivity were calculated using the following equation:

wjðcrÞ ¼ ð1þ crÞ �wj (2)

where wjðcrÞ is the particular weight change from the OATmethod; cr is the change rate of the weight, which can be set to

y analysis for crop-land suitability evaluation.

Table 1Criteria data sources and processing.

Criteria Input dataset Data source Format Processing

Soil texture (ST) Soil type maps of Yiliregion (1:100000)

Institute of Geographical Sciencesand Natural Resources Research

Polygon Format transformation

Soil depth (SD) Soil sampling points Fieldwork by the research team Point Kriging interpolationSoil organic matter (SOM) Soil sampling points Fieldwork by the research team Point Kriging interpolationSand dune waviness (SDW) Topographic maps

of Yili region (1:50000)Institute of Geographical Sciencesand Natural Resources Research

Raster Selection of the sand distributionfrom the land use map andcalculation of the relative heightfrom the DEM digitized fromthe topographic maps

Land use map of Yiliregion in 2008

Data Center for Resources and EnvironmentalSciences, Chinese Academy of Sciences

Soil erosion (SE) Topographic mapsof Yili region (1:50000)

Institute of Geographical Sciencesand Natural Resources Research

Raster Calculation of the gully densityfrom the above DEM

Water supply and drainage (WSD) Land resource mapof China (1:1000000)

Institute of Geographical Sciencesand Natural Resources Research

Raster Resampling

Groundwater depth (GD) Equi-potential line mapof the groundwater

Department of Water Resources of XinjiangUygur Autonomous Region in China

Poly-line Format transformation

E. Xu, H. Zhang / Applied Geography 45 (2013) 1e94

�5%, �10%, .. �100% in our study; and wj is the original weightof the j-th criterion from the AHP. The equation used for the otherweights is given as follows:

wiðcrÞ ¼ wi �1�wj

1�wj(3)

where wiðcrÞ is the other weight adjusted for wj, i.e., isj; and wi isthe original weight of the i-th criterion from the AHP.

Next, 280 LSE results were generated for the sensitivity analysis.The equation used for this calculation is given as:

R�wj; cr

� ¼ wj � cj þXnisj

wi � ci (4)

where R(wj,cr) is the simulated LSE result with wj as the changerate, cj is the standard score of the i-th criterion, and ci is the otherstandard score of the criteria, i.e., isj.

Local area uncertaintyThe uncertainty of the simulated results was represented by the

change rate. Based on the GIS, the uncertainties of every pixel in theregion using all step sizes for a particular weight were spatiallyvisualized in the GIS. Thus, the local area difference visualization of

Table 2Classification threshold values of criteria and standard scores for suitability evalu-ation of wheat.

Highsuitability

Moderatesuitability

Marginalsuitability

Unsuitability

ST Criteria Loam Medium loam,sandy loam

Clay, heavyloam

Sand

Score 100 70 50 0SD Criteria (m) >0.8 0.5e0.8 0.3e0.5 <0.3

Score 100 70 40 10SOM Criteria (%) >4.0 2.0e4.0 0.5e2.0 <0.5

Score 100 80 50 10SDW Criteria (m) Non-sand <1 1e2 >2

Score 100 50 20 0SE Criteria

(km/km2)<0.6 0.6e1.0 1.0e1.2 >1.2

Score 100 50 30 10WSD Criteria No ponding Short-term

seasonalponding

Seasonalponding

Long-timeponding

Score 100 60 30 10GD Criteria (m) >6 3e6 1e3 <1

Score 100 60 40 10

a particular weight with a given change rate could be identified forthe sensitivity analysis.

The equation used for the change rate is given as follows:

Ck�wj; cr

� ¼ Rk�wj; cr

�� R0R0

� 100% (5)

where Ckðwj; crÞ is the change rate of the LSE result with a wjchange rate for the k-th pixel, Rkðwj; crÞ is the simulated LSE resultof the k-th pixel calculated by Eq. (4), R0 is the original value of theLSE result calculated by Eq. (1), and k is the order number ofrandom pixels.

Summary sensitivity analysisFor decision-making, summary sensitivity indicators for the

entire regionwere generated by traditional methods (i.e., the meanof the absolute change rate, MACR) and spatially explicit measures(i.e., EMD). High MACR and EMD values indicate high sensitivity.

The MACR was calculated by the following equation:

MACR�wj; cr

� ¼XNk¼1

1N�����Rk

�wj; cr

�� R0R0

����� 100% (6)

where MACRðwj; crÞ is the mean absolute value of the change ratewith wj as a change rate, and N is the number of pixels.

The EMD is a distance function used for evaluating the dissim-ilarity between two histograms and differs from the traditionalmeasures (Levina & Bickel, 2001; Rubner et al., 2000). Based on asolution for finding the minimal cost in a transportation problem,the EMD transforms one histogram into the other (Cha & Srihari,2002).

Given two histograms P and Q, the EMD as defined by (Rubneret al., 2000) is:

EMDðP;QÞ ¼0@ minn

fijo X

i;j

fijdij

1A,0

@Xi;j

fij

1A (7)

Table 3Weights for suitability evaluation of wheat.

ST SD SOM SDW SE WSD GD

Weight 0.1866 0.2076 0.1369 0.1027 0.1132 0.1095 0.1435

Fig. 3. Criteria used in the suitability evaluation of wheat. Suitability classes are based on the threshold values in Table 2.

E. Xu, H. Zhang / Applied Geography 45 (2013) 1e9 5

s:t: fij �0X

fij � PiX

fij �Qi

Xfij ¼ min@X

Pi;X

QjA

j i i;j

0i j

1

where ffijg represents the flows. Each fij represents the amounttransported from the ith supply to the jth demand.We refer to dij asthe ground distance between bin I and bin j in the histograms.

Fig. 4. Land suitability evalu

The EMD has been used in image retrieval (Rubner, Guibas, &Tomasi, 1997; Rubner et al., 2000), comparison of diffusion tensormagnetic resonance images (Jiao et al., 2010), estimation ofspatial rainfall distributions (van den Berg, Vandenberghe, DeBaets, & Verhoest, 2011), and comparison of priority maps formanaging woody invasive alien plants (Roura-Pascual et al.,2010). The values of the original and simulated LSE results are

ation result for wheat.

Fig. 5. Mean absolute values of the change rate for the land suitability evaluationunder simulations.

E. Xu, H. Zhang / Applied Geography 45 (2013) 1e96

similar to the gray values of ordinary images. Next, the EMDcomputations of these two histograms of the LSE maps describethe spatial differences in both, which are similar to the imagecontrast. A higher EMD value indicates a greater change in the LSEmaps. Therefore, we used the EMD as a spatially explicit metricfor sensitivity analysis to compare the simulated LSE results withthe changing weights and the original results. We used an effi-cient algorithm (Pele & Werman, 2009) in MATLAB to computethe EMD values.

Fig. 6. Change rate maps of the wheat suitability

Results

The standard criteria maps (Fig. 3) were aggregated usingweights to create the final land suitability map for wheat in thenewly reclaimed region (Fig. 4). The areas that are highly suitablefor wheat are primarily located in the southeastern and north-eastern regions. In contrast, relatively unsuitable land is located tothe west of the region and along the banks of the Yili River withdifferent restrictive suitability factors. The major restrictions onwheat cropping in the northwest are the texture of the sandy soiland low soil organic matter, whereas in the southwestern region,these restrictions include thin soil and low soil organic matter. Soilerosion, water supply and drainage problems, and high ground-water depth restrict wheat cropping along the banks of the YiliRiver.

Based on the OAT method, the simulated LSE maps for wheatwere generated with the weight of each criterion changedfrom �100% to 100% with a step size of 5%. These simulated un-certainty maps can be used for sensitivity analysis. The MACRssummarize themean of the absolute change rates for the pixels’ LSEvalues based on the changing weights (Fig. 5). These values show alinear increase with an increase in the rate of change of the weightsbut with different gradients for different criteria. The MACRs of thesame criteria are almost equal using the same absolute rates ofchange but with positive and negative values, which indicatessimilar sensitivities for positive and negative weight changes. Ahigh gradient, which indicates a greater change in the LSE valueswith the changing weights, indicates high sensitivity of the crite-rion for LSE. The ranking of the MACRs for all criteria is as follows:SD > ST > SOM > SE > GD > WSD > SDW. The rankings of SD and

scores when criteria weights change by 75%.

Fig. 7. Change rate maps of the wheat suitability scores when the SOM weight changesby (a) þ50%, and (b) �50%.

Fig. 8. Earth Mover’s Distances for the land suitability evaluation under simulations.

E. Xu, H. Zhang / Applied Geography 45 (2013) 1e9 7

ST follow the order of the criteria weights. Similarly, WSD and SDW,which are assigned low weights, are low-sensitivity criteria. Theweight of a 75% change is taken as an example. The SD, which is thecriterion most sensitive to weight changes, has a MACR of 5.8% ifthe weight changes by 75%, whereas SDW is the least sensitivecriterion with a MACR of just 2.6%. The MACR of the simulated re-sults is significantly lower than the rate of change of the weights(Fig. 5), which indicates that the LSE result is relatively robust.

The rates of change of the LSE values with a 75% change inweight are displayed for spatial visualization in Fig. 6. This figureshows a relatively large spatial variation in the LSE values, whichindicates that sensitivity analysis should be carried out. In thissituation, the uncertainty maps are displayed for local comparison.For example, the rates of change of all pixels with a 75% change inthe SD range from �17.1% to 31.8% are displayed in Fig. 6 andcompared with the MACR of 5.8%. Areas with high LSE values arerelatively robust, which makes sense because these areas have highvalues for all criteria scores and are not likely to be affected by asingle parameter. However, the areas with relatively low LSE valuesare more sensitive, especially when the changing weight of thecriterion matches the corresponding restrained criterion for theseareas. The distribution of sensitivities of different changes inweights is associated with the distribution of suitability classes forthe corresponding criterion (Figs. 3 and 6). Taking the SOMmap of aweight changed by 50% as an example of a locally visualizedcomparison (Fig. 7), the areas with highly decreased LSE values aregenerally located where the SOM suitability class is unsuitable. Incontrast, the areas with highly increased LSE values have high SOMscores and vice versa (Fig. 7A). In addition, the uncertainty maps ofthe same criterion with the same absolute change rate but withpositive and negative values show a highly similar distribution(Fig. 7A and B). The difference is that the positive and negative LSEvalues change. This change indicates similar sensitivities in thesame pixel for weights with the same changing value, albeit posi-tive and negative.

The EMDwas used as a spatially explicit metric in the sensitivityanalysis to identify new information from the spatial dimension forLSE. The EMD is a metric that takes both the numerical values andthe distribution into consideration. The EMDs show a linear in-crease with little fluctuation as the rate of change of the weights

increases. The change in EMD represents the difference in the nu-merical LSE values for the same criterion. Therefore, for differentcriteria, a high gradient indicates high sensitivity. The ranking ofthe EMDs is SOM > SE > ST > SD > GD > WSD > SDW (Fig. 8),which differs from the ranking of the MACRs. In this situation, GD,WSD, and SDW are the same three criteria with the lowest sensi-tivity, but the order of the other criteria has changed. This result canbe attributed to the spatial distribution of the corresponding cri-terion. According to Fig. 3, SOM, SE, and ST display a more discretedistribution than that of the others, which may result in greaterspatial variations in the LSE values as the weights of the corre-sponding weights change. Unlike the other criteria that include ahighly suitable class, the SOM criterion contains four suitable classareas staggered across the region. This result can explain why theEMD of SOM is the highest out of all criteria, although the MACR ofSOM ranks third; it is also the reason why the SE ranks secondaccording to the EMD but ranks fourth according to the MACR.Furthermore, it appears that the SOM has a more staggered spatialdistribution than the SE based on the local area difference visuali-zation, which results in the sensitive ranks of these two criteria.

In contrast, SD, which is considered to be the most sensitiveparameter according to the MACR, exhibits less sensitivity than theabove mentioned three criteria. This observation may be attributedto the fact that the distribution of SD is similar to the distribution inthe original evaluation map. However, SDW has the lowest MACRvalue because the concentrated distribution in the areas of the foursuitable classes have the least spatial variation; areas with low-suitability SDW classes are concentrated in the northwest of theregion, whereas the other largely contiguous area is highly suitablefor wheat. Therefore, the differences in spatial distribution betweenthe LSE maps (which are not easily detected by the local area dif-ference visualization alone but are easily ignored by the traditionalmetrics) can be explored using the EMD.

Thus, LSE for wheat in our study area is robust yet relatively andlocally sensitive to weight changes. This observation should be awarning to experts and multiple decision-makers that the sensi-tivities of the weights for SD and SOM should be taken intoconsideration. Careful assignment of these two most sensitivecriteria is beneficial for validation of the MCDM and robustness ofthe LSE results.

Discussion

We used the MACR and EMD as different metrics to minedifferent information from the sensitivity analysis in the LSE. The

E. Xu, H. Zhang / Applied Geography 45 (2013) 1e98

MACRs provide information on the numerical changes in the LSEvalues, indicating the sensitivity of criteria associated with the sizeof the weights for different criteria. This result is consistent withthat of a previous study (Chen et al., 2010). However, the EMDsexplore the spatial information between maps, which indicate thesensitivity differences of the criteria primarily attributed to thespatial distribution of the corresponding criteria. Although only asingle weight of the criterion is changed and other weights changewith equal proportions using the OAT method, the impact of thespatial difference of the criteria propagated on the distinction offinal sensitivity is output via the changing weights. This new in-formation from the spatial dimension provided by the EMD cannotbe detected by traditional methods (Benke et al., 2011; Butler et al.,1997; Lowell et al., 2011; Maguire et al., 1991; Pelizaro et al., 2011;Tenerelli & Carver, 2012).

To make informed decisions, it is necessary to understand therobustness of the evaluation and have confidence in the decision-making (Chen et al., 2011). Sensitivity analysis can provide usefulinformation for the LSE. In our study, the comparison between theMACR of simulated results and the rate of change of the weightsindicates that the LSE for wheat is relatively robust. Spatial visu-alization shows that areas with high LSE values have low un-certainties, which validates the application of the LSE results forland allocation. The relative robustness can be partly attributed tothe design of the MCDM because the evaluation results are sensi-tive to different aggregation techniques (Zanakis, Solomon,Wishart, & Dublish, 1998). The LSE focuses on selection of suit-able areas for a given crop for potential land source development(Shi, 1986). The additive model yields a more comprehensive resultby combining each criterion and is suitable for the LSE; therefore,this method is always chosen as the aggregation technique in theLSE (Al-Mashreki et al., 2011; Chen et al., 2010; Pelizaro et al., 2011).It makes sense that the additive model is not susceptible to theinfluence of the individual factors, especially because the weightsof the model (range from 0.1 to 0.2) are relatively close in value.

The summary sensitivity analysis can provide a straightforwardresult (i.e., an exact MACR and EMD value in our studies), whichdecision-makers prefer to use in planning. The MACR with a defi-nite statistical value can be easily understood together with therobustness of the LSE results. However, the question arises as tohow the EMD (as a new metric in our study) can provide usefulinformation to assist planners in their decision-making. Similar toother map comparison applications (van den Berg et al., 2011; Jiaoet al., 2010; Roura-Pascual et al., 2010), our results show that theEMD can effectively highlight the differences between the simu-lated land suitability maps and the original map for sensitivityanalysis. Different approaches for cataloging uncertainties haveresulted in misunderstandings with respect to which uncertaintiescan be resolved by MCDM (Mosadeghi et al., 2012). Compared withthe MACR, the EMD presents a different conclusion with respect tothe sensitivity results, which appears to cause confusion for deci-sion-makers.

The EMD explores the spatial information of sensitivities for theLSE from the results. We propose that our framework for sensitivityanalysis should integrate these two metrics for the MCDM.Consider the following example using SOM and SD. The SOM wasidentified as the most sensitive factor by the EMD, which meansthat it causes the largest spatial variation in the LSE values undersimulation. These variations in different pixels indicate relativepriority changes in the different pixel LSE values, which results in alarge contrast in the histograms between the original and simu-lated LSE maps. Next, these variations are similar to the imagecontrast and can be detected by the EMD. Furthermore, the prioritychanges between the pixels’ LSEs cause a decision change for thearea selected for wheat cropping. Therefore, the EMD exploration of

the spatial sensitivity of LSE with weight changes can assist in landallocation planning. In contrast, SD was considered the most sen-sitive factor according to the MACR, which indicates a significantvalue variation in the LSE values under simulation. These variationsrepresent a change in the pixels’ LSE values compared withthemselves. However, the priorities of the pixels’ LSE values remainrelatively stable when the SD weight changes, which is why theEMD ranked fourth out of all criteria. Therefore, these two metrics,which incorporate different aspects of the information, can be usedto detect sensitive parameters in our framework for validation ofMCDM.

Application of the EMD requires further study to confirm itsvalidity in sensitivity analysis of the LSE and to investigate its role insupporting decision-making. Although we evaluated only thesensitivity of the weights, the EMD can be applied to other sourcesof sensitivity (Burgman, 2005; Chen et al., 2011; Reshmidevi et al.,2009; Wood et al., 2009). However, there is no rigorous statisticalmethod for testing whether the LSE maps are significantly differentusing the EMD. In our study, the EMD can indicate the relativesensitivity of all criteria but cannot determine whether a change inthe weight significantly influences the spatial change in the results.Monte Carlo simulation, which is used for significance tests inspatial analysis (Schabenberger & Gotway, 2004), could be apromising approach for improving our method.

Conclusion

A better understanding of the robustness of the LSE results canprovide a better aid for its effective implementation in land-useplanning. We proposed a framework based on spatially explicitsensitivity analysis using the EMD as a new metric. Application ofour framework in the newly reclaimed region of Yili confirmed itseffectiveness in the validation of MCDM for LSE. This result in-dicates that the LSE for wheat is robust according to the MACR, butlocal areas are relatively sensitive to changing weights according tothe spatial visualization. Based on our framework, the MACRsummarizes the information in the numerical value variations,whereas the EMD incorporates new information from the spatialvariations for sensitivity analysis in the MCDM. In this case, SD andSOM are the twomost sensitive criteria in terms of weight changes.These two indicators explore different information with respect tosensitivity and result in different sensitivity orders for the sevencriteria. Integration of these two criteria was proposed for valida-tion of MCDM. All the criteria provide a spatially explicit approachfor the LSE and informed decision-making. The improvementsmentioned above should be incorporated in future studies.

Acknowledgments

This work was supported by the National Natural ScienceFoundation of China (41071065). We thank Karen Bradshaw inLiwen Bianji (Edanz Group China) for English editing. We are mostgrateful for the excavation of soil profiles and the collection of soilsamples performed by Wang Lixin, Yang Yang, Ma Hanqing, andZhang Ying.

References

Al-Mashreki, M. H., Akhir, J. B. M., Rahim, S. A., Lihan, K. M. D. T., & Haider, A. R.(2011). GIS-based sensitivity analysis of multi-criteria weights for land suit-ability evaluation of sorghum crop in the Ibb Governorate Republic of Yemen.Journal of Basic and Applied Scientific Research, 1(9), 1102e1111.

Benke, K. K., Pelizaro, C., & Lowell, K. E. (2009). Uncertainty in multi-criteria eval-uation techniques when used for land suitability analysis. In Crop modeling anddecision Support (pp. 291e298). Berlin: GER: Springer.

E. Xu, H. Zhang / Applied Geography 45 (2013) 1e9 9

Benke, K. K., Steel, J. L., & Weiss, J. E. (2011). Risk assessment models for invasivespecies: uncertainty in rankings from multi-criteria analysis. Biological In-vasions, 13(1), 239e253.

van den Berg, M., Vandenberghe, S., De Baets, B., & Verhoest, N. (2011). Copula-based downscaling of spatial rainfall: a proof of concept. Hydrology and EarthSystem Sciences, 15(5), 1445e1457.

Blaser, A. D., Sester, M., & Egenhofer, M. J. (2000). Visualization in an early stage ofthe problem-solving process in GIS. Computers & Geosciences, 26(1), 57e66.

BojóRquez-Tapia, L. A., Cruz-Bello, G. M., & Luna-González, L. (2012). Connotativeland degradation mapping: a knowledge-based approach to land degradationassessment. Environmental Modelling & Software, 40, 51e64.

Burgman, M. (2005). Risks and decisions for conservation and environmental man-agement. Cambridge, UK: Cambridge University Press.

Butler, J., Jia, J., & Dyer, J. (1997). Simulation techniques for the sensitivity analysis ofmulti-criteria decision models. European Journal of Operational Research, 103(3),531e546.

Ceballos-Silva, A., & López-Blanco, J. (2003). Evaluating biophysical variables toidentify suitable areas for oat in Central Mexico: a multi-criteria and GISapproach. Agriculture, Ecosystems & Environment, 95(1), 371e377.

Cha, S.-H., & Srihari, S. N. (2002). On measuring the distance between histograms.Pattern Recognition, 35(6), 1355e1370.

Chen, H., Wood, M. D., Linstead, C., & Maltby, E. (2011). Uncertainty analysis in a GIS-based multi-criteria analysis tool for river catchment management. Environ-mental Modelling & Software, 26(4), 395e405.

Chen, Y., Yu, J., & Khan, S. (2010). Spatial sensitivity analysis of multi-criteria weightsin GIS-based land suitability evaluation. Environmental Modelling & Software,25(12), 1582e1591.

Crosetto, M., Tarantola, S., & Saltelli, A. (2000). Sensitivity and uncertainty analysisin spatial modelling based on GIS. Agriculture, Ecosystems & Environment, 81(1),71e79.

Daniel, C. (1958). On varying one factor at a time. Biometrics, 14(3), 430e431.Delgado, M. G., & Sendra, J. B. (2004). Sensitivity analysis in multicriteria spatial

decision-making: a review. Human and Ecological Risk Assessment, 10(6), 1173e1187.

Elaalem, M., Comber, A., & Fisher, P. (2011). A comparison of Fuzzy AHP and ideal pointmethods for evaluating land suitability. Transactions in GIS, 15(3), 329e346.

FAO. (1976). A framework for land evaluation (Soils Bulletin No. 32). Rome: Food andAgriculture Organisation of the United Nations.

Feick, R., & Hall, B. (2004). A method for examining the spatial dimension of multi-criteria weight sensitivity. International Journal of Geographical InformationScience, 18(8), 815e840.

Fuller, M. M., Gross, L. J., Duke-Sylvester, S. M., & Palmer, M. (2008). Testing therobustness of management decisions to uncertainty: everglades restorationscenarios. Ecological Applications, 18(3), 711e723.

Hallisey, E. J. (2005). Cartographic visualization: an assessment and epistemologicalreview*. The Professional Geographer, 57(3), 350e364.

Helton, J. C. (1993). Uncertainty and sensitivity analysis techniques for use in per-formance assessment for radioactive waste disposal. Reliability Engineering &System Safety, 42(2), 327e367.

Heumann, B. W., Walsh, S. J., & McDaniel, P. M. (2011). Assessing the application of ageographic presence-only model for land suitability mapping. Ecological Infor-matics, 6(5), 257e269.

Hossain, M. S., & Das, N. G. (2010). GIS-based multi-criteria evaluation to landsuitability modelling for giant prawn (Macrobrachium rosenbergii) farming inCompanigonj Upazila of Noakhali, Bangladesh. Computers and Electronics inAgriculture, 70(1), 172e186.

Jiao, F., Phillips, J. M., Stinstra, J., Krger, J., Varma, R., Hsu, E., et al. (2010). Metrics foruncertainty analysis and visualization of diffusion tensor images. In Medicalimaging and augmented reality (pp. 179e190). Berlin, GER: Springer.

Kumar, S., Patel, N. R., Sarkar, A., & Dadhwal, V. K. (2013). Geospatial approach inassessing agro-climatic suitability of soybean in rainfed agro-ecosystem.Journal of the Indian Society of Remote Sensing, 1-10. http://dx.doi.org/10.1007/s12524-12012-10249-12529.

Larichev, O. I., & Moshkovich, H. M. (1995). ZAPROS-LMdA method and system forordering multiattribute alternatives. European Journal of Operational Research,82(3), 503e521.

Levina, E., & Bickel, P. (2001). The earth mover’s distance is the Mallows distance:some insights from statistics. In , Proceedings. Eighth IEEE International Confer-ence on: Vol. 2. Computer vision, 2001. ICCV 2001 (pp. 251e256). IEEE.

Ligmann-Zielinska, A., & Jankowski, P. (2008). A framework for sensitivity analysisin spatial multiple criteria evaluation Geographic information science (pp. 217e233). Berlin: GER: Springer.

Ligmann-Zielinska, A., & Jankowski, P. (2012). Impact of proximity-adjusted pref-erences on rank-order stability in geographical multicriteria decision analysis.Journal of Geographical Systems, 14(2), 167e187.

Ligmann-Zielinska, A., Jankowski, P., & Watkins, J. (2012). Spatial uncertainty andsensitivity analysis for multiple criteria land suitability evaluation. In Seventhinternational conference on geographic information science.

Lowell, K., Christy, B., Benke, K., & Day, G. (2011). Modelling fundamentals and thequantification and spatial presentation of uncertainty. Journal of Spatial Science,56(2), 185e201.

Maguire, D. J., Goodchild, M. F., & Rhind, D. W. (1991). Geographical informationsystems: Principles and applications. In Applications (Vol. 2). London, UK: Long-man Scientific and Technical.

Marinoni, O. (2004). Implementation of the analytical hierarchy process with VBAin ArcGIS. Computers & Geosciences, 30(6), 637e646.

Merritt, W. S., Croke, B. F. W., & Jakeman, A. J. (2005). Sensitivity testing of a modelfor exploring water resources utilisation and management options. Environ-mental Modelling & Software, 20(8), 1013e1030.

Mosadeghi, R., Warnken, J., Tomlinson, R., & Mirfenderesk, H. (2012). Uncertaintyanalysis in the application of multi-criteria decision-making methods inAustralian strategic environmental decisions. Journal of Environmental Planningand Management1e28. http://dx.doi.org/10.1080/09640568.09642012.09717886(ahead-of-print).

Nisar Ahamed, T. R., Gopal Rao, K., & Murthy, J. S. R. (2000). GIS-based fuzzymembership model for crop-land suitability analysis. Agricultural Systems,63(2), 75e95.

Ohta, K., Kobashi, G., Takano, S., Kagaya, S., Yamada, H., Minakami, H., et al. (2007).Analysis of the geographical accessibility of neurosurgical emergency hospitalsin Sapporo city using GIS and AHP. International Journal of Geographical Infor-mation Science, 21(6), 687e698.

Pannell, D. J. (1997). Sensitivity analysis of normative economic models: theoreticalframework and practical strategies. Agricultural Economics, 16(2), 139e152.

Pele, O., & Werman, M. (2009). Fast and robust earth mover’s distances. In Com-puter vision, 2009 IEEE 12th International conference on (pp. 460e467). IEEE.

Pelizaro, C., Benke, K., & Sposito, V. (2011). A modelling framework for optimisationof commodity production by minimising the impact of climate change. AppliedSpatial Analysis and Policy, 4(3), 201e222.

Pereira, J. M. C., & Duckstein, L. (1993). A multiple criteria decision-makingapproach to GIS-based land suitability evaluation. International Journal ofGeographical Information Science, 7(5), 407e424.

Rabitz, H. (1989). Systems analysis at the molecular scale. Science, 246(4927), 221e226.Refsgaard, J. C., van der Sluijs, J. P., Højberg, A. L., & Vanrolleghem, P. A. (2007).

Uncertainty in the environmental modelling processea framework and guid-ance. Environmental Modelling & Software, 22(11), 1543e1556.

Reshmidevi, T. V., Eldho, T. I., & Jana, R. (2009). A GIS-integrated fuzzy rule-basedinference system for land suitability evaluation in agricultural watersheds.Agricultural Systems, 101(1), 101e109.

Roura-Pascual, N., Krug, R. M., Richardson, D. M., & Hui, C. (2010). Spatially-explicitsensitivity analysis for conservation management: exploring the influence ofdecisions in invasive alien plant management. Diversity and Distributions, 16(3),426e438.

Rubner, Y., Guibas, L. J., & Tomasi, C. (1997). The earth mover’s distance, multi-dimensional scaling, and color-based image retrieval. In Proceedings of theARPA image understanding workshop (pp. 661e668).

Rubner, Y., Tomasi, C., & Guibas, L. J. (2000). The earth mover’s distance as a metricfor image retrieval. International Journal of Computer Vision, 40(2), 99e121.

Saaty, T. L. (2008). Decision making with the analytic hierarchy process. Interna-tional Journal of Services Sciences, 1(1), 83e98.

Saltelli, A., Chan, K., & Scott, E. M. (2000). Sensitivity analysis (Vol. 134). New York,USA: Wiley.

Schabenberger, O., & Gotway, C. A. (2004). Statistical methods for spatial data analysis(Vol. 64). Boca Raton, USA: Chapman and Hall/CRC.

Shi, Y. l. (1986). Land resources studies in three decades. Natural Resources, 3, 54e57(in Chinese).

Shi, R. X., Yang, X. H., & Wang, L. X. (2009). Soil nutrient properties and landdevelopment in the newly reclaimed area of Yili, Sinkiang Municipality, China.Resources Science, 31(12), 2016e2023 (in Chinese).

Tenerelli, P., & Carver, S. (2012). Multi-criteria, multi-objective and uncer-tainty analysis for agro-energy spatial modelling. Applied Geography, 32(2),724e736.

Vaidya, O. S., & Kumar, S. (2006). Analytic hierarchy process: an overview of ap-plications. European Journal of Operational Research, 169(1), 1e29.

Vitek, J. D., Giardino, J. R., & Fitzgerald, J. W. (1996). Mapping geomorphology: ajourney from paper maps, through computer mapping to GIS and virtual reality.Geomorphology, 16(3), 233e249.

Wood, M. D., Beresford, N. A., Barnett, C. L., Copplestone, D., & Leah, R. T. (2009).Assessing radiation impact at a protected coastal sand dune site: an inter-comparison of models for estimating the radiological exposure of non-humanbiota. Journal of Environmental Radioactivity, 100(12), 1034e1052.

Zanakis, S. H., Solomon, A., Wishart, N., & Dublish, S. (1998). Multi-attribute decisionmaking: a simulation comparison of select methods. European Journal ofOperational Research, 107(3), 507e529.

Zoras, S., Triantafyllou, A. G., & Hurley, P. J. (2007). Grid sensitivity analysis for thecalibration of a prognostic meteorological model in complex terrain by ascreening experiment. Environmental Modelling & Software, 22(1), 33e39.