geoprocesamiento en amazon

8/13/2019 Geoprocesamiento en Amazon

1/37

GEO PROCESAMIENTO EN WEB


2/37

Introduccin

Sources images:alkoholikuin.blogspot.comnro.nao.ac.jp

Informacin Geogrfica

publicacion

Internet

ProcessingDesktop applications

UploadDownload

Cloud computingProcessing

publicacion

Escenario A

C h

a n g

i n g

Escenario B


3/37

Evolucin Geopro

(Brauner et al., 2009)

Geoprocessing

Web Processing Services(WPS)

Improved WPS

Llevar los servicios donde se encuentra lainformacin (moving code paradigm)

Utilizar la tecnologia Grid Computing

Incorporar Service Oriented Architecture(SOA) caracteristicas

Parallelized WPSOn

Cloud Computing


4/37

Web Services Framework Of OGC GeoprocessingStandards

Web Processing Services (WPS)


5/37

Web Processing Services (WPS)

Interface estndar que permite enmascarar procesos,algoritmos y operaciones en la Web de una formadefinida y estructurada para ser encontrados y usados porclientes y otros procesos

WPS

GetCapabilities

GetDescription

Execute

Disponible servicios

Informacin acerca del servicio

Ejecuta

Current version 1.0.0


6/37

WPS Frameworks

Deegree

52North

PyWPS

Zoo

Extensions Grid Capabilities

GridGain

UNICORE


7/37

Cloud Computing


8/37

Definicin

Cloud computing is a technology that uses the Internet and

central remote servers to maintain data and applicationsWikiInvest


9/37


10/37

Proveedores de Cloud Computing

Amazon Web Service (AWS)Google App EngineMicrosoft AzureGoGridRackspace

o Amazon Elastic Compute Cloud(EC2)

o Elastic Block Store (EBS)o Multiple Locationso Elastic IP Addresso Auto Scalingo Elastic Load Balancingo Amazon Simple Storage Service

(S3)o Amazon CloudFronto Amazon Relational Database

Service


11/37

Aplicaciones Geospaciales en la Web

Publicaciones de Web Map Services (WMS) on Google App Engine(GAE)Implementacin de indices utilizando Hadoop technology sobre 110-millones de parcelas en una nube privadaGoogle maps utiliza cloud computing para soportar miles de usuarios

WPS service en AWS.. =>


12/37

WPS en Grid


13/37


14/37

El principio de los mtodos

o Dependencia de los vecinoso Proceso estacionario

Cmo es obtiene s depende del metodo

Kriging => Ordinary KrigingUniversal Kriging

Radial Basis Function (RBF)

Inverse Distance Weight (IDW)


15/37

Methods

InverseDistance Weight

(IDW)

Ordinaryand

UniversalKriging

Radial BasisFunction

(RBF)

Semivariogram Fitting theempirical toTheoreticalmodel.

LinearExponentialSpherical..,

Ordinary Kriging

Universal Kriging andits variance s 2

k power of interpolation

221 cr r

22

2

1

cr

r

Multiquadric :

Inverse multiquadric :

Thin plate spline :Multilog :Natural cubic spline :Spline with tension :

( r i) is the radial basis function

z is the evaluationof all points in the functionused

Preprocessing

Least Squares

Least Squares

None

CalculatingGetting s


16/37

Web Processing Services en Grid

Interpolacin

Caso de estudio


17/37

Cross Validation

IDW

Kriging(Ordinary andUniversal)

RBF

Number of neighborhoods from 5 to 12, andthe power value change from 0.1 to 10 atsteps of size 0.1.

Number of neighborhoods from 5 to 12,number of lags (7-15), models: Linear,Spherical and Exponential

Number of neighborhoods from 5 to 12, thesmooth factor from 0.1 to 0.5 at steps of size0.1 and sub models (Multiquadric, Inversemultiquadric, Thin plane spline, Multilog,Natural cubic spline)


18/37

Distribucin del proceso deinterpolacin

Pixels to beinterpolatedby each node

Points to besent to eachnode


19/37

Libreria para paralelizar procesos

Classes with

parallelizationfeatures


20/37

Adicionar servicios en paralelo

Profile by default in the52North Framework


21/37

Estructura de usada

Tomcat configuration Master Node (GridGain Node)

Master Node

Nodes


22/37

WPS services parallelized

Entradas:Data: WFS with GML and SHP-ZIP formatField: Contain the attribute to do the

interpolationMethod: Ordinary Kriging, UniversalKriging, IDW, RBF

Salida:RMS: Error of the best method in thecross validationStdRMS: Standardized ErrorCorrelation coefficient

Parameters: Parameters found.Cross Validation Graph: URL with crossvalidation graphFitting graph: Show the error behavioraccording to the method selectedIteration: URL with a log file with theiterations summary.

Entrada:Data: WFS with GML and SHP-ZIP formatField: Contain the attribute to dointerpolationMethod: This input receives astring with the method andparameters for executing theinterpolationResolution: Spatial Resolution

Salida:Result : WMS with reference tocoverage on GeoserverDuration: process duration

Validacin cruzada Interpolacin


23/37

Instalando a WPS en Amazon AWS(Linux)

Crear una cuenta enAmazon AWShttp://aws.amazon.com/account/

Instalar AWS service APIObtener credenciales variables (Linux)Crear una instancia
http://aws.amazon.com/account/http://aws.amazon.com/account/


24/37

Consola de Amazon AWS

Amazon AWS tiene 2433 instancias publicas


25/37

Utilizando 10 instancias


26/37


27/37

Informacin usada

[1] http://www.aemet.es/es/servidor-datos/acceso-datos/listado-contenidos/detalles/datos_observacion

Meteorological Agency of Spain (AEMET).


28/37

Datos de elevacin

The DEM is published by OpenTopo,created in 2008 with LIDAR, resolution of 0.5 meters,Universal Traversal Mercator (UTM) region 12 North,datum WGS84

Evaluacin con 1000 y 10000puntos


29/37


30/37

Software y hardware AWS

Master NodeSoftware

AMI Operating system fedoraTomcat 6.Java SDK 1.6.0.20.GridGain 2.1.1Geoserver 2.252North WPS Framework RC652North WPS OpenLayer client

Hardware simulated (1 micro Instance )Micro Instance 613 MB of memory 32-bitplatform

Hardware simulatedHigh-CPU Medium Instance 1.7 GB ofmemory, 5 EC2 Compute Units (2 virtualcores with 2.5 EC2 Compute Units each),350 GB of local instance storage, 32-bitplatform

Nine Nodes

Software

AMI Operating system fedora

Tomcat 6.

Java SDK 1.6.0.20.

GridGain 2.1.1

Hardware simulated (1 micro Instance )

Micro Instance 613 MB of memory 32-bit platform


31/37

Evaluacin de la libreria

y = 1.0075x

0

2

4

6

8

10

12

14

0 5 10 15

V a l u e I n t e r p o l a t e d

b y W P S

( G e o s t a t

i s t c a l

l i b r a r y

Value Interpolated by WPS (Geostatistcallibrary)

Comparison between Geos. libraryand ArcGIS Method Kriging

y = 0.9856x

0

2

4

6

8

10

1214

0 5 10 15

V a l u e

I n t e r p o l a t e d

b y W P S

( G e o s t a t

i s t c a l

l i b r a r y

Value Interpolated by Geostatisticalextension ArcGIS

Comparison between Geos. libraryand ArcGIS Kriging Univeral

Method

y = 0.9994x

0

2

4

6

8

10

12

14

0 5 10 15

V a l u e


b y W

P S

( G e o s t a t

i s t c a l

l i b r a r y

Value Interpolated by Geostatisticalextension ArcGIS

Comparison between Geos. library andArcGIS

IDW method

y = 0.9994x

0

2

4

6

8

10

1214

0 5 10 15

V a l u e


b y W

P S

( G e o s t a t

i s t c a l

l i b r a r y

Value Interpolated by Geostatisticalextension ArcGIS)

Comparison between Geos. library andArcGIS

RBF Multiquadratic Method

MethodGeostatistical Library ArcGIS (Geostatistical Extension)

RMSStandardized

RMS RMSStandardized

RMS

IDW1.607 - 1.607 -

Power=2

Ordinary Kriging1.587 0.881 1.605 0.829 Exponential(R:261138;S:5.4;N:

1.97)

Universal Kriging1.593 0.873 1.623 0.820 Exponential(R:29138;S:4.4;N:1.

67)

RBF1.648 - 1.648 - Model: Multiquadratic;

Factor=0


32/37

Comparasion interpolacin

RBF Kriging Ordinario

Small differences


33/37

Evaluation in WPS OpenLayer client

CrossValidation WPS service

Interpolation WPS service


34/37

1 2 3 4 1 2 3 4 1 2 3 4

Res: 2 Res: 5 Res: 10IDW 34686 28323 18286 17968 7152 5833 4497 4460 3688 4037 4391 2820Kriging 26132 23232 18487 14955 4464 4676 4289 3748 3414 2401 2602 2088

KrigingUniversal 34588 28482 21773 18639 7116 5742 4511 4593 3284 2856 2592 2586Spline 35908 26000 16980 19056 7389 5590 4290 4609 3760 4298 4391 2920

0

5000

10000

15000

20000

25000

30000

35000

40000

D u r a t

i o n

I n t e r p o l a t

i o n

M i l l i s e c o n d s

Utilizando intranet 4 nucleos(4 nodos, 1.000 points)

1.000.000 pixels 160.000 pixels 40.000 pixels


35/37


36/37

Datos a evaluar

Relacin numero de nodos y tiempo de interpolacin

Entorno de evaluacin:

Evaluacion: 2011-02-22 T 9:00:00 Z - 2011-02-22 T22:00:00 ZNumero de puntos: 10000Resolucin: 2 meters (1000000 pixels), 5 m (160.000pixels) and10 m (40.000 pixels).Numero de nodos(micro instances): 1-10Repeticiones: 10Solicitudes secuenciales 15 segundos


37/37

Resultados de la evaluacin

Method ResolutionNumberof Pixels

MaxDuration

Numberof Nodes

MaxDifference

PercentageDifference

Kriging 2 1000000 265467 10 -221610(a) -83%

Kriging 5 160000 14295 7 -10628(a) -74%

Kriging 10 40000 3311 8 -1136(a) -34%

IDW 2 1000000 227569 10 -187858(a) -83%

IDW 5 160000 11435 10 -8024(a) -70%

IDW 10 40000 2848 9 -713(a) -25%

Method ResolutionNumber of

PixelsMax

Duration

After thisnode (Notfind signif.

p-value

geoprocesamiento en amazon

Documents