d6.2 data-source integration framework

ThisprojecthasreceivedfundingfromtheEuropeanUnion’sHorizon2020researchandinnovationprogramundergrantagreementNo700475

beAWAREEnhancingdecisionsupportandmanagementservicesinextremeweather

climateevents

700475

D6.2Data-SourceIntegrationFramework

Disseminationlevel: Public

Contractualdateofdelivery: Month15,31March2018

Actualdateofdelivery: Month15,31March2018

Workpackage: WP6:PublicSafetyAnsweringPointTask: T6.2DataSourceIntegrationFramework

Type: Report

ApprovalStatus: Final

Version: V1.0

Numberofpages: 49

Filename: D6.2_beAWARE_Data-SourceIntegrationFramework_2018-03-31_v1.0.docx

AbstractThis deliverable reflects thework performed in task T6.2 dedicated to ensure robust andreliable integration of data sources. In particular, the T6.2 objectives are to develop asuitable support framework to createan integration layer that canbedeployedon topofvaryingphysicalinfrastructurestoexpeditetheintegrationofexistingandnewdatasources.Byprovidingthisabstractionfromtheunderlyingdatasources, itwillprovideaframeworktointelligentlymanagethedataandcontrolflowsbetweenthebeAWAREplatformandthephysical environment. To support the integration of existing sensors, camera, video andother structured data, a set of reusable data source connectors will be developed. ThisdeliverablewilldescribetheDataSourceIntegrationFrameworklayer.

Ref. Ares(2018)1756440 - 31/03/2018

ThisprojecthasreceivedfundingfromtheEuropeanUnion’sHorizon2020researchandinnovationprogramundergrantagreementNo700475

Theinformationinthisdocumentreflectsonlytheauthor’sviewsandtheEuropeanCommunityisnotliableforanyusethatmaybemadeoftheinformationcontainedtherein.Theinformationinthisdocumentisprovidedasisandnoguaranteeorwarrantyisgiventhattheinformationisfitforanyparticularpurpose.Theuserthereofusestheinformationatitssoleriskandliability.

Co-fundedbytheEuropeanUnion

D6.2–V1.0

Page3

History

Version Date Reason Preparedby

0.1 December1,2017 Deliverableoutline Y.Mordecai(MSIL)

0.2 January15,2018 Deliverabledraft Y.Mordecai(MSIL)

0.3 February15,2018

PartnerContributionIntegration–Cycle1 Y.Mordecai(MSIL)

0.4 March15,2018 PartnerContributionIntegration–Cycle2 Y.Mordecai(MSIL)

0.5 March19,2018 Pre-SubmissionReview B.Kantsepolsky(MSIL)0.6 March28,2018 Peer-ReviewedVersion Y.Mordecai(MSIL)1.0 March29,2018 SubmittedVersion Y.Mordecai(MSIL)

D6.2–V1.0

Page4

Authorlist

Organization Name ContactInformationMSIL YanivMordecai [email protected] BorisKantsepolsky [email protected] MichalShany [email protected] IsraelAltar [email protected] BennyMandler [email protected] AnastasiosKarakostas [email protected] SteliosAndreadis [email protected] EmmanouilMichail [email protected] JürgenMoßgraber [email protected] PhilippHertweck [email protected] HylkevanderSchaaf [email protected] TobiasHellmund [email protected] StamatiaDasiopoulou [email protected] JensGrivolla [email protected] MicheleFerri [email protected] DanieleNorbiato [email protected] FrancescaLombardo [email protected] GiovanniTomei [email protected]

Reviewers

Partner Name ContactInformationMSIL BorisKantsepolsky [email protected] AnastasiosKarakostas [email protected] GerasimosAntzoulatos [email protected] IosifVourvachis [email protected]

D6.2–V1.0

Page5

ExecutiveSummary

beAWARE is a highly-interconnected systemof systems and interdependentmodules thatfeed each other with information regarding evolving and ongoing extreme climateemergencies.Thisincludesacontrolcentersolution,amobileapplication,variousmediaandsocialmediaanalysisservices,sensoranalysisservice,weatherandclimateforecastservices,crisis classification and reasoning services,multilanguageanalysis and generation services,andsupportinginfrastructure.

beAWARE integrates data from multiple sources, including field reports, sensormeasurements,weather data,media, analysis results, and socialmedia posts. In addition,thesystemhastogenerateinformationandinsightaswellasactionsandinstructionswithinthe system and for its users and stakeholders. For this reason, a suitable data sourceintegration framework has been defined to govern the data and information exchangearchitectureandrealization.

Thepurposeof thedata-source integration framework is therefore to assure and supportinteroperability among various stakeholders and operational actors – authority officials,emergencymanagementofficers,analysts, first responders,andcitizens.This isdonebyarobust,dynamic,andflexibleinfrastructurethatallowsinterconnectivityamongthevarioussubsystems,modules, and services. On top of this infrastructure, the system enables andimplements multiple end-to-end interactions, leveraging data exchange, storage, access,acquisition,andvisualizationmechanisms.

This document is themain reference for thedata exchange architectureof thebeAWAREplatformasimplementedtowardbeAWARE'sfirstprototype,dueJune2018.Theframeworkhasbeendefinedtobesufficientlyrobusttoaccommodatepossibleupdatesandextensionsofdataexchangeandcollaboration,astheprojectprogressestowardsthefinalversion,dueDecember2019.

Thisdocumentprovides:a)apublicoverviewoftheoperationaluserrequirementsentailingimpact related todata anddata-sourcemanagement and integration;b) apublic detaileddescriptionofthecommondataintegrationinfrastructure,includingmessaging,dataaccessandquerying,andfilestorageservices;andc)aconfidential,detailedspecificationofeachmodule’sconnectivityrequirements,inputs,outputs,anddataaccessmechanisms.

The operational section is public because it relies on publicly-available operational userrequirements, which constitute common knowledge of public interest. The operationalrequirements have been analyzed for impact on data-source and data user integration,

D6.2–V1.0

Page6

interaction, and interoperability. Themainmodules and entitieswere laid out to supporttheserequirements.Thisprocessanditsoutcomesareofuseasageneralreferenceforthecommunity.

Thesectiondiscussingthegenericinfrastructuresupportingdataexchangeandintegration,includingmessaging,mediastorage,databases,etc.,isalsopublicsinceitreliesonpublicly-available open-source products, and constitutes a benchmark that we advocate to theemergencymanagementtechnologycommunity.

In the applicative layer, the beAWARE modules' utilization of the generic infrastructureaccountsforthevariousconstraints,challenges,technicalrequirements,anddatacontent.This information remains confidential, in order to prevent direct exposure of technicaldetails, data structures, and access control mechanisms, and in order to protect thebeAWAREpartners'intellectualproperty.

This deliverable is the product of a year-long effort to define, organize, manage, design,implement,anddemonstratethedata-sourceintegrationframework,aspartoftaskT6.2inWP6, under the responsibility of MSIL. This effort was heavily supported by IBM in thegenericinfrastructureaspects,andinvolvedalltheothertechnicalpartners–AAWA,CERTH,FMI, IOSB, and UPF. Each technical partner helped derive and define its technicalconnectivityandfunctionaldata-exchangerequirements.Theconsortiumasawholeworkedtogethertoensureend-to-endflowofinformationthroughaseriesofteleconferences,face-to-facemeetings, partner-to-partner discussions and end-to-end integration sessions. Theoperational partners – HRT, PLV, and FBBR – contributed to this effort by reviewing thesystem requirements and overall architecture throughout the project, and ensuring theircompliance with the operational requirements. In addition, PLV and HRT provided usefulcommentsandreviewsonthisdeliverablepriortoitssubmission.

D6.2–V1.0

Page7

AbbreviationsandAcronyms

alt altitude API ApplicationProgrammingInterface ASR AutomaticSpeechRecognition CAP CommonAlertProtocol CDR CentralDataRepository CRCL CrisisClassification CSV Comma-SeparatedValues DB Database GIS GeographicInformationSystem GPS GlobalPositioningSatellite HTM(L) HypertextMarkupLanguage HTTP HypertextTransferProtocol ID identifier IoT InternetofThings IP InternetProtocol JSON JavaScriptObjectNotation KBS Knowledge-BaseServices lat latitude long longitude MRG MultilanguageReportGeneration MSB MessageBus MTA MultilanguageTextAnalysis ODI OpenDataInterface PCI PeripheralComponentInterconnect PSAP PublicSafetyAnsweringPoint Pub/Sub Publish—Subscribe REST Representationalstatetransfer SDK SoftwareDevelopmentKit SMA SocialMediaAnalytics SoS SystemofSystems SQL StructuredQueryLanguage TCP TransmissionControlProtocol TR TechnicalRequirement UDP UserDatagramProtocol UI UserInterface UR UserRequirement URI UniformResourceIdentifier URL UniformResourceLocator USB UniversalSerialBus WFS WeatherForecastServices WP WorkPackage XML ExtensibleMarkupLanguage XSD XMLSchemaDefinition

D6.2–V1.0

Page8

PartnerAcronyms

AAWA AUTORITÀDIBACINODEIFIUMIISONZOTAGLIAMENTOLIVENZAPIAVEBRENTABACCHIGLIONECERTH CENTREFORRESEARCHANDTECHNOLOGYHELLASFBBR FREDERIKSSUND-HALSNÆS:FIRE-&RESCUESERVICEFMI FINNISHMETEOROLOGICALINSTITUTEHRT HELLENICRESCUETEAMIBM IBMISRAEL-SCIENCE&TECHNOLOGYLTD.IOSB FRAUNHOFER-GESELLSCHAFTZURFOERDERUNGDERANGEWANDTENFORSCHUNGE.V.MSIL MOTOROLASOLUTIONSISRAELLTDPLV AYUNTAMIENTODEVALENCIAUPF UNIVERSITATPOMPEUFABRA

D6.2–V1.0

Page9

TableofContents

1 Introduction[PUBLIC] 131.1 Scope............................................................................................................................131.2 "beAWARE"..................................................................................................................131.3 TheneedforaData-SourceIntegrationFramework......................................................131.4 Outline..........................................................................................................................14

2 Data-SourceIntegrationRequirements[PUBLIC] 162.1 Scope............................................................................................................................162.2 UserRequirementsforData-SourceIntegration............................................................16

2.2.1 UserRequirementsfortheFloodScenario.................................................................172.2.2 UserRequirementsfortheFireScenario....................................................................212.2.3 UserRequirementsfortheHeatwaveScenario..........................................................24

2.3 OperationalRoles.........................................................................................................292.4 Modules........................................................................................................................292.5 OperationalEntities......................................................................................................302.6 DataAccessModalities.................................................................................................33

3 DataSourceIntegrationInfrastructure[PUBLIC] 363.1 Scope............................................................................................................................363.2 CentralMessageBus–MSB..........................................................................................37

3.2.1 Overview.....................................................................................................................373.2.2 SupportedEntities.......................................................................................................373.2.3 ConnectivityDiagram..................................................................................................383.2.4 FeaturesandCapabilities............................................................................................383.2.5 DeploymentandOperation.........................................................................................393.2.6 DataAccessConventions............................................................................................393.2.7 Authentication.............................................................................................................393.2.8 CommonTopicHeader................................................................................................393.2.9 Topics..........................................................................................................................40

3.3 CentralDataRepository–CDR......................................................................................423.3.1 Overview.....................................................................................................................423.3.2 SupportedEntities.......................................................................................................423.3.3 ConnectivityDiagram..................................................................................................433.3.4 FeaturesandCapabilities............................................................................................433.3.5 DeploymentandOperation.........................................................................................43

D6.2–V1.0

Page10

3.3.6 DataAccessConventions............................................................................................433.4 NoSQLDatabase...........................................................................................................44

3.4.1 Overview.....................................................................................................................443.4.2 SupportedEntities.......................................................................................................443.4.3 DeploymentandOperation.........................................................................................443.4.4 DataAccessConventions............................................................................................44

4 Module-LevelDataAccessMechanisms[CONFIDENTIAL]45

5 Conclusion[PUBLIC] 47

6 References 49

D6.2–V1.0

Page11

ListofFigures

Figure3-1.ConnectivityDiagram-MSB..................................................................................38Figure3-2.ConnectivityDiagram-CDR..................................................................................43

D6.2–V1.0

Page12

ListofTables

Table2-1.InitialUserRequirements–FloodScenario[2]......................................................17Table2-2.InitialUserRequirements–FireScenario[2].........................................................21Table2-3.InitialUserRequirements–HeatwaveScenario[2]...............................................24Table2-4.OperationalRoles...................................................................................................29Table2-5.Modules..................................................................................................................29Table2-6.OperationalEntities................................................................................................30Table2-7.DataAccessModalities...........................................................................................33Table3-1.CommonHeaderforbeAWAREMessageBusTopics.............................................40Table3-2.beAWAREMessageBusTopics...............................................................................41

D6.2–V1.0

Page13

1 Introduction[PUBLIC]

1.1 Scope

This document constitutes a reference for the data integration and exchangeframeworkforbeAWARE.

1.2 "beAWARE"

The beAWARE Project is an EU-funded collaboration (#700475) of partners fromseveral countries in Europe to deliver a prototype disastermanagement system forextreme weather conditions. The Project is focused on Flood, Forest Fire, andHeatwavescenarios,andisintendedfordeploymentandtestingofthesescenariosinVicenza(Italy),Valencia(Spain),andThessaloniki(Greece),respectively.Theendusersare the Alto Adriatico Water Authority (AAWA), Valencia Local Police (PLV), andHellenic Rescue Team (HRT), respectively. In addition, the Frederiksborg FireDepartment (FrederiksborgBrandOGRedning, FBBR) contributed to theoperationalrequirementsfortheFirescenario.

The technical partners involved in the project include: Centre for Research andTechnology – Hellas (CERTH) who is also the coordinator of the project; MotorolaSolutions Israel (MSIL),who is also the technicalmanager of the project; IBM IsraelHaifaResearchLabs(IBM);FinnishMeteorologicalInstitute(FMI);FraunhoferInstitutefor Optronics, System Technologies and Image Exploitation (IOSB); and UniversitatPompeuFabra(UPF).

The beAWARE system is an end-to-end solution for collecting information frommultipledatasources–suchasendusers,socialnetworks,sensors,anddataproviders–analyzingit,predictingandassessingemergencies,alertingthepublic,andmanagingfirstresponders'activities.

1.3 TheneedforaData-SourceIntegrationFramework

Themultitudeofdataoriginatorsandconsumers inflexibleconfigurationswithintheBeAWAREplatformmandates a robust framework for data connectivity, integration,processing,andexchangeamongtheinvolvedparties,modules,andservices.

Thedata-sourceintegrationframeworkincludesthreelayers,followingtheoutlineoftheModel-BasedInteroperabilityEngineering(MoBIE)framework[1]:

D6.2–V1.0

Page14

• An Interoperability Layer, in which the multiple operational stakeholders cancollaborate,coordinate,constructandtransferknowledge,andexchangeusefulinformationinordertocarryoutthemissionsincurredbynaturalemergencyanddisaster.

• An Interconnectivity Layer, in which the services and systems supporting theoperational stakeholders can jointly process and streamline content,information,anddata, inordertosupportoperationalproceduresandbusinessprocessesenabledbythesesystems.

• An Intercommunication Layer, inwhich themultiple input-output interfaces ofthe constituent systems can send and receive standardized data structures aspartof the informational transactionsandservicesprovidedby theconstituentsystems.

Thepartnersengagedinthedevelopmentoftheindividualsubsystemswilladdressthedevelopmentof appropriatemechanisms in their respectiveworking area to complywiththisdataintegrationframework.

Similarly, the enduserswill further build the operational transactions andprotocolsbased on the common language and guidelines described and specified in thisdocumentandaspartofongoingwork.

Thehighpotentialfordynamicsinthedataexchangeamongthepartiescallsforsolidandyetrobustguidelinesandformalismstoensuretheflexibilityandextendibilityoftheplatformtoaccommodatetheexpecteddynamics.

This report is aligned and coordinated with D7.2 – System Requirements andArchitecture, and adheres to theBeAWAREprinciple of unified “publish—subscribe”messagingbus.Still,thisreportdoesnotfocusonthetechnicalitiesofdatamessagingbutontheoverallprinciplesandconventionsthatensureappropriateandconstructiveutilizationof assets suchas themessagebusandmessageprotocolby the technicalpartnersandtheirprovidedsystems.

1.4 Outline

Thisdocumentisstructuredasfollows:

• Section 2 [Public] discusses the requirements for a data-source integrationframework.

D6.2–V1.0

Page15

• Section 3 [Public] discusses the data source integration infrastructure, whichenablestheinteroperabilityamongthebeAWAREmodulesandsubsystems.

• Section 4 [Confidential] discusses the module-level data access mechanisms,receivable and producible data artifacts applicable to each module, and theinteractionswithothermodules.

• Section5[Public]concludesthisreport.

D6.2–V1.0

Page16

2 Data-SourceIntegrationRequirements[PUBLIC]

2.1 Scope

In this section we review and analyze the requirements for data integration andinteractionamongthebeAWAREmodules.Inaddition,wederivedataintegrationandinteractionneedsandfunctionalitiesatthebeAWARE-levelandatthemodule-level.

2.2 UserRequirementsforData-SourceIntegration

Stakeholders of the operational scenarios that beAWARE is required to respond tohavedefinedvariousoperationalandfunctionalrequirementsforthesystem,inorderto support the roles, responsibilities, and activities of human agents during theoccurrenceofthescenario.ThecompletelistofinitialUserRequirementsisdefinedinDeliverableD2.1[2],whichwaspublishedinM6.

Severaluserrequirementsconcerntheexchangeofdataand informationamongtheauthorities,decisionmakers,controlcenteroperators,crisisanalysts,firstresponders,thepublic,andautomatedanalyticservices.

For the purpose of this report, the initial user requirements provided by beAWAREpartners are considered as a general reference for an overall understanding of userneeds, expectations, intentions, and constraints with respect to the data sourceintegration framework. The possible impact of each user requirement on dataintegration and information-based interaction has been analyzed throughout thecourseoftheprojectandthearchitectingoftheframework.Theresultsofthemultipleteleconferences and emails exchange discussions are summarized in the followingsections.Theimpactisspecifiedintherightcolumnineachoftheuserrequirementstablesbelow.

User requirements whose impact resembles that of previously analyzed userrequirements refer to the original user requirements from which the impact wasderived(forinstance,UR#202pointsthereadertoUR#128forimpact).Thegoalwastominimize multiple definitions of the same impact, and to consolidate use cases asderivedfromvarioususerrequirements.Forexample,providingasuitablemechanismfortransferringpublicalertsfromthecontrolcentertothecitizensisthesameforalltheuserrequirementsthatrefertogeneratingpublicalerts,regardlessofthetypeofhazardtheyintendtowarnabout,thespecificityoftheapplicablelocationorarea,andthecontentofthealert.

D6.2–V1.0

Page17

Weemphasizethattheimpactsderivedfromtheuserrequirementsarestillorientedasmuchaspossibletotheproblemdomain,andaremostlyinteroperability-driven,sothat they reflect operational needs for connectivity and data exchange asmeans tosupport interactions among operational users, e.g. to allow information sharingbetweentheauthorityandthepublic,orthefirstrespondersandthecontrolcenter.Thus, these impacts are still not allocated to specific modules or data accessmechanisms,andservetogetherasthebasisforthedefinitionofthisarchitecture.

2.2.1 UserRequirementsfortheFloodScenario

Inthissectionwesummarizetheanalysisofdatasourceintegrationandconnectivityimpacts for the user requirements defined for the Flood scenario (SCN#1). Asexplained, the impacts are specified on the rightmost column for each userrequirement,andrequirementswithanimpactsimilartotheonealreadydefinedforpreviouslyanalyzed requirements refer to theuser requirementwhichhas the sameimpact.Sincethisisthefirstscenarios’userrequirementsthatwereanalyzed,mostofthe user requirements have a distinct impact in the connectivity and data exchangeaspects.Anegligentportionoftherequirementsrefertomodule-intrinsiccapabilitiesthat do not impact the overall connectivity or require special data exchangemechanisms.Wehavekepttheserequirementsforreference.Afewrequirementsaresufficientlysimilartobecoveredbyasimilarconnectivityorintegrationfeature,suchasthepublicalertormetricvisualization.

Table2-1.InitialUserRequirements–FloodScenario[2]

UR# Requirementname

Requirementdescription DataSourcesandDataIntegrationImpact

UR_101 Typeofvisualization

Displayinformationtoauthoritiesinaweb-gisplatform(citizenandfirstrespondersreportsbycalls,apps,

socialmedia)

Transferpositiondataforfirstresponders,andsocialmediareports

UR_102 MapoftheAMICOFloodEWSresults

Displayreliableandtrustfulfloodforecasts,potentialdangerous

situationsandtheforecastedlevelofrisktotheauthorities,basedontheresultsoftheEarlyWarningSystem

AMICO(improvedwiththeassimilationofSatellitedata(snow

cover,soilmoisture,etc.)andMeteorologicalforecastsdatawithafinerspatialresolutionprovidedby

FMI)

Transferfloodforecastmetricsandfloodriskassessmentmetricsto

authorities

D6.2–V1.0

Page18

UR# Requirementname


UR_103 Floodwarnings Provideauthorities/citizenswithautomaticwarningsonriverlevelsovertoppingsomepredefinedalertthresholds,basedonforecastresults

Transferpublicalertsfromauthoritiesto

citizens.

UR_104 Send/receiveemergencyreports

Allowcitizenstosendtext,images,audioandvideomessagesfromtheirmobilephones(forthedifferentoperativesystems)andfromtheir

socialmediaaccounttotheauthorityduringbadweatherconditionswhen

theGPSsignalislow.

Transfermultimedia-enrichedincidentreportsfromcitizens’mobiledevicestoauthorities

UR_105 Sendtaskreports AllowFirstResponderstosendreportsabouttheirassignmentsfromtheirmobilephonetolocalauthorities

Transfermultimedia-enrichedincidentreportsfromfirstresponders’mobiledevicesto

authoritiesUR_106 Visualizevideo

camerasDisplaystreamedvideofromvideocamerastotheauthorities/citizens

Transfervideostreamsfromcamerasto

authoritiesUR_107 Localizevideo,

audioandimagesProvideauthoritieswiththeabilitytolocalizevideos,audioandimagessentbycitizensfromtheirmobilephones

Transfercitizenreportpositiondatato

authorities

UR_108 Localizetaskstatus

Provideauthoritieswiththeabilitytolocalizefirstrespondersreports

regardingthestatusoftheirassignedtasks

Transferteamstatusandpositionreportto

authorities

UR_109 Localizetweets ProvideauthoritieswiththeabilitytolocalizeTwittermessagesconcerning

afloodevent

SeeUR_101

UR_110 Localizecalls ProvideauthoritieswiththeabilitytolocalizePhoneCallstoanemergencynumberconcerningafloodevent

SeeUR_105

UR_111 Detectfloodedelementsfrom

video

Provideauthoritieswiththeabilitytodetectandcountfloodedelements(e.g.carsandpeopleinsidethe

river)fromvideoandimagessentfrommobilephonesandsocialmedia

Transfervideo/imageanalysisreportsandmetricstoauthorities

UR_112 Detectelementsatriskfromreports

Provideauthoritieswiththeabilitytodetectthenumberofelementsatriskandthedegreeofemergencyfromtextsentbythemobileapporsocial

media

Transfersocialmediaanalysisreportsandmetricstoauthorities

D6.2–V1.0

Page19

UR# Requirementname


UR_113 Detectelementatriskfromcalls

Provideauthoritieswiththeabilitytodetectthenumberofelementatriskandthedegreeofemergencyfrom

emergencycalls

Transferaudiocallanalysisreportsandmetricstoauthorities

UR_114 Detectwaterdepthandvelocity

Provideauthoritieswiththeabilitytodetectwaterdepthandwatervelocityfromvideoandimagessentbythe

mobileappandsocialmedia

SeeUR_111

UR_115 Realtimefloodmapping

Displayfloodedareasinrealtimetoauthorities/citizens

Transferareapositiondata(polygons)to

authoritiesUR_116 Warningpeople

approachingfloodareas

Provideauthoritieswiththeabilitytowarnpeopleindangerwithwarning

messages,whentheyareapproachingafloodedarea

SeeUR_103

UR_117 Manageassignmentsincaseofnewemergencies

Provideauthoritieswiththeabilitytomanagefirstresponderassignments

Transfertasksfromauthoritiestofirstresponderteams

UR_118 Riverovertopping Provideauthorities/citizenswiththeabilitytoknowiftheriverlevelisovertoppingpredefinedalert

thresholds

Transfersensoranalysisreportsandmetricsto

authorities

UR_119 Manageassignmentsbasedonriver

levelovertopping

Provideauthoritiestheabilitytoassigntasktofirstresponderteams

relatedtotheovertoppingofpredefinedriverlevelthresholds

SeeUR_117

UR_120 Mapofrescueteamsandtaskevaluation

Displaytoauthoritiesthelocationintimeoffirstresponderteamsinallthemunicipalityandprovidetheabilitytoevaluateinrealtimetheexecutionof

theassignedtaskswithaglobalvisualizationoftheactivities

performed

SeeUR_108

UR_121 Detectrainfallvolumeandduration

Provideauthoritieswiththeabilitytodetectrainfallvolumeandduration

fromvideos(fixedandmobilecameras,socialmediaandthemobile

app)

SeeUR_111

D6.2–V1.0

Page20

UR# Requirementname


UR_122 Rainfallwarnings Provideauthorities/citizenswiththeabilitytoknowinrealtimeiftherainfallintensityisovertoppingpredefinedalertthresholds

SeeUR_118

UR_123 Detectembankmentexceeding

Provideauthoritieswiththeabilitytodetectfromvideo,automatically(fixedandmobilecameras,socialmediaandmobileapp),ifariverembankmentisovertoppingand/o

breaking

SeeUR_111

UR_124 Embankmentwarnings

Provideauthorities/citizenswiththeabilitytoknowinrealtimeifariverembankmentisovertoppingand/orbreaking;thecomprehensiveand

reliablereal-timeinformationaboutthesituation,especiallythebreachenlargementanddischarge,the

spatialandtemporaldevelopmentoftheinundationandthedamages

SeeUR_118

UR_125 Trafficwarnings Provideauthoritieswiththeabilitytosendwarningstocitizensinordertoavoidinterferencesinsidetheareainvolvedbycivilprotectionactivities

SeeUR_103

UR_126 MapofSatellitedataandweather

forecasts

Displayupdatedsatelliteimagesandweatherforecasts.

Transfersatelliteimagesandweatherforecaststo

authoritiesUR_127 Filters Provideadvancedfiltersinthedata

managementplatform(visualizeandlistinformationselectedby

filters/query)

N/A

UR_128 Evaluationofthelevelofrisk

Provideauthoritieswiththeabilitytoevaluatetheforecastedlevelofrisks(basedonalltheavailabledataset)

Transferforecastandanalysismetricsto

authoritiesUR_129 Automatic

translationfromaforeignerapplicant

Makethecommunicationbetweenpeoplewithdifferentlanguageseasier

Transfertextasneededtoautomatictranslation

services

UR_130 TrafficStatus Displaytotheauthoritiesthecurrenttrafficsituationsothattheycandecidewheretodirectthefirst

respondersorinformthemofwhichroutestoavoid

Transfertrafficdatatoauthorities

D6.2–V1.0

Page21

UR# Requirementname


UR_131 Trafficwarnings Provideauthoritieswiththeabilitytosendwarningstocitizensinordertoavoidacertainareathatisjammed

withtraffic

SeeUR_103

2.2.2 UserRequirementsfortheFireScenario

Inthissectionwesummarizetheanalysisofdatasourceintegrationandconnectivityimpacts for the user requirements defined for the Fire scenario (SCN#2). Again, theimpacts or pointers to previously analyzed requirements with a similar impact(including user requirements for SCN#1) are specified on the rightmost column foreachuserrequirement.Sincethisisthesecondscenariotobeanalyzed,andsincethisscenario isabitmorerestricted inscope,mostof theuserrequirementsarealreadycoveredbypreviouslydefinedconnectivityorintegrationfeatures.

Table2-2.InitialUserRequirements–FireScenario[2]

UR# Requirementname

Requirementdescription DataSourcesandDataIntegration

ImpactUR_201 Detectionof

peopleandgoodsindanger

Displayinformationauthorities/firstresponderstodetectpeople,carsand

buildingsindanger.

SeeUR_128

UR_202 Detectionofcriticalaspects

Provideauthorities/firstresponderswithinformationinordertodetectthefollowing

kindofsituation,process,materialorconditionthatcancauseawildfireor

intensifyitsdamagingimpacts:drought,airtemp.andweatheraspects,fuelaccumulationspots,crowds,etc.

SeeUR_128

UR_203 Studyofthesmoke

behaviour

Provideauthorities/firstresponderswithinformationonthesmokebehavior

(vertical/inclined,column,smokecolor…).

SeeUR_111

UR_204 Identificationofthefuelbeing

burned

Provideinformationtoauthorities/firstresponderstoknowthetypeoffuelbeingburnedbythecolourandtheshapeofthe

smoke

SeeUR_111

UR_205 Analysisofadvancingfire

Provideauthorities/firstresponderswithananalysisoftheadvancingfire(flameprogression,heightandlength).

SeeUR_115

D6.2–V1.0

Page22

UR# Requirementname


ImpactUR_206 Specific

weatherdataProvideauthorities/firstrespondersandcitizenswithspecificweatherdataofthe

Devesaplace,asithasaspecificmicroclimatethatmightbedifferentfrom

otherplaces.

SeeUR_126

UR_207 Aerialimages Displayauthorities/firstresponderstovisualizeaerialimagesofthesmokeandthetrajectoryflames.Itwillprovideinformationabouttheextensionandthedamages(kindofdamages,andsoon),thetrackingofthefire,vehiclesandpeoplearoundthespot,in

ordertofindoutpossiblesuspectsorvictims.Furthermore,iftheseaerialimagesprovidethermalinformationitcanbeusedforlookingoverthefireperimeteronceithasbeenextinguished,inordertolocate

sleeperfireandavoidpossiblereproduction.

SeeUR_111,UR_104

UR_208 Accesstoroadtrafficcameras

Allowauthorities/firstresponderstohaveaccesstothecameraslocatedatCV-500(LaDevesamainroad)andCV-5010.Althoughtherearealreadyinstalledcameras,theyare

managedandvisualizedbyautonomicresources.

SeeUR_106

UR_209 Electronictrafficpanels

Displayauthorities/firstresponderstodisplayinelectronictrafficpanelsusefulinformationandevacuationinstructionsin

case.

SeeUR_103

UR_210 Mobileapplication

Enablecitizenstocommunicateafirealert,detectedneglectsorotherrisksituations

andevensendvisualdatathroughamobileapplication.

SeeUR_104

UR_211 Locationofvehiclesandpersonnelinvolved

Displayauthorities/firstresponderstovisualizeGPSlocationand/orrealtimefootageofvehiclesandpersonnelontheincidentsite.Transmittedtoanonlinemapwherethecoordinationcentrescanfollowboththedevelopmentoftheincidentandthelocationandamountofresources.Theonlinemapwillalsoprovidethepossibilityofinteractingwiththepoliceandother

agenciesinvolved

SeeUR_108

UR_212 Trafficwarnings Sendingwarningstocitizensinordertoavoidinterferencesinsidethearea.

SeeUR_103

D6.2–V1.0

Page23

UR# Requirementname


ImpactUR_213 Recommendati

onsSendingrecommendationstocitizens. SeeUR_103

UR_214 Warnings Sendingwarningsofpre-emergencyalertstocitizensbyauthorities

SeeUR_103

UR_215 Evacuationorders

Orderingevacuationsofcitizensatrisk. SeeUR_103

UR_216 Internalsharingofinformation

Sharingdata(images,videos,geolocation,reports)regardingtheforestfireamong

authorities&firstresponders

Transfermultimedia-enrichedincidentupdatesfrom

authoritiestofirstresponders

UR_217 Twitteranalysisandwarning

Warningauthorities/firstrespondersaboutTwittermessagesconcerningtheforestfire

event.

SeeUR_112

UR_218 Automaticdetectionsystem

Havinganautomaticdetectionsystemoftheforestfire,whichisconnectedto

firefightersandpoliceofficers

SeeUR_128

UR_219 Coordinationand

communicationbetweendifferentresources

Providecommunicationbetweenauthoritiesandfirstresponders,inordertoimprove

theircoordination.

SeeUR_216,UR_108,UR_117

UR_220 Improvementofthesignalfortelephonesandemergency

communication

Provideauthorities/firstresponderswithanaccuratecoverageoftelephonemobilelinesandemergencycommunicationduetothereiscurrentlyalackofsignalinsomespotsof

thearea.

SeeUR_115

UR_221 Geolocalitationoftelephone

calls

Togeolocalizeamobilephonecitizencallbysendingarequestpermissionmessageto

thecitizen,whowouldaccepttobetrackedtemporarily.

SeeUR_104

UR_222 Filteroftheemergencymessages

Transferemergencycallsbywriting(onlyminoremergenciesoronlyinformationcall).Theaimistosaveoperatortimeandavoid

losingemergencycalls

SeeUR_104

UR_223 Automaticselectionofthe

levelofemergency

Thiscanbedonewiththeoperator’ssupervision.Theaimistosavetimeand

avoidlosingemergencycalls

SeeUR_128

D6.2–V1.0

Page24

UR# Requirementname


ImpactUR_224 Automatic

translationfroma

foreignerapplicant

Makethecommunicationbetweenpeoplewithdifferentlanguageseasier

SeeUR_129

UR_225 Quicksearchofeventsandapplicants

Datastorage,inordertoimproveindexationofinformationrelativetoeventsand

applicants

Transfersearchrequestsandresultsbetweenauthoritiesanddatasources

2.2.3 UserRequirementsfortheHeatwaveScenario

Inthissectionwesummarizetheanalysisofdatasourceintegrationandconnectivityimpacts for the user requirements defined for the Heatwave scenario (SCN#3).Similarly, the impactsorpointers topreviously analyzed requirementswitha similarimpact(includinguserrequirementsforSCN#1)arespecifiedontherightmostcolumnforeachuserrequirement.Sincethisisthethirdscenariotobeanalyzed,mostoftheuser requirements are already covered by previously defined connectivity orintegrationfeatures,mostofwhichwereidentifiedforSCN#1.Asmallportionoftherequirementsthatwerenotidentifiedashavingconnectivityordataexchangeimpactshavebeenmarkedas“N/A”.

Table2-3.InitialUserRequirements–HeatwaveScenario[2]

UR# Requirementname


ImpactUR_301 Realtime

weatherforecast

Providetheauthoritieswithrealtimeweatherforecastinrelationtothe

progressionoftheheatwavephenomenon

SeeUR_126

UR_302 Automaticwarning

beAWAREsystemtogenerateandprovidetheauthoritieswithanautomaticwarningwhenanimminentheatwavephenomenon

isforecasted

SeeUR_128

D6.2–V1.0

Page25

UR# Requirementname


ImpactUR_303 Riskassessment

foraforestfireProvidetheauthoritieswitharisk

assessmentregardingtheprobabilityofaforestfiretooccurduringorinthe

upcomingperiodafteraheatwave.Therelevantauthoritieswillhavean

assessmentofafireriskbasedontheweatherforecastduringaheatwaveand

especiallyduringthefollowingdays

SeeUR_128

UR_304 Heatwaveintensity

Providetheauthoritieswithariskassessmentregardingtheintensityofthe

phenomenoninthecity.

SeeUR_128

UR_305 Possiblelocationsforincidents

Displaytotheauthoritiesvisualinformationaboutpossiblelocationsinthecity(or

outsidethecity)whereasituationthatwillrequirerescueteaminterventionismorelikelytodevelop(forexample,basedon

pastexperience,trafficjamand/oraccidentswillbemorelikelytooccuratamainstreetintersection/publicpark/

entrancetohospitalsorbanks…etc.).Insuchcasesadecisionmightbemadetosendrescueteamsinadvancetoshortenresponsetimeif/whenanincidentoccurs

Transferpredictedincidentstoauthorities

UR_306 Numberofpeopleaffected

Providetheauthoritieswithanestimationofthepeoplethatmightbeaffectedfrom

thephenomenonandinwhichareas

SeeUR_118

UR_307 Powerneeds Providetheauthoritieswithanestimationonthepowerneedsduringaheatwave

basedonitsforeseenprogression

SeeUR_118

UR_308 Infrastructureoverload

Providetheauthoritieswithanestimationofdamage/overloadtothecity’s

infrastructure(phonelines,electricity,water,etc)

Transferinfrastructuredatato

authorities

UR_309 FalseAlarms Providetotheauthoritieswithaproceduretoconfirmnecessityofrescueteamssotheyarenotsentneedlesslytooneplaceinsteadofsomewhereelsewheretheyareneededmoreurgently,thereforetheability

tohandlefalsealarms.

N/A

D6.2–V1.0

Page26

UR# Requirementname


ImpactUR_310 City-wide

overviewoftheevent

Providetheauthoritieswithacity-wideoverviewoftheevent–allowdecisionmakingauthoritiesanoverallviewofall

incidentshandledatanypointintime/seewhereallrescueteamsarelocatedinreal-timetoallowthemtomakeinformed

decisionsregardingwhotosendwhere…etc

N/A

UR_311 InformationStorage

Providetheauthoritieswithaccesstoallhistoricalinformationbyprovidingstorage

forallinformationforfuturelessons-learnedpurposes,sothataftertheheatwavesituationisover,decisionmakingauthoritiescanreviewtheinformationgatheredandhandledduringtheevent,andset-upbetterprocedurestohandle

futureeventsmoreefficiently

Transferdatatoacentralarchive

UR_312 Warningcitizens ProvidecitizenswithwarningsthroughthebeAWAREapp,ofanimminentheatwaveandalistofproactivemeasuresandhowto

reduceitseffects

SeeUR_103

UR_313 Firstrespondersstatus

Providetotheauthoritiesthecurrentstatusandlocationofallfirstresponderswhen

theyareperformingtheirtasks

SeeUR_108

UR_314 Assigntaskstofirstresponders

Allowauthoritiestoassignadditionaltaskstothosefirstresponderswhoareavailableoreveninstructthosewhoareableto

assistotherresponders

SeeUR_119

UR_315 TrafficStatus Displaytotheauthoritiestomonitorthecurrenttrafficsituationsothattheycan

decidewheretodirectthefirstrespondersorinformthemwhichroutestoavoid

SeeUR_130

UR_316 Capacityofreliefplaces

Showtheauthoritiesthecurrentstateoftheavailablecapacityofallreliefplaces

providedtothepublic

Transferassetdatatoauthorities

UR_317 Areaswithpoweroutage

Displaytotheauthoritiestheareaswherethereisapoweroutage.

Transferpowerstatustoauthorities

UR_318 Trappedcitizens Allowauthoritiestoknowiftherearepeopletrapped(e.g.inanelevator)and

displaywhere

SeeUR_118

UR_319 Trappedeldersathome

AllowauthoritiestoknowifthereareelderpeopletrappedinhouseswithoutanA/C

anddisplaywhere

SeeUR_118

D6.2–V1.0

Page27

UR# Requirementname


ImpactUR_320 Hospital

availabilityShowtheauthoritiesthecurrentavailability

ofthehospitals.SeeUR_316

UR_321 Affectedarea Providetheauthoritieswithapredictionoftheaffectedarea

SeeUR_115

UR_322 InformationforincidentstatusfromSocialMedia

Provideinformationtotheauthoritiesregardingpotentialrisksincasethereisasituationinthecity(egcaraccident,etc.)

gatheredfromsocialmedia

SeeUR_101

UR_323 InformationforHospitalStatusfromSocialMedia

Provideinformationtotheauthoritiesregardingovercrowdedhospitalsandplacesofferedtothepublicwitha/c,gathered

fromsocialmedia

SeeUR_101

UR_324 Informationforexistingsituation

inthefromSocialMedia

Provideinformationtotheauthoritiesregardingexistingtrafficconditionsalloverthecitygridgatheredfromsocialmedia

SeeUR_130

UR_325 Suggestedplacesforrelief

Provideinformationtocitizensregardingthesuggestedplacesforreliefthroughan

app.

SeeUR_103

UR_326 Typeofvisualization

Displayinformationtotheauthorities/citizensalltheinaweb-gisplatform

N/A

UR_327 Sendemergencyreports

Allowcitizenstosendtext,imagesandvideomessagesfromtheirmobilephone(forthedifferentoperativesystems)andfromtheirsocialmediaaccounttothe

authority.

SeeUR_101

UR_328 Sendtaskreports

AllowFirstResponderstosendreportsabouttheirassignmentsfromtheirmobile

phonetolocalauthority

SeeUR_105

UR_329 Visualizevideocameras

Displaystreamedvideofromvideocamerastotheauthorities/citizens

SeeUR_106

UR_330 Localizevideoandimages

Provideauthoritieswiththeabilitytolocalizevideosandimagessentbycitizens

fromtheirmobilephones

SeeUR_107

UR_331 Localizetaskstatus

Provideauthoritieswiththeabilitytodetectthelocationoffirstresponders

SeeUR_108

UR_332 Localizetweets ProvideauthoritieswiththeabilitytolocalizeTwittermessages

SeeUR_109

UR_333 Localizecalls ProvideauthoritieswiththeabilitytolocalizePhoneCallstoanemergencynumberconcerningcitizenswhoare

trapped

SeeUR_110

D6.2–V1.0

Page28

UR# Requirementname


ImpactUR_334 Manage

assignmentsincaseofnewemergencies

Provideauthoritieswiththeabilitytomanagefirstresponderassignments

SeeUR_117

UR_335 Mapofrescueteamsandtaskevaluation

Displaytoauthoritiesthemovementsoffirstresponderteamsinallthemunicipalityandprovidetheabilitytoevaluateinrealtimetheexecutionoftheassignedtaskswithaglobalvisualizationoftheactivities

performed

SeeUR_120

UR_336 Trafficwarnings Provideauthoritieswiththeabilitytosendwarningstocitizensinordertoavoidacertainareathatisjammedwithtraffic

SeeUR_103

UR_337 Locationofvehiclesandpersonnelinvolved

Allowauthorities/firstresponderstovisualizeGPSlocationand/orrealtimefootageofvehiclesandpersonnelontheincidentsite.Transmittedtoanonlinemapwherethecoordinationcentrescanfollowboththedevelopmentoftheincident,andthelocationandamountofresources.Theonlinemapwillalsoprovidethepossibilityofinteractingwiththepoliceandother

agenciesinvolved

SeeUR_105

UR_338 Warnings Allowauthoritiestosendwarningsofpre-emergencyalertstocitizens.

SeeUR_103

UR_339 Evacuationorders

Allowauthoritiestoorderevacuationsofcitizensatrisk.

SeeUR_103

UR_340 Internalsharingofinformation

Allowauthoritiesandfirstresponderstosharedata(images,videos,geolocation,

reports)

SeeUR_117

UR_341 Twitteranalysisandwarning

Allowauthorities/firstresponderstobewarnedbyTwittermessagesconcerningtrafficjam,availabilityofplacesofrelief,potentialhazardsorpeopleindanger

SeeUR_101

UR_342 Coordinationand

communicationbetweendifferentresources

Providecommunicationbetweenauthoritiesandfirstresponders,inorderto

improvetheircoordination.

SeeUR_105

D6.2–V1.0

Page29

2.3 OperationalRoles

Basedonthedescriptionofthescenariosanduserrequirements,asetofoperationaluser roles has been defined, in order to cover all the participants of the actualscenarios.TheoperationalroleshavebeendefinedandlistedinTable2-4.

Table2-4.OperationalRoles

ROLE LOCATION INTERFACE RESPONSIBILITIES SCENARIOSEmergencyManager(H)

Command&ControlCenter

PSAP Overseetheemergencysituation Flood,Fire,Heatwave

IncidentManager(H)


PSAP Manageandprioritizeincidents Flood,Fire,Heatwave

OperationsManager(H)


PSAP Assigntaskstoteamsandmonitortheirprogress

Flood,Fire,Heatwave

Analyst(H) Command&ControlCenter

AMICO OverseeFloodForecasting Flood

FirstResponder

(H)

Field FRAPP Reportonnewandexistingincidents,executeassignedtasks,

reporttaskprogress,receiveoperationalalerts

Flood,Fire,Heatwave

Citizen(H) Field SCAPP Reportonnewandexistingincidents,Receivepublicalerts

Flood,Fire,Heatwave

Sensor(M) Field SENSAN Providesensormeasurements Flood

SocialNetwork(M)

Cloud SMA Generatetweets Flood,Fire,Heatwave

2.4 Modules

Basedontheanalyzeduserrequirements,wehaveformedalistofmodulesthatneedtocollaboratethroughthebeAWAREsystem.Althoughsomeof thesemoduleswereconceptually defined at the inception of the project, some emerged as critical andnecessary during our joint work and analysis. For instance, a Media Hub, whichmanagesrequestsformediaanalysisformultiplemediatypes,hasbeenidentifiedasacriticalnodeintheoverallarchitectureinordertocreatearobustcapabilitytohandleincidentreportswithmultiplemediaattachmentsofvarioustypes.

Table2-5.Modules

FULLNAME OWNER ALIAS TYPEAMICOFloodPredictionServices AAWA AMICO Subsystem

AutomaticSpeechRecognition CERH ASR BackendModule

CentralDataRepository IBM CDR StorageService

CentralMessageBus IBM MSB MessagingService

D6.2–V1.0

Page30

FULLNAME OWNER ALIAS TYPECrisisClassification CERTH CRCL BackendModule

ImageAnalytics CERTH IMGAN BackendModule

KnowledgeBaseServices CERTH KBS BackendModule

KnowledgeBaseRepository IOSB KB StorageService

MediaHub CERTH MEDHUB IntegrationService

MobileApp.forCitizens/FirstResponders IOSB APP UserApplication

MultilingualReportGenerator UPF MRG BackendModule

MultilingualTextAnalysisService UPF MTA BackendModule

PublicSafetyAnsweringPoint MSIL PSAP UserApplication

SensorAnalytics IOSB SENSAN StorageandAnalysisService

SocialMediaAnalysisServices CERTH SMA BackendModule

VideoAnalytics CERTH VIDAN BackendModule

WeatherForecastServices FMI WFS DataService

2.5 OperationalEntities

Basedontheanalyzeduserrequirements,wehaveformedalistofoperationalentitiesidentifiedforexchangeamongthevariousdataoriginatorsandrecipients(modules)inbeAWARE.

Table2-6.OperationalEntities

OperationalEntity Description Originators Recipients

Weather WeatherforecastprovidedbyFMIbasedontheHIRLAMmeteorologicaldata

analysissystem,ina7-kmgridresolution

WFS AMICO

FloodForecast AMICOimportsmeteorologicalforecastsprovidedbyFMI,tobeusedasinputfortheFloodForecastingModel.Thenthemodelrunshydrological-hydraulic

simulationtoobtainasresultsthetimeseriesofforecastedwaterlevelineachriversection.EverynewresultofAMICO

isimportedbySENSANandmadeavailablefortheotherbeAWARE

modules

AMICO SENSAN,CRCL

D6.2–V1.0

Page31


Pre-EmergencyFloodEarlyWarning

Duringthepre-emergencyphase,CRCLchecksfromAMICO’sresultimportedinSENSANifthepredictedwaterlevel

exceedssomefixedthresholdinoneormoreriversectionandestimatestheseverityleveloftheforecastedcrisis.If

thisexceedingoccurs,CRCLgeneratesandproceedsearly

warningmessagestoPSAPinordertoalertitforextremeweatherconditionsanddangeroussituationsbeforeor

duringacrisis

CRCL PSAP

FloodCrisisLevel(UC102)

Duringtheemergencyphase,CRCLestimatesseveritylevelofeveryincidentreportfromoptionalfieldsintheappfilledbyendusers(categoryofincidentreport,estimateofthewaterlevel,elementsinthefloodedarea),fromsocialmediapostwhicharedefinedrelevantbySMA,andfromreal

measuredriverlevels

CRCL PSAP

HeatwaveIndex IndicatorofpotentialforheatwavefortheThessalonikiregion,basedontemperaturesrelativetoseasonal

temperatures,windspeed,humidity,andotherfactors

FMI SENSAN,CRCL

FireIndex Indicatorofpotentialforeruptionofforestfire,targetedfortheDevesaareainValenciaandfortheThessaloniki

forestarea

FMI SENSAN,CRCL

Alert Generalalertsenttothepublicortospecificgroupsoffirstresponders(policeofficers,firefighters,etc.)

PSAP APP

Tweet Twitterpoststhatrefertofloods,firesandheatwaves

Twitter SMA

Textualtweetcontents

MessagesfromTwitterpoststhatrefertofloods,firesandheatwaves

SMA MTA

Visualtweetcontents

ImagesfromTwitterpoststhatrefertofloods,firesandheatwaves

SMA MEDHUB

Incident-RelatedTweetCluster

Spatiotemporalclustersoftweetsthatrefertoanincident

SMA KBS

FieldReport Reportsentbyeithercitizenorfirstresponderusingthemobileapp.

APP KBS,MTA

D6.2–V1.0

Page32


TextualMessage Textualpartofafieldreportsentbyeithercitizenorfirstresponderusingthe

mobileapp.

APP MTA

SensorMeasurement

Measurementmadebyasensor SENSOR SENSAN

SensorEvent EventdetectedbySensorAnalytics. SENSAN KBIncident TheKBSmodulegeneratesincident

reportsthatcontainaggregatedknowledgefromothermodules(suchastheAPP,IMGAN,VIDAN,etc.)andsends

ittoPSAP.

KBS PSAP

Team Groupoffirstresponderswhomoveandacttogether.

APP PSAP

Task Assignmentofarequesttohandleteamanincidentorpre-emergencysituation(e.g.evacuatetrappedcitizens,placesandbags,extinguishfire)toateamof

firstresponders.

PSAP APP

MultimediaObject

Image,video,oraudiofilecapturedbymobileapporonsocialmedia.

APP MEDHUB,VIDAN,

IMGAN,ASRAnalyzed

Image/VideoImagesandVideosoriginallyextractedfromtweetsorposteddirectlythroughtheAPP,thatafterproperanalysisshowdetectedtargetslikepeopleorvehicles

indanger

IMGAN/VIDAN KB

TranscribedSpeech

Transcriptionsofaudiorecordingsprovidedbymobileapp.AudiofilesareanalyzedbytheASRmoduleandtexttranscriptionsaresavedinMongoDBalongwithaudiotimestampand

languageinformation.

ASR MTA

AnalyzedText Eventsalongsideinvolvedentities,locationandtemporalaspectsthatareextractedfrom(thetextualpartsof)

tweetsandfieldreports,aswellasfromtranscribedspokencommunications

MTA KB;SMA(onlyfor

informationextracted

fromtweets)KBgraph Ontologicalrepresentationsofthe

inferredsituation(incidents,vulnerableobjects,severity,etc.)

KBS MRG

D6.2–V1.0

Page33


VerbalisedReport

Textualreportthatdeliversinthetargetedlanguage(English,Greek,ItalianorSpanish),theKBinferences

regardingtheunfoldingsituation(whatishappening,whatentitiesare

impacted,etc.)

MRG KB

2.6 DataAccessModalities

Based on the analysis of User Requirements, we have concluded that severalmodalitiesofdataaccessarerequired,assummarizedinTable2-7.

Eachmodalityprovidesdataaccessinadifferentmanner.

Table2-7.DataAccessModalities

DataAccessModality Definition SuitableUses Examples Comments

Database Raw,unchangingdata,availableinrelational(SQL)orelastic(NoSQL)

databases

Accesstodatawhichismostlystatic,publishedperiodically,

updatedinlowfrequency,andqueryable.

WeatherForecast,GlobalParameterSets,

CommonKnowledge

Datamaybeupdatedeveryseveralhoursandgrabbedbyits

consumersperiodicallyinasuitablesampling

frequencyFileServer Raw,unchanging

data,availableinstructuredformats(XML,JSON,CSV,

etc.)

Accesstodatawhichismostlystatic,publishedperiodically,

updatedinlowfrequency,and

batch-downloadable.

ForFutureUse Datamaybeupdatedeveryseveralhoursanddownloadedby

itsconsumersperiodicallyinasuitablesampling

frequency

Content-basedRouting

Mechanismthatreceivesmessages

fromdataproviders,

determinestherelevantrecipients

accordingtomessagecontent(mostlyheaderfields)andrelaysthemessagetoits

intendedrecipients

Deterministicrouting,complex

filter-basedrouting,

workflow-basedrouting,

deterministicmessagestructures

(identifiableas"topics")

Anyinformationentitywithasinglewell-

knownrecipient,ormultiple

potentialwell-knownrecipients

basedonmessagecontent

Routingrulesaredeterminedbya

singleauthorityanddictatedtotheroutingsystem.

Routingcanbefully-controlledbydata

originatorbydeterminingtherecipientsin

specifically-definedmessageattributes.

D6.2–V1.0

Page34


Publish&Subscribe

Mechanismthatreceivesmessagesfromrecipient-agnosticdata

providersundervarioustopics,and

redirectsthemessagestoanyrecipientthat

subscribedtoitstopic.

Dynamicrouting,dynamicdata

provider/consumercomposition,dynamictopiccomposition,recipient-

agnosticdataexchangepolicy

Anytopicwithadynamiclistof

multiplepotentialrecipient-agnostic

producersandrecipientsmultiplepotentialrecipients

Routingisdeterminedbytopicsubscriptionsonly.Producerscannot

controltherecipientlistforthedatathey

provide.

ObjectStorage

Storageforlargeandstaticfiles

Filesharing,fileexchange

Video,image,audio,anddata

files

AccesstothefilesisthroughURLs,file

uploadingisaccessedthroughaRESTful

APIRESTfulAPI Dataisprovidedas

aresponsetoanHTTPrequest,orpushedthroughan

HTTPrequestfollowedbyan

acknowledgementresponse

On-linequeryingofdynamic

information,dataproviderexposinga

genericinterfaceforexternalconsumerstoqueryit,dataconsumerexposinga

genericinterfaceforexternal

providerstopushdatatoit

Sensordatarequest(pull),FloodForecast(pull),real-timeweatherdatarequest(pull),Googlesearch

results(pull),on-linelogging

service(push),messaging

service(push)

Pull:APIservesfordataacquisitionbyrequesterfromprovideruponrequest(withorwithoutquery

arguments)throughdataresponse.

Push:APIservesfordatapostingby

providertoreceiveruponrequest(whichincludesthedata),followedbyan

acknowledgementorrejectionresponse

fromreceiverDirectdatacommunicati

on

Dataisprovidedtorecipientthrough

anad-hocinterfacewithapredefined

structureknowntoproviderandreceiver

Dedicatedorlegacyinterface

over,e.g.Ethernet

(TCP/UDP),serial(RS232/RS422etc.),PCI/PCIx,

USB,etc.

Videostreams,serialrawdatastreamsfrom

devices,hardware,sensors

D6.2–V1.0

Page35


Datadisplay Visualizationofdataand

informationtoauserona

computermonitor,mobiledevice,ordedicateddisplay

Interactionwithusers

Incidentreporting

(mobile),Incidentlists(desktop),on-mapeventdisplay(both),GPSsignalindication(dedicated)

D6.2–V1.0

Page36

3 DataSourceIntegrationInfrastructure[PUBLIC]

3.1 Scope

The purpose of this chapter is to describe the central mechanisms for data sourceintegrationinbeAWARE,basedonthepreviously-defineddataaccessmodalities.

Foreachinfrastructuremechanism,weprovideanoverview,high-leveldescriptionofitsarchitecture,andthetypesofsupporteddataand informationentities,aswellastherelevantusecases,modes,anddataaccessconventionsapplicabletothiskindoftechnologyorapplication.

SincebeAWAREisacomplexsystemwithmanymodulesdevelopedseparatelyacrossEurope, it became evident quite early in the architectural process that a centralmessagingservicemustbeset-up,ratherthanallowameshofpeer-to-peerinterfacesamongthemultiplebeAWAREmodules.Moreover,all thecommunicationmustpassthrough a centralmanager,which provides governance, routing, delivery assurance,andmonitoring.Forthisreason,themainelementoftheinfrastructureisthemessagebus(MSB).

In addition to theMSB,wehave identified the need for a centralmedia repository,thatwillbeusedforstoring images,videos,audio files,anddatasets,sothatratherthanroutethiskindofinformationasrawattachmentstomessages,referencestothemediafileswillbeprovidedalongwiththemessages,suchthatmediaaccesswillonlybeonthebasisofnecessity,andnetworktrafficwouldbeminimized.

Anothermeansofdatastorage,mostlyforplaindataobjects,hasbeenputinplaceinthe formofaNoSQLdatabase,whichallowseasyaccess tonon-relationaldata.Thissolutionismostlyusedforstoringsocialmediapostsandtheiranalysisresults,whosestructurecouldbeflexibleandlesssuitableforarelationalSQLdatabase.

Asalloftheseservicesrelyonexisting,off-the-shelftechnologies,thissectionispublic.Westronglyadvocatetheutilizationofsuchacombinationofsolutionsaspartofanoverallarchitectureforacomplexsystemofsystems ingeneralandespecially foranemergencymanagementsolutionlikebeAWARE.

D6.2–V1.0

Page37

3.2 CentralMessageBus–MSB

3.2.1 Overview

TheCentralMessageBus isusedasthemainvehicleforexchanging informationandnotificationsamongdifferentcomponentsintheplatform.Followingamicro-servicesapproacheachcomponentisautonomoustoalargeextentandallinteractionsamongcomponentsareachievedvia thecentralmessagebus.Hence,eachcomponent thathasinformationthatcouldpotentiallybeofinteresttoothercomponentswillsendamessage through themessagebus, andeach component that needs tobe awareofspecificnotificationsfromothercomponentsshallsubscribetoreceivemessagessenton a specific topic. Different components interacting via the message bus need toagree on a topic name and message format so that information is received andunderstood.

The communication bus is realized by using an instance of aMessageHub service1,deployedinIBM’sBlueMixcloud.Theback-endisbasedonaclusterofApacheKafkaservers, and the interactionwith the service is realizedusing standardKafka clients,embeddedwithinthevariouscomponents.

3.2.2 SupportedEntities

Mostofthesystemcomponentsmakeuseofthecommunicationbusasthisistheonlywayforcommunicatingbetweendifferentcomponents.

InarepresentativeflowanewpieceofinformationflowsintothesystemviatheAPP.The APP's back-end uses the message bus as a producer to alert all interestedcomponentsastotheexistenceofanewinputdataitem.Theinterestedcomponents,mainly the components in charge of analyzing the incoming data, receive thecorrespondingmessageviathemessagebuscallbackmechanism,bysubscribingtoitstopicasconsumers,andproceedtoanalyzethenewdatabasedonthecontentofthereceivedmessage.Oncetheanalysisisfinished,onceagainthemessagebusisusedbythe analyzing module, now as a producer of a new topic, to inform interestedcomponentsthatnewanalyzedinformationisavailable.OneofthesubscriberstothiskindofinformationistheKBS.TheKBSturnsasaproducertotheMRGasaconsumer,

1 https://www-03.ibm.com/software/products/en/ibm-message-hub

D6.2–V1.0

Page38

viathebus,foranewreporttobeconstructed.Inturn,MRG,asaproducer,sendsthereportthroughthemessagebustotheKBS,whichreceivesitasaconsumer.Then,theKBSpublishesanewmessagewhichisconsumedbythePSAP,viathemessagebus.

3.2.3 ConnectivityDiagram

Belowisagenericconnectivitydiagram,showingtheMSBasthecentralmodule.SincetheMSB is agnostic to the topics it processes and to the clients that produce andconsume messages based on these topics, the diagram shows them as generic –Producers1,2,and3;Consumers1,2,and3;andTopics1,2and3.

Figure3-1.ConnectivityDiagram-MSB

3.2.4 FeaturesandCapabilities

The abstraction provided is that of a "publish—subscribe" (Pub/Sub) middleware.Pub/Subisamessagingpatterninwhichsendersofmessages,calledpublishers,sendmessageswithoutspecifyingthereceivers,butinsteadcategorizepublishedmessagesinto topics. In turn, potential receivers, called subscribers, subscribe to the topics inwhich they have interest, and subsequently only receive messages they havesubscribedto,withoutknowledgeofwhichpublishersoriginatedthem.Thus,differentcomponentsdonotneed tobe awareof eachother, or evenbe active at the sametime. Inaddition,thePub/Submechanismiscontent-agnosticandendpoint-agnostic,i.e.itdoesnotneedtoknowinadvancewhoarethesendersandreceivers,andwhatmessages they exchange. The central message bus infrastructure ensures that allmessagespublishedonaspecifictopicreachallsubscribersofthatspecifictopic.

D6.2–V1.0

Page39

3.2.5 DeploymentandOperation

TherealizationofthecentralmessagebusisviaaclouddeploymentofMessageHub,which isamanagedApacheKafkaclusterdeployedon the IBMcloud (BlueMix).Theserviceitselfisprovidedbyaclusterof5Kafkabrokers(kafka01,..,kafka05).

The specific connection details are provided to the components running inside theplatform'sKubernetescluster(amanagedsetofcontainerizedapplicationsthatmakeup a logical unit). The connection details include a namespace wide secretcorresponding to the deployment namespace, namely prod. The secret contains anAPI-keyandthelistofKafkabrokers.

3.2.6 DataAccessConventions

Access to the message bus capability is provided by using standard client librarieswhich are available in multiple languages. In addition, there is a REST interfaceavailableaswellforcommunicationwiththemessagebus.

3.2.7 Authentication

All Kafka agents (producers and consumers)must authenticate themselveswith thecentral service in order to receive or producemessages. The authentication is doneusing a certificate assigned to the agent, in order to ensure that only allowed datasourcesinteractwiththebeAWAREmessagebus.

3.2.8 CommonTopicHeader

All topics of messages going through the beAWAREMessage Bus share a commonheader, which includes conventional topic, message, and origin identifiers. TheattributesintheheaderarelistedinTable3-1.

ThetopicheaderstructuregenerallyconformstotheCommonAlertProtocol(CAP)[3].ForeveryattributewealsospecifytheequivalentCAPattribute.CAPisastandardizedformat for distributing alerts, warnings, and notifications, especially in emergency-related systems. Since CAP is in principle an XML standard, we created a JSONtemplateforourheaderthatcorrespondstotheXMLSchemaDefinition(XSD)ofCAPalerts. More information about CAP can be be found here: http://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2-os.html.

D6.2–V1.0

Page40

Table3-1.CommonHeaderforbeAWAREMessageBusTopics

AttributeName AttributeDescription CAPEquivalent MandatorytopicName nameoftopicofmessage

(hierarchicallyallowed)n/a Y

topicMajorVersion topicmajorversion(Vin“V.u”) n/a YtopicMinorVersion topicminorversion(uin“V.u”) n/a Y

sender systemthatoriginatedthemessage sender YmsgIdentifier uniquemessageidentifier identifier Y

sentUTC UTCtimeinwhichmessagewassent

sent Y

status Thecodedenotingtheappropriatehandlingofthealertmessage

(REQUIRED)

status Y

actionType typeofactiontoapplytothealert msgType YspecificSender specificenpoint/user/device/ip

addresswhooriginatedthemessage

source N

scope intendeddistributionofthealertmessage(REQUIRED)

scope Y

district specificdistrictinwhichthemessageisrelevant

restriction Y

recipients specificrecipients'name,id,ipaddress,etc.withinthesubscribertowhomthemessageisintended

addresses N

code identifieroftherealitytowhichthemessageapplies

code Y

note textualnotificationforvariouspurposes,e.g.errormessage(when

actionType==“Error”)

note N

references optionalidentifierofpreviousmessagestowhichthecurrent

messagerefers,orURIinwhichthecontentofthecurrentmessageisstored(multiplevaluesallowed)

references N

3.2.9 Topics

The beAWARE message bus topics are listed in Table 3-2 below. For each topic, aunique identifier in the formatTOPXXX_TOPIC_NAMEhasbeendefined.Each topic'sproducers(orpublishersinthePub/Subterminology)andconsumers(subscribers)arelisted. While the producer is not familiar with the subscribers, the subscribers canknow the producer's identity thanks to its specification in the common header"Originator"attribute. Inaddition,an indirectly-routedmessagecanhavea "sender"

D6.2–V1.0

Page41

whichisnotthe"originator".AlinktothedetailedstructureofeachtopicisavailableforauthorizedreaderswithaccesstothebeAWAREWIKIrepository.

Table3-2.beAWAREMessageBusTopics

TopicID Description Producers(Publishers)

Consumers(Subscribers)

LinktoTemplate

TOP001_SOCIAL_MEDIA_TEXT

Thistopicisusedtosendandreceiverecentlycrawled

Twitterpoststhatcontaintext

SMA MTA TOP001

TOP002_SOCIAL_MEDIA_IMAG

E

ThistopicisusedtosendandreceiverecentlycrawledTwitterpoststhatcontain

images,representedbytheirids

SMA MTA,IMGAN TOP002

TOP003_SOCIAL_MEDIA_REPORT

UsedbySMAforcommunicatingTwitterreports

totheKBS.

SMA KBS TOP003

TOP010_AUDIO_ANALYZED

VoicerecordinghasbeenanalyzedbyASRmoduleand

transcriptionhasbeenstoredinRawStorage

Mobileapp ΜΤΑ TOP010

TOP017_video_analyzed

Avideohasbeenprocessedandtheanalysisfileshavebeen

storedandlinkedtothismessage

VIDAN PSAP top017

TOP018_image_analyzed

Animagehasbeenprocessedandtheanalysisfileshavebeen

storedandlinkedtothismessage

IMGAN PSAP TOP018

TOP021_INCIDENT_REPORT

Reportincidentfrommobileapp

APP KBS,MTA,MEDHUB,IMGAN,

VIDAN,ASR

TOP021

TOP022_PUBLIC_ALERT

Newpublicalertforthemobileapp

PSAP APP TOP022

TOP023_TASK_ASSIGNMENT

Newtaskcreatedforfirstresponders

PSAP APP TOP023

TOP028_TEXT_ANALYSED

ExtractedinformationhasbeenstoredintheKB

MTA KBS TOP028

TOP030_REPORT_REQUESTED

Setoftriplescapturingthecontenttobecommunicated

KBS MRG TOP030

TOP040_TEXT_REPORT_GENERATED

Verbalisedreport(alarm,message,update)inthe

recipientslanguagehasbeenstoredintheCDR.

MRG KBS TOP040

D6.2–V1.0

Page42

TopicID Description Producers(Publishers)

Consumers(Subscribers)

LinktoTemplate

TOPC051_FLOOD_FORECAST

NotificationonnewAMICOforecastresults

AMICO CRCL TOP051

TOP101_INCIDENT_REPORT

IncidentReporttoPSAP KBS PSAP top101

TOP102_TEAM_REPORT

TeamReporttoPSAP APP PSAP top102

TOP103_TASK_REPORT

TaskreporttoPSAP APP PSAP top103

TOP104_METRIC_REPORT

MetricdatatoshowonPSAPdashboard

CRCL PSAP top104

3.3 CentralDataRepository–CDR

3.3.1 Overview

Thecentraldatarepository(CDR)isintendedtostorelargefilesthatneedtobesharedbetweendifferentplatformcomponents.

For this purpose, we deployed an instance of the IBMObject Storage in the cloud,providingunstructuredclouddatastorage,foreasystoringandaccessingcontentfromvariousapplications.


Mostof theplatformcomponents interactwiththeCDRforuploadingandaccessingmedia or data files, or to perform both operations. In a representative flow a newvideofileisingestedintothesystemviatheAPP.Theapplicationback-endstoresthevideofile intheCDRanddeclarestheexistenceofthenewfileusingamessagesentvia the MSB. The MEDHUB, which subscribed to receive notifications on the sametopic,proceedstoaccessthestoredfileandanalyzeit.Oncetheanalysisiscompleted,anewfile isstored intotheCDR includingconclusionsreachedbythecorrespondinganalysis component, and information is shared with other system components in asimilarmanner.

D6.2–V1.0

Page43

3.3.3 ConnectivityDiagram

Figure3-2.ConnectivityDiagram-CDR

3.3.4 FeaturesandCapabilities

TheCDRismeantasthecentralsecureaccesspointforstoringandaccessingdatabydifferent platform components using a REST interface. It is manifested as a cloud-enabledfilesystem,providingsharingandcollaborationcapabilities.TheCDRservesasa scalable persistent storage layer for the beAWARE platform. Thus, a secure andprotected data repository serving different platform components working withdifferentkindsoffilesisprovided.

ReadilyavailableSDKsandclient librariesexist foravarietyofpopularprogramminglanguages.Amanagementconsolecanbeusedtoconfigurethegenericserviceforthespecific access patterns. In addition, access policies can be configured based on theneedsofthedifferentsystemcomponents.


Therealizationof theCDR isviaaclouddeploymentof IBM’sObjectStorageserviceinstance.


Currentlytherearetwomainwaystoaccessthecentraldatarepository,directlyviaaclient library or through a helper application which is deployed as a cloud foundryapplication(https://object-store-app.eu-gb.mybluemix.net)insidetheprojectspacein

D6.2–V1.0

Page44

the IBM cloud. The convenience function exposes a REST interface for uploading,retrieving,anddeletingfiles.

3.4 NoSQLDatabase

3.4.1 Overview

The NoSQL capabilities are mainly used for storing and accessing relevant tweetsinformation. The information is stored as JSON documents, and pointers to it arepassedbetweencomponentsthroughthemessagebus.

For the realization of the NoSQL capabilities we use an instance of a MongoDBdeployedonBlueMix(https://www.compose.com/databases/mongodb).


The main users of this capability are the SMA and MTA components. In arepresentativeflow,thetweetercrawlerfeedstheSMAcomponentwithnewtweets.If theSMAdetermines thatacertain tweet is relevant, it stores the tweet ina JSONformat in theNoSQLDB, andpublishes the corresponding link via themessagebus.The textanalysiscomponent in turnpicksup this information fromthemessagebusandextractstherelevanttweetinformationfromtheDBusingthesuppliedlink.


TheNoSQLcapabilityintheplatformisrealizedviaaclouddeploymentofaMongoDBinstance in IBM’s cloud.AKubernetes secret, namedmongo-bw2-secret, is injectedinto the platform namespace providing the required connection information in theformofthecorrespondingURI.


TheDBcanbeaccessedusingstandardclientlibrarieswhichareavailableinmultiplelanguages.

D6.2–V1.0

Page45

4 Module-Level Data Access Mechanisms[CONFIDENTIAL]

The purpose of this chapter is to describe themechanisms for data access in eachsubsystemandmoduleinbeAWARE.Foreachmodule,weprovide:

• anoverviewofthemoduleanditspurposeormainfunctionality,

• a connectivity diagram that shows the inputs (on the left-hand side of thediagram) to the module from other modules and mediators (as specified inSection3), and itsoutputs toothermodulesandmediators (on the right-handsideofthediagram),

• a list of receivable inputs including themapping of the operational entity to atechnical data or information entity, and the relevant data accessmechanismwhichfacilitatesit,

• a listofproducedoutputs including themappingof theoperationalentity toatechnical data or information entity, and the relevant data accessmechanismwhichfacilitatesit,

• a subset of themodule's technical requirements from the full list (specified in[4]) that entail interoperability, connectivity, or data access requirements orchallenges,

• a list of mechanisms and interfaces of the module for receiving input orgeneratingoutput.

Altogether, all inputs into a module must be mirrored by an output from anothermodule, and vice versa, such that eventually all the outputs by any module in thesystemareusedbyatleastoneothermodule.Inputsfromexternaldatasourcessuchas social networks or sensors are indicated accordingly, and regarded as boundary-crossinginputs.Similarly,outputstoexternaldatareceiverssuchasoperationalusersareregardedasboundary-crossingoutputs.

Theservicesandmodulesdescribednextrelyondedicatedinnovativetechnologiesorheavily-customized assets, and may be subject to intellectual property. In addition,specificsofthedataexchangeamongthemodulesarenotnecessarilyrelevantinanycontextorforanyapplication,andstemfortheinternaldefinitionsandspecificationsofthebeAWAREplatform.Duetotheseconsiderations,thissectionisconfidential,andavailableonlytoEuropeanCommissionofficersandbeAWAREConsortiumMembers.

D6.2–V1.0

Page46

The full content of this section is available in Deliverable D6.6 – Data-SourceIntegrationFramework–ConfidentialVersion.

D6.2–V1.0

Page47

5 Conclusion[PUBLIC]ThisdocumenthaspresentedthebeAWAREdata-sourceintegrationframework,aimedtoproviderobust interoperabilityamongtheoperationalagents,connectivityamongthe subsystems andmodules, and data exchange and information-based interactioncapabilitiesacrossthesystem.

We have started with identifying the user requirements which are relevant forinteroperability and connectivity, and the common available data accessmodalities,methods,andsolutions.

Itbecameapparentveryearlyinthearchitectingandhigh-leveldesignprocessthatthesystem architecture must rely on a robust and extensible connectivity and dataexchangeinfrastructure,includingassetslikeamessagebus(Kafka),centraldatabase(MongoDB),andcentraldatarepository(ObjectStore).

Inaddition,eachsubsystemandmodulehasbeendesignedtointeractseamlesslywiththe infrastructure as well as with some of the other modules via alternativeconnectivitychannels,forenhancedperformanceandflexibility.Forinstance,thereisa direct interface between the Flood Forecasting service (AMICO) and theWeatherForecastingservicetoobtainup-to-dateandarea-specificweatherforecasts

We have presented the high-level specification of the common infrastructurecomponents, including the supported data access modalities and exchangedoperational and informational entities. In addition,we have presented the technicalrequirementsandsolutionspecificationforconnectivityanddataexchangeservicesineachofthebeAWAREmodulesandcomponents(availableintheconfidentialversiononly).

Finally,wehaveelaboratedontheinformationalentitiesthatwehaveidentifiedoverthe course of the project. We have specified their attributes and appropriate dataaccessmodalitiesandutilizingfunctionalflows.Theseflowsprovideend-to-endvalueby supporting informationexchangeand information-basedcollaborationamong theoperationalusers.

Thisdocumentmarks theoutcomeof the intensiveworkof all technicalpartners aspart of task T6.2, with continuous feedback from end-users, in order to ensure thefacilitationofa robustdata-source integration infrastructure, and the intelligentandefficient utilization of the infrastructure by themodules developed by the partners.During the project it became increasingly critical to ensure the commonality andstandardizationofdataexchangeprocesses,governthedefinitionofnewdatatopics

D6.2–V1.0

Page48

and ensure compliance across the board, especially for the seamless exchange andmanipulationof informationwhilepreservingacommonsemantic language. Inmostcases,itwasfoundthattheKafka-basedMessageBuswasthemostsuitablemeansofexchangingdataamongthemodules,includingbothinformativedataandnotificationsregardingtheavailabilityofdata inotherchannels. Insomecases,RESTfulAPIswereset-up or exploited for on-demand data requisition and retrieval, which allows forsynchronousinteractionsandtransactions,orforasynchronousrequestsofdatawhennotifications are provided. In addition, during the course of the project and theformulationofthesolution,itbecameclearthatastoragesolutionforlargemediafiles(audio, video, image, and dataset) is necessary, and that links to media would beprovidedaspartofKafkamessagesratherthanrawattachments.

Futurepost-projectorPhase-IIextensionsoftheframeworkmayincludemonitoringofdata acceptance via functional acknowledge messages that will be generated byconsumers as topicked messages in their own right in response to incomingoperational topics. This would allow assurance of data availability to the relevantstakeholders,whetherbytheoriginatoroftheinformationorbyaneutralmonitoringauthoritythatwouldoverseeandlatersupervisetheflowofinformationinthesystem.Additionalenhancementsmayincludedifferentiationofservicelevelsaccordingtothecriticality and frequency of various topics, as well as the adoption of topic filteringconventionsinordertoallowhierarchicalprocessingofinformation(e.g.separationofincident topics by category in order to support easy routing of incident updates torelevantprocessingservices).

Toconclude,thecurrentinfrastructurehasproventobeanoptimalsolution,intermsofcost-benefitbalance,andinthecontextofsupportingthechallengingintegrationofthe beAWARE platform. We continue to monitor the correct utilization of theinfrastructureby thebeAWAREmodules, criticalextensionsandenhancements,and,whenthevolumeandthroughputofdatareachacriticalmass,alsotheevaluationofreliability,performance,availability,affordability,scalability,andoverallservicelevel.

D6.2–V1.0

Page49

6 References

[1] Y.Mordecai,O.Orhof,andD.Dori,“Model-Based InteroperabilityEngineering inSystems-of-Systems and Civil Aviation,” IEEE Trans. Syst. Man Cybern. Syst. (toAppear.,vol.TBD,no.TBD,p.TBD,2016.

[2] beAWARE,“D2.1UseCasesandInitialUserRequirements.”

[3] E. Jones,W.Systems,andJ.Westfall,“CommonAlertingProtocolVersion1 .2,”no.July,pp.1–47,2010.

[4] beAWARE,“D7.2SystemRequirementsandArchitecture.”

d6.2 data-source integration framework

Documents