methods and algorithms supporting combat …

26th Geographic Information Systems Conference and Exhibition “GIS ODYSSEY 2019” Conference proceedings

37

METHODS AND ALGORITHMS SUPPORTING COMBAT SITUATION RECOGNITION AND RECONNAISSANCE UTILISING

MOBILE PLATFORMS AND GIS SERVICES

Mariusz Chmielewski, Ph.D. Military University of Technology Faculty of Cybernetics Warsaw, Poland e-mail: [email protected]

Michał Sobolewski, M.Sc. Military University of Technology Faculty of Cybernetics Warsaw, Poland e-mail: [email protected]

Abstract

The paper presents description of developed mobile software designed as extension module for supporting land forces and territorial defence in battlespace data collection, situation awareness assessment and task guidance. The application and implemented algorithms serve as mechanisms for capturing recognised objects but most of all positioning them based on image based calculations and sensor-based data. The paper contains also calculation examples and evaluation of location precision method. The data is supplemented by set of in-application examples. The architecture of application contains all required perspectives and documents mainly the logic and physical models. Developed tools deliver mechanisms supporting Network Centric Warfare characteristics to individual soldier and commander creating network of battlespace entities capable of data acquisition, object identification (reconnaissance) as well as reasoning about current environment state, mission and possible courses of action (CoA). Designed layered architecture based on mobile and SOA technologies is a proof-of-concept demonstrator utilised in several projects as a integration platform but most of all as a decision support tool assisting military during live exercises. mCOP and tCOP systems have been designed and prepared to be powered from external data sources such as heterogeneous C4I systems and other military-government data sources, using NATO and open standards (JC3IEDM, NFFI, TSO). Experiences and gathered data demonstrate novel approach which deliver advanced algorithms for evaluation of soldier's environment to support his understanding and awareness of combat situation. Such features are crucial for rapid situation development and engagements therefore handheld tools which can efficiently and reliably render tactical and operational pictures are very useful and required. New types of conflicts introduce threats, which require not only operational forces involvement but also territorial defence and population. Therefore, there is a need for ICT tools which will enable wide spread secure communication between forces and possibly civilians, automate reporting hostile data, and scenario description. All these requirements have been implemented in mCOP mobile toolkit supported by central server facilities and C2 portal – tCOP. Demonstrated results will present novel approach to mobile device utilization during combat missions for automatic data acquisition and rendering. Such solutions in the future could supplement weapon systems in order to increase efficiency of tactical and topographical orientation within the combat environment. The paper will evaluate characteristics of mobile application, which are able to support combat operations performed by operational forces and territorial defence. Discussed ideas will also contain discussion on how to construct specialised, possibly classified software deployed on commercially available hardware platforms and what advantages such approach can deliver to support operations in hybrid conflicts.

Key words: situation awareness, COP, common operational picture, military DSS, mobile system, tactical support, territorial defense, NATO NEC

Introduction and research motivation

Handhelds, their hardware capabilities and software platforms provide new opportunities to support also combat mission responsibilities (CHMIELEWSKI, 2017). Exceptional computing power, can be successfully applied for complex sensor data processing, instant location monitoring, data fusion and


38

decision support. This delivers new means for utilising commercially available platforms integrated with external communication equipment (tactical radios, 5G LTE network systems) as specialised platforms for personal mission assistance. Supporting military individuals and decision makers with network technology has been deployed since the early 90s but only now the interoperability of hardware platforms, specialised military communication systems, mobile devices deliver means for the development of smartphone integrated specialised applications. Development of technology in the domain of software but most of all hardware and telecommunication deliver powerful platforms widely available to civilian market consumers. These types of technologies can be adopted to support security applications, after introduction of security mechanisms but most of all specialised approach to integrate specialised command support software with commercial handheld platform. The requirements of such integration depend on the data classification levels and in some cases deployment of such software is restricted based on the recommendations from the intelligence services. This paper describes utilisation of mCOP (CHMIELEWSKI, 2017) mobile software platform as a visualisation toolkit integrated with ontology based reasoning services provided by integration layer of tCOP platform and serving as asymmetric threat identification and recognition mechanisms. The mentioned mechanisms are webservices, utilising Protégé OWL API (HORRIDGE, BECHHOFER, 2011) for operating on ontology modules supported by OWL 2.0 semantic reasoners Pellet (SIRIN et al., 2007), HermiT (GLIMM et al., 2014), which provide model consistency checks, classification of concepts (terminology) and individuals (data). Developed method of data integration and hidden information association search engine delivers services for criminal or terrorist threat activity selection and recognition (CHMIELEWSKI, 2017).

The idea behind such tools as mCOP is to collect and distribute operational data of conducted mission utilizing GIS functionalities and augmenting the data with additional information on terrain, hostile and friendly forces. The range of processed data and produced tactical information depends on the level of command, battlespace dimension but most of all available data sources. The difficulty, while developing operational rendering environments comes from the variety of formal NATO standards and specificity of tactical symbology (e.g. Std-2525, APP-6C) on top of which, a set of map data standards need to be integrated (UK MoD, 2005). The geoinformation varies depending on the battlespace dimension or in case of cyberspace even limits such relationship. TCOP and mCOP products deliver functionality for all battlespace dimensions and crisis management due to the target audience – territorial defence units and land forces. The software needs to be secure and easily configurable to be rapidly deployed on soldier’s and commander’s handhelds (mCOP) and in lower level command centres (tCOP). mCOP platform delivers software, not hardware platform, therefore, it utilizes existing communication infrastructure intentionally. Considering the operational background of territorial defence such feature can be also an advantage as software that can operate in the civilian and military communication environment. Hidden combat actions performed by military units disguised as ordinary civilians using such software platforms (mCOP) can deliver operational capabilities integrated within commercial smartphones seamlessly used in combat or intelligence operations. Presented research concentrates on developing specialized combat functionality which directly supports battlespace data acquisition, processing and distribution among friendly side assets gaining decision superiority. This task is performed using handheld devices connected to secured tactical networks through application of specialized network interfaces.

Analytics for combat support

In order to prove applicability of developed analytical methodology and tools a demonstration environment has been designed and established featuring hypothetical, operationally justifiable scenarios. Based on these operational scenarios, a set of requirements have been formulated in order to design, configure and integrate military and crisis management tools. Selected software platforms supply consistent data flow process, consisting of:

- data generation – responsible for delivering decision situation events and output data in the form of time-constrained messages (simulation software);

- data integration and filtration – offering algorithms and mechanisms for combining data about units and objects (locations, composition, tasks performed), data filtering based on the semantics and levels of security of units;

- data visualization – constructing an operational image presenting civil and military elements supplemented with GIS data and products of decision support procedures;

- knowledge acquisition and reasoning – the highest level mechanisms responsible for supplementing data instances to the knowledge base and launching reasoning rules, classifying objects and identifying indirect, hidden information. The designed web applications (tCOP) and services (tactical symbology, NFFI-TSO translators)

have been developed for military commands and crisis management centres (decision nodes). TSO (CEN,


39

2009) and NFFI (NSA, 2008; NOGALSKI, NAJGEBAUER, 2011) integration standards act as scenario data inputs, which further are filtered and merged into one Common Operational Picture product. The mobile applications support individual combatants (reconnaissance and reporting nodes) providing on-site communications, Common Operational Picture development, instant Situation Awareness (US DOD, 2009) and simple reporting tools (ADatP3).

As a result provided mobile DSS (decision support system) environment and reasoning services delivered:

- Planning and executing of the Comprehensive Approach (CA) driven EU-led Crisis Management Operations (CMOs) requires extensive information sharing between all actors involved (international, governmental, non-governmental, private, etc.) leading to civil-military coordination, cooperation and collaboration and creation of the Shared Situational Awareness;

- There is a need for commonly agreed procedural and technical standards and tools for information exchange between civil and military parties supporting creation of a equally understood joint SSA for the purpose of EU-led CMOs; o Available information exchange standards (NFFI, TSO, JC3IEDM, App6A, XMPP) provided by

constructed environment according to defined services; o Semantic integration of information resources across information domains with the purpose

of semantic alignment; o closure of information gaps, deconflicting, removing inconsistencies etc.; o fusion of information and knowledge acquisition;

- PL NEC DEMO provides various presentation tools for different users (Portals, Tactical and Mobile tools);

- Access to COP (Common Operational Picture) products has been developed using distributed, access anywhere within the tactical communication network mechanisms (web-based, mobile-device-based).

Methods for reconnaissance support

Smartphone’s camera, besides the ability to capture images and record video, enables the possibility to measure the relative distance between designated target and the operator. Estimation of the distance is possible with the use of basic calculations supported with specific camera parameters. Presented research proves that it is possible to use a camera located in a smartphone for the purpose of building a tool designed for surveillance and object detection. Mobile tool is perfect to perform stealthy measurements. This utility may be used for example by Territorial Defense Forces, civilians or soldiers who use mobile devices to support their actions during military operations. It is necessary to provide a security layer to secure mobile applications from violating security rules during operations. Usefulness of reconnaissance executed by mobile tool is achieved using simple to use measurements requiring only simple and quick interactions (point, adjust bounding mesh, estimate relative height – Fig. 2) and then asynchronously send to the command center’s data fusion server. mCOP tool, based on geolocation data, azimuth and distance to object may determine position of the reported object.

This work presents two techniques used to calculate the object’s distance. Method A utilises Thales theorem and known object height, and alternatively, Method B uses object size difference projected on digital sensor gathered from separate snapshots. These methods can measure distance in two dimensions of an object: width or height. For the purpose of this article, the author used height. Method A uses proportion of focal length and the height of the object on the image sensor is the same as the

proportion between distance to the object real height:

, where denotes real height of

object, denotes distance to this object, denotes object height on digital sensor, denotes focal

length. After transformation, we can get a formula to calculate distance:

. It is possible to read the

value of focal length from camera parameters provided by Camera API. In this case, where to distance measurement is used a smartphone, focal length value is constant. Real object height can be entered manually or automatically detected (OpenCV). The object’s height projected in the sensor can be computed. In this case, it is necessary to fit grid to the object seen on the preview. The discrepancy between sensors format (4:3) and preview format (16:9) must be included in calculations, due to the Android system transformation of image taken from the sensor to image projected on the preview. Consequently, image on preview has another dimension. Transition of image on the sensor to image on

preview consist of software crop top and bottom of this image:

, were denotes height of

sensor,

denotes object height on preview in mCOP application, denotes height of this preview,


40

denotes difference in height between format 4:3 and 16:9. Formula after calculation described transitions:

(

)

and considers image sensor height. It is rarely specified in the documentation provided

by the smartphone producer. Android version 5.0 (API 21) [21] and above deliver a new version of Camera API (Camera2), providing information about camera sensor size. However, on Android devices with software below API 21, sensor size must be manually calculated or inserted. Crop factor: it describes camera sensor size as compared to a reference format. This term is applied to digital cameras (located in smartphones), relative to 35 mm film format (43.3 mm diagonal). The most commonly used definition of

crop factor is the ratio of a 35 mm frame diagonal to the image sensor diagonal:

, where

denotes crop factor, denotes diagonal of 35 mm film format, denotes diagonal of the sensor.

Another definition of CF:

, where

denotes 35 mm equivalent focal length. Thus,

knowing the focal length and 35 mm equivalent focal length allow calculating CF.

Fig. 1. Diagram showing the calculation concepts for Method A (left) and Method B (right) describing distance measurement techniques. Source: own work.

We can obtain a diagonal measure of the sensor:

. In order to find sensor dimensions

(format 4:3), we can use Pythagorean theorem: ( ) ( ) ,

,

, where

denotes width of sensor. Information about the camera angle of view is located in Camera API parameters. The angle of view can be computed from the effective focal length and chosen dimension. After formula

transformation, we can get the height of the sensor as follows: (

), (

)

. Method B is using another assumption followed by additional measurement of the distance taken from a location to which a precise location is known. In order to measure the distance to objects which height is not known, we can utilize proportion. Method B requires two separate measurements (surveys) taken from different locations. The proportion between heights of an object on sensor and distance difference

obtained from these two measurements is the same. Following formulae describe Method B:

,

,

,

, where denotes distance difference between two

measurements, denotes object height on the sensor from first measurement, denotes object height on the sensor from the second measurement.

Location measurement reports evaluation

Measurements was carried out on several devices, with following characteristics: focal length 2,38 [mm], image height 3120 [px], sensor height 2,21 [mm]. Presented in Table 1 measurements (1-6) were conducted outdoors in combat-like conditions, where an operator performed, a task of identifying and recognising camouflaged soldier.

The device was held by an operator with 1.80m height, at his eye level. As can be seen, Method A has much more accuracy of measure in studies 1-6. This is due to inaccurate identification of distance difference between two measurements, which is generally hard to do in field conditions. Confirmation for this is the difference in accuracy of obtained results in measurement 4 and others. In this case (4) distance difference was accurately measured but in other cases only estimated. Measurements 7-8 was conducted under laboratory conditions, at small distances, and the observed object’s size was 10x10cm.


41

Table 1. measurements in method A and method B. MA denotes Method A survey, MA denotes Method B survey, X denotes Method A survey taken from captured object picture, Y denotes Method B survey taken from captured object picture, sensor data (4160x3120px)

No D

[m] MA [m]

MB [m]

m [m]

X [m]

Y [m]

δA [%]

δB [%]

1 23,05 22,90 23,90 ~2 [1,96] 22,96 23,50 0,65% 3,69%

2 22,37 22,33 18,99 ~2 [1,88] 22,35 21,24 0,18% 9,88%

3 15,15 15,06 14,65 ~2 [2,01] 15,04 14,87 0,59% 3,30%

4 14,12 13,95 12,99 2 13,97 14,10 1,20% 8,00%

5 6,05 5,98 5,95 ~2 [1,95] 5,97 5,96 1,16% 1,65%

6 6,09 5,97 5,95 ~2 [2,06] 5,98 5,96 1,97% 2,30%

7 0,50 0,50 0,50 0,10 0,50 0,50 0% 0%

8 0,30 0,30 0,30 0,10 0,30 0,30 0% 0% Source: own work.

Fig. 2. mCOP view showing interface with sliders to input height and object size on preview(1 – slider to input object height, 2 – slider to fit grid with viewed object, 3 – cropping grid, 4 – relative distance to object, 5 – object height, 6 – size of cropping grid). Source: own work.

Table 2. Measurement specific data evaluated for measurements delays and distance

Operation delays Measurements method A

No

[px] Operation

time [s]

[px] Distance

D [m] 1 900 3,02 800 2025,97 2 800 3,16 900 1800,87 3 700 3,42 1000 1620,78 4 600 3,81 1001 1619,16 5 500 4,24 1002 1617,55 6 400 4,93 1003 1615,93 7 300 5,87 1200 1350,65 8 200 7,08 1500 1080,52

Source: own work.


42

Reference measurements have been taken with a stabilised smartphone and measuring tape. In those tests, a similar level of precision has been obtained. Inaccuracy of outdoor tests is mainly caused by the imprecision of entered parameters and distance difference (for method B). Another factor increasing measurement error is imperfect cropping of the viewed object by an operator. Operator by putting more effort into grid adjustment substantially increase the time of measurement. Solution for this problem could be application of image processing libraries, which after a suitable period of training the model could accurately recognize and mark object automatically. In order to measure objects over 30m away image stabilization is necessary. Simulation was carried out with Sigma 150-600 f/5-6.3 DG OS HSM lens: focal length: 600 [mm], image height [px] 3120 [px], sensor height 4,62 [mm], object height 4 [m]. In order to maximize precision of measurement, it is proposed to use coupled with smartphone optical system. In case of deviations in measurements, to resolve problem of inconsistent data, the author propose the triangulation method, which can be developed also on smartphone. This method uses triangle properties, particularly the method that allows calculating the height of a triangle based on its angles.

Processing a few measurements allows creating a triangle. On the basis of the azimuth in which the phone is directed and the precise distance of the measurements points it is possible to calculate the

angles and distance to the object. Following formulae describe triangulation method:

( ), where

d denotes distance to object (triangle height), l denotes distance between points where measurements were taken. Obtained value of the distance to the detected object is transferred to the headquarters, where it is applied to the actual tactical or operational situational image and then resend to all subordinates. This common knowledge is the essence of the situational awareness required on the battlefield. Responsibility of each commander is raising situational awareness of their subordinates by all means. As already shown, presented in this article mechanisms provides the ability of realization of such requirements.

Situation awareness and task guidance

Situation awareness on the battlefield can be obtained by utilisation of mCOP application to assess elements of tactical situation in real time. This considers also topographical information similar to the exact location in terrain and azimuth to the given location or object.

Fig. 3. Mission guidance mode, demonstrating checkpoint direction indicator supported by TARV view during combat activity (1- map component, 2 – guiding indicator) – battlespace elements - data layers can

be adjusted (switched on/off) and configured (alfa channel). Source: own work.

Such feature can be especially useful for reconnaissance units to know where they are, where are Blueland and Redland forces. Those capabilities provide one of mCOP’s activities with AR view (Fig. 4, 5, 6).


43

Fig. 4. Tactical orientation mode demonstrating operational picture view and map-compass supported by TARV during combat activity – additional mechanisms for AR view configuration has been introduced (buttons 1-3) allowing user to preconfigure preferred datasets and HUD functions. Source: own work.

Fig. 5. TARV based combat scenario view construction, viewing battlespace entities supplemented with equipment and supplies data and enhancing evaluation capabilities. Source: own work.

The main part of computations on data derived from device’s sensors are made by Route Trace Component. This component is strictly associated with the activity mentioned above. The arrow that indicates direction and azimuth to the given point in the terrain is being painted on the screen of the device by OpenGL ES component, which is also responsible for calculating the rotation of this arrow. These calculations are based on data Obtained from the device’s position as well as motion sensors. Those two components combined gives the functionality of route management. In real scenario using it, soldier can place checkpoints on the map. Their coordinates allow determining where azimuth arrow should be


44

pointing at. In addition, this activity contains also map component which in real time indicates the exact location of the soldier. Thanks to those, scout doesn’t have to pay too much attention to route tracking and topographical orientation because it is sufficient to follow the prescribed itinerary in application, which was inherently set in such a way as to avoid enemy safely and concealed, providing an opportunity for accurate observation.

NEC domain concepts and terminology

Network Enabled Capabilities or Network Centric Warfare (US DoD, 2009) are military operations doctrines relying on information technology development and battlespace communication availability. Implementation of technology in operations helps to achieve decision superiority and thus higher efficiency in combat. Latest advances in military communications has been aimed at formulating, standardizing and integrating decision support and weapon control systems – C2IS, C4ISR (US DoD, 2009), (UK MoD, 2005). Such tools are used for rapid data acquisition, integration and visualisation which in result support quicker response to dynamically changing situation thus increasing the efficiency of decision making processes.

One of most important and valuable features of any C4ISR system is the COP – Common Operational Picture – a fused, integrated image of GIS data with military and civilian assets presenting both static and dynamic characteristics of elements and environment. The production of fused COP image can be achieved through application of GIS terrain data representation standards and tactical symbology standard like APP6C. As a result COP is rendered 2D or 3D scenario snapshot of a given timeframe and map viewport exposing semantics, functions and behaviour of battlespace-crisis entities. The vision of Network Enabled Capabilities (UK MoD, 2005) is to deliver seamless access to rich and timely information to all war fighters and decision-makers at every echelon in the military hierarchy. This enables all scenario participants, including individual soldiers, vehicles, command centres to share the battlespace information. NATO Network Enabled Capabilities techniques and technologies deliver following capabilities (UK MoD, 2005):

- Efficient networked mechanisms – improving battlespace data flow and information sharing between all entities within the operational scope and considering all security and efficiency restrains;

- Information sharing and superiority – improvement of data availability, accuracy and quality, resulting in wider operational scenario understanding and leading to more efficient situational awareness development (faster, more accurate);

- Shared Situational Awareness – tailored, not restrained data sharing among all scenario participants, describing actual understanding of environment and scenario. Process enables and ensures collaboration (data sharing) between sensors, actors and decision nodes improving their operational synchronization, thus reducing the communication delays, which in result speed up decision process and increase mission efficiency. Such information is fused (integrated) based on several data sources (battlespace observers or

sensors) and combined into a coherent, accurate picture of the battlespace. Central and crucial objective of Network Enabled Capabilities concept is to organize and standardize C2IS system compatibility and interoperability. In order to organize efficient data sharing all levels of integration (technical, syntactic and semantic) must be provided and integrated. Developed tCOP and mCOP (Fig. 1) software comply to described above concepts assuming that information exchange between the nodes of the system not only provide the data and information, but also deliver filtering functionalities. Advances in mobile technologies, and in particular programmable and extendable handheld platforms provide technical capabilities for designing personal and professional tools for military. Overcoming security (CHMIELEWSKI, 2009), issues and relying on reliable, secure networks a commander can organize an efficient tactical network, which after application of software can automate many activities supporting battlespace orientation and increasing situation awareness. Described in this work system components tCOP and Fig.4 mCOP concentrate on providing such features. Functionality constructs are intentional due to the target audience of mCOP – territorial defence units and land forces.


45

Fig. 6. Functional mCOP forms presenting (left) semantic query results evaluating threat levels for monitored critical infrastructure elements, (centre) downloaded combat scenario supplemented with distance measurements, (right), detailed view of location threat level indicators evaluated with rule-based reasoning. Source: own work.

Shared Situation Awareness for asymmetric threats and hybrid conflicts

Situation Awareness (SA) can be defined as a perception of environmental elements within a volume of time and space, the comprehension of their semantics, and the projection of their status in the near future (ENDSLAY et al., 2000). SA is directly associated with the perception of environment, which is a key factor for decision-makers in complex, dynamic processes such as military command and control. For hybrid conflicts SA should emphasize and extract unknown or not recognised entities supplemented with environmental characteristics preparing forces for more robust and high-detail operations (on lower levels). Available models of SA (ENDSLAY et al., 2000; CHMIELEWSKI, 2009) illustrate several important variables describing individuals, tasks, and environmental factors.

The majority of such factors has been utilized to design the ontology – UBOM – Unified Battlespace Model (CHMIELEWSKI, 2009) which integrates utilised ontology modules JC3IEDM, NFFI, ADatP3 (selected reports), POLREP, Location. Implemented model provides accordingly three stages of reasoning. The first stage of ontology processing, correlates the information describing registered battlespace objects evaluates location, composition and classifies their affiliation, potential parameters as well as threat status – calculated based on (CHMIELEWSKI, SAPIEJEWSKI,2018; TAYLOR, 1980). The second stage aggregates the battlespace information providing inference for echelon composition based on unit’s doctrinal patterns and their behaviour. Based on inferred situation additional military entity relationships are introduced marking and describing the operational functions of elements in the formation. This stage also is aimed at mapping ontology modules thus expressing data from one standard (terminology) in the other model. The semantic bridges (CHMIELEWSKI, GAŁKA, 2010) have been prepared for every pair of ontology modules. The third stage of reasoning introduces possible behaviour of recognized units and objects formulating course of action of units involved in the scenario.

SA benefits from COP products and requires additional information supplementing the actor’s orientation within the scenario. SA combines data gathered from all available battlespace dimensions Ground, Air, Maritime, Logistics and Cyber (Fig. 7) provided by set of recognized operational and utility pictures (US DOD, 2009; CHMIELEWSKI, 2009):

- RAP (Recognised Air Picture) – containing all available objects within particular airspace supplemented with affiliation, type, evidential records and performed tasks;

- RMP (Recognised Maritime Picture) – describing all relevant and available maritime objects within defined naval area supplemented with object features, conducted tasks and activities;


46

- RGP (Recognised Ground Picture) – containing all available objects (installations, units, warfare, obstacles) within particular area supplemented with affiliation, type, evidential records and tasks, operational responsibilities, etc.;

- RLP (Recognised Logistics Picture) – defining, underlining available aspects of logistics operations supporting all battlespace domains;

- RCP (Recognised Cyberspace Picture) – combining available data describing cyber and info space with regard to communication networks, protocols and systems;

- Intelligence data – combining classified, hidden data gathered from all available data sources; - Results of decision support algorithms – for environment (e.g. terrain) and operational

elements characteristics evaluation – enriching the overall picture and supplementing SA.

Fig. 7. Integration of battlespace data with semantic unification and projection in GIS based map stack. Integration of data is performed using JC3IEDM terminology and semantic migration tools. Source: own work.

The major difficulty while designing unified COP environment comes from diversity of object metadata descriptions and availability of unified symbology, therefore semantic mapping into standardised APP6A symbols is crucial and can be automated (CHMIELEWSKI, GAŁKA, 2010). Based on the experiences of ontology usages an inference Description Logic based engine can be applied to automate individual mapping across separate domain models.

Ontology-based support of reasoning. Expert system implementation

Described in this work system components tCOP and mCOP concentrate on providing reasoning power supplementing situation awareness features within CMO. The software need to be secure and easily configurable to be rapidly deployed on soldier’s and commander’s handhelds (mCOP) and in lower level command centres (tCOP). The deployment requirements for the presented platform has been minimized and the installation mechanisms fully automated in order to simplify configuration and administration activities. Provided platform concentrates on software not hardware platforms. It utilizes existing infrastructure intentionally, which can be perceived as an disadvantage. However, in case of territorial defence units, which operate in civilian environment and restricted funding, exploiting existing, personal mobile devices and restricted computationally platforms is an advantage. Hidden combat actions


47

performed by military units disguised as ordinary civilians using such software platforms (mCOP) can deliver operational capabilities integrated within commercial smartphones seamlessly used in combat or intelligence operations. Described research contains notes on lessons learned during the design, development and testing (TRL VII, VIII) of the software during EDA NEC capabilities demonstrations in reference to demonstration of Situation Awareness development techniques and other NEC capabilities delivered by the mCOP and tCOP platforms.

Previously described military concepts provide crucial tools for combat support and reconnaissance activities. Delivering such information capabilities provides advantages for coordination an control of executed missions or performed information gathering activities (terrorist threats). Dismounted Soldier Solution (US DOD, 2002) family of tools deliver personalized system for individual soldier use during combat, supporting geomonitoring, observation and surveillance equipment, weapons, communications and most of all C2 data presentation and management. mCOP resembles in some manners DSS solutions but it concentrates more on developing configurable COP and SA products as the handheld platform can be easily accessed by soldier to consult and orientate within the current battlespace situation. (Fig. 1, Fig. 6, Fig. 7). Described software platforms deliver data and tactical information to soldiers and commanders on the battlefield in order to simplify topographical and tactical orientation within the current combat situation. Such capabilities delivered on a secured mobile platform can increase also the situational awareness of crisis management forces and intelligence. Any formation with mCOP support – Android API level 21 – secured platform, approved credentials may access crisis management information distributed within the secure operational network.

Fig. 8. mCOP TARV composite view rendering recognised crisis management situation supplemented with SA products and CMOKMT entities sharing threat status for monitored units, forces and locations (1 – location and data

quality status bar, 2 – elevation and tilt, 3 – compass and azimuth data, 4 – entity details within a tactical scenario, 5 – situational objects with evaluated state and threat data for monitored locations), 6 – value pointing parallel level of

handheld in relative to the earth. Source: own work.

The mCOP semantic visualizer application provides following main groups of functionalities:

- Rendering of fused, integrated operational and GIS data used for COP product development within the scope of a given unit (individual) with respect to data security level and required information scope.

- Unification of civil and military tactical (situational) symbology providing clear, readable and efficient tool for reflecting current, rapidly changing tactical picture supplemented with intelligence reports and threat level data.

- Data acquisition mechanisms recording operational officer (or equipment) location, movement characteristics and health state. Application simplifies the process of reconnaissance data reporting, introducing new means for locating, describing and documenting objects in interest.

1

2

3

4

5

6


48

- Operational data integration and unification mechanisms fusing crisis management, military and intelligence data and utilising ontology mapping features. The mCOP application provides set of Android components (services, activities) forming a

decision support tool integrated into tactical communication network and tCOP command post server application. The mobile application has been closely integrated with handheld’s OS (smartphone and tablet) to deliver COP products for lower level commanders and group leaders. mCOP delivers military and civilian operational pictures with standardized GIS data sources and operational symbology standards (STD2525, APP6A, crisis management). mCOP supports rendering of units, equipment and installations, supplemented with their affiliation, type, status, composition, potentials. Furthermore, detailed combat information contains not only static data, but also description of unit’s behaviour in time (tasks). The application delivers also simple GIS based calculators for distance and area measurements, terrain profile evaluation with.

All COP and SA use tactical symbols (graphical representation) of operational scenario, which provides consistent, effective and clear way of expressing static and dynamic properties of objects within the battlespace. COP image (Fig. 3), (Fig. 4), (Fig. 7) has been supplemented and extended to support intelligence data (provided in form of reports and data structures).

Fig. 9. TARV composite view rendered for border guard operations and crisis management sharing situational

elements, POI. Source: own work.

In order to achieve such efficiency CIV&MIL operational specialist need to be educated and trained in the area of:

- Expressing static planned and current situation (monitored locations) with regard to friendly, hostile, neutral and unknown objects (installations, units, equipment), fusing all battlespace dimensions (land, air, maritime) and reports;

- Supplementing scenario data with dynamic information on planned and current tasks and activities of selected elements formulating tactical and operational responsibilities and obligations;

- Reading and interpreting scenario static and dynamic data describing all battlespace dimensions and inferring possible results of combat, (e.g. estimating attrition rates, human and equipment loses, etc.);

- Validating tactical and operational picture elements – evaluating their threat factors based on the doctrinal unit composition and actual potential of components (equipment, personnel). Building an augmented reality view (Fig. 9) required utilization of the sensor, battlespace and GPS

data combined with camera display. Construction of augmented reality view utilizes location of battlespace elements, location and inertial sensors (HALL et al., 2001). The algorithm utilizes iteratively executed subroutines aimed at evaluation of observer’s location and it’s viewing azimuth (field of projection) and locations of battlespace objects (CHMIELEWSKI, 2017). In order to present only valuable


49

battlespace information a set of filtering rules has been defined associated with the distance to units, their type, movement and potential characteristics.

Research environment features for combat situation recognition and reconnaissance

The demonstration environment (CHMIELEWSKI, 2017) has been divided into three components: preparation and implementation (of required tools and prototypes), integration testing and calibration of tools (validation and verification phase), final demonstration with evaluation of operational adequacy. The research team and demonstration attendees, concluded prepared scenarios and federation of tools, formulating the key environment demonstration findings:

- effective information exchange between all CMO actors, requires introduction of joint civil/military reporting procedures, interoperable tools and technical standards, in SOA environment supported by web-based technologies;

- promote and request “need to share” data schemes especially, while cooperating on human intelligence and analyse crisis management data;

- knowledge management enables CMO-wide semantic integration of heterogeneous datasources thus requiring standards for representing joint “situational pictures”;

- information assurance requires existence of EU federated CIS network and use of cross-domain multi-level security services and tools, application of “need to share” principle with mutual trust in exchanging information within CMO-wide Communities of Interest (CoI) (UK MoD, 2005) and new information security management principle concentrated on protection of information and not the CIS domains. In response conducted research and gathered experiences helped to elaborate set of

recommendations for further research and NEC strategy applications: - standardization of reporting procedures, tools and technical standards enabling information

exchange between civil and military parties should be initiated; - application of SOA and use of open communication standards in order to increase the

interoperability potential. On lower level of command, that uses wireless communications, this could however decrease the information delivery success rate. This effect should be further investigated;

- use of EU federation core CIS in CMO with cross-domain multilevel security services and tools enabling trusted information sharing within Communities of Interest and intensive development of these new solutions;

- promote semantic technologies and reasoning mechanisms for hidden information retrieval and automatic threat level evaluation, allowing management of big volumes of data and provide automated expert support;

- promote formulation of cyber defence activities, procedures to develop cyber situational awareness tools applied in cyberspace and possible hybrid conflict monitoring. Planning and conducting EU-led Civil-Military Operation requires extensive information exchange

between all actors involved (civil, military, international, governmental, non-governmental, private, etc.) leading to coordination, cooperation and collaboration and creation of the Shared Situational Awareness.

Conclusions and further extensions

Presented algorithms deliver quantitative methods for combat entity evaluation to support decision makers and individual soldiers in the development of their situation awareness, which incorporates their understanding of the environment and combat situation. Demonstrated features of mCOP application provide the information infrastructure for military and civilian crisis operation support, delivered as a handheld tool integrated with tactical network. Developed algorithms and software utilize mission critical elements rendering to present an updated image of surrounding units and combat elements. A designed novel approach to utilise TARV has proven efficient tool to visualise operational picture and assist combat personnel during a mission by decreasing significantly required time for topographical and tactical orientation – key combat (decision maker) responsibilities. The architecture of the mCOP environment have been iteratively verified and validated in several possible communication configurations utilizing both WI-FI-based tactical networks as well as mobile GSM-LTE solutions. The application of augmented reality tactical view resulted in the decrease of delays in elementary SA tactical action thus increasing the decision making efficiency (only in exercise scenarios). This has been recorded in the military operations as well as in crisis operations (border guards). Based on gathered exercise data the tactical and topographical orientation (key activities during combat), have been significantly shortened. This features provides faster response rates for the operator. The results demonstrate that time required to perform selected tasks decreased by almost 46%. Moreover, the reconnaissance module


50

achieved almost 75% better accuracy of asset location evaluation. The simplicity of the reporting functionality supports also usability and accessibility of mechanisms. Recorded trials involved tasks performed outdoors based on simulated data transmitted into mCOP environment. The application operators Military University of Technology officer cadets, were appointed to perform tactical and topographical actions based on the projected dynamically changed situation. Each mCOP operator was monitored by auditing observer assisted with automatic time reporting for SA related activity time measurement. Presented capabilities and properties of constructed software tools demonstrate the applicability of mobile software for operational forces but most of all territorial defence formations. mCOP platform utilizes Android platform and any commercially available smartphone can be prepared for its deployment after fulfilling all security requirements and configuration procedures. The application of intelligence report fusion and ontology-based reasoning capabilities provides functional support for Network Enabled Capabilities tool implementations. As an outcome implementation of the mCOP platform usage results in the increase of the decision making efficiency while providing configurable and adjustable language for classification rule definitions. In conclusion the developed ontology based GIS tools can be utilised in areas where the domain descriptions provide reasoning capabilities and where the operational specialists require dynamic extensions of rules used in inference – asset classification, battlespace object’s risk assessment, etc.

Acknowledgements: This work was partially supported by research projects EU Network Enabled Capability Demonstration (EU NEC Demo) “Shared Situational Awareness in EU-led Crisis Management Operation (CMO)” awarded by EDA PT NEC and project DOBR/0023/R/ID3/2013/03 and supported by the National Center For Research and Development mCOP.

References

CEN, 2009, Disaster and emergency management – Shared situation awareness – Part1: Message structure. CACI International, 2009. Dealing With Today’s Asymmetric Threat to U.S. and Global Security,

https://www.asymmetricthreat.net/pdf/symposium2_report.pdf (access 01.05.2018). CHMIELEWSKI, M. 2009. Ontology Applications for Achieving Situation Awareness in Military Decision

Support Systems, ICCCI 2009, LNCS, 5796: 528-539, DOI: 10.1007/978-3-642-04441-0_46. CHMIELEWSKI, M. 2014. Development of Operational Picture in DSS Using Distributed SOA Based

Environment, Tactical Networks and Handhelds; Information Systems Architecture and Technology: Selected Aspects of Communication and Computational Systems, Wrocław.

CHMIELEWSKI, M., GAŁKA, A. 2010. Semantic Battlespace Data Mapping Using Tactical Symbology, Advances in Intelligent Information and Database Systems. Studies in Computational Intelligence, 283: 157–168.

CHMIELEWSKI, M., KUKIELKA, M. 2016. Applications of RFID technology in dismounted soldier solution systems – study of mCOP system capabilities, CSCC 2016, MATEC Web of Conferences 7 04014, DOI: 10.1051/matecconf/20167604014.

CHMIELEWSKI, M., 2008, Data fusion based on ontology model for common operational picture using OpenMap and Jena semantic framework, Proceedings of the Military Communications and Information Systems Conference, Cracow, 2008

CHMIELEWSKI, M. 2012. Ontology-based indirect association assessment method using graph and logic reasoning techniques, PhD Thesis, Military University of Technology, Warsaw.

CHMIELEWSKI, M., 2017. Situation Awareness tools supporting soldiers and low level commanders in land operations. Application of GIS and augmented reality mechanisms, Geographic Information Systems Conference and Exhibition GIS ODYSSEY 2017, Trento, Italy.

CHMIELEWSKI, M., SAPIEJEWSKI, K., Augmented reality mechanisms in mobile decision support systems supporting combat forces and terrain calculations-a case study. In: Proceedings of the Geographic Information Systems Conference and Exhibition GIS ODYSSEY 2018.

ENDSLEY, M.R., GARLAND, D.J. 2000. Situation Awareness, Analysis and Measurement, CRC Press. HALL, D.L., LLINAS, J. 2001. Handbook of Multisensor Data Fusion, Boca Raton: CRC Press. GLIMM, B. et al., 2014. HermiT: an OWL 2 reasoner. Journal of Automated Reasoning, 53.3: 245-269. HORRIDGE, M.; BECHHOFER, S. 2011. The owl api: A java api for owl ontologies. Semantic Web, 2.1: 11-21. MUT (Military University of Technology), 2014. EU NEC DEMO project summary wiki page – presentation of

mCOP and tCOP capabilities, http://uranus.wat.edu.pl:8808/wiki/index.php/EU_NEC_DEMO (access: 27.08.2019).

NOGALSKI, D., NAJGEBAUER, A., 2011, Semantic mediation of NATO C2 systems based on JC3IEDM and NFFI ontologies. In: RTO Symposium “Semantic and Domain based Interoperability” RTO-MP-IST-101. Oslo.


51

NC3A (NATO Consultation, Command and Control Agency), 2006. NATO Common Operational Picture (NCOP) – NC3A.

SIRIN, E. et al., 2007. Pellet: A practical owl-dl reasoner. Web Semantics: science, services and agents on the World Wide Web, 5.2: 51-53.

STANAG, NSA. 5527, 2008: NATO Friendly Force Information (NFFI). NATO Standardization Agency. UK MoD (UK Ministry of Defence), 2005. Network Enabled Capability, Joint Service Publication 777,

http://barrington.cranfield.ac.uk/resources/papers/JSP777 (access: 01.04.2017). US DoD (US Department of Defense), 2009. Department of Defense Dictionary of Military and Associated

Terms. Joint Publication 1-02, US Department of Defense, 17 March 2009. TAYLOR, J.G. 1980. Lanchester Type Models of Warfare vol. I. Operations Research, Naval Postgraduate

School Monterey http://www.dtic.mil/dtic/tr/fulltext/u2/a090842.pdf (access 16.01.2018).

methods and algorithms supporting combat …

Documents