computational science and engineeringin the ﬁeld of human grasping, there exists extensive...

Computational Science and Engineering(International Masterrsquos Program)

Technische Universitat Munchen

Masterrsquos Thesis

Formal specification of the semantics ofgrasping using the Web Ontology Language

(OWL) and integration into a robotprogramming GUI

Benjamin Aaron Degenhart



Masterrsquos Thesis

Formal specification of the semantics of grasping usingthe Web Ontology Language (OWL) and integration into

a robot programming GUI

Author Benjamin Aaron DegenhartExaminer Prof Dr-Ing habil Alois Christian KnollAssistant advisor Alexander Perzylo fortiss GmbHSubmission Date March 15 2019

I hereby declare that this thesis is entirely the result of my own work except where other-wise indicated I have only used the resources given in the list of references

March 15 2019 Benjamin Aaron Degenhart

Acknowledgments

I want to thank my advisor Alexander Perzylo from fortiss for his competent helpful andfriendly guidance throughout my entire Masterrsquos Thesis processAlso I want to thank Stefan Profanter and Ingmar Kessler both from fortiss as well foroffering their expertise on certain topicsFuthermore thanks to Anabele-Linda Pardi for proofreading and giving general advice

vii

Abstract

In this thesis the semantics of grasping get developed first conceptually then modelledontologically and finally implemented in fortissrsquo Robot Instruction Framework The ad-vantage of doing this on a symbolic level compared to classical grasp planning approachesis that it saves a lot of computational effort and allows semantically meaningful reasoningabout grasp modes By logically guaranteeing incompabilities the remaining search spacefor subsymbolic methods shrinks considerably The basic idea is to compare the graspingcapabilities of a gripper with the geometric conditions of the object to grasp in order tofind matching candidates - the applicable grasp modes Basic sanity checks like valueswithin ranges are performed and feasiblity scores per grasp mode allow ranking the re-sulting list of matches

ix

Legend

If not stated otherwise all figures and screenshots were created by me

Variable names or file extension are displayed as such

Bold and italic will occasionally be used to highlight keywords

In the digital version of this document URLs in footnotes and references to other sectionsof the document are active links

x

Contents

Acknowledgements

vii

Abstract

ix

1 Introduction

111 Motivation and Goal

112 Scope Of Tasks And Structure Of This Thesis

2

2 Related Work

3

3 Theory

531 Theoretical Basis

532 What Are Ontologies

533 Inference Enriches The Ontology And Yields Insights

634 Existing Ontologies From fortiss

7341 OntoBREP Reasoning About CAD Data

7342 Semantic Process Description

9

4 Developing The Semantics Of Grasping

1141 Starting With A Taxonomy

1142 The Concept To Find Valid Grasp Modes

1143 What The Gripper Offers Grasping Capabilities

1244 What The Object Offers Geometric Conditions

1345 Getting Applicable Grasp Modes

14

5 Implementation

1951 Part 1 Work On The Knowledge Base

19511 Additions To The Existing Ontologies

21512 SPARQL Queries Working With The Modelled Ontologies

25513 GraphDB Plugin To Compute Overlapping Area

3452 Part 2 Integration Into The Existing Software

37521 fortissrsquo Robot Instruction Framework

37522 Parsing OntoBREP In The GUI

38523 Two New GUI Elements For The Semantics Of Grasping

39

6 Discussion Conclusion And Future Work

49

Appendix

55

Bibliography

63

xi

1 Introduction

The Robotics division at fortiss has been involved in the project SMErobotics1

from 2012to 2016 together with 24 other institutions companies and universities It is funded by theEuropean Unions Seventh Framework Program under grant agreement 2877872

underthe full title of ldquothe European Robotics Initiative for Strengthening the Competitiveness ofSMEs in Manufacturing by integrating aspects of cognitive systemsrdquo

The main motivation is to make the use of robots more affordable for small and medium-sized enterprises (SMEs) see the project website or this article [13

] for more informationincluding research on the considerable economic potential of this technology and how thisapproach can substantially reduce the level of robotics expertise required A big part of for-tissrsquo contribution is to work on a semantic description language (ontology) for robot-basedindustrial automation This research continues throughout other projects with multipleindustry- and research partners currently in the project Data Backbone3

(from Jan 2018 toMar 2020) which is about ldquothe data infrastructure for continuous production without sys-tem interruptionsrdquo and is funded by the Bavarian Ministry of Economic Affairs and MediaEnergy and Technology with project support by Zentrum DigitalisierungBayern Amongother proof of concept implementations fortiss developed a software with a web-basedgraphical user interface (GUI) that allows programming robots via an intuitive interfacethat uses semantic reasoning in the background The Software can either simulate an as-sembly process or control a real robot to do so after a user created the process Detailsof its implementation can be found in section 521

The focus is on allowing non-expertsaccess to robot programming by means of intuitive interfaces


The aspect of this software system that I worked with in the context of this thesis is thegrasp planning In the demo-workflows (using the above mentioned GUI) that were de-veloped so far all of the robot grasping actions were hardcoded This was because morecentral parts of the system were prioritized when it came to developing semantics andreasoning capabilities for them Hardcoded in this case means that predefined numericalvalues for the grasping poses (relative coordinates and angles) are prepared for all possi-ble assembly-tasks of objects within a small demo-set of objects that exist in both physicalform as well as in virtual form in a workcell that an assembly process gets deployed to

1httpwwwsmeroboticsorg

2httpscordiseuropaeuprojectrcn101283factsheeten

3httpswwwfortissorgenresearchprojectsdata-backbone

1

1 Introduction

However all the spatial geometrical (see section 341

) and procedural information (seesection 342

) required to define grasping procedures already resides in the ontology There-fore it makes perfect sense to keep expanding the ldquovocabularyrdquo that this semantic descrip-tion language can ldquospeakrdquo One way to achieve this is by developing the semantics ofgrasping

Therefore is the goal of this thesis to specify the semantics of grasping to build them intofortissrsquo ontology and to develop an addition to the GUI in the form of a grasp planningtool that integrates into the existing assembly-process workflow As a result of this workno more hardcoded values will be necessary and a generic way to handle grasp planningis enabled

This work builds on the research fortiss has done over the past few years in collaborationwith Prof Knoll from the Technical University of Munich about using ontologies in robot-supported assembly


My task was to develop a formal specification of the semantics of grasping using the WebOntology Language (OWL) and as proof of concept integrate that into fortissrsquo robot pro-gramming GUI

The next chapter 2

will give some insights into related work regarding grasping in thecontext of cognitive robotics Then chapter 3

will introduce the theoretical building blocksnecessary for understanding the next part That is in chapter 4

the development of thesemantics of grasping derived from a grasping taxonomy of industrial robots In the firstpart of chapter 5

these semantics will be expressed through ontological modelling to aug-ment the existing ontologies The second part of chapter 5

is then to integrate the semanticsof grasping into the existing software system at fortiss and to develop new elements forthe workflow in the GUI that allow users to make use of the new functionality

The repeating pattern throughout the chapters will be to look at what the gripper has tooffer what the object has to offer how the two come together to form actionable graspmodes and then to check this for valid values and optionally rank them by feasibilityThis pattern stays the same throughout the phases of conceptual explanation ontologicalmodelling required queries and implementation

2

2 Related Work

In the field of human grasping there exists extensive research Probably the most notableone is the Grasp Taxonomy by Feix et al 2015 [6

] It had the goal to determine the largestset of different grasp types that can be found in literature Overall 33 different grasp typeswere identified and arranged into the ldquoGRASPrdquo taxonomy A comparison sheet can befound on the project website1

They were building on their own work from 2009 [7

] andamong others they were exploring Napierrsquos work from 1956 [11

] Which was an earlyattempt to classify human grasps by their needs for precision or power Prof Kragic whowas also involved in the GRASP taxonomy worked together with others on task-basedrobot grasp planning [20

] They developed a probabilistic framework for the representa-tion and modelling of robot-grasping tasks This is the contrary direction though that isbeing taken in this thesis - the driving principle here is to cover as much as possible on apurely symbolic level by means of ontological reasoningRegarding ontologies for robotics the ldquoIEEE Standard Ontology for Robotics and Au-tomationrdquo by Schlenoff et al 2012 [18

] is a crucial contribution to the field It came outof a working group whorsquos goal it was ldquoto develop a standard ontology and associatedmethodology for knowledge representation and reasoning in robotics and automation to-gether with the representation of concepts in an initial set of application domainsrdquo [8

]then builds on this Core Ontology for Robotics and Automation (CORA) and introduces aset of ontologies that complement it with notions such as industrial design and position-ing Furthermore it contained some updates to CORA aiming to improve the ontologicallyrepresentations of autonomy and of robot partsAnother notable development is ldquocloud roboticsrdquo an emerging field in the intersection ofrobotics and cloud computing It gives robots access to greater processing power and stor-age capacity than they could host themselves Goldberg and Kehoe published a surveyof related work in 2013 [9

] Beetz et al introduced openEASE ldquoa web-based knowledgeservice providing robot and human activity datardquo2

in 2016 [2

] Later they have shownuse cases for it namely [3

] talks about using cloud robotics to provide a computationally-expensive ldquomental simulation servicerdquo that robotic agents can use to predict consequencesof their actions right before the execution of everyday manipulation tasks Later in [4

]they develop the ldquoopenEASE robot interfacerdquo and a framework for robots to reason ona central cloud application that has encyclopedic knowledge in the form of ontologies aswell as execution logs from other robots Their use case was an exchange of knowledge be-tween different environments regarding the opening trajectories of a fridge using episodicmemoriesThe ldquoRobotics and Semantic Systems grouprdquo3

around Prof Malec at Lund University isworking on applying semantic techniques in robotics and automation in order to facilitate

1httpgraspxiefnet

2httpwwwopen-easeorg

3httprsscslthse

3

2 Related Work

high-level interfacing and for improving human-robot interaction They call the combi-nation of semantic technologies with industrial automation ldquosemantic systemsrdquo In [12

]Malec et al present ldquoa knowledge integration framework for roboticsrdquo with the goal toto represent store adapt and distribute knowledge across engineering platforms Theymake use of the Automation Markup Language AutomationML4

as well as RDF triplesand SPARQL endpoints In [21

] Malec et al present an approach to support seman-tic capture during kinesthetic teaching (also called ldquomanual guidancerdquo) of collaborativeindustrial robots The system captures the users utterances or typings during the taskdemonstration In that way the user can create skills and reference systems from scratchby extending the vocabulary of actions and objects right as they are used in context Thisapproach lets users bootstrap system knowledge and is a promising answer to the fact thatmost industrial robot systems being deployed today contain no domain knowledge andas such canrsquot aid their operators in setup and defining use cases In terms of robots learn-ing from demonstrations Argall et al [1

] did a comprehensive survey in 2009 of robotLearning from Demonstration (LfD)Another take on the topic of teaching industrial robots is explained in [17

] coming fromthe chair of Cognitive Systems led by Prof Cheng at the Technical University of Mu-nich Among other things they work on a novel semantic-based method to obtain generalrecognition models through teaching by demonstration Their latest journal article [5

] onthe topic is done together with Prof Beetz whorsquos work is cited above The term ldquopurposivelearningrdquo is used to describe the notion of robots to learn from observation The semantic-based techniques they developed focus on extracting meaningful intentions from humanbehaviors and secondly on the ability to transfer past experiences into new domains

4httpswwwautomationmlorg

4

3 Theory


Building systems that allow the programming of robots with much less expertise requiresshifting considerable parts of this expertise towards the system Things that are obviousenough to be automatically derived should not bother the operator Anomalies errors andthe need to handle uncertainties are inevitable yet many categories can be accounted forautomatically if the system has an ldquounderstandingrdquo of what is going onTo achieve this level of understanding a programming paradigm is necessary in whichcommon-sense knowledge as well as the robotics- and domain-specific knowledge is mod-elled explicitly using semantics via ontologies In this way the system can understand in-terpret and reason about itself The semantic description language developed by fortiss isbased on the Web Ontology Language1

(OWL) which is a natural choice as many ontolo-gies have already been developed as OWL-ontologies The QUDT ontology2

for instancestandardizes data types and units and is one of many generic ontolgoies about common-sense knowledge Even though there exist basic ontologies for the robotics domain [18

] therequirements of SMEs and their specific domains are not adequately considered There-fore fortiss chose to augment base ontologies with domain-specific knowledge in order tocreate cognitive robotic systems that are specialized for a domain


The term ontology is used in philosophy to describe the study of being

rdquoIt studies concepts that directly relate to being in particular becoming existencereality as well as the basic categories of being and their relations [] ontology oftendeals with questions concerning what entities exist or may be said to exist and howsuch entities may be grouped related within a hierarchy and subdivided according tosimilarities and differencesrdquo3

Itrsquos meaning in computer- and information science is similar

rdquoAn ontology encompasses a representation formal naming and definition of thecategories properties and relations between the concepts data and entities that sub-stantiate one many or all domains Every field creates ontologies to limit complexity

1httpswwww3orgTRowl2-primer

2httpwwwqudtorg

3httpsenwikipediaorgwikiOntology

5

3 Theory

and organize information into data and knowledge As new ontologies are made theiruse hopefully improves problem solving within that domainrdquo4

There are many terms to get familiar with when working with ontologies Firstly it is ageneral concept and ontologies could just as well be written with pen and paper How-ever once a formal language is used to describe an ontology such as OWL 25

in our caseit becomes very useful OWL is an extension of RDFS which in itself builds on top ofRDF the Resource Description Framework RDF is organized in triples each RDF triplehas a subject a predicate and an object The predicate is also called property sometimesand connects the subject and the object When modelling an ontology one creates classeswithin a hierarchy of classes Thing is the root-class of all others From a class arbitrar-ily many individuals can be instantiated These classes need to have concrete values forobject- and data properties that their parent class restricted them to Object properties linkto other individuals whereas data properties link to literals like a string or a double valueProperties are also organized in a hierarchy and can have characters like Symmetric orFunctional They can also be declared as the inverse of other properties An example fromthe OntoBREP ontology (more in section 341

) is the object property represents that isthe inverse of representedBy So when a face is representedBy a wire automaticallywithout explicitly stating so the wire represents the face And this relation can be activelyused when doing inference via SPARQL queries (more in section 512

) for instance

Ontologies can be perfectly stored in simple text-files usually with the extension owlHowever in most use cases they need to be ldquoaliverdquo in a production environment Adatabase needs to store them and answer to queriesIn general it can be said that the more knowledge is present in a cyber-physical system(CPS) in the form of ontologies the more synergies arise from the the possibility of auto-matically linking and combining knowledge sources through a language based on a logicalformalism Among multiple use cases this can be utilized for developing intuitive inter-faces for end-users who can be relieved of having to know all technical and programmaticdetails and rely on using the semantic language tailored to the task The complexity be-hind simple instructions is being handled by the reasoning system in connection to theknowledge base Instructions are being multiplied out to be used by the concrete hard-ware all variables get parameterised and additional statements get automatically inferredif applicable


Because OWL 2 is based on a logical formalism inferring becomes possible

rdquoInference is the derivation of new knowledge from existing knowledge and axiomsIn an RDF database such as GraphDB inference is used for deducing further knowl-edge based on existing RDF data and a formal set of inference rulesrdquo6

4httpsenwikipediaorgwikiOntology_(information_science)

5httpswwww3orgTRowl2-overview

6httpgraphdbontotextcomfreedevhubinferencehtml

6


In the context of this work the default ruleset RDFS-Plus (Optimized) was used Anoption would have been to extend the OWL2-RL-one with custom rules in a pie-fileA reasoner is the tool used for inference In Protege see section 51

ldquoPelletrdquo and ldquoHermitrdquoare usually used whereas GraphDB see section 51

has its ownReasoners are good at ldquounderstandingrdquo transitive properties across large graphs of con-nections The simplest example of a transitive property is this if a gt b and b gt c then itcan be inferred that a gt c

Categories Of Computation Symbolic vs Subsymbolic

As will be seen later the distinction between symbolic and subsymbolic levels is crucial incategorizing types of computations It means drawing a line between logical symbolic op-erations that require no arithmethic and all other mathematical operations where numer-ical values are involved When working with ontologies this distinction nicely describeseverything that can happen ldquocleanlyrdquo by reasoning in the ontology in comparison to morecomputationally intensive operations involving ldquonumber crunchingrdquo The classical graspplanning for instance involves random sampling of millions of options to find useful onesIn contrast to that a reasoning setup like described in this thesis can find a useful set ofoptions with very little computational effort in comparison However even then there willbe a last step of finding the concrete pose for a gripper that has to be computed subsym-bolically So there is a balance to be found between these two domains


fortiss has developed many ontologies so far for various projects and purposes Two willbe highlighted in the following as my work builds on them


A prerequisite to reason about geometrical objects is to describe spatial information se-mantically Perzylo et al 2015 [15

] present an approach for leveraging Computer-aideddesign (CAD) data to a semantic level by using OWL to define boundary representations(BREP) of objects In this way each object is defined as a compound of topological entitieswhich are represented by geometric entities This ontology OntoBREP is available on-line7

The advantage over polygon-based geometry models is that the exact mathematicalrepresentation are stored From these representations approximations (like meshing thegeometry using triangulation) can be generated at the level of detail most suitable to anapplicationrsquos requirements In [15

] it is stated that

This separation of represented data and use-case dependent calculations is as im-portant as the separation of data and the software that was used to generate it Theseparadigms allow flexible sharing and re-use of information

7httpsgithubcomOntoBREP

7

3 Theory

From the perspective of Product Process Resource (PPR)-based modelling OntoBREP canbe seen as an ldquoenabling technologyrdquo that serves both the product-side (both for direct de-scription of CAD objects as well as basis for further object models) as well as the resource-side by describing the specifics of resources like the fingers of a gripper involved in agrasping capability for instance

Figure 31

shows an overview of the BREP structure as modelled in the OntoBREP-ontologyListing 1

in the appendix shows excerpts of a cubeowl file in OntoBREP format (whichis expressed in XML) How to use this format to retrieve information and infer insightswill become clearer in chapter 4

Figure taken from [15

]Topological entities in blue geometric entities in red Class hierarchy in Protege

Figure 31


] overview of semanticmodels and interrelations


] industrial use case ofassembling a gearbox in three steps

Figure 32

8



Perzylo et al 2016 [14

] introduced a novel robot programming paradigm namely seman-tic process descriptions for small lot production The programming happens at object levelabstracted away from raw coordinates and basic commands Tasks get only parameter-ized and possibly automatically augmented with further inferred tasks when being de-ployed on a concrete workcell Constraints between geometric entities as well as individ-ual assembly steps get captured on an ontological level This reduces the human expertiserequired to define production processes dramatically Statistics on estimated time-savingsby using this method instead of traditional programming can be found in [13

]Figure 32

shows two illustrations taken from [14

] The workflow for the user in the fron-tend will be showcased in section 5211

9


In this chapter the development of the semantics of grasping will be explored from itsstarting point as taxonomy to the conceptual building blocks necessary to semanticallyfully describe grasp modes that a gripper can perform on an object


The difference between taxonomy and ontology is that taxonomies are more similar to en-riched classifications and mainly serve as information to humans An ontology howeverhas more information about the behavior of the entities and the relationships betweenthem Furthermore it can be processed by machines to do operations like checking forlogical consistency using a semantic reasoner and run inference for drawing automaticconclusions

To get an idea about gripper types on the market their grasp modes and relevant parame-ters research into industrial grasping was conducted via the product specifications of var-ious manufacturers see figure 41

The purpose was not to condense this into a conclusiveand comprehensive taxonomy but merely to inform a meaningful approach to developinga grasping ontology in the next step


From this taxonomy-overview the overall concept took form It become clear that thegripper and the object both have to ldquoofferrdquo something to end up with a list of options thatare ideally even ranked by usefulness The solution is to semantically enrich both partiesin such a way that matches can be found between them on a symbolic level From look-ing at the grasping taxonomy it becomes clear that a gripper can be described as havingcapabilities when it comes to grasping A parallel gripper for instance has the ability tograb an object by approaching it with its two fingers in parallel from outside Howeverdepening on the shape of the fingers the same gripper could potentially also grab objectsfrom inside if there is space to insert the closed fingers and open them until they clamp theobject from inside This gripper can then be said to offer two ldquograsping capabilitiesrdquo Buthow does the gripper know if one or more of its capabilities can be used on a given objectThatrsquos where the offer from the object comes in To be properly grasped in parallel fromoutside the object ideally has two parallel planar faces that the gripper can close in onThese ldquofeaturesrdquo from the object can be called ldquogeometric conditionsrdquo An object can havearbitrarily many of them These two concepts can now be brought together by a simpleldquorequiresrdquo relation A grasping capability from the gripper requires a certain geometric

11


Figure 41 Sketch of a robotic grasping taxonomy

condition to be present in the object otherwise it can not be performed

This is the concept given a gripper annotated with its grasping capabilities (ie parallelfrom outside) and given an object that is annotated with itrsquos geometric conditions (ie twoparallel planes facing outwards) matches can be deduced where each represent a possi-ble grasp mode Note that throughout this thesis both grasping capability as well as graspmode will be used The first is meant more as the passive capability of the gripper whereasthe second is the result of a match with an objects geometric condition and can be activelyacted upon

Figure 42

illustrates this paradigmatically The conceptual elements will be discussed inmore detail throughout this chapter Itrsquos implementation will be presented in the followingchapter 5


We decided to narrow down the development of grasping semantics in the context of thisthesis to a small set of 5 meaningful grasp types and parameters This can be taken asbasis for extending it further see section 6

The parameters were restricted to those di-rectly relating to a grasp mode General ones like the maximum payload were ignoredThe following five grasp modes were chosen including the relevant parameters from thegripperrsquos side

12


Figure 42 Examples of matches

bull Planar parallel from inside grasp the range fitting between the planar outer surfaces of the grippers fingers

bull Planar parallel from outside grasp the range fitting between the planar inner surfaces of the grippers fingers

bull Cylindrical parallel from inside grasp the range the grippers cylindrical halves can get in and then span up

bull V-shaped notch from outside grasp the range of radii this grippers notch-halves can close in on

bull Planar surface vacuum suction grasp the surface area of the suction cup when engaged

With ldquoV-shaped notchrdquo is meant that itrsquos not ellipses cut out on the grippers finger thatwould only allow a form-fitting grasp Instead it is two V-shaped surfaces that allow arange of cylinder radii to fit in The gripper used in the implementation part 3D-printedby fortiss has this feature thatrsquos why it is included here More about that in section 511

Figure 43

shows three of these grasp modes in action The screenshots are taken insimulation-mode in the GUI-frontend of fortissrsquo Robot Instruction Framework which willbe presented in more detail in section 521

Now that grasp modes are defined in this way a strategy is needed about how to assessthe object that is to be grasped How can the system filter for valid grasp modes whenconfronted with a specific object Our answer to this is to annotate objects with theirldquoGeometric Conditionsrdquo


Geometric conditions can be thought of as features that an object exhibits When humansare planning grasping actions we scan the object for features that we can utilize Mostlywithout giving it much conscious thought The handle of a cup for instance calls for acertain way to hold the cup Just like the cylindrical outwards facing surface of a glas callsfor a certain way

13


Parallel from outside Cylindrical from inside V-shaped notch from outside

Figure 43 Examples of grasp modes

In the following list the grasp modes as shown above in section 43

are extended bythe types of geometric conditions and their parameters that objects need to possess to bematched with the respective grasping capability

bull Planar parallel from insideoutside graspTwo parallel planar faces inwardsoutwards Two planar parallel faces must bepresent that are both oriented either inwards or outwards The orthogonal distancebetween them is required for comparison with the grippers range Also the overlap-ping area of the two planes (upon projection into the same plane) must be at least gt 0for a proper grasp to be applied

bull Cylindrical parallel from inside grasp V-shaped notch from outside graspCylindrical face inwardsoutwards An inward or outward facing cylindrical facemust be present Its radius is required for comparison to the grippers range

bull Planar surface vacuum suction graspPlanar face outwards A planar surface must be present Its surface area must bebig enough to place the grippers suction cup

Another conceptual element that was introduced alongside the annotation of geometricconditions is the ability to flag faces as noGrasp This concept means that the faces are notsuitable for grasping To stick with the analogy of human grasping some faces of objectsshould better not be touched like the hot body of a pan Or maybe some fresh paint thatis drying during the assembly


Each grasping capability requires a specific geometric condition to be applicable For morecomplicated abilities in the future multiple required conditions will likely be necessary

Figure 44

shows three matches The screenshots are taken in the grasp planner a new stepadded to the workflow in the GUI described in detail in section 5232

14


Planar parallel from outside grasp matched with its required geometric condition oftwo parallel planar faces outwards (2 screenshots to show all 4 involved faces)

V-shaped notch from outside grasp matchedwith Cylindrical face outwards

Cylindrical parallel from inside grasp matchedwith Cylindrical face inwards


If a match is found based on the geometric condition a value check must be passed beforecounting as suitable match

Value Checking To Filter Out Invalid Matches

As discussed in the previous sections 43

and 44

a grasping capability might have a rangeof valid values that it requires values of geometric condition to be in All parallel graspmodes for instance have a minimum and a maximum defined by the gripper span Thedistance between parallel planes or the radius of the cylindrical face to be matched withhas to fit inside this range This first check is of boolean nature matches that have falsein any of the value checks are disregarded going forward The second round of checkshowever is of subsymbolic nature

15


Feasibility Scores To Rank Matches

The matches which passed the value check can now be inspected as to how good of acandidate they are An object that fits right into the middle of a grippers span might bescoring highest whereas one that requires an almost maximum opening scores lower Asanother example two parallel planes that have a high overlapping area are preferred overa pair with a low overlap as it would result in a weaker grasp Both the reasons for scoringhigh or low as well as the numerical method to compute these scores will vary dependingon the use case One could think of a metric related to how much time a grasp modecosts in relation to an object cooling off in an assembly process or a metric capturing howwell the coefficients of friction of the surfaces participating in a grasp mode fit together Abasic metric would also be assigning lower scores as the objectrsquos mass comes close to thegrippers maximum payloadThe modelling of this conceptual part must therefore be kept as generic as possible

Is A Grasp Mode Actually Doable - Subsymbolic Postprocessing

This part of the concept was not achieved in the implementation part of this thesis

After passing the value check 45

and getting a feasiblity score 45

assigned a match con-sists of a theoretically possible grasp mode that the gripper can conduct on the objectHowever orientation of objects within the workcell (the object might be resting on the sur-face one is trying to grasp it with) ldquothings in the wayrdquo and factors like gravity or frictionhave not been taken into account yet and might very well render this grasp mode unprac-tical or even completely useless for a real execution

Figure 45

shows a Planar parallel from outside grasp that matched correctly with its requiredgeometric condition of two parallel planar faces outwards However this grasp mode wouldbe impossible to execute

Figure 45 Example of a logically valid match that is physically impossible to execute(2 screenshots to show all 4 involved faces)

For a human these types of ldquomismatchesrdquo might be very obvious but some of the spatialpossibilities that make things unpractical are hard or impossible to capture on a logical

16


level It requires ordered by increasing computational workload subsymbolic postpro-cessing like constraint solving collision checking and potentially even physics simulationOnly then can the list of matches be narrowed down to the physically possible ones

17

5 Implementation

This chapter explains in detail the newly implemented developed semantics of grasping aspresented in the previous chapter 4

The first part section 51

covers the work done on theknowledge base meaning the ontological embodiment of the developed concepts - plusthe SPARQL queries operating on this ontology The second part 52

is about integratingthis into the frontend and develop two new GUI elements that users can use to plan graspmodes

A focus of the frontend-implementations was to keep things at a lightweight level ina sense that all the ldquointelligencerdquo should stem from the knowledge base and SPARQL-queries that operate on it The guiding principle here is to separate knowledge from pro-gram code as consequently as possible

The approach to additions to the knowledge base is more generous Here we encouragestoring more then absolutely necessary The reason being that it might well be useful inother reasoning situations later on and storing triples is comparatively ldquocheaprdquo Section5121

covers some examples of this purposeful redundancy GraphDB itself also followsthis philosophy by immediately materializing inferred triples

GraphDB supports inference out of the box and provides updates to inferred factsautomatically Facts change all the time and the amount of resources it would taketo manually manage updates or rerun the inferencing process would be overwhelmingwithout this capability This results in improved query speed data availability andaccurate analysis1


fortiss follows the Product Process Resource (PPR) modell and as such their core ontolo-gies cover these three areas As tools for modelling Protege and GraphDB were used

Modelling tool Protege

Protege2

was used as the tool for modelling As an example for its various useful editing-and development features figure 51

shows inferred object properties in yellow back-ground color that show up once a reasoner is started Pellet in this case


2httpsprotegestanfordedu

19

5 Implementation

Figure 51

(a) shows an individual of the Wire class Face1 is boundedBy Wire1 - andbecause bounds is marked as inverse of boundedBy this relation shows up here with-out being explicitly modelled Edge1 shows up twice because firstElement is a childproperty of contains If one would ask for all edges contained by this wire (eg with aSPARQL expression as such Wire1 cadcontains edge ) Edge1 would thereforealso show up in the results However only if the triple store hosting these triples sup-ports inference Figure 51

(b) shows an individual of the Point class This one containsno explicitly modelled object properties at all just the three coordinates as data proper-ties However since this point is referenced from triangles via contains and vertices viarepresentedBy these links show up inferred via their inverse object properties con-tainedBy and represents A query on a triple store that supports inference couldutilize these relations by querying ldquoupwards starting from the pointrdquo instead of findingthe way towards it by its parents

(a) A Wire individual (b) A Point individual

Figure 51 Reasoner inferring object properites in Protege

Graph Database GraphDB

GraphDB3

is a semantic graph database also called RDF triplestore Since it is used infortissrsquo robot instruction framework (see section 521

) it made sense to use it all along fordeveloping and testing SPARQL queries before embedding them in the frontend Protegealso supports SPARQL queries however there are occasional differences in how the re-spective reasoning-engines interpret patterns Figure 52

shows an exemplary query to getthe geometric condition individuals attached to the object CubeBlock This query canalso be found in the appendix 5

We had to use GraphDB version 880 or higher because only since then math functions arenatively supported More about that in section 512

Figure 53

shows another feature of GraphDBrsquos Workbench the ldquoVisual graphrdquo It allowsexpanding nodes manually from a starting node This can be useful to get an overview ofhow nodes are connected and if certain links have been established In this example onecan see parts of the OntoBREP graph of the object CubeBlock Furthermore a geometric

3httpgraphdbontotextcom

20


Figure 52 Screenshot of the SPARQL-tab in Workbench GraphDBrsquos web-based adminis-tration tool Showing 5 out of 12 rows in the results-table 12 because a cubehas 3 pairs of planar parallel outward faces each of which is represented by ageometric condition individual that has 4 parameters attached

condition individual in green can be seen linking two planar faces together that are par-allel to each other and facing outwards This example will be discussed further in section5121


The implementation work is bundled in one OWL-file even so some elements would bet-ter be located with the existing ontologies However during development this encapsu-lated one-file approach was useful as proof-of-concept Later on once decided where thepieces should go they will likely disseminate into a new ontology dedicated to graspingand into existing onesThis is a quick run-through of the modelling-concept before it is presented in more detailthroughout the remainder of this chapter Each GraspingCapability has the following

bull the object property requiresGeometricCondition links to a geometric condi-tion class

bull it contains one or more FaceLists which group together faces of a finger that are

21

5 Implementation

Figure 53 Screenshot of the Visual graph feature in GraphDBrsquos Workbench

involved in a particular grasp

bull an optional link needsValueCheck to members of the class ValueCheck - if presentthis demands data property values of a geometric condition to be in a specific rangefor a match to be valid eg if an object fits between the grippers finger

bull a ValueCheck might further contain a link to a FeasibilityCheck that assigns ascore to how well a value fits into a range based on the given evaluation method

5111 Modelling Geometric Conditions

Figure 54

shows the addition of GeometricCondition to the overall class hierarchyNote that the example of an individual would normally not appear materialized on mod-elling level like that It would be inserted within the graph database later on via queriesfrom the frontend It was added only to be shown here for explanatory purposesNote that having the radius as a property of the geometric condition individual seemsredundant because that value is already attached to the CylindricalSurface that rep-resents Face8 in that case One way to lift this information up to the level of the geometriccondition for more convenient reasoning is via a property chain see 33

However in thiscase redundant information is acceptable as other geometric conditions have properties

22


Positioning in the class hierarchy The TwoParallelPlanarFaces class

The CylindricalFace classAn individual of the CylindricalFaceInwards class

Figure 54 Ontological modelling of geometric conditions in Protege

that are not directly copied from their faces like distanceEuclidean between two par-allel planes And as ldquoprimitive shape matchingrdquo should be supported in the future (see6

) it could be the case that the radius is not retrievable as simple as looking two moresteps further down - the surface could be very complicated and the radius is the one of theprimitive cylindrical shape that was matched into it

5112 Modelling Grasping Capabilities

Figure 55

shows the addition of new GraspingCapabilitys to the existing ones Unlikegeometric condition individuals grasping capability individuals get defined manually be-fore importing the OWL-files into the graph database Eventually this is something themanufacturer of a gripper should provide alongside the overall specificationsldquoSubClass Of (Anonymous Ancestor)rdquo in Protege are subclass-statements this class inher-its from its parents contains some FaceList for instance is the general statement inheritedfrom GraspingCapability whereas contains exactly 2 FaceList is the concretizationinherited from ParallelGraspFaceList bundles faces that are involved in a grasp mode from one finger The ldquoV-shapednotch from outside grasprdquo for instance has 4 faces involved in total 2 on each finger - ascan be seen in figure 44

Each FaceList-individual keeps a link to itrsquos finger via the objectproperty containedBy The ValueCheck concept will be presented in the next sectionNote that the grasping capability individual does not contain a requiresGeometric-Condition-link That is because this value stays the same for all individuals of that class

23

5 Implementation

Positioning in the class hierarchyThe PlanarParallelFromOutsideGrasp

class

An individual of the PlanarParallelFromOutsideGrasp class

Figure 55 Ontological modelling of grasping capabilities in Protege

and therefore doesnrsquot have to be copied to its individuals The SPARQL query making thematches will retrieve this info from the individualrsquos class rather than the individual itself

5113 Modelling Value- And Feasibility Checks

As can be seen on the previous figure 55

a grasping capability might have an object prop-erty needsValueChecking that points to an individual of the class ValueCheck fromwhich for now only the subclass ValueWithinRange exists

Figure 56

shows both this class as well as an individual of it The individual shown isattached to the grasping capability Planar parallel from outside grasp and defines the rangethe grippers finger can have between them The valueToCheckProperty points in thiscase to the data property distanceEuclidean that the required geometric condition ofTwo parallel planar faces outwards possesses If this value is within the range the value checkpasses with trueThe ValueWithinRange-individual in figure 56

also points to a FeasibilityCheck viathe data property valueCheckFeasibilityScoreMethodIn this case itrsquos the RelativeDifferenceToMeanFeasibilityCheck Figure 57

showsboth its class and itrsquos individual containing an actual SPARQL query as string Notice the$REPLACE$-string in the query - the frontend will replace this with pairs of geometric con-dition individuals and their value within range individuals before sending it to the graphdatabase The returning table contains said pairs ordered by their feasiblity score as com-

24


Figure 56 Modelling of value check

puted by the method used This allows a ranking of the matches found for the gripper andthe object to grasp The full query can be found at 511

Figure 57 Modelling of feasibility score

When further developing this grasping-ontology it will make sense to replace anyURIfrom both valueToCheckProperty and valueCheckFeasibilityScoreMethodwithconcrete classes that have yet to be modelled


The Modelling described above needs to be queried to retrieve triples or insert new onesThis is done in GraphDB using the SPARQL 11 Query Language4

We do make our-selves dependent on Ontotextrsquos GraphDB5

as semantic graph database because of the useof the property sesamedirectType and the use of standard math functions6

(such asfpow(23)) provided by Ontotext Such dependencies on a specific product are unde-sirable but we consider them acceptable for the time beingNote that all geometry-related queries expect the CAD data to be present in the OntoBREPformat as explained in section 341

In the following important parts from the queries regarding the annotation of geometric

4httpswwww3orgTRsparql11-query


6httpgraphdbontotextcomdocumentationstandardusing-math-functions-with-sparqlhtml

25

5 Implementation

conditions matching them with grasping capabilities checking for values within rangesand assessing feasibility scores will be highlighted

5121 Annotate Objects with their Geometric Conditions

As mentioned at the beginning of this chapter the geometric conditions and with somemore effort also the grasping capabilities wouldnrsquot necessarily have to be stored explicitlyanyhwere It would be possible to derive them on the fly by huge queries that includeeverything required However it makes sense to store these explicitly before looking formatches To annotate a gripper with its grasping capabilities for instance could be seen asthe task of the manufacturer And having the geometric conditions of an object availablemight be useful later on for other purposes as it enriches the available knowledge on CADobjects When identifying the geometric condition of parallel planar faces for instanceit is necessary to find normal vectors that are in opposite direction towards each otherThis ldquobyproductrdquo of the computation can be stored as well as it might be useful for otherqueries in the future

As described in section 44

faces can be annotated as noGrasp to mark them unsuitablefor grasping Since this marking is done before annotating geometric conditions we couldhave decided not to create geometric condition individuals if at least one of the involvedfaces has a noGrasp flag Yet also here we choose to insert the geometric conditionsanyways and leave the task of filtering the noGrasp-ones out for the matching processlater on The reason being that we might want to make a GUI addition in the future whereusers can see invalid grasp modes to identify ways to make them valid for instance Thequery to mark faces as noGrasp is one line per face see figure 51

1 PREFIX cad lthttpkblocalrestkbcadowlgt2 INSERT 3 ltcubeowlFace1gt cadnoGrasp trueˆˆxsdboolean 4 ltcubeowlFace2gt cadnoGrasp trueˆˆxsdboolean 5 WHERE

Listing 51 Flag faces as noGrasp (URI shortened)

As explained in section 43

we have 5 grasping capabilities with a required geometric con-dition each Inward or outward facing surfaces can be distinguished within one queryleaving us with three queries needed to annotate 5 different geometric conditions

The query to annotate planar parallel faces inwardsoutwards will be discussed as a walk-through in the following (and can be found in full at listing 2

) The queries to annotatecylindrical face inwardsoutwards and the one to annotate planar faces outwards can befound in the appendix at 3

and 4

Walk-trough of the query to insert TwoParallelPlanarFaces individuals

26


First prefixes are being declared at the top of the listing 52

They serve as convenience tomake the query less visually complex and at the same time show the involved ontologiesat one central place of the query similar to includes at the top of a C++ class for instanceBy having declared the PREFIX cad for example we can address the object propertyboundedBy from the CAD ontology in the following manner cadboundedBy instead ofhaving to use its full uri lthttpkblocalrestkbcadowlboundedBy

gt everytime

1 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt2 PREFIX owl lthttpwwww3org200207owlgt3 PREFIX f lthttpwwwontotextcomsparqlfunctionsgt4 PREFIX cad lthttpkblocalrestkbcadowlgt5 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt6 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt

Listing 52 TwoParallelPlanarFaces-query start prefixes

Next with listing 53

we look at how to select pairs of faces within an object which arerepresented by planes and prepare their normal vectors for mathematical comparison Thefollowing query starts from an object which means all objects at once based on thepattern restricting what connections define an object Running the query on one specificobject would mean replacing object with an uri like lthttpkblocalrestkbbd-thesis-devowlCubeBlock

gt Within each object all pairs of faces get selectedThrough filtering out by comparing the uri-strings reverse-order duplicates are avoidedwhen we have A-B we do not want B-A because they are the sameOnce it is ensured that the face-representations are both of type Plane (a planar surface)we retrieve the xyz components of their respective normal vectors

1 INSERT 2 WHERE 3 object smeroboticsshape compound 4 compound cadcontains solid 5 solid cadboundedBy shell 6 shell cadcontains face1 face2 7 FILTER(STR(face1) lt STR(face2)) 8 face1 cadrepresentedBy plane1 9 plane1 sesamedirectType cadPlane

10 face2 cadrepresentedBy plane2 11 plane2 sesamedirectType cadPlane 12 plane1 caddirectionNormal p1dN 13 plane2 caddirectionNormal p2dN 14 p1dN cadx p1x cady p1y cadz p1z 15 p2dN cadx p2x cady p2y cadz p2z

Listing 53 TwoParallelPlanarFaces-query continuation select pairs of planes and theirnormal vectors

At this point we have not ensured that the planes are parallel to each other in 3D If theyare their normal vectors share the same direction However we are only interested in

27

5 Implementation

the cases where the normal vectors point towards each other or looking into opposite di-rections The first means we have a planar parallel inwards situation whereas the secondindicates an outwards situation Therefore we determine the angle between the first vectorand the negation of the second vector If they are indeed in exactly opposite direction thisangle yields 0 (it would be 180 without the negation of the 2nd vector)The cosine of an angle between two vectors is equal to their dot product divided by theproduct of their magnitude

cosα =~a middot~b~a middot ~b

(51)

Since we are dealing with numerical values the angle will only be exactly 00 in edge-casestherefore an suitable range should be used instead see listing 54

1 BIND(p1x -p2x + p1y -p2y + p1z -p2z AS dotProduct) 2 BIND(fsqrt(fpow(p1x 2) + fpow(p1y 2) + fpow(p1z 2)) AS p1dNmag) 3 BIND(fsqrt(fpow(p2x 2) + fpow(p2y 2) + fpow(p2z 2)) AS p2dNmag) 4 BIND(facos(dotProduct (p1dNmag p1dNmag)) AS angle) 5 BIND(IF(angle lt 0 -angle angle) as angle) 6 FILTER (angle lt ftoRadians(1))

Listing 54 TwoParallelPlanarFaces-query continuation filter out pairs of planes that arenot parallel to each other facing both inwards or both outwards

Now the geometric conditions as such are established all face-pairs not filtered out yet areinwards our outwards facing parallel planes What is left to figure out if they are inwardor outward oriented and the two parameters the distance between the planes and theiroverlapping area when projected into the same plane Also since a new individual will becreated a unique URI must be assembled for itThe distance between the parallel planes corresponds with the length of a perpendicularfrom one plane to the other For that we create a line equation from a point from one plane(since every Plane has a cadposition already this can be utilized) in the direction ofits normal vector By inserting this line into the scalar equation of the other plane we get theintersection point and thus can compute the distance between the two plane-intersectionpoints

To get a scalar equation of Plane1 in the form ax + by + cz = d based on its point-normalform only the d-coefficient is missing since a b and c are equivalent with the componentsof the normal vector We can get d by using a known point (denoted with subscript 0) in theplane as such a(xminusx0)+b(yminusy0)+c(zminusz0) = 0 and rearranging it to d = ax0+by0+cz0Called dCoefficient in the query 55

Next the line-equation is defined from a point in Plane2 in the direction of the normal

28


vector of Plane1 Using the variable names as in the query the equation becomes

l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)

This line equation inserted into the scalar equation of Plane1 and multiplied out can begrouped into all scalar parts and all parts tied to λ called scalarPart and lambdaPartin the query Solving the equation for λ (lambda) gives the factor by which to multiplyPlane1rsquos normal vector p1dN to get the intersection point with Plane1 when placing it atp2pos denoted as iXiYiZ The distanceBtwnPlanes between the planes isthen equal to the euclidean distance between p2pos and this computed intersection point

1 p1 cadposition p1pos p2 cadposition p2pos 2 p1pos cadx p1posX cady p1posY cadz p1posZ 3 p2pos cadx p2posX cady p2posY cadz p2posZ 4 BIND(p1posX p1dNX + p1posY p1dNY + p1posZ p1dNZ AS dCoefficient) 5 BIND(p2posX p1dNX + p2posY p1dNY + p2posZ p1dNZ AS scalarPart) 6 BIND(fpow(p1dNX 2) + fpow(p1dNY 2) + fpow(p1dNZ 2) AS lambdaPart) 7 BIND((dCoefficient - scalarPart) lambdaPart AS lambda)8 BIND(p2posX + lambda p1dNX AS iX) 9 BIND(p2posY + lambda p1dNY AS iY)

10 BIND(p2posZ + lambda p1dNZ AS iZ) 11 BIND(fsqrt(fpow(p2posX - iX 2) + fpow(p2posY - iY 2) + fpow(p2posZ - iZ 2))

AS distanceBtwnPlanes) rarr

Listing 55 TwoParallelPlanarFaces-query continuation distance between the planes

A useful side effect of having calculated λ is that it also indicates if the planes are facingeach other or have their backs towards each other Listing 56

uses this to distinguishbetween the inward or outward facing class and to prepare parallelismType to beused as a new object property between the two faces The other parameter needed is theoverlapping area of the face pairs This however would be unnecessarily complicatedto compute using SPARQL and consequently a Java-based plugin was developed for thispurpose It is called via a predefined predicate-keyword and assigns the computed valueto overlappingArea More details about the workings of this plugin can be found insection 513

Next a unique URI for these new geometric condition individuals must beconstructed newGeoCondInstanceIri is a concatenation of its class the name of theobject and the name of the involved faces

1 BIND(IF(lambda gt 0 graspisBackToBackParallelTo graspisFrontToFrontParallelTo) ASparallelismType)rarr

2 BIND(IF(lambda gt 0 graspTwoParallelPlanarFacesOutwardsgraspTwoParallelPlanarFacesInwards) AS geoCondClassIri) rarr

3

4 newGeoCondInstanceIri lthttpwwwfortissorgkbcomputeOverlappingAreaOfPolygonsgt (f1f2) overlappingArea rarr

5

6 BIND(IRI(CONCAT(STR(geoCondClassIri) _ STRAFTER(STR(object) ) _STRAFTER(str(f1) ) - STRAFTER(str(f2) ))) AS newGeoCondInstanceIri) rarr

29

5 Implementation

Listing 56 TwoParallelPlanarFaces-query continuation inwards or outwards overlap-ping area and building new URIs

Finally it is time to insert the newly computed knowledge as triples into the graph databaseListing 57

shows the INSERT block of this query Note that it sits above the WHERE blockthat started in listing 53

These insertion patterns tie the two normal vectors together viaisOppositeDirectionOf and the two faces via isBackToBackParallelTo or is-FrontToFrontParallelTo Furthermore it creates the new geometric condition indi-vidual and ties the two faces as well as the two parameters to it

1 INSERT 2 p1dN graspisOppositeDirectionOf p2dN3 f1 parallelismType f2 4 newGeoCondInstanceIri rdftype owlNamedIndividual5 rdftype geoCondClassIri6 cadcontains f1 f27 caddistanceEuclidean distanceBtwnPlanes8 graspoverlappingArea overlappingArea 9

Listing 57 TwoParallelPlanarFaces-query continuation insert new triples

5122 Get Info About The Gripper And Its Grasping Capabilities

As the workflow in the two query-responsible new JavaScript-classes Annotator andGraspPlanner in the Frontend will show later in section 523

info about the gripperwill be retrieved before matching grasping capabilities with geometric conditions to getactionable grasp modes Query 11

in the appendix fetches the spatial orientation of thegrippers parts to be displayed correctly in the graps planner This is however rather spe-cific to fortissrsquo demo-setup and will not be discussed in detail

Listing 12

in the appendix shows the SPARQL-query used to retrieve parameters of thegripper as a whole (which at this point is only title and description but might encompassvalues like maximum payload in the future) plus its annotated grasping capabilities withtheir involved gripper-faces and the required geometric condition for eachA separate query for getting these infos from the gripper would not have been necessaryThe matching query explained in the next section could have retrieved this info alongwith the geometric condition infos However that would have expanded that query sub-stantially and since it is potentially the most time consuming one it gives a smootherfrontend-experience for the user to see the grasping capabilities appear first and then aftera while to see them getting these categories filled with matches

5123 Matching With Value Checks

The full matching-query can be found in the appendix at 6

The frontend only has to fill inthe URIs for the object to grasp and for the gripper

30


In most if not all match-cases of geometric condition and grasping capability it needs to bechecked if values are within suitable ranges The concept of ValueCheck is kept genericenough to accommodate a wide range of possible checks ValueCheck-individuals havea rangeMin and a rangeMax value and two data properties that contain URIs value-ToCheckProperty points to the data property in the geometric condition whose valuemust be between min and max And valueCheckFeasibilityScoreMethod option-ally points to members of the class FeasibilityCheck The idea with feasibility checksis that after the matches with all values within the respective ranges are established asecond query can be run to assess how well that value scores given the method As proofof concept for this thesis for instance the RelativeDifferenceToMeanFeasibility-Check was implemented to give advantage to cylinder-radii and parallel-plane-distancesthat lie right in the middle of a grippers span for a particular grasping capability

Listing 58

presents the query after the first half in which grasping capabilities of the grip-per were matched with the geometric conditions of the object If the geoCondParam-Value currently investigated has a cadnoGrasp object property tied to it its value getsrecorded in noGraspFace Since this property is only used for faces it is implicitly clearthat the geoCondParamValue in that moment is a face bound to the geometric conditionvia cadcontains SPARQL queries are like a pattern-mask where everything in thegraph database sticks that fit into this pattern - a fundamentally different logic comparedto simply looping through something and distinguishing items one by oneThe next block handles the value checking logic It is completely optional as we do notrequire grasping capability to demand value checks for matches with their desired geo-metric conditions If the grasping capability individual does indeed point to an individ-ual of the class ValueCheck via the object property graspneedsValueChecking thenecessary values get extracted Namely min and max of the range that the grasping capa-bility demands the valueToCheckProperty of the geometric condition to be in Alsothe valueCheckFeasibilityScoreMethod gets added to the results to be used in asecondary round of queries more on that in the next section Since the valueToCheck-Property is modelled as data property of type xsdanyURI it must first be casted toan active IRI - only then the query-pattern can narrow the results using the FILTER()-notation down to the geometric conditionrsquos property that needs assessment Once it isensured we are looking at the right property the value must be within min and max If itdoes valuePassesCheck becomes true otherwise false The last step is to give apotential noGrasp flag a chance to turn the value check to false If no value is boundto valuePassesCheck (meaning no graspneedsValueChecking for this particularparameter) the value check defaults to true

1 OPTIONAL 2 geoCondParamValue cadnoGrasp noGraspFace 3 4 OPTIONAL 5 graspCapI graspneedsValueChecking valueCheckingType 6 valueCheckingType grasprangeMin rangeMin7 grasprangeMax rangeMax8 graspvalueToCheckProperty valueToCheckPropertyStr9 graspvalueCheckFeasibilityScoreMethod valueCheckFeasibilityScoreMethod

10 BIND(IRI(valueToCheckPropertyStr) AS valueToCheckProperty) 11 FILTER(geoCondParamName = valueToCheckProperty)

31

5 Implementation

12 BIND(IF(geoCondParamValue gt= rangeMin ampamp geoCondParamValue lt= rangeMax truefalse) AS valuePassesCheck) rarr

13 14 BIND(IF(EXISTSgeoCondParamValue cadnoGrasp noGraspFace false

IF(BOUND(valuePassesCheck) valuePassesCheck true)) AS valuePassesCheck) rarr

Listing 58 Matching-query 2nd half (see fully query at 6

)value checking

Next the results of running the matching query for a simple cube and the gripper SchunkWSG50-110Gearbox (used by fortiss in their demo-setup) is presented Listing 59

shows snippetsof the ldquoRaw Responserdquo in JSON format that also a GET-request via GraphDBrsquos REST APIwould yield more about that in section 522

The JSON-response format contains two top-level sections the header containing the col-umn titles as they would appear on top of the results rendered as table and the resultswhich is an array of bindings to the parameter names of the header Note that a bindingmust not have values for all header variables Non-values are an option and are renderedas empty cells in the table formatThe first binding shows one of the 24 resulting rows Just 7 out of 10 columns have val-ues That is because it is a face which is not subject to a range-check It is attached viacadcontains to the geometric condition individual and is in this case flagged as no-Grasp this renders valuePassesCheck as false The second binding has values in 9out of 10 columns the missing one is due to no value in the noGraspFace-column as thisrow is not dealing with a face but with the parameter caddistanceEuclidean Andsince its value of 50 is well within the ValueWithinRange-individual ldquoSchunkWSG50-110Gearbox PlanarClosingRangerdquo (it has a range of 0 minus 110 which is not neccessary toinclude in the results because a previous query already transported this info to the fron-tend see 5122

) it passes the value check with true Said individual also points to afeasibility scoring method that will come to use in the next step initiated by the frontendupon receiving this very results

1 2 head 3 vars [graspCapI geoCondI matchingGeoCond geoCondParamName4 geoCondParamValue noGraspFace valueCheckingType5 valueToCheckProperty valuePassesCheck6 valueCheckFeasibilityScoreMethod]7 results 8 bindings [9 valuePassesCheck datatype boolean

10 value false 11 matchingGeoCond type uri12 value TwoParallelPlanarFacesOutwards 13 graspCapI type uri14 value PlanarParallelFromOutsideGrasp-LargeBackPair 15 geoCondI type uri16 value TwoParallelPlanarFacesOutwards_CubeBlock_Face2-Face4 17 geoCondParamValue type uri18 value httpkblocalrestkbcadcubeowlFace4 19 geoCondParamName type uri20 value httpkblocalrestkbcadowlcontains 21 noGraspFace datatype boolean

32


22 value true 23 24 valueToCheckProperty type uri25 value httpkblocalrestkbcadowldistanceEuclidean 26 valuePassesCheck datatype boolean27 value true 28 matchingGeoCond type uri29 value TwoParallelPlanarFacesOutwards 30 valueCheckFeasibilityScoreMethod datatype anyURI31 value RelativeDifferenceToMeanFeasibilityCheck 32 graspCapI type uri33 value PlanarParallelFromOutsideGrasp-LargeBackPair 34 valueCheckingType type uri35 value SchunkWSG50-110Gearbox_PlanarClosingRange 36 geoCondI type uri37 value TwoParallelPlanarFacesOutwards_CubeBlock_Face1-Face3 38 geoCondParamValue datatype double39 value 500 40 geoCondParamName type uri41 value httpkblocalrestkbcadowldistanceEuclidean 42 ]

Listing 59 Partial results from a matching query (edited for better readability)

Note that matches do not get stored in the knowledge base at this point They ldquoliverdquo onlyin the results table and in how the frontend displays them Storing these would requiremodelling them in the ontology and deciding for a place to store them with the mainrepository in the process-ontologies that get created for each process or somewhere elseThe advantage would be a very desirable reduction of code-complexity in the frontend asthe matches are being constructed there Furthermore it would save inference-time whenthe same gripper asks for grasp modes with the same object again they would already bestored

5124 Compute Feasibility Scores

Once the matching-results (see listing 59

) reach the frontend a second round of queries areprepared to cover the subsymbolic calculations regarding feasiblity scores First a singlequery gets dispatched to fetch the rule-strings (in itself SPARQL queries with sections to bereplaced) combining all feasiblity methods that appeared in the matching-results see 510

At this point only RelativeDifferenceToMeanFeasibilityCheck is implementedas proof of concept though

1 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt2 SELECT WHERE 3 VALUES (method) 4 (graspRelativeDifferenceToMeanFeasibilityCheck) (graspSomeOtherFeasiblityCheck)5 6 method grasphasRuleString ruleString 7

Listing 510 Query to fetch the SPARQL queries for the feasiblity scoring methods

33

5 Implementation

Once those queries are received the frontend can replace $REPLACE$ with those casesthat were indicated as candidates for feasibility scoring For each a pair of the geometriccondition individual in question as well as the value within range individual see listing511

One result from the previous json-results in listing 59

for instance would yield thispair for insertion at the replace-position

(ltTwoParallelPlanarFacesOutwards_CubeBlock_Face1-Face3gt

ltSchunkWSG50-110Gearbox_PlanarClosingRangegt)

1 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt2 SELECT geoCondI valueWithinRange feasibilityScore WHERE 3 VALUES (geoCondI valueWithinRange) 4 $REPLACE$5 6 valueWithinRange graspvalueToCheckProperty valueToCheckStr 7 BIND(IRI(valueToCheckStr) AS valueToCheck) 8 geoCondI valueToCheck value 9 valueWithinRange grasprangeMin rangeMin

10 grasprangeMax rangeMax 11 BIND((rangeMin + rangeMax) 2 AS mean) 12 BIND(abs(value - mean) AS absDiffToMean) 13 BIND(absDiffToMean mean AS relDiffToMean) 14 BIND(1 - relDiffToMean AS feasibilityScore) 15 ORDER BY DESC(feasibilityScore)

Listing 511 SPARQL query hosted as string in the data property hasRuleString

In this case the math for calculating the relative difference to the mean is fairly simple andcould easily have happend in the frontend as well However the guiding principle is tokeep as much knowledge as possible independent of code and therefore it also makessense to store algorithms in the knowledge base Future feasibility scoring algorithmsmight also well be more complicated or call a java-based plugin

Another feasibility score that would make sense to implement next for instance is theoverlapping area of not only the parallel planes with each other but also how well theyoverlap with the surfaces of the grippers finger This would clearly be something to solvevia a plugin rather than a lengthy and complicated SPARQL query


GraphDB offers a Java-based Plugin API7

to allow extending the functionality of the graphdatabase This was used to compute the overlapping area of planar parallel faces whenprojected into the same plane This value must at least greater than zero for a proper graspto be applied as mentioned in section 44

A GraphDB Plugin extends the class PluginBase that is provided by Ontotext Further-more the interfaces PatternInterpreter or ListPatternInterpreter need to be

7httpgraphdbontotextcomdocumentationstandardplug-in-apihtml

34


implemented - the first to interpret simple triple patterns whereas the second can readmultiple inputs In the case of the PolygonAreaPlugin developed in the context ofthis thesis both interfaces were used to achieve the syntax seen in listing 56

namely theldquopassingrdquo of the two faces as ldquomethod variablesrdquo and then assigning the computed valueto overlappingArea This was the line in question from listing 56

where it is used tocalculate the overlapping area of two parallel planes and record that value in the respec-tive geometric condition individual

geoCondI ltcomputeOverlappingAreaOfPolygonsgt (f1 f2) overlappingArea

The comma is just a SPARQL-shortcut for reusing the subject and predicate for a newtriple statement with the object that follows Next I present what happens in the Plu-gin upon being called from a SPARQL query like shown in the line above The URIwwwfortissorgkbcomputeOverlappingAreaOfPolygonswas freely choosen as pred-icate to trigger the plugin Since the call with the two faces happens before the one to assignoverlappingArea nothing else happens in that first step but storing the two faces in aHashMap with the URI of the geometric condition individual (geoCondI) as key

facesOfGeoCondIndividualsput(entitiesget(subject)stringValue() objects)

Upon receiving the second call the facesOfGeoCondIndividuals HashMap is used tore-associate these two faces with the geometric condition individual By using the samemath as was used for computing the distance between parallal planes (see listing 55

) twoperpendicular points on the planes are identified and then used as origins in each planes2D-representation These 2D-polygons can then be fed to the intersection() methodof the JTS Topology Suite8

and the result of that consequently to the getArea() methodto get the value we want to be assigned to overlappingArea More explanations inthe comments of listing 512

Note that the predicate URI got translated to itrsquos internalrepresentation as long value by the variable name predicateId

1 Override2 public StatementIterator interpret(long subject long predicate long object long

context Statements stmts Entities entities RequestContext rc) rarr3 if (predicate = predicateId) 4 return null5 6 if (entitiesget(subject) == null ||7 facesOfGeoCondIndividualscontainsKey(entitiesget(subject)stringValue()))8 -1 indicates the computation has failed9 return createLiteral(entities subject predicate -1)

10 11 try 12 long[] faceIDs =13 facesOfGeoCondIndividualsget(entitiesget(subject)stringValue())14 iriToId a MapltString Longgt mapping IRIs to long IDs the plugin uses15 Face face1 = new Face(getLogger() stmts entities iriToId faceIDs[0])16 Face face2 = new Face(getLogger() stmts entities iriToId faceIDs[1])17 face2computePlaneEquation()18

19 The idea is to project the geom center point of face1 via its normal

8httpsprojectseclipseorgprojectslocationtechjts

35

5 Implementation

20 vector to face2 The point where it hits face2 plus the point where it21 started from are then treated as origins on the respective planes in 2D22 Point face1GeomCenter = face1computeGeometricCenter()23 double scalarPart = face2planeEqScalar - (face2planeEqX1 face1GeomCenterx +

face2planeEqX2 face1GeomCentery + face2planeEqX3 face1GeomCenterz)rarr24 double lambdaPart = (face2planeEqX1 face1dirNormalx + face2planeEqX2

face1dirNormaly + face2planeEqX3 face1dirNormalz)rarr25 double lambda = scalarPart lambdaPart26 Point pointWhereNormalLineFromFace1GeomCenterHitsFace2 =

PointpointPlusVector(face1GeomCenter VectorscalarMultiply(face1dirNormallambda))

rarrrarr

27

28 creates points2D list with the respective points as origins29 face1compute2Dpoints(face1GeomCenter)30 face2compute2Dpoints(pointWhereNormalLineFromFace1GeomCenterHitsFace2)31

32 from the JTS Topology Suite package comvividsolutionsjtsgeom33 use the points2D list to build a polygon34 Polygon poly1 = face1get2dPolygon()35 Polygon poly2 = face2get2dPolygon()36 Geometry intersection = poly1intersection(poly2)37

38 return createLiteral(entities subject predicate intersectiongetArea())39 catch (Exception e) 40 getLogger()info(egetMessage())41 return createLiteral(entities subject predicate -1)42 43

Listing 512 interpret() method of the GraphDB plugin to calculate overlapping areaof parallel planes

Before compute2Dpoints() computes the 2D-representation of the plane the 3D-pointson the plane already got collected in the Face() constructor by iterating over the edgescontained by the facersquos wire and collecting the points representing their bounding verticesfollowing the OntoBREP structure (341

) of facesWhat happens in compute2Dpoints() then is that for each of these 3D-points a linearequation system of the following form needs to get solved for point2DdirXx dirXy dirXz

dirYx dirYy dirYzdirZx dirZy dirZz

lowastpoint2Dx1point2Dx2point2Dx3

=

origin2DToPoint3Dxorigin2DToPoint3Dyorigin2DToPoint3Dz

(53)

Where dirX dirY and dirZ are the normalized three direction vectors that are part ofdefining every Plane according to the OntoBREP format origin2DToPoint3D is thevector from the previously computed origin (for plane 1 its geometric center and for plane2 its intersection with the perpendicular from the geometric center of plane 1) to the pointin question on the plane These 3D-vectors from the artificial origin to each point arethe coordinates in the 2D system we are interested in After solving the linear systemtwo components of point2D will be non-zero These are the components of the pointin 2D that we can use to construct the 2D-polygon representing this 3D-plane from theperspective of an observer floating directly above it whatever orientation it has in 3D-space Java-listing 513

shows this part in compute2Dpoints() using the linear algebra

36


tools from Apache Commons Math9

1 for (Point point3D points3D) 2 Vector origin2DToPoint3D = VectorfromTo(origin2D point3D)3 RealMatrix coefficients = new Array2DRowRealMatrix(new double[][] 4 dirXnormx dirXnormy dirXnormz5 dirYnormx dirYnormy dirYnormz6 dirZnormx dirZnormy dirZnormz false)7 DecompositionSolver solver = new LUDecomposition(coefficients)getSolver()8 RealVector constants = new ArrayRealVector(new double[]9 origin2DToPoint3Dx origin2DToPoint3Dy origin2DToPoint3Dz false)

10 RealVector solution = solversolve(constants)11 12

Listing 513 Solving a linear system for each point in compute2Dpoints()


This part presents how the SPARQL queries from part 1 of this implementation chapterfound their way into the frontend First though a short introduction to the Robot Instruc-tion Framework fortiss has been developing


As mentioned before fortiss developed an intuitive robot programming interface a ldquoRobotInstruction Frameworkrdquo The underlying academic work is spread over various papersProfanter et al [16

] for instance describes a user study conducted to analyze the preferredinput modalities for task-based robot programming Kraft and Rickert [10

] compare a pre-vious version of the GUI to the current one The redesign was driven by applying userexperience (UX) paradigms to human-robot-interactionBehind a browser-based GUI the middleware ties together various subsystems written indifferent programming languages All configuration steps the user makes are directly rep-resented by entries in a GraphDB database The user can conveniently configure assembly-processes The workflow is simple and intuitive the user places objects on a workspace tothen create assembly-tasks for combinations of them For each task constraints are addeduntil the desired position is exactly defined The overall assembly-process then consists ofthe chronological execution of those tasks This can be done entirely virtually for develop-ment purposes or it gets wired to a real robot where a perception system detects objects onthe table

Figure 58

shows a simplified overview of the software components All parts communi-cate via a middleware however the new additions drawn in blue do not use the middle-ware The SPARQL queries from the frontend directly land in the graph database via GETand POST requests - asynchronously by using Ajax-calls

9httpcommonsapacheorgpropercommons-mathuserguidelinearhtml

37

5 Implementation

Figure 58 Schematic overview of the software components

The proof of concept for the semantics of grasping explained throughout chapter 4

wasto implement it in this software

5211 Workflow In The GUI

Figure 59

shows the workflow in the robotic workcell GUI Outlined in blue are the twoconceptual new elements that got added more about both in section 523

bull the Annotator that lets users mark noGrasp-faces and inserts geometric conditions

bull the Grasp planner that shows actionable grasp modes based on value-checked matchesof grasping capabilities with geometric conditions ranked by feasibility score


To further the direct communication between frontend and graph database it made senseto get rid of a json-format that was used for parsing CAD data in the GUI so far Backin 2016 fortiss developed a converter from step files to json however that is no longermaintained and is not seen as the path forward anymore Instead that converter is beingrestructured to convert to owl-files following the OntoBREP format This follows the phi-losophy of concentrating all knowledge in ontologies and shedding the need for auxiliarycode-based steps or formats as much as possible All the info required exist in the Onto-BREP format it makes perfect sense to only use this as source for rendering 3D objectsAlongside itrsquos use for inferring geometric conditionsIn the context of this thesis the class BrepOwlLoader in the TypeScript file brepowl-loaderts was developed to achieve this Given an URI of the object to fetch it willdispatch four SPARQL queries they can be found in the appendix in section 6

38


bull query 7

collects URIs of all faces wires edges vertices and points of the object

bull query 8

fetches the triangulations of faces each row in the results correlates with onetriangle and gives its three point-URIs

bull query 9

now collects all points with their coordinates that are appearing in triangles -because of the way triangulation works this necessarily also includes all points thatrepresent vertices

bull query 10

finally fetches the surface-info about all faces of the object what type is thesurface and what are itrsquos direction vectors

Upon receiving the results for these queries a BrepObject is assembled containing edgesfaces and vertices From these a THREEObject3D is generated that can be viewed andbe interacted with tools the JavaScript library Threejs provides heavily augmented withcustom functionality throughout multiple TypeScript classes

The plan is to expand the use of this new BrepOwlLoader to also parse the CAD data ofthe workcell and the robot arm ideally eventually the entire software can handle CAD datain OntoBREP format This would greatly simplify the conversion-pipeline of handling newobjects in the system


From all of the theoretical- and implementation-work described so far the user only getsto see changes in the GUI-workflow Flowchart 59

highlighted the two places across theworkflow were new elements were introduced The purpose of these additions is to sup-port the user in specifying grasp modes in the assembly-process Grasp mode candidatesare being semantically generated using the queries and mechanisms described These getpresented to the user as ranked options to choose from

5231 Part 1 Annotator

The addition of the annotator is outlined on the left side of the flowchart 59

It has twotasks giving the user the opportunity to flag faces as noGrasp and to send the query toautomatically infer geometric conditions Since both of these tasks relate to objects alonea natural place to locate this functionality is the ldquoUpload New Partrdquo tab next to the defaulttab of ldquoSelect Existing Partrdquo that slides open on the right side of the GUI when creating anew project for instance Since annotating conditions has to happen only once per objectwhy not do it when the object enters the pool of objectsWhen the annotator opens the first task is to select noGrasp-faces if applicable as shownin figure 510

A click on the face directly on the object adds it to the selection a 2nd clickon that face or clicking on its name in the selection removes it from the selectionThe click on ldquoApply no-grasp annotationsrdquo (it is saying ldquoSkip no-grasp annotationsrdquo if noface is selected) sends off the query to insert 51

-flags to the selected faces Then it poststhree queries to inspect the object for the five different geometric conditions currently sup-ported as explained in section 5121

Two parallel planar faces inwardsoutwards Cylindrical

39

5 Implementation

face inwardsoutwards and Planar face outwards

Once these four queries are completed another one gets send to fetch the geometric con-ditions that were just inserted see listing 5

These get displayed sorted by category asshown in figure 511

As it can get a lot of results per category depending on the object acategory collapses if it has more then 4 entries One of them is grayed out in the screenshotbecause it contains a noGrasp-faceIn the future one might want to add an option here to manually remove inferred conditionsthat are actually not suitable Or even manually insert missing ones although a modelling-mechanism would have to be developed to support that kind of flexibilityTo showcase parts of the TypeScript code 514

shows the nested queries being set off byclicking on the ApplySkip no-grasp annotation button as described above

1 thisannotatorListToolbaronApplyNoGraspFaceSelectionsubscribe(2 (faceIris string[]) =gt 3 thisannotatorrunMarkNoGraspFacesQuery(faceIris)then(() =gt 4 thisannotatorrunInferGeometricConditionsQueries()then(() =gt 5 thisannotatorgetGeometricConditionsQuery()then(6 (geoConds GeometricCondition[]) =gt 7 thisannotatorListToolbarlistGeoConds(geoConds)8 ))))

Listing 514 Snippet from the Annotator3DComponent-class

To give insight into one of these methods TypeScript-listing 515

shows the method thatruns the three queries to infer geometric condition individuals The queries are not shownthey can be found in the appendix at 6

1 public runInferGeometricConditionsQueries() Promiseltvoidgt 2 return new Promiseltvoidgt((resolve reject) =gt 3 let twoParallelPlanarFacesQuery = 4 let cylindricalQuery = 5 let planarFacesQuery = 6 let promises = []7 promisespush(thisrunQuery(twoParallelPlanarFacesQuery))8 promisespush(thisrunQuery(cylindricalQuery))9 promisespush(thisrunQuery(planarFacesQuery))

10 Promiseall(promises)then(() =gt 11 resolve()12 )catch(() =gt 13 reject()14 ))15 private runQuery(query string) Promiseltvoidgt 16 return new Promiseltvoidgt((resolve reject) =gt 17 $ajax(18 url httplocalhost7200repositorieskbstatementsupdate=19 type POST dataType json data query20 contentType applicationsparql-update21 complete function (msg) 22 if (msgstatus === 204) 23 resolve()24 else 25 reject()26 ))

40


Listing 515 runInferGeometricConditionsQueries()-method in the Annotator-class

5232 Part 2 Grasp Planner

The addition of the grasp planner is outlined on the lower part of the flowchart 59

Whena user is done defining tasks of the assembly process the click on ldquoSimulate amp Runrdquo opensa modal window showing the tasks as squares horizontally instead of the vertical list fromthe previous screen The only option at this point is to ldquoChoose a workcellrdquo Upon doingso the process gets deployed on the selected workcell This entails a range of actions

bull the ontological description of the workcell getrsquos loaded

bull the controller for the robot arm gets added both the ontological description as well asits CAD data (which in the future will only be parsed from the ontology see section522

)

bull the perception systems and the object detectors get activated if none is connectedto the middleware (in case of pure virtual development for instance) a dummy-perception serves as fallback

bull then the process gets loaded into the workcell and initialized meaning all values thatdid not need to be specified up to know need concrete values fitting to the workcellenvironment

bull as part of initializing the process new tasks might be inferred in the knowledge base- examples of inferred task are ldquoMove robot homerdquo at the end of a process or a ldquoGraspand Releaserdquo task for potentially every object if the perception system is not preciseenough (after a grasp and release a fuzzy location of an object is known preciselynamely in the middle of the gripper span as it closed on the object)

bull as the frontend gets notice of the updated process newly inferred tasks get insertedas squares with a green frame into the task list

Figure 512

shows the tasklist after a workcell was chosen and the process got deployedon it The grasp-icons open the grasp planner on a specific object Since in most assemblytasks two or more (in case of assembling assemblies) objects are involved choosing whichone to plan a grasp for also counts as decision to have the robot move this object insteadof the other one in an assembly step Currently the solution to distinguish the objects isvia a mouse-over tooltip as seen in the screenshot - other solutions to this user interfacechallenge will likely be considered in the futureFigure 513

shows the grasp planner after having pressed a gripper icon in figure 512

Bydefault it opens with the matches sorted by category with the grasping capabilities beingthe categories Note that it deals with a different object then the ones seen previouslyNamely one used in development constructed to be simple but still exhibit all 5 currentlysupported geometric conditions

41

5 Implementation

At the stage as shown in figure 513

all 5 (currently - each new category of feasiblity scor-ing method added in the future will add one query) queries that the grasp planner isequipped with have already been sent to the graph database and have returned with re-sults Namely the following shown with their TypeScript method name

bull getGripperDataFromWorkcell() fetches the CAD data of the gripper to be dis-played in 3D see listing 11

more info in section 5122

bull getGrippersGraspingCapabilities() gets the grippers parameters and its grasp-ing capabilities including which of its faces are involved see listing 12

more info insection 5122

bull inferMatches() compares the grippers grasping capabilities with the objects geo-metric conditions the matches that pass the value checks are actionable grasp modessee listing 6

bull fetchFeasibilityScoreMethodSPARQLqueries() collects all feasibility scor-ing methods mentioned in value checks of the previous results and fetches their rule-string which in itself is a SPARQL query were certain contents needs replacing be-fore dispatching it see listing 510

bull run the queries fetched in the method above after inserting custom info at the placesmarked for replacing example in listing 511

Figure 514

shows the grasp planner with the matches sorted by feasibility score The ideabeing that in the future automatically the highest one will be used by default if the userdoes not choose to select a grasp mode by handNote that matches who didnrsquot pass any of their value checks are not displayed This canhowever be realized in the future as a feature for operators to investigate why a certaingrasp mode might not show up Those invalid matches are already included in the resulttable there is just a user interface for it missing in the frontend

In terms of TypeScript code listing 516

shows the method inferMatches() of the classGraspPlanner The query-string itself is left out it can be found in listing 6

What hap-pens here is looping through the results table and sorting things to the respective matchesNotice the use of the classes Match GeometricCondition and ValueCheck Thesewere created to assist the sorting of data throughout the frontend Moreover the mapselfgrippersGraspCaps makes use of the class GraspingCapability

1 public inferMatches() Promiseltvoidgt 2 let query = 3 return new Promiseltvoidgt((resolve reject) =gt 4 let self = this5 $getJSON(GraspPlannerrepositoryURL query query infer true)6 done(function(response) 7 let rows = responseresultsbindings8 if (rowslength === 0) reject() return 9 let allMatches = new Mapltstring Matchgt()

10 for (let i = 0 i lt rowslength ++i) 11 let row = rows[i]

42


12 let matchKey = rowgraspCapIvalue + + + rowgeoCondIvalue13 if (allMatcheshas(matchKey)) 14 let match = new Match(rowgraspCapIvalue rowgeoCondIvalue)15 matchgraspCap = selfgrippersGraspCapsget(rowgraspCapIvalue)16 matchgeoCond = new GeometricCondition(rowgeoCondIvalue)17 allMatchesset(matchKey match)18 19 let match = allMatchesget(matchKey)20 if (rowgeoCondParamNamevalue === httpkblocalrestkbcadowlcontains)

rarr21 matchgeoCondaddFace(rowgeoCondParamValuevalue rownoGraspFace

rownoGraspFacevalue === true false)rarr22 else 23 matchgeoCondaddParam(rowgeoCondParamNamevalue

rowgeoCondParamValuevalue)rarr24 25 if (rowvalueCheckingType) 26 let valueCheck = new ValueCheck(rowvalueCheckingTypevalue

rowvalueToCheckPropertyvalue rowvaluePassesCheckvalue === truerowvalueCheckFeasibilityScoreMethodvalue)

rarrrarr

27 if (valueCheckvaluePassesCheck) 28 matchhasInvalidValue = true29 30 matchvalueCheckspush(valueCheck)31 32 33 allMatchesforEach((match Match key string) =gt 34 if (matchhasInvalidValue) selfmatchesset(key match) 35 )36 Msgsuccess(selfmatchessize + valid matches (37 + (allMatchessize - selfmatchessize) + invalid))38 resolve()39 )40 fail(function () 41 reject()42 ))

Listing 516

To give an insight into the workings of AngularJS in the HTML of the frontend listing 517

shows the part in graspmodelisttoolbarcomponenthtml that is responsible forrendering the matches on the right side of the grasp planner when viewed by feasibilityscore matchesRanked is an array of matches sorted by decreasing feasiblity score

1 lt-- By feasibility score --gt2 ltdiv class=group clearfix layer-1 [ngClass]= expand byCategory === falsegt3 ltdiv class=header (click)=toggleViewMode()gt4 ltspan class=namegtBy feasibility scoreltspangt5 ltspan class=arrowgtlti class=fa-angle-downgtltigtltspangt6 ltdivgt7 ltdiv class=group-contentgt8 ltdiv class=partsgt9 ltdiv ngFor=let match of matchesRanked class=part [ngClass]=selectedGeoCond

match===selectedMatchgtrarr10 ltspan class=hint geoCondSpangt11 ltspan class=smallFont feasibilityScoregt12 score matchfeasibilityScoreAverage | number 12-213 ltspangtltbrgt14 ltspan ngFor=let face of matchgeoCondfacesgt15 faceirisplit()[1]

43

5 Implementation

16 ltspangtltbrgt17 ltspan class=smallFont ngFor=let param of matchgeoCondparametersgt18 paramnamesplit()[1] paramvalue | number 12-2ltbrgt19 ltspangt20 ltspangtltdivgtltdivgtltdivgtltdivgt

Listing 517 HTML-part responsible for rendering the matches sorted by feasiblity score(edited for better readability)

The next step would be to get from a grasp mode to concrete poses for the gripper to ap-proach during execution This however was unfortunately not achieved in the timeframeof this thesis It will involve consulting the constraint solver and the collision checker tosee if a grasp mode is physically possible

Also as mentioned at the end of section 5123

currently matches as standalone individu-als are not modelled ontologically yet They live only in result tables and frontend-displayIt is certainly worth considering permitting them a triple store presence of their own Thefact that pseudo-IDs for matches became necessary for handling them in the frontend-code as seen in code snippet 516

indicates this further

44


Figure 59 Workflow in the GUI

45

5 Implementation

Figure 510 First step in the Annotator mark noGrasp-faces

Figure 511 Second step in the Annotator list inferred geometric conditions

46


Figure 512 Task list augmented with inferred tasks after deploying the process on a work-cell and icons to open the grasp planner

Figure 513 Grasp Planner one match selected sorted by category

47

5 Implementation

Figure 514 Grasp Planner one match selected sorted by feasibility score

48


Discussion

The result of the conceptual and implementation work done in this thesis is a foray intothe semantics of grasping in an industrial context The concept the modelling and theimplementation all of it will likely undergo further evolution yet the work in this thesisrepresents a significant contribution

Before there was no semantic concept dealing with grasping In fortissrsquo software the graspmodes were hardcoded for a small demo-set meaning exact numerical poses for every ob-ject and combination of objects was deposited This is not a sustainable strategy whenopening up to other scenarios and further objects The assembly-combinations that wouldhave to be covered by manual pose-definitions would increase exponentially In contrastto this now it is possible to infer grasp modes based on the one-time annotation of a grip-pers grasping capabilities and a one-time annotation of an objects geometric conditions

The effects on a users relation to such a software is also worth considering Now there is aprocess to witness an object getting enriched by making its features explicit even thoughunexpected things may come to light Also the process of going through grasp modescould be enjoyable as one might be comparing the options to how one might grab this ob-ject with their own hands

These being the potentially positive effects of it the argument can also be made that thisadds to the overall time required to define assembly processes Both the object annotationas well as choosing a grasp mode is something that requires active user input - unless itbecomes further automatized that is However things like marking faces unsuitable forgrasping and annotating the grasping capabilities of a gripper can also be assets that canbe shared with others who work with similar setups It could be included in standardshow companies offer their products enriched by ontological descriptions of its specifica-tions and features

As a tangible example of the added value one might think of an engineer that has themost experience with robotic grasping in the context of the companys specific assembly-requirements As valuable as such skillsets are it creates a strong dependency on thatperson Painfully large parts of knowledge might go missing at least temporarily whensuch key-persons retire or leave a company Even a proper handover canrsquot replace decadesworth of intuition grown in a particular production context The approach of ontologi-cal domain modelling seeks to precisely address this The knowledge migrates from theldquoheads of expertsrdquo into the ontologies And as such can continously be refined and sharedwith others in the same field Standards can emerge that might further the developments

49


in adjacent fields etc

Conclusion

All in all the formal specification of the semantics of grasping using the Web OntologyLanguage (OWL) and its integration into fortissrsquo robot programming GUI were successfulin most parts Although the final step from a valid grasp mode to the required poses ismissing and would have rounded the development off in a nice way as we could see arobot moving around objects and grasping them the way the new concept ldquotold them tordquoinstead of using hardcoded valuesThe concept of matching a grippers grasping capabilities with an objects geometric condi-tion while considering value ranges and rank them by feasiblity is a novel one as such Itemerged from the immersion in the grasping taxonomy research and the thought-processon how to model grasp modes as generic and convenient as possible

Future Work

The trend in industry is to dilute conservative multitier models where each level can onlytalk to its neighbouring ones Instead at times we might want to puncture through from atop level to sensor data on a very low level The guiding principle of this work to separatethe knowledge base and the code as much as possible is playing into that Furthermore byshifting maximally much ldquointelligencerdquo from the code to the knowledge base we becomemore independent on any concrete implementations and can quickly change translate oradapt the implementations

Another trend is ldquoZero Programmingldquo the ldquoautomation of the automationrdquo or also calledthe knowledge based digital engineering Having reasoning tools that can on a symboliclevel guarantee incompabilities is a crucial puzzle piece in this way forward as it candrastically limit the search spaces that subsymbolic methods have to comb through andcan give users semantically meaningful insights

Full Stack Of Grasp Mode Validation

As stated in the conclusion the full stack of grasp mode validation was not explored forthis thesis Here is an overview of the different levels ordered from symbolic to more andmore subsymbolic

Symbolic level On the symbolic level matching grasp modes can be found as explainedthroughout this thesis

Constraint solver However to reach concrete poses for the gripper the constraint solvermust perform arithmetic operations that take the geometric restrictions and finds concretemanifestations that satisfy all constraints Basically enough constraints must be defined tonarrow down the degrees of freedom to zero

50

Collision Checker The next level of vetting a grasp mode for its actual usefulness is col-lision checking A collision checker answers the question if one volume is ingested byanother or not If this passes without problems there is still one further level

Physics simulation Namely considering forces like friction and gravity that occur in re-ality Only a physics simulation can answer seamingly simple questions like will theassembly the gripper wants to grasp fall apart the moment it is lifted Think of trying tograb a ball bearing on a mechanical tree in order to lift this assembly The effect will be totake the bearing off the tree and not move both

One could draw parallels to human grasping We intuitively assess the geometric featuresof objects in order to derive grasping strategies This constitutes the logical reasoning partOnce we touch it further subsymbolic optimizations are being done by small repositioningand applying pressure based on friction weight and the centre of mass

Ideas For Improvements

There are many meaningful threads to continue in various directions from this work Bothin terms of furthering the concept the modelling and the implementation

The very obvious one is to complete the workflow-pipeline and take the step(s) from graspmode to concrete gripper poseAlso high on the priority list I would suggest to model a match-class in the ontology andcreate individuals upon running the matching query A decision where to store thesewould have to be madeUsing OntoBREP for all CAD-usages across the back- and frontend will bring clarity to theformats required by the system and standardize a CAD-format that is maximally enrichedby being able to hold both symbolic as well as approximated representationsDevelop more classes of geometric conditions and grasping capabilities beyond the 5 basicones chosen for this thesis Also the modelling needs to be adjusted to support grasp-ing capabilities in demanding multiple geomtric conditions not just one as currently thecase For instance the capped V-shaped notch from outside grasp (the lower one of thetwo notches on the gripper fingers as seen on the gripper-screenshots) actually needs anoutward facing cylinder plus a restriction on the cylinder in one vertical directionValue checks must also support other types than ranges (eg must be higher then or mustbe unequal to) and more feasiblity scoring methods must be developed based on demandBoth SPARQL- and Plugin-basedThe computation of overlapping area must be extended to the gripper finger and the ob-jects face or faces to asses their feasibility Currently it only calculates the overlapping areaof two parallel planes upon projection in the same planeThe user interface regarding how to open the grasp planner should be given some morethought With the two icons currently it is a bit cumbersome to plan a grasp on the desiredobject and it is also not clear what that means in terms of picking order during assemblySupport the automatic selection of the grasp mode with the highest score by default Con-fidence could be indicated by coloring the grasp planner icon

51


In the annotator screen the user might want to have the option to manually remove geo-metric conditions that were automatically inferred Maybe even allowing the insertion ofown custom ones Even so that might be not trivial to modelJust like the grasp planner should be default select the best option the annotator shouldalso do its job silently and without a mandatory intervention by the user The marking offaces as noGrasp is optional in any caseFinding a convenient way to also give the user insight into invalid grasp modes if desiredIncluding the reasons for failing value checks gives the user a chance to investigate issueswith the object or the gripperSupporting properties of the gripper as a whole like a maximum payload would becomeimportant at some point These might then have to be matched with properties of geomet-ric conditions or with the objects properties as a whole like its massIt would be great to be able to annotate grasping capabilities also more or less automati-cally like geometric conditions Or at least update them upon changes When the geometryof a finger changes for instance or the gripper span changes this should cause immediateupdates in the grasping capabilities tooIt might be worth considering to mergemixaugment OntoBREP with ldquoGeoSPARQL - AGeographic Query Language for RDF Datardquo1

for convenient access to complex geometricalfunctionsOther graph databases might have to offer more useful functionality Stardog2

for instanceseems to have better support for propert chains and custom rules It is not available forfree thoughEventually the grasp planner might also have to consider the target location where to placethe object There might be physical constraints there making match found on the logicallevel impossible to execute For instance donrsquot grasb the ball bearing from outside if itneeds to go into the pipe - the gripper finger would be in the wayAlong that line but thinking even further ahead - considering the next tasks ahead couldbring great benefits Which grasp mode is best suited to manoeuver the gripper alreadyin a pose that is useful for the next assembly task Which grasp mode on the other handmakes certain things impossible in the next taskThe concept of ldquoprimitive shape matchingrdquo (eg as described in Somani et al [19

]) couldhelp with organic objects with potentially too many surface features to ldquoreasonable reasonrdquoabout them Subsymbolic algorithms would match the best fitting primitive objects intothe organic shape and these would then be used for further reasoning about the object likematching with grasping capabilities

1httpswwwopengeospatialorgstandardsgeosparql

2httpswwwstardogcom

52

Appendix

53

1 lt-- Establishing the link to the cubes namespace --gt2

3 ltowlNamedIndividual rdfabout=devowlCubegt4 ltrdftype rdfresource=devowlCubegt5 ltsmeroboticsshape rdfresource=cubeowlCompound1gt6 ltowlNamedIndividualgt7

8 lt-- Compound1 contains Solid1 contains Shell1 contains 6 faces9 The first one --gt

10

11 ltowlNamedIndividual rdfabout=cubeowlFace1gt12 ltrdftype rdfresource=cadowlFacegt13 ltcadboundedBy rdfresource=cubeowlWire1gt14 ltcadrepresentedBy rdfresource=cubeowlPlane1gt15 ltcadtriangulatedBy rdfresource=cubeowlTriangulation1gt16 ltcadhasForwardOrientationgttrueltcadhasForwardOrientationgt17 ltowlNamedIndividualgt18

19 lt-- Plane1 has a position and vectors for x y and z direction --gt20 lt-- Triangulation1 contains two Triangles The first one --gt21

22 ltowlNamedIndividual rdfabout=cubeowlTriangle1gt23 ltrdftype rdfresource=cadowlTrianglegt24 ltcadfirstNode rdfresource=cubeowlPoint4gt25 ltcadsecondNode rdfresource=cubeowlPoint2gt26 ltcadthirdNode rdfresource=cubeowlPoint1gt27 ltowlNamedIndividualgt28

29 lt-- Wire1 contains 4 edges The first one --gt30

31 ltowlNamedIndividual rdfabout=cubeowlEdge1gt32 ltrdftype rdfresource=cadowlEdgegt33 ltcadadjacentEdge rdfresource=cubeowlEdge2gt34 ltcadboundedBy rdfresource=cubeowlVertex1gt35 ltcadboundedBy rdfresource=cubeowlVertex2gt36 ltcadrepresentedBy rdfresource=cubeowlLine1gt37 ltcadhasForwardOrientationgtfalseltcadhasForwardOrientationgt38 ltowlNamedIndividualgt39

40 lt-- Vertices are represented by Points that contain the coordinates --gt41 lt-- Line1 has a direction vector and a position --gt

Listing 1 Excerpts of a cube in OntoBREP format (URIs are displayed shortened)

SPARQL queries

SPARQL queries regarding geometric conditions

1 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt2 PREFIX owl lthttpwwww3org200207owlgt3 PREFIX f lthttpwwwontotextcomsparqlfunctionsgt4 PREFIX cad lthttpkblocalrestkbcadowlgt5 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt6 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt7 INSERT

55

8 p1dirN graspisOppositeDirectionOf p2dirN9 f1 parallelismType f2

10 newGeoCondInstanceIri rdftype owlNamedIndividual11 rdftype geoCondClassIri12 cadcontains f1 f213 caddistanceEuclidean distanceBtwnPlanes14 graspoverlappingArea overlappingArea 15 16 WHERE 17 lthttpkblocalrestkbbd-thesis-devowlCubeBlockgt smeroboticsshape compound 18 object smeroboticsshape compound 19 compound cadcontains solid 20 solid cadboundedBy shell 21 shell cadcontains f1 f2 22 FILTER(STR(f1) lt STR(f2)) 23 f1 cadrepresentedBy p1 p1 sesamedirectType cadPlane 24 f2 cadrepresentedBy p2 p2 sesamedirectType cadPlane 25 p1 caddirectionNormal p1dirN 26 p2 caddirectionNormal p2dirN 27 p1dirN cadx p1dirNX cady p1dirNY cadz p1dirNZ 28 p2dirN cadx p2dirNX cady p2dirNY cadz p2dirNZ 29 BIND(p1dirNX -p2dirNX + p1dirNY -p2dirNY + p1dirNZ -p2dirNZ AS

dotProduct) rarr30 BIND(fsqrt(fpow(p1dirNX 2) + fpow(p1dirNY 2) + fpow(p1dirNZ 2)) AS

p1dirNlength) rarr31 BIND(fsqrt(fpow(p2dirNX 2) + fpow(p2dirNY 2) + fpow(p2dirNZ 2)) AS

p2dirNlength) rarr32 BIND(facos(dotProduct (p1dirNlength p2dirNlength)) AS angleBtwnDirNs) 33 BIND(IF(angleBtwnDirNs lt 0 -angleBtwnDirNs angleBtwnDirNs) as angleBtwnDirNs) 34 FILTER (angleBtwnDirNs lt ftoRadians(1)) 35 p1 cadposition p1pos 36 p2 cadposition p2pos 37 p1pos cadx p1posX cady p1posY cadz p1posZ 38 p2pos cadx p2posX cady p2posY cadz p2posZ 39 BIND(p1posX p1dirNX + p1posY p1dirNY + p1posZ p1dirNZ AS

p1coordFormRightSideValue) rarr40 BIND(p2posX p1dirNX + p2posY p1dirNY + p2posZ p1dirNZ AS

scalarPartLeftSide) rarr41 BIND(fpow(p1dirNX 2) + fpow(p1dirNY 2) + fpow(p1dirNZ 2) AS

lambdaPartLeftSide) rarr42 BIND((p1coordFormRightSideValue - scalarPartLeftSide) lambdaPartLeftSide AS

lambda)rarr43 BIND(p2posX + lambda p1dirNX AS iX) 44 BIND(p2posY + lambda p1dirNY AS iY) 45 BIND(p2posZ + lambda p1dirNZ AS iZ) 46 BIND(fsqrt(fpow(p2posX - iX 2) + fpow(p2posY - iY 2) + fpow(p2posZ - iZ

2)) AS distanceBtwnPlanes) rarr47 BIND(IF(lambda gt 0 graspisBackToBackParallelTo graspisFrontToFrontParallelTo) AS

parallelismType)rarr48 BIND(IF(lambda gt 0 graspTwoParallelPlanarFacesOutwards

graspTwoParallelPlanarFacesInwards) AS geoCondClassIri) rarr49 BIND(IRI(CONCAT(STR(geoCondClassIri) _ STRAFTER(STR(object) ) _

STRAFTER(str(f1) ) - STRAFTER(str(f2) ))) AS newGeoCondInstanceIri)

rarrrarr

50 newGeoCondInstanceIri lthttpwwwfortissorgkbcomputeOverlappingAreaOfPolygonsgt(f1 f2) overlappingArea rarr

51

Listing 2 SPARQL query to annotate objects with the geometric condition of TwoParal-lelPlanarFacesInwardsOutwards

56

1 PREFIX cad lthttpkblocalrestkbcadowlgt2 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt3 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt4 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt5 PREFIX owl lthttpwwww3org200207owlgt6 INSERT 7 newGeoCondInstanceIri rdftype owlNamedIndividual8 rdftype geoCondClassIri9 cadcontains face

10 cadradius radius 11 12 WHERE 13 lthttpkblocalrestkbbd-thesis-devowlCylinderBasicgt smeroboticsshape compound

rarr14 object smeroboticsshape compound 15 compound cadcontains solid 16 solid cadboundedBy shell 17 shell cadcontains face 18 face cadrepresentedBy surface surface sesamedirectType cadCylindricalSurface 19 surface cadradius radius 20 face cadhasForwardOrientation orientation 21 BIND(IF(orientation = true graspCylindricalFaceOutwards

graspCylindricalFaceInwards) AS geoCondClassIri)rarr22 BIND(IRI(CONCAT(STR(geoCondClassIri) _ STRAFTER(STR(object) ) _

STRAFTER(str(face) ))) AS newGeoCondInstanceIri) rarr23

Listing 3 SPARQL query to annotate objects with the geometric condition of Cylindrical-FaceInwardsOutwards

1 PREFIX cad lthttpkblocalrestkbcadowlgt2 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt3 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt4 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt5 PREFIX owl lthttpwwww3org200207owlgt6 INSERT 7 newGeoCondInstanceIri rdftype owlNamedIndividual8 rdftype graspPlanarFaceOutwards9 cadcontains face

10 11 WHERE 12 lthttpkblocalrestkbbd-thesis-devowlCubeBlockgt smeroboticsshape compound 13 object smeroboticsshape compound 14 compound cadcontains solid 15 solid cadboundedBy shell 16 shell cadcontains face 17 face cadrepresentedBy plane plane sesamedirectType cadPlane 18 face cadhasForwardOrientation true 19 BIND(IRI(CONCAT(STR(graspPlanarFaceOutwards) _ STRAFTER(STR(object) ) _


Listing 4 SPARQL query to annotate objects with the geometric condition of PlanarFace-Outwards

57

1 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt2 PREFIX cad lthttpkblocalrestkbcadowlgt3 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt4 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt5 PREFIX owl lthttpwwww3org200207owlgt6 SELECT DISTINCT geoCondI geoCondClass paramName paramValue noGrasp WHERE 7 object smeroboticsshape compound 8 FILTER(object = lthttpkblocalrestkbbd-thesis-devowlCubeBlockgt)9 compound cadcontains solid

10 solid cadboundedBy shell 11 shell cadcontains face 12 geoCondI rdftype graspGeometricCondition 13 geoCondI cadcontains face 14 geoCondI sesamedirectType geoCondClass FILTER (geoCondClass =

owlNamedIndividual) rarr15 geoCondI paramName paramValue 16 FILTER(paramName = rdftype) 17 OPTIONAL 18 paramValue cadnoGrasp noGrasp 19 20

Listing 5 SPARQL query to fetch geometric conditions of an object together with theirparameters

SPARQL queries to match grasping capabilities with geometric conditions

1 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt2 PREFIX cad lthttpkblocalrestkbcadowlgt3 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt4 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt5 PREFIX owl lthttpwwww3org200207owlgt6 PREFIX rdfs lthttpwwww3org200001rdf-schemagt7 PREFIX xsd lthttpwwww3org2001XMLSchemagt8 PREFIX f lthttpwwwontotextcomsparqlfunctionsgt9 SELECT DISTINCT graspCapI geoCondI matchingGeoCond geoCondParamName geoCondParamValue

noGraspFace valueCheckingType valueToCheckProperty valuePassesCheckvalueCheckFeasibilityScoreMethod WHERE

rarrrarr

10 object smeroboticsshape compound 11 FILTER(object = lthttpkblocalrestkbbd-thesis-devowlGraspDevObjgt)12 object smeroboticsshape compound 13 compound cadcontains solid 14 solid cadboundedBy shell 15 shell cadcontains objectFace 16 geoCondI rdftype graspGeometricCondition 17 geoCondI cadcontains objectFace 18 geoCondI sesamedirectType geoCondClass FILTER (geoCondClass =

owlNamedIndividual) rarr19

20 lthttpkblocalrestkbsmeroboticsowlSchunkWSG50-110Gearboxgt sesamedirectTypegripperClass FILTER(gripperClass = owlNamedIndividual)rarr

21 gripper sesamedirectType gripperClass 22 gripper grasphasGraspingCapability graspCapI 23 graspCapI sesamedirectType graspCapClass FILTER(graspCapClass =

owlNamedIndividual)rarr24 graspCapClass rdfssubClassOf rest1 25 rest1 owlonProperty grasprequiresGeometricCondition 26 rest1 owlonClass requiredGeoCondClass

58

27 FILTER(geoCondClass = requiredGeoCondClass) BIND(geoCondClass ASmatchingGeoCond) rarr

28

29 geoCondI geoCondParamName geoCondParamValue FILTER(geoCondParamName = rdftype)30 OPTIONAL 31 geoCondParamValue cadnoGrasp noGraspFace 32 33 OPTIONAL 34 graspCapI graspneedsValueChecking valueCheckingType 35 valueCheckingType grasprangeMin rangeMin36 grasprangeMax rangeMax37 graspvalueToCheckProperty valueToCheckPropertyStr38 graspvalueCheckFeasibilityScoreMethod valueCheckFeasibilityScoreMethod 39 BIND(IRI(valueToCheckPropertyStr) AS valueToCheckProperty) 40 FILTER(geoCondParamName = valueToCheckProperty) 41 BIND(IF(geoCondParamValue gt= rangeMin ampamp geoCondParamValue lt= rangeMax true

false) AS valuePassesCheck) rarr42 43 BIND(IF(EXISTSgeoCondParamValue cadnoGrasp noGraspFace false

IF(BOUND(valuePassesCheck) valuePassesCheck true)) AS valuePassesCheck) rarr44

Listing 6 SPARQL query to match geometric conditions of an object with the graspingcapabilities of gripper and consider value checks

BrepOwlLoader SPARQL queries

1 PREFIX cad lthttpkblocalrestkbcadowlgt2 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt3 SELECT face wire edge vertex point WHERE 4 lthttpkblocalrestkbbd-thesis-devowlCubeBlockgt smeroboticsshape compound 5 compound cadcontains solid 6 solid cadboundedBy shell 7 shell cadcontains face 8 face cadboundedBy wire 9 wire cadcontains edge

10 edge cadboundedBy vertex 11 vertex cadrepresentedBy point 12

Listing 7 SPARQL query to get all URIs from a facersquos wirersquos subelements

1 PREFIX cad lthttpkblocalrestkbcadowlgt2 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt3 SELECT face point1 point2 point3 WHERE 4 lthttpkblocalrestkbbd-thesis-devowlCubeBlockgt smeroboticsshape compound 5 compound cadcontains solid 6 solid cadboundedBy shell 7 shell cadcontains face 8 face cadtriangulatedBy triangulation 9 triangulation cadcontains triangle

10 triangle cadfirstNode point1 cadsecondNode point2 cadthirdNode point3 11

59

Listing 8 SPARQL query to get triangulation points of faces

1 PREFIX cad lthttpkblocalrestkbcadowlgt2 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt3 SELECT DISTINCT point coords WHERE 4 lthttpkblocalrestkbbd-thesis-devowlCubeBlockgt smeroboticsshape compound 5 compound cadcontains solid 6 solid cadboundedBy shell 7 shell cadcontains face 8 face cadtriangulatedBy triangulation 9 triangulation cadcontains triangle

10 11 triangle cadfirstNode point1 BIND(point1 AS point)12 13 UNION 14 triangle cadsecondNode point2 BIND(point2 AS point)15 16 UNION 17 triangle cadthirdNode point3 BIND(point3 AS point)18 19 point cadx x point cady y point cadz z 20 BIND(concat(str(x) str(y) str(z)) AS coords)21

Listing 9 SPARQL query to get points and their coordinates

1 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt2 PREFIX owl lthttpwwww3org200207owlgt3 PREFIX cad lthttpkblocalrestkbcadowlgt4 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt5 SELECT face surface surfaceType positionCoords dirXCoords dirYCoords dirZCoords

radius WHERE rarr6 lthttpkblocalrestkbbd-thesis-devowlCubeBlockgt smeroboticsshape compound 7 compound cadcontains solid 8 solid cadboundedBy shell 9 shell cadcontains face

10 face cadrepresentedBy surface 11 surface sesamedirectType surfaceType 12 FILTER (surfaceType = cadTriangulation ampamp surfaceType = owlNamedIndividual) 13 surface cadposition position 14 position cadx Px cady Py cadz Pz 15 BIND(concat(str(Px) str(Py) str(Pz)) AS positionCoords)16 surface caddirectionX dirX caddirectionY dirY caddirectionZ dirZ 17 dirX cadx DXx cady DXy cadz DXz 18 BIND(concat(str(DXx) str(DXy) str(DXz)) AS dirXCoords)19 dirY cadx DYx cady DYy cadz DYz 20 BIND(concat(str(DYx) str(DYy) str(DYz)) AS dirYCoords)21 dirZ cadx DZx cady DZy cadz DZz 22 BIND(concat(str(DZx) str(DZy) str(DZz)) AS dirZCoords)23 OPTIONAL 24 surface cadradius radius 25 26

60

Listing 10 SPARQL query to get the surfaces representing faces

Other SPARQL queries

1 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt2 PREFIX swdl lthttpkblocalrestkbswdlowlgt3 PREFIX cad lthttpkblocalrestkbcadowlgt4 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt5 SELECT WHERE 6 lthttpkblocalrestkbworkcellfortiss_cobot_workcellowlFortissCoBotWorkcellgt

(swdlsucceedingJoint | swdlsucceedingLink) gripper rarr7 gripper rdftype smeroboticsGripper 8 gripper rdftype lthttpkblocalrestkbsmeroboticsowlSchunkWSG50-110Gearboxgt 9 gripper (swdlsucceedingJoint | swdlsucceedingLink) node

10 node ˆ(swdlsucceedingJoint | swdlsucceedingLink) parent 11 OPTIONAL 12 node swdltransformation transMatrix 13 transMatrix14 cada11 a11 cada12 a12 cada13 a13 cada14 a14 15 cada21 a21 cada22 a22 cada23 a23 cada24 a24 16 cada31 a31 cada32 a32 cada33 a33 cada34 a34 17 cada41 a41 cada42 a42 cada43 a43 cada44 a44 18 19 OPTIONAL 20 node smeroboticsmodelUrl modelUrl 21 22

Listing 11 SPARQL query to get the spatial orientation of the grippers parts for displayingit in the grasp planner

1 PREFIX grasp lthttpkblocalrestkbbd-thesis-devowlgt2 PREFIX cad lthttpkblocalrestkbcadowlgt3 PREFIX rdf lthttpwwww3org19990222-rdf-syntax-nsgt4 PREFIX owl lthttpwwww3org200207owlgt5 PREFIX rdfs lthttpwwww3org200001rdf-schemagt6 PREFIX smerobotics lthttpkblocalrestkbsmeroboticsowlgt7 PREFIX dcterms lthttppurlorgdctermsgt8 SELECT graspCapI title graspCapClass requiredGeoCondClass faceList

gripperFingerOfFaceList face cappedByFace paramName paramValue WHERE rarr9 gripper rdftype owlNamedIndividual

10 gripper rdftype lthttpkblocalrestkbsmeroboticsowlSchunkWSG50-110Gearboxgt 11 gripper rdftype gripperClass 12 gripperClass sesamedirectType smeroboticsGripper 13 gripper rdftype owlClass 14 15 gripper grasphasGraspingCapability graspCapI 16 graspCapI dctermstitle title 17 graspCapI sesamedirectType graspCapClass FILTER(graspCapClass =

owlNamedIndividual)rarr18 graspCapI cadcontains faceList 19 faceList cadcontains face 20 faceList cadcontainedBy gripperFingerOfFaceList

FILTER(gripperFingerOfFaceList = graspCapI) rarr21 OPTIONAL 22 faceList graspcappedBy cappedByFace

61

23 24 graspCapClass rdfssubClassOf restriction 25 restriction owlonProperty grasprequiresGeometricCondition 26 restriction owlonClass requiredGeoCondClass 27 UNION 28 gripper paramName paramValue 29 FILTER(paramName = rdftype ampamp paramName = rdfssubClassOf ampamp paramName =

grasphasCapability ampamp paramName = grasphasGraspingCapability)rarr30 31

Listing 12 SPARQL query to get the parameters of the gripper and its grasping capabilitiesincluding their required geometric conditions and the involved faces

62

63

Bibliography

[1] Brenna D Argall Sonia Chernova Manuela Veloso and Brett Browning A surveyof robot learning from demonstrationrdquo Robotics and Autonomous Systems 57(5)469 ndash483 2009

[2] M Beetz D Beler J Winkler J Worch F Blint-Benczdi G Bartels A Billard A KBozcuolu N Figueroa A Haidu H Langer A Maldonado A L P UrecheM Tenorth and T Wiedemeyer Open robotics research using web-based knowl-edge services In 2016 IEEE International Conference on Robotics and Automation (ICRA)pages 5380ndash5387 May 2016

[3] A K Bozcuolu and M Beetz A cloud service for robotic mental simulations In 2017IEEE International Conference on Robotics and Automation (ICRA) pages 2653ndash2658 May2017

[4] A K Bozcuolu G Kazhoyan Y Furuta S Stelter M Beetz K Okada and M InabaThe exchange of knowledge using cloud robotics In 2018 IEEE International Conferenceon Robotics and Automation (ICRA) pages 1ndash8 May 2018

[5] Gordon Cheng Karinne Ramirez-Amaro Michael Beetz and Yasuo Kuniyoshi Pur-posive learning Robot reasoning about the meanings of human activities ScienceRobotics 4(26) 2019

[6] J Schmiedmayer HB Dollar AM Kragic D Feix T Romero The grasp taxonomy ofhuman grasp types In Human-Machine Systems IEEE Transactions on 2015

[7] Thomas Feix Roland Pawlik and Heinz-Bodo Schmiedmayer The generation of acomprehensive grasp taxonomy In RSS Robotics Science and Systems Workshop on Un-derstanding the Human Hand for Advancing Robotic Manipulation RSSrsquo09 Seattle USA01 2009

[8] Sandro Fiorini Joel Carbonera Paulo Gonalves Vitor Jorge Vitor Rey Tamas Haideg-ger Mara Abel Signe Redfield Stephen Balakirsky and Veera Ragavan Sampath Ku-mar Extensions to the core ontology for robotics and automation Robotics andComputer-Integrated Manufacturing 33 09 2014

[9] Ken Goldberg and Ben Kehoe Cloud robotics and automation A survey of relatedwork Technical Report UCBEECS-2013-5 EECS Department University of Califor-nia Berkeley Jan 2013

[10] Martin Kraft and Markus Rickert How to teach your robot in 5 minutes Applyingux paradigms to human-robot-interaction 08 2017

[11] J R Napier The prehensile movements of the human hand 38-B902ndash13 12 1956

65

Bibliography

[12] J Persson A Gallois A Bjoerkelund L Hafdell M Haage J Malec K Nilsson andP Nugues A knowledge integration framework for robotics In ISR 2010 (41st Interna-tional Symposium on Robotics) and ROBOTIK 2010 (6th German Conference on Robotics)pages 1ndash8 June 2010

[13] A Perzylo M Rickert B Kahl N Somani C Lehmann A Kuss S Profanter A BBeck M Haage M R Hansen M Roa-Garzon O Sornmo S Gestegard RobertzU Thomas G Veiga E A Topp I Kessler and M Danzer Smerobotics Smart robotsfor flexible manufacturing IEEE Robotics Automation Magazine pages 1ndash1 2019

[14] A Perzylo N Somani S Profanter I Kessler M Rickert and A Knoll Intuitiveinstruction of industrial robots Semantic process descriptions for small lot produc-tion In 2016 IEEERSJ International Conference on Intelligent Robots and Systems (IROS)pages 2293ndash2300 Oct 2016

[15] A Perzylo N Somani M Rickert and A Knoll An ontology for cad data and ge-ometric constraints as a link between product models and semantic robot task de-scriptions In 2015 IEEERSJ International Conference on Intelligent Robots and Systems(IROS) pages 4197ndash4203 Sept 2015

[16] S Profanter A Perzylo N Somani M Rickert and A Knoll Analysis and seman-tic modeling of modality preferences in industrial human-robot interaction In 2015IEEERSJ International Conference on Intelligent Robots and Systems (IROS) pages 1812ndash1818 Sept 2015

[17] Karinne Ramrez-Amaro and Gordon Cheng Accelerating the teaching of industrialrobots by re-using semantic knowledge from various domains 05 2018

[18] C Schlenoff E Prestes R Madhavan P Goncalves H Li S Balakirsky T Kramerand E Miguelez An ieee standard ontology for robotics and automation In 2012IEEERSJ International Conference on Intelligent Robots and Systems pages 1337ndash1342Oct 2012

[19] Nikhil Somani Caixia Cai Alexander Perzylo Markus Rickert and Alois Knoll Ob-ject recognition using constraints from primitive shape matching In Proceedings of the10th International Symposium on Visual Computing (ISVCrsquo14) pages 783ndash792 Las VegasNV USA December 2014 Springer

[20] D Song C H Ek K Huebner and D Kragic Task-based robot grasp planning usingprobabilistic inference IEEE Transactions on Robotics 31(3)546ndash561 June 2015

[21] M Stenmark M Haage E A Topp and J Malec Supporting semantic capture dur-ing kinesthetic teaching of collaborative industrial robots In 2017 IEEE 11th Interna-tional Conference on Semantic Computing (ICSC) pages 366ndash371 Jan 2017

66

Acknowledgements

Abstract

Introduction

Motivation and Goal

Scope Of Tasks And Structure Of This Thesis

Related Work

Theory

Theoretical Basis

What Are Ontologies

Inference Enriches The Ontology And Yields Insights

Existing Ontologies From fortiss

OntoBREP Reasoning About CAD Data

Semantic Process Description

Developing The Semantics Of Grasping

Starting With A Taxonomy

The Concept To Find Valid Grasp Modes

What The Gripper Offers Grasping Capabilities

What The Object Offers Geometric Conditions

Getting Applicable Grasp Modes

Implementation

Part 1 Work On The Knowledge Base

Additions To The Existing Ontologies

SPARQL Queries Working With The Modelled Ontologies

GraphDB Plugin To Compute Overlapping Area

Part 2 Integration Into The Existing Software

fortiss Robot Instruction Framework

Parsing OntoBREP In The GUI

Two New GUI Elements For The Semantics Of Grasping

Discussion Conclusion And Future Work

Appendix

Bibliography



Masterrsquos Thesis

Formal specification of the semantics of grasping usingthe Web Ontology Language (OWL) and integration into

a robot programming GUI

Author Benjamin Aaron DegenhartExaminer Prof Dr-Ing habil Alois Christian KnollAssistant advisor Alexander Perzylo fortiss GmbHSubmission Date March 15 2019



Acknowledgments


vii

Abstract


ix

Legend





x

Contents

Acknowledgements

vii

Abstract

ix

1 Introduction



2

2 Related Work

3

3 Theory







9







14

5 Implementation









39


49

Appendix

55

Bibliography

63

xi

1 Introduction













1

1 Introduction








The next chapter 2







2

2 Related Work










in 2016 [2





1httpgraspxiefnet


3httprsscslthse

3

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography



Acknowledgments


vii

Abstract


ix

Legend





x

Contents

Acknowledgements

vii

Abstract

ix

1 Introduction



2

2 Related Work

3

3 Theory







9







14

5 Implementation









39


49

Appendix

55

Bibliography

63

xi

1 Introduction













1

1 Introduction








The next chapter 2







2

2 Related Work










in 2016 [2





1httpgraspxiefnet


3httprsscslthse

3

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

Abstract


ix

Legend





x

Contents

Acknowledgements

vii

Abstract

ix

1 Introduction



2

2 Related Work

3

3 Theory







9







14

5 Implementation









39


49

Appendix

55

Bibliography

63

xi

1 Introduction













1

1 Introduction








The next chapter 2







2

2 Related Work










in 2016 [2





1httpgraspxiefnet


3httprsscslthse

3

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

Legend





x

Contents

Acknowledgements

vii

Abstract

ix

1 Introduction



2

2 Related Work

3

3 Theory







9







14

5 Implementation









39


49

Appendix

55

Bibliography

63

xi

1 Introduction













1

1 Introduction








The next chapter 2







2

2 Related Work










in 2016 [2





1httpgraspxiefnet


3httprsscslthse

3

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

Contents

Acknowledgements

vii

Abstract

ix

1 Introduction



2

2 Related Work

3

3 Theory







9







14

5 Implementation









39


49

Appendix

55

Bibliography

63

xi

1 Introduction













1

1 Introduction








The next chapter 2







2

2 Related Work










in 2016 [2





1httpgraspxiefnet


3httprsscslthse

3

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

1 Introduction













1

1 Introduction








The next chapter 2







2

2 Related Work










in 2016 [2





1httpgraspxiefnet


3httprsscslthse

3

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

1 Introduction








The next chapter 2







2

2 Related Work










in 2016 [2





1httpgraspxiefnet


3httprsscslthse

3

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

2 Related Work










in 2016 [2





1httpgraspxiefnet


3httprsscslthse

3

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

2 Related Work









4

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

3 Theory












2httpwwwqudtorg


5

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

3 Theory





) for instance








6













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography













] it is stated that



7

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

3 Theory


Figure 31





Figure 31





Figure 32

8





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography





]Figure 32



9









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography









11





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography





Figure 42





12









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography









Figure 43





13












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography












Figure 44


14









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography









and 44


15









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography









Figure 45




16



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography



17

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation












Protege2





19

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation

Figure 51






GraphDB3





Figure 53



20








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography








21

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation






Figure 54



22








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography








Figure 55



23

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation


class







Figure 56




24















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography















25

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation












and 4


26




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography




gt everytime










27

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation



(51)







28



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography



l ~x =

p2posXp2posYp2posZ

+ λ

p1dNXp1dNYp1dNZ

(52)











3


5


29

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation











Listing 12





30



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography



Listing 58




31

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation





)value checking







32











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography











33

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation
















34

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

















35

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation





rarrrarr

27









=


(53)



36












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography












Figure 58



37

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation





Figure 59






38


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography


bull query 7


bull query 8


bull query 9


bull query 10













39

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation












40








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography








)





Figure 512




41

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation










Figure 514







42







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography







rarrrarr


Listing 516





43

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation







44



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography



45

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation



46




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography




47

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

5 Implementation


48


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography


Discussion






49



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography



Conclusion


Future Work







50







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography







51







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography







2httpswwwstardogcom

52

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

Appendix

53




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography




10








SPARQL queries



55














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography














rarrrarr


51


56











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography











57








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography








rarrrarr






58


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography


28











59








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography








60












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography












61




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography




62

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

63

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

Bibliography












65

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

Bibliography











66

Acknowledgements

Abstract

Introduction

Motivation and Goal


Related Work

Theory

Theoretical Basis

What Are Ontologies











Implementation










Appendix

Bibliography

computational science and engineeringin the ﬁeld of human grasping, there exists extensive...

Documents