event composition operators
TRANSCRIPT
Event Composition Operators: ECO Setareh Rafatirad
Department of Computer Science University of California, Irvine
Irvine, CA, USA
Amarnath Gupta San Diego Supercomputer Center University of California, San Diego
La Jolla, CA, USA
Ramesh Jain Department of Computer Science
University of California, Irvine Irvine, CA, USA
ABSTRACT Events are at least as important as objects in modeling the dynamic universe. Modeling the real world and weaving the web of events require Composite Events that are valid constitution of atomic and composite sub-events. The progress in event composition has been limited to construction of some entities and relationships in upper ontologies such as Event–Model-F, and DSO ontology that are extensions of BFO,DOLCE, and E-Model. For building a composite event from atomic events, different operators must be used to provide right attributes for this event. Event Composition Operators on all attributes of events should be defined. The knowledge of context semantics of sub-event facets should be used for valid population of the resulting composite event attributes. Our contribution and focus in this paper is to define and implement such event composition operators on spatial and temporal aspect of events.
Categories and Subject Descriptors I.2.4 [ARTIFICIAL INTELIGENCE]: Knowledge Representation Formalisms and Methods – Semantic Networks; H.5.1 [INFORMATION INTERFACES AND PRESENTATION]: Multimedia Information Systems - Evaluation/methodology
General Terms Measurement, Standardization
Keywords Event Representation and Event models, Event Composition Operators, Spatial facet, Temporal facet
1. INTRODUCTION Many applications require modeling physical world that we live in. Dynamic real world is modeled using both objects and events [1]. Objects have received significant attention for modeling in computer science, but events received only limited attention. Recently strong event models have been proposed to model the concept of events in diverse applications. Events are not just spatiotemporal data. The structure and semantics of events plays key role in modeling and can be represented as a directed graph. Events are either atomic or composite. Atomic events have been
broadly discussed in the literature whereas construction of composite events is still in very early stages. We focus on the issues of event composition using the subevent-of relationship between events. In order to represent the possible semantics of a composite event, the event attributes should be computed as a function of its subevent attributes through some form a recursive union operation. The subevent-of relationship induces a directed acyclic graph over events (an atomic event can be a subevent of two distinct composite events). The hierarchy in Figure 1 is an example of a composite event. It shows the meeting schedule of “Jon” extracted from his calendar. The schedule indicates the following list of individual meetings on June 21, 2009: Atomic-event e1:“Meeting with Mina” from 10:15 a.m. to 12 p.m. in Peet’s Café; Atomic-event e2: “Meeting with Tom and Brian” from 12 p.m. to 2 p.m. in Peet’s Café; Atomic-event e3: “Meeting with COM group” from 2 p.m. to 4 p.m. at Jon’s house.
Figure 1:Jons' Meeting schedule
We will use format <YYYY-MM-DDThh:mmTZD> or <YYYY-MM-DDTZD> for time and <lat,long> for space representation. We assume the Time-Zone for all the temporal attributes in all examples of this paper is the same. Meeting schedule in Figure 1 includes two main composite events: “Meeting with TECH group” that is Jon’s meeting schedule with technical people consisted of Mina, Tom, and Brian; and “Meeting with Jon’s group” that is Jon’s meeting schedule with both TECH and COM (i.e. commercial people) groups. We take into account an EventBase that is an information system where events are stored along with their properties and inter-relationships. Formally EventBase can be modeled as a node and edge labeled RDF graph where a node may represent an event or object, and an edge may denote heterogeneous types of inter-node (e.g. sub-event-of) and property relationships (e.g. has-location).
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. EiMM’09, October 23, 2009, Beijing, China. Copyright 2009 ACM 978-1-60558-754-7/09/10...$10.00.
65
Space and time are intrinsic data properties of an event graph and as we will show in sections 2.1 and 2.2, they can be presented by heterogeneous types. Our data model yields an induced sub-graph from this RDF graph. Out of several event-specific relationships we focus on the sub-event-of relationship. Each event node may be subevent-of one or more composite events but the interesting observation is that the outcome of focusing on this relationship is a hierarchical graph similar to Figure 1 in which the solid edges denote subevent-of relationship (e1 subevent-of e12, e2 subevent-of e12, and e3 sub-event-of e123). In this paper, we consider the universe of subgraphs induced by the sub-event-of relationship of an event graph. For the event nodes in these subgraphs we only consider the space and time data properties. The space and time property relationships we use here are has-location, and occurs-in-time to access the spatial location and temporal attribute associated with event nodes-- the dashed edges in Figure 1 represent time and space property relationships in describing the spatiotemporal semantics of the leaf level nodes (i.e. atomic events). In these subgraphs we observe that the leaf level nodes have observed values of space and time. In order to express the spatial and temporal semantics of a composite event we need to evaluate the space and time properties of the non-leaf node at an arbitrary level of depth. Going back to the example we would like to evaluate the space and time attributes of nodes e123 and e12. We propose the following scenario: Suppose Jon is invited to attend a ceremony on the same day he is supposed to meet his group. Thus he needs to make a survey about the important constraints of his meeting schedule to respond to the invitation. Two important constraints for Jon are time and space. The ceremony is going to be held in Santa Monica, so he needs to see how far is the location of his meeting to the location of the ceremony on atlas, and how long the meeting lasts. For this reason he needs to answer three questions: “where is the location of e123”, “when does e123 take place”, and “how long e123 lasts”. With this information he decides whether or not he will have sufficient time to arrive Santa Monica on time. Also based on the location of e123 and the distance from this location to Santa Monica, he may decide to commute the road with his own car or just catch a bus. Now the problem is the absence of spatiotemporal semantics of composite event e123. A recursive union aggregation mechanism can fill out such absence. Thus the next level of the hierarchy is checked. In this level there is still another composite event e12 with empty spatiotemporal attributes in addition to the atomic event e3.The spatiotemporal attributes of e12 have to be aggregated from its lower level atomic events e1 and e2 to ultimately fill the spatiotemporal attributes of e123 (bottom-up aggregation). To compute the time and location of e12 we need a temporal operator that returns time interval [2009-06-21T10:15TZD,2009-06-21T14:00TZD] as a function of time intervals associated with e1 and e2, and a spatial operator that returns location “Peet’s Café”: <lat:33.6491284,long:-117.8395201> as a function of <lat,long> pairs associated with e1 and e2. Similarly for finding the location and time of composite event e123 temporal and spatial operators are needed to return time interval [2009-06-21T10:15TZD,2009-06-21T16:00TZD] and locations “Peet’s Café “ and “Jon’s house” as a function of temporal and spatial attributes of e12 and e3. In this simple example we considered only a small hierarchy of events. In many routine applications, one has hierarchies that may span several – maybe hundreds – level of events. In all these compositions one has only the attributes, such as time and
location, for only atomic events. These attributes for all higher level events must be computed from the attributes of atomic events. Thus for composing events we need a framework which offers variety of tools to provide valid event-attributes. The sub-event-of relationship is an aggregating relationship, i.e. the property Pi of node n can be computed using a recursive union of the property values (i.e. attributes) of the descendants of node n reachable via edges labeled sub-event-of. In some cases this recursive union is simple to compute-- Suppose in Figure 1 e1 and e2 are each associated with bunch of photos. We present the experiential property of the events by PE: e1. PE denotes the photos associated with e1 (e1.PE=<photo-1, photo-3,photo-5>); similar statement is true for e2 (e2.PE=<photo-2,photo-4>). To populate the experiential property of e12, a simple union operator returns the union of photos: e12. PE = e1. PE U e2.PE,thus e12. PE =< photo-1, photo-2, photo-3, photo-4, photo-5>. Sometimes the outcome might be a bag union. This type of union may be useful in inferring participant semantics of events, e.g. the number of times participant x is tagged in the photos associated with e12. However For spatial and temporal data properties the problem is more complex. It is important to solve this problem against queries that are posted against non-atomic events. This solution must involve different computations in addition to simple union operation. Our contribution in this work is introducing operators with such capabilities. In this paper we propose ECO -- a set of Event Composition Operators. Such operators propagate properties of atomic events to composite events. The composition operation for a composite event is done via operators functioning on properties of descendant sub-events, and then combining the results obtained from each operator. In order to realize what types of operators are required, we need to identify common properties of sub-events e.g., we need operators for spatial and temporal facets. This requires an event model to capture such facets. In this paper we use Event Model, called E-Model, proposed by Westerman and Jain [2] that introduces six common facets for any arbitrary event: Temporal, Spatial, Informational, Causal, Structural, and Experiential. Figure 2 shows this model:
• Temporal facet: time is a built-in attribute of any arbitrary event. In fact what mainly distinguishes events from objects is the temporal facet. Being concerned about the starting time and duration of meetings with “TECH” and “COM” group stated earlier are examples of an application interest in the underlying events’ temporal aspects. Continuous or instantaneous, divergent temporal measurement units might correspond to arbitrary events.
• Spatial facet: Being concerned about the location in which “meeting with TECH group” has taken place is an application interest example in the spatial aspect of the underlying event. Spatial facet provides cognizance about location.
• Informational facet: It covers various information such as event type, and information about people and objects that are engaged with the event.
• Causal facet: It reflects the cause for an event to be created, destroyed, or changed.
• Structural facet: The main reason of building ECO is the structural facet of events that refers to different levels of abstractions in terms of event composition.
• Experiential facet: This is the media referencing aspect of E-Model to various types of media, from discrete (i.e. images) to
66
continuous (i.e., video), and other complex and sensor data. It denotes multimedia cognizance.
In this work atomic and composite notions for events are taken from Westerman and Jain [3]. They defined atomic events as classes of events which are unbreakable into any further sub-events.
Figure 2:E-Model [2]
An extension to E-Model is Event-Model-F introduced by Scherp et. al. [4] that includes some of the important notions represented in DOLCE ontology [12] e.g., perdurant, endurant, participant, etc. In addition to that it introduces a mereology pattern for composition of events. Some of the current models have formalized Allen’s time [5] and RCC-8’s space [6] calculus as well. One may borrow the above related techniques to apply on top of the results obtained from composition operators for a consolidation of available techniques. Basically there are two implications of composition operation. We use the popular example of adding apples and oranges. Using Addition operator for three apples and two oranges either results in union of 3 apples and 2 oranges, or 5 fruits. The similarity of these two results is that initially we take the simple union of apples and oranges in both cases. However in the second case we go one level above in terms of resolution and exploit a domain knowledge semantic that states “Apple is-a fruit AND Orange is-a fruit”. This paper only focuses on the former case while techniques like mapping (i.e., changing the uniting representation of the underlying element; e.g., 30-minutes is half of an hour) and change-of-resolution (i.e. merging mechanism similar to the one used in geographical maps; e.g., merging States to represent a Country) can be applied on top of the results produced by the composite operators. In this paper we address composition operators for only two facets – spatial and temporal. No framework has been proposed so far to construct the attributes of composite events as a function of their sub-events. We show how these operators become pragmatic in event composition and development of future event processing tools via some examples. Applications decide which set of operators to use. The case study at the end of the paper addresses the utilization of spatial and temporal composite operators.
2. EVENT COMPOSITION OPERATORS In order to develop the required framework for composing events, we need to define operators for the six event facets but in this paper, we will only address time and space. We introduce Ospace= {Sspace, Rspace,, Lspace, Hspace} representing Spot, Region, Line, and Hybrid spatial operators; and Otime= {Mtime, Itime, THtime} representing moment/time-point[7], time-Interval, and Time-Hybrid temporal operators. Figure 3 shows two events e1 and e2 are composed via spatial and temporal operators to form composite event E.
e1 e2
E
ECO
Figure 3: Event Composition
We define operators independent of domains. However in various domains different operators are used manually or automatically. The types of attributes for space or time are identified and proper operators are utilized to combine those types. For instance, to compose two atomic events, both with spatial attribute spot into a composite event with spatial attribute line; spatial operator Lspot (L indicates the output is a line) which is a member of Lspace family (introduced later) is used. Please note that the main focus here is providing tools to be utilized by various applications. We are not advocating the mechanism for the selection of operators. The way some of the operators are implemented is done via reusing other operators. We will explain the logic behind the notations in the next sections.
2.1 Spatial Operators Let’s have a brief understanding of space and time. Spot, region, and line are space elements. Spot is a single point in the space represented by a <lat,long> pair, e.g., Jon’s-house has-spatial-representation spot-house, spot-house has-latitude “33.645115”, and spot-house has-longitude “-117.831877”. Thus spot-house has-lat-long <33.645115,-117.831877>. We treat a spot as a 0-D space indicated in Figure 4-(a). Line is a 1-D space represented by a collection of spots(i.e., <lat,long> pairs) which comprise connected collection of straight lines indicated in Figure 4-(b). Line may be correlated with time since it sometimes represents a directed path starting from a source (src) and ending to a destination (dst): Figure 4-(c). Thus here the order of src, dst, and other spots located between src and dst on the line trajectory matters i.e. it implies a temporal order (the temporal location of src is always before the temporal location of dst). Region on the other hand is a 2-D space and is represented by pairs of <lat,long> indicated in Figure 4-(d), e.g., his-farm has-spatial-representation region-farm, region-farm contains (spot-1 and spot-2), spot-1 has-lat-long <33.647639,-117.831774>, spot-2 has-lat-long <33.66355,-117.70679>. We can express region as collection of spots in the space. In this paper we only consider 2-D space for line and region. Extensions to consider altitude as the third dimension appear to be straightforward and are not considered in this paper for brevity. We define Hybrid as a set containing union of space elements, e.g. a collection of a spot, a region, and a line. These concepts are used in Formulation of spatial operators. Having the assumption that the Hybrid set in Figure 5 is obtained as a result of a spatial operator, the collection of “spot-1 is-entrancePoint-of region-1”, and “line-1 is-disjoint-with region-1” is an example of applying higher level composition techniques on the Hybrid set to stitch spot-1, region-1, and line-1 together and ultimately obtain the structure
67
in Figure 5. Some of these relationships like entrancePoint-of express semantics of domain knowledge. RCC-8 relationships (e.g. disjoint-with) on a hybrid set can as well create a SpatialGraph similar to Figure 5 showing a stitched holistic view of the elements based on inter-space-element relationships, though they don’t need domain knowledge. In order to understand the utility of Hybrid type, we use an example. Let’s imagine that a robbery has taken place in a jewelry store. There are multiple spatial locations associated with composite event robbery: a spot at which the thief has been shot, and the region in which the evidence of robbery is present. Police needs to realize the semantic relationship between these two spatial elements for its investigation. Without the existence of the hybrid set which consists of that spot and region, no tool can express semantics of the domain knowledge to state the spot is actually the entrancePoint-of that region. Thus not only RCC-8 relationships can be expressed on a hybrid element, but also semantic relations derived from domain knowledge can be applied on the sub-elements of a hybrid element.
Figure 4: Spatial Elements
Figure 5:SpatialGraph
We categorize the main spatial operators into four groups Sspace, Rspace, Lspace, and Hspace such that the associated derivations of each group are supposed to return a spot, region, line, and hybrid representation of their subordinate elements respectively. The spatial operators using the introduced notions (i.e., spot, line, region, and hybrid) are listed below: Sspace= SS ; Rspace= {RS , RL , RR , RS-R , RL-R , RS-H , RL-H , RR-H , RH}; Lspace= {LS , LS-L , LL , LS-H , LL-H , LH}; Hspace= {HS ,HS-R ,HS-L ,HS-H ,HR ,HR-L ,HR-H ,HL ,HL-H, HH}; We have used “S” as “spot”, “R” as “region”, “L” as “line”, and “H” as “hybrid” for the sake of brevity in the notations. The term space used in these notations refers to the type of input parameters on which these operators function on. For instance SS indicates the input parameters of this operator are of type spot and the operator is expected to return a spot, on the other hand RS-R indicates that
the type of input parameters are of type spot and region, and the operator is expected to return a region (i.e. output). However according to the inter-space-element relationships in Table 1,Table 2, and Table 3 these operators can be converted into each other. The tables show spatial operators return heterogeneous spatial properties as a function of their input parameters. In each formulation, the spatial relationships between the input parameters yields a different sub-formula (e.g. in case of operator SS in Table 1, if spot parameters are disjoint, sub-formula-(a) is used, while if they are equal, sub-formula-(b/c/d) may be used) . The asterisk symbol in all the tables of this paper implies applying multiple operators to the input parameter(s).
2.1.1 Operator Definition The result obtained by an operator is a function of its properties i.e., being commutative, reflexive, and associative. All the spatial operator types except Lspace are reflexive in a subset of their family. The remaining subset in each type that functions on heterogeneous space-elements can not have the reflexive property. For instance SS-R is not reflexive since it can only be valid to function on a spot and region, not on a spot and itself. All the spatial operator types except Lspace that is correlated with time are commutative. In contrast with the addressed properties, associative property can not be determined in the level of operators as it is purely based on the application specification, thus we don’t talk about that here. Spatial operators are described in Table 1, Table 2, Table 3, and Table 4. Please consider the following abbreviations for RCC-8 spatial relations: EQ = EQuals, EC = Externally-Connected, IC=Internally Connected, DIS = DISjoint, CON = CONtains, OV = partially-OVerlaps. Note that is-<relationship-name> opposites the implication of the relationship i.e., r1 is-CON r2 means r1 is-contained-in r2 OR r2 contains r1. We describe family of Sspace, Rspace, Lspace, and Hspace in the following sections.
2.1.1.1 Sspace Family Example: Figure 6 shows composing 2 events on their spatial aspect spot-1 and spot-2 using operator SS described in Table 1. The underlying operator takes the spatial attributes of both events as its input parameters.
Figure 6:(a)before composition; (b)after composition
Having descriptions indicated in the Figure 6-(a), and “spot-1 is-disjoint-from spot-2”, concludes SS(spot-1 DIS spot-2) = HS(spot-1 DIS spot-2) = <spot-1 U spot-2> which is produced in Figure 6-(b); the hybrid operator is explained in Table 4. In this example SS-(a) is used.
Table 1:Sspace Family Sspace Formulation
SS (S1,S2)
a)SS (S1 DIS/EC S2)= HS(S1, S2), operator SS is converted to operator Hs; b)SS(S1 EQ S2) = S1 OR S2; c) SS(S1 EQ S2) = S1;presicion heterogeneity d) SS(S1 EQ S2) = S2;presicion heterogeneity
(a) spot : 0-D (b) line : 1-D
(c) line correlated with time (d) region : 2-D
(a) (b)
68
There is an interesting observation in parameters of operators SS-(b,c,d) in Table 1. In all of these three operators the parameters are equal but it does not mean they have the same level of precision. Given two spots S1 and S2 with <lat,long> values <33.64,-117.83> and <33.64763,-117.83177> respectively, one can realize that these spots are at different level of precision. Operators SS-(c,d) are created for this type of situation. It is up to the application whether it prefers to have the finer or courser precision. If it prefers finer precision, SS-(d) is used; otherwise SS-(c) is used (coarser precision). SS-(b) on the other hand is used when the precision level of parameters is the same.
2.1.1.2 Rspace Family 2.1.2 Example: Assume atomic events e1 and e2 have spatial attributes line1 and line2 in Figure 7. Line1 and line2 have 2 intersected spots in common.
Figure 7: Region surrounded by two lines
If the application needs to have the spatial attribute of the composite event in form of a region, operator RL should be used for the computation. Operator RL-(a) in Table 2 with parameters line1 and line2 returns the colored region “Region-1”. Please note that for each line, all the spots on the trajectory of the line are taken into account to surround the colored region in Figure 7.
Table 2:Rspace Family
Rspace Formulation
RS(S1,S2) a)RS(S1 DIS/EC S2)= R that is a square or rectangle region including S1 and S2 ; b)RS(S1 EQ S2)=not valid;
RL(L1,L2)
a)RL(L12≥I L2) = R i.e. the region that is
surrounded by L1-and-L2 if these 2 lines intersect in at-least 2 spots; b)else, RL(L1,L2)= HL(L1,L2)
RR(R1,R2)
a)RR(R1 IC/is-CON R2)= R2; b)RR(R1 EQ R2)=R1 OR R2; c)RR(R1 EC R2)=(R1 U R2); d)RR(R1.OV.R2)=(R1UvR2)-(R1∩R2); e)RR(R1 DIS R2)= HR(R1, R2).
RS-R (S,R) a)RS-R(S DIS/EC R)=HS-R(S,R) ; b)RS-R(S IC/is-CON R) =R.
RL-R(L,R) a)RL-R(L-is-CON/IC/EC-R)=R;we assume line does not have width, only length. b)RL-R(L OV R)= HL-R(L,R).
RS-H (S,H)
a)RS-H(SєH)=RS*RS-R*RR*RL-R*RL (H); b)else RS-H(S,H) = HS-H(S,H).
RL-H(L,H)
a)RL-H(LєH)=RS* RS-R* RR* RL-R* RL(H); b)else RL-H(L,H) = HL-H(L,H).
RR-H(R,H)
a)RR-H(RєH)=RS*RS-R*RR*RL-R*RL(H); b)else RR-H(R,H) = HR-H(R,H).
RH(H1,H2) RH(H1,H2)= applying RS*RS-R* RR* RL-R* RL ((H1 U H2)- (H1 ∩ H2))
2.1.2.1 Lspace Family In the case study section we will have an example of Lspace family for a trip application. Thus we don’t bring any example here.
Table 3:Lspace Family
Lspace Formulation
LS(S1,S2) a)LS(S1 DIS/EC S2)=L[src: S1, dst: S2]; b)LS(S1 EQ S2)= not valid;
LS-L(S,L) a)LS-L(S is-CON/IC L)= line; b)LS-L(S EC L)=L’[src:S, trajectory: {L.src U all SiЄ(L.trajectory)},dst:L.dst]; c)LS-L(S DIS L)= HS-L (S,L);
LL(L1,L2)
a)LL(L1 DIS L2)= HL(L1,L2); b)LL(L1 EQ L2)= L1 OR L2; c) LL (L1 is-CON/IC L2)=L2; d)LL(L1-EC-L2)=L[src:L1.src, trajectory: {all Si-Є-(L1.trajectory)U-L1.dst U-L2.src U all S’i-Є-(L2.trajectory)}, dst: L2.dst]; e)Lline(L1.OV.L2)=L[src:L1.src,trajectory:{ all.SiЄ(L1.trajectory)-U-L1.dst-UL2.srcU all.S’iЄ(L2.trajectory)-((SiЄ(L1.trajectory)U-L1.dst))∩(L2.srcU-S’iЄ(L2.trajectory)))}, dst:L2.dst].
LS-H(S,H) a)LS-H(S є H)= LS* LS-L* LL(H); b)else LS-H(S,H) )= LS* LS-L * LL(S,H).
LL-H(L,H) a)LL-H (L є H)= LS* LS-L * LL(H); b)else LL-H(L , H) )= LS* LS-L * LL(L,H).
LH(H1,H2) LH(H1,H2)=LS*LS-L*LL((H1U-H2)-(H1∩H2)).
2.1.2.2 Hspace Family Example: Earlier in the robbery example (section 2.1) we emphasized the utility of Hybrid type. Figure 6 is an example of utilizing HS operator in Table 4 as a member of Hspace family that returns union of the spots.
Table 4:Hspace Family
Hspace Formulation
HS(S1,S2) a) HS(S1 DIS/EC S2)= (S1 U S2); b) HS(S1 EQ S2)= S1 OR S2.
HS-R(S,R) HS-R(S,R)=(S U R);
HS-L(S,L) HS-L(S,L)=(S U L);
HS-H(S,H) a) HS-H(S є H) = H; b) else HS-H(S,H) =(S U H).
HR(R1,R2) a) HR(R1 DIS/EC R2)=(R1 U R2); b) HR(R1 EQ R2)= R1 OR R2.
HR-L(R,L) HR-L(R,L)=(R U L);
HR-H(R,H) a) HR-H(R є H) = H; b) else HR-H(R,H) = (R U H).
HL(L1,L2) a) HL(L1 EQ L2)=L1 OR L2 ; b) HL(L1,L2)=(L1 U L2).
HL-H(L,H) a) HL-H(L є H) = H; b) else HL-H(L,H) = (L U H).
HH(H1,H2) HH(H1, H2)= (H1 U H2) – (H1 ∩ H2). 2.2 Temporal Operators We consider time as a linearly ordered property. Temporal operators have classification Otime= {Mtime, Itime, THtime}. M
69
implies moment or an instant time-point on a timeline, while I as interval is the continuous representation of set of moments with two outstanding moments: Mstart and Mend representing the starting and ending moments of the interval respectively such that Mstart ≠ Mend. We define THybrid as a set containing union of time elements, e.g. a collection of a moment, and an interval. These concepts are used in Formulation of temporal operators. Similar to the Hybrid type, THybrid type is also beneficial in terms of allowing other tools express the semantics of domain knowledge on the sub-elements of a thybrid set. For instance in the composite event robbery stated in section 2.1, according to the domain knowledge some tool can express that “the moment at which the thief is shot” is-the-important-moment-of “the interval associated with composite event robbery”. Without Thybrid type, representing such domain knowledge semantics is impossible. We represent a time interval by notation “[m1-m2]” to include the temporal boundaries. We categorize the main temporal operators into three groups Mtime (all its derivations return moment representation of their subordinate elements), Itime (all its derivations return interval representation of their subordinate elements), and THtime (all its derivations return temporal hybrid representation of their subordinate elements. Applying Allen’s relationships on a thybrid set obtained from a temporal operator can create a TemporalGraph similar to Figure 8. The TemporalGraph presents a stitched holistic view of the elements. The temporal operators using the introduced notions (i.e., moment, interval, and thybrid) are listed below: Mtime= MM; Itime={ IM , II , IM-I, IM-TH, II-TH , ITH}; THtime={ THM , THI ,THM-I, THM-TH , THI-TH , THTH }.
Figure 8: TemporalGraph
We have used “M” as “moment”, “I” as “interval, and “TH” as “thybrid” in the temporal operators’ notations for brevity. The term time used in temporal notations refers to the types of parameters on which temporal operators function on. For instance MM indicates that the input parameters of this operator are of type moment and the operator is expected to return a moment, on the other hand IM,I indicates that the type of input parameters are of types moment and interval, and the operator is expected to return an interval. However sometimes according to inter-time-line-element relationships, temporal operators may be converted into each other. You may find such examples in the next tables.
2.2.1 Operator Definition Temporal operators are formulated in Table 5,Table 6, and Table 7. Abbreviations used in notations (in next tables) are based on Allen’s Time Calculus: BF=before, DIS=disjoint, AF=after, EQ=equals, MT=meets, ST=starts, OV=overlaps, DUR=during, FIN=finishes, CON=contains. Temporal operators return heterogeneous temporal properties as a function of their input parameters. In each formulation, the temporal relationships between the input parameters yields a different sub-formula (e.g. in case of operator MM in Table 5, if input parameters are equal, sub-formula-(a) is used, while if
they are disjoint, sub-formula-(b) is used) . We describe family of Mtime, Itime, and THtime in the following sections.
2.2.1.1 Mtime Family Example: In the meeting example in Figure 1 at the beginning of the paper, if Jon wants to know the moment of start-up discussions in composite event “Meeting with TECH group”, using MM operator from Table 5 leads him to answer: MM(e1.Mstart ,e2.Mstart) = MM(2009-06-22T10:15TZD, 2009-06-22T12:00TZD) = THM(2009-06-22T10:15TZD, 2009-06-22T12:00TZD)= <2009-06-22T10:15TZD U 2009-06-22T12:00TZD>.
Table 5:Mtime Family Mtime Formulation
MM(M1,M2) a)MM(M1 EQ M2)=M1 OR M2; b)MM(M1 MT/DIS M2)=THM(M1,M2).
Figure 9 shows the union of start-up moments in composite event “Meeting with TECH group” on a timeline. Such composed result if provided to higher level application dependent composition techniques, can lead to interesting information. For instance if the application needs to know at which season of the year “Jon” has this meeting series, a change-of-resolution technique returns <season: Spring>.
Figure 9: Union of moments on a Timeline
2.2.1.2 Itime Family Example: The famous example for this family of temporal operators is comprising two disjoint moments into a single time interval using IM operator described in Table 6.
Table 6:Itime Family
Itime Formulation
IM(M1,M2) a) IM(M1 EQ M2)=no valid interval since both moments are temporally projected to the same temporal location; b) IM(M1 MT/DIS M2)=[M1, M2] if M1<M2.
II (I1,I2) a) II (I1 EQ I2) =I1 OR I2; b) II(I1 OV I2)=[I1.Mstart,I2.Mend]; c) II (I1 DUR/ST/FIN I2) = I2; d) II (I1 DIS I2) = THI (I1,I2).
IM-I(M,I) a) IM-I(M DUR/ST/FIN I)=I; b) IM-I(M MT I) =I’[M,I.Mend]; c) IM-I(M DIS I) = THM-I(M,I).
IM-TH(M,TH) a) IM-TH(M є TH) = IM* IM-I*II (TH ); b) else IM-TH(M,TH)=IM* IM-I* II(M U TH).
II-TH (I,TH) a) II-TH (I є TH) = IM* IM-I* II (TH); b) else II-TH (I,TH) = IM* IM-I* II(I U TH).
ITH(TH1,TH2) ITH(TH1,TH2) =IM* IM-I* II ((TH1 U TH2) -( TH1 ∩ TH2)).
disjoint-from
important-moment-of overlaps
is-before
Time-line
i1
i2
i3 m1
m2
2009-06-22T12:00TZD 2009-06-22T10:15TZD
70
In Figure 9 for instance, the two moments indicated in the timeline can be the starting and ending moments of a time interval: IM(2009-06-22T10:15TZD, 2009-06-22T12:00TZD) =
[2009-06-22T10:15TZD, 2009-06-22T12:00TZD].
2.2.1.3 THtime Family Example: The corresponding example is expressed earlier in Figure 9. The set (thybrid) of two moments is visualized on the timeline as a result of operator THM.
Table 7:THtime Family
THtime Formulation
THM(M1,M2) a)THM(M1 DIS/EC M2)=(M1 U M2); b)THM(M1 EQ M2)= M1 OR M2.
THI (I1,I2) a)THI(I1 DIS/EC I2)= (I1 U I2); b)THI(I1 EQ I2)= I1 OR I2.
THM-I(M,I) THM-I(M,I) = (M U I).
THM-TH(M,TH) a)THM-TH(M є TH)=TH; b)else THM-TH(M,TH)=(M U TH).
THI-TH(I,TH) a)THI-TH(I є TH)=TH; b)else THI-TH(I,TH)=(I U TH).
THTH(TH1,TH2) THTH(TH1,TH2)=(TH1 U TH2) – (TH1 ∩ TH2).
3. CASE STUDY In this section we show how these operators function within different events. We use an example of a trip scenario as a common example.
Trip Scenario: Given the event hierarchy in Figure 11, the goal is to populate the spatial and temporal attributes of the graph hierarchy in a bottom-up fashion. The spatial and temporal attributes of the atomic events presented as the leaves of the graph in Figure 11 are specified in the following way: e01(<M01>-<S01>): Arriving(<M01><S01>), <M01> = <2008-04-09T09:59TZD>, <S01> = <19.017656,72.856178>; e02(<I02>-<S02>):Taking-picture-of-Marine-Dr(<M02><S02>), <I02> = <[2008-04-09T10:00TZD,2008-04-09T15:00TZD]>, <S02> = <19.017656,72.856178>; e03(<I03>-<S03>):Taking-picture-of-ChinaWall(<I03><S03>), <I03> = <[2008-04-25T12:00TZD,2008-04-25T16:00TZD]>,- <S03> = <39.9081726, 116.3979471>; e04(<I04>-<S04>):Calling-cab(<I04><S04>), <I04> = <[2008-04-25T15:50TZD,2008-04-25T15:55TZD]>, <S04> = <39.9081726, 116.3979471>; In Level-1 of Figure 11 we have composite events e11 and e12: e11(<?I11>-<?S11>)=Visiting-Marine-Dr(<?I11> <?S11>); e11 subsumes (e01 U e02); e12(<?I12>-<?S12>)=Visiting-ChinaWall (<?I12> <?S12>); e12 subsumes (e03 U e04). The following process shows the temporal and spatial population of composite events e11 and e12:
<I11> = IM-I(<M01> MT <I02>) , <I11> =<[2008-04-09T09:59TZD,2008-04-09T15:00TZD]>;
<S11> = SS(<S01> EQ <S02>), <S11> = <19.017656,72.856178>;
<I 12> = II(<I03> CON <I04>), < I 12> = <[2008-04-25T12:00TZD,2008-04-25T16:00TZD]>; <S12> = SS(<S03> EQ <S04>), <S12> = <39.9081726, 116.3979471>; We have Level-1 populated in terms of time and space so far -- Figure 12. Now let’s go another level up which is Level-2 in the given graph. The only composite event present at this level is e2:Trip(<?I> <?L>). We populate the temporal and spatial attributes of this composite event using the composite sub-events e11 and e12. The final result is indicated in Figure 13. <I2>=II(<I11> DIS <I12>)=THI(<I11> DIS <I12>)=<I11> U <I12>, <I2>= < I11-U- I12 >; <L 2>= LS(<S11> DIS <S12>), <L2> = < L[src: <S11>,dst: <S12>] > ; This result is delineated in Figure 10.
Figure 10: Spatial property of e2
This example signifies the important role of spatial and temporal composition operators in filling the corresponding aspects in different levels of composition in event web with different space-time characteristics. Following Figures 11,12, and 13 we can observe that different levels of abstraction in terms of event composition can be populated with this approach. Our future work is to introduce new composite operators functioning on other remaining aspects of events in order to make ECO a powerful tool.
Figure 11: The space-time attributes of composite event “Trip” must be aggregated from the leaf events. We have used nicknames for the space and time property values (e.g. M2 as
temporal attribute of e2); in the Trip Scenario we have mentioned the numerical space and time property values.
71
Figure 12:Populating Level-1. The evaluated numerical values are given in Trip Scenario.
Figure 13:Populating Level-2. The evaluated numerical values are given in Trip Scenario.
4. CONCLUSION Composite operators on event facets open a wide range of research towards construction of event web [8]. It is going to be a huge step in terms of representing semantics of video/audio data. Video annotations used in indexing and making inferences about data [9] is going to be exploited as the under-process metadata for operators in the composition process and thus associating composite events to video streams. Also MPEG-21 multimedia framework can incorporate ECO to use composite events in order to specify content access rules [10]. Composition operators will play important role in research areas such as story telling, summarization, authoring events, and in field of news [11] since these areas need to include composite events in their domain.
5. REFERENCES [1] A.Quinton,“Objects and Events”, Mind, vol. 88, no. 350,
1979, pp. 197–214. [2] U.Westerman, R.Jain, “Toward a Common Event Model for
Multimedia Applications”, IEEE MultiMedia, Jan 2007. [3] U.Westermann,R.Jain, “Events in Multimedia Electronic
Chronicles (E-Chronicles)”, International Journal of Semantic Web and Information Systems (IJSWIS), vol.2 No. 2 pp. 1-27, July 2006.
[4] A.Scherp,T.Franz,C.Saathoff,S.Staab,” F—A Model of Events based on the Foundational Ontology DOLCE+DnS Ultralite”, K-CAP, 2009.
[5] J.F. Allen,“Maintaining knowledge about temporal intervals”, Comm. ACM 26 ,1983.
[6] S.Li, H.Wang,“RCC8 binary constraint network can be consistently extended”, Artificial Intelligence,2006.
[7] P.Terenziani, R.T.Snodgrass, “Reconciling Point-Based and Interval-Based Semantics in Temporal Relational Databases: A Treatment of the Telic/Atelic Distinction”, IEEE Transactions on Knowledge and Data Engineering, May 2004.
[8] R.Jain, “Toward Event Web”, IEEE Computer Society, Sep 2007.
[9] A.R.J.Francois,R.Nevatia,J.Hobbs,R.C.Bolles,”VERL: An Ontology Framework for Representing and Annotating Video Events”,Journal IEEE MultiMedia,2005.
[10] J. Hunter, “Reconciling MPEG-7 and MPEG-21 Semantics through a Common Event-Aware Metadata Model,” Computing Research Repository (CoRR), Oct. 2002.
[11] D.Pack, R. Singh, S.Brennan, R.Jain, “An event model and its implementation for multimedia information representation and retrieval”, Proc. 2004 IEEE Int’l Conf.Multimedia and Expo(ICME),IEEE press,2004,pp. 1611-1614.
[12] C. Masolo, S. Borgo, A. Gangemi, N. Guarino, A. Oltramari, ”WonderWeb deliverable D18 ontology library”, IST WonderWeb project, 2003.---
72