how to act on inconsistent news: ignore, resolve, or reject

www.elsevier.com/locate/datak

Data & Knowledge Engineering 57 (2006) 221–239

How to act on inconsistent news: Ignore, resolve, or reject

Anthony Hunter *

Department of Computer Science, University College London, Gower Street, London WC1E 6BT, UK

Received 3 February 2005; received in revised form 3 February 2005; accepted 27 April 2005

Available online 31 May 2005

Abstract

Inconsistencies frequently occur in news about the real-world. Some of these inconsistencies may be

more significant than others, and some news may contain more inconsistencies than others. This creates

problems of deciding whether to act on these inconsistencies, and if so how. Possible actions on an incon-

sistency in a news report include ignore the inconsistency, resolve the inconsistency, and reject the report.

To support this, we extend and apply a general characterization of inconsistency, based on Belnap�s four-

valued logic. For conflicts arising between the news and background knowledge, we analyse coherence and

significance of the corresponding the four-valued models for that knowledge and show how this analysiscan indicate an appropriate course of action.

� 2005 Elsevier B.V. All rights reserved.

Keywords: Analysing inconsistency; Measuring inconsistency; Contradiction; Inconsistency tolerance; Information

integration; Knowledge fusion

1. Introduction

News reports are often inconsistent. Some tolerance of inconsistency is therefore normally neces-sary. However, it is not always clear how much tolerance should be excercised. Sometimes, it is nec-essary to reject news reports. Sometimes, it is necessary to resolve some of the inconsistencies in the

0169-023X/$ - see front matter � 2005 Elsevier B.V. All rights reserved.

doi:10.1016/j.datak.2005.04.005

* Tel.: +44 02076797295; fax: +44 02073871397.

E-mail address: [email protected]

URL: www.cs.ucl.ac.uk/staff/a.hunter

mailto:[email protected]

www.cs.ucl.ac.uk/staff/a.hunter

222 A. Hunter / Data & Knowledge Engineering 57 (2006) 221–239

news report. At other times, news reports are accepted with inconsistencies, but treated with caution.Having some understanding of the ‘‘degree of inconsistency’’ of a news report can help in decidinghow to act on it. To do this systematically calls for a formal framework for considering inconsistency.

In order to analyse and reason with information in news, we assume the news reports are pro-vided in the form of structured news reports. These are XML documents, where the text entriesare restricted to individual words or simple phrases, such as names and domain-specific terminol-ogy, and numbers and units. It is assumed structured news reports do not require natural lan-guage processing. Though we do assume structured news reports can be obtained as outputfrom information extraction technology [1]. Much information can be represented in the formof structured news reports, including weather reports, business reports, summaries of scientificarticles, results from clinical trials, market intelligence, operations reports within organisations,etc. We are therefore taking a very general view on news to include any kind of regular supplyof information that conforms to some stereotypical format.

Each structured news report is isomorphic to a logical term. So structured news reports can beused, in conjunction with some domain knowledge, for logical reasoning as part of some softwaretool. Intelligent tasks based on structured news reports include: (1) forming a merged structurednews report from a set of heterogeneously sourced structured news reports [2,3]; (2) looking forinteresting news by looking for violations of expectations [4,5]; and (3) constructing argumentsfor and against conclusions based on conflicting news reports [6]. Central to all these intelligenttasks is the need to address inconsistencies that arise.

An important aspect of analysing an inconsistency is evaluating its significance. As an illustra-tion of the need to evaluate significance, consider two news reports on a World Cup match, wherethe first report says that Brazil beat Germany 2-0, and the second report says that Germany beatBrazil 2-0. This is clearly a significant inconsistency. Now consider two news reports on the samefootball match, where the first report says that the referee was Pierluigi Collina and the secondreport says that the referee was Ubaldo Aquino. This inconsistency would normally be regardedas relatively insignificant.

Moreover, inconsistencies between information in a news report and domain knowledge can tellus important things about the news report. For this we use a significance function to give a valuefor each possible inconsistency that can arise in a news report in a given domain. We may then usethis in various ways: (1) to reject reports that are too inconsistent; (2) to highlight unexpectednews; (3) to focus on repairing significant inconsistencies; and (4) to monitor sources of informa-tion to identify sources that are unreliable.

In this paper, to analyse the significance of inconsistencies in structured news reports, we willadapt and extend an approach to measuring inconsistency based on four-valued models [7]. In thisadaptation, we use Belnap�s logic for finding the models for a set of inconsistent formulae. Wethen extend this framework to support the analysis of heterogeneously sourced information suchas news from different newsfeeds.

2. Structured news reports

Syntactically, a structured news report is a data structure containing a number of grammati-cally simple phrases together with a tag (giving semantic information) for each phrase. As an

A. Hunter / Data & Knowledge Engineering 57 (2006) 221–239 223

illustration, see Example 1. Each phrase that is tagged is a textentry. The set of tags in a structurednews report is meant to parameterize a stereotypical situation, and so a particular structured newsreport is an instance of that stereotypical situation. For example, news reports on corporateacquisitions can be represented as structured news reports using tags including buyer, seller,acquisition, value, and date. Each phrase in structured news report is very simple, such asa proper noun, a date, or a number with unit of measure, or a word or phrase from a prescribedlexicon. For an application, the prescribed lexicon delineates the types of states, actions, and attri-butes, that could be conveyed by the structured news report.

The definition for a structured news report is very general. In practice, we would expect a DTDfor a given domain. So for example, we would expect that for an implemented system that mergesweather reports, there would be a corresponding DTD. One of the roles of a DTD, say forweather reports, would be to specify the minimum constellation of tags that would be expectedof a weather report. We may also expect integrity constraints represented in classical logic tofurther restrict appropriate structured news reports for a domain.

Each structured news report is isomorphic with a ground atom of classical logic where each tag-name is a function or predicate symbol and each textentry is a constant symbol: The root tagnameis the predicate symbol for the atom, and the other tagnames are function symbols for the atom.This isomorphism is illustrated in the following example.

Example 1. Consider the following structured news report.

hreporti
hlogihstationiInvernessh/stationihdatei12/2/03h/dateih/logihrainfalli2.3cmh/rainfalli
h/reporti

This can be represented by the following atom.

report(log(station(Inverness),date(12/3/02)),rainfall(2.3cm))

Representing each structured news report by a logical atom means that they can then be di-rectly used with other information in a knowledgebase via logical inference. To show this we pro-vide a few simple definitions. From now on, a report will be taken to mean an atom that representsa structured news report.

Definition 1. A grounding is an equality predicate where the first argument is a variable and thesecond argument is a ground term. A grounding set is a set of groundings which can be substitutedinto an unground term to give a grounded term. Let a be a formula and let U be a grounding set.Ground(a,U) gives the result of substituting each variable x in a with term t where the groundingx = t is in U.

Example 2. Let a(b(x),c(y),d(z)) be a formula where x, y and z are variables. Let the ground-ing set U be {y = john,z = betty}.

GroundðaðbðxÞ;cðyÞ;dðzÞÞ;UÞ ¼ aðbðxÞ;cðjohnÞ;dðbettyÞÞ


We use access rules, defined next, to reason with information in reports.

Definition 2. An access rule is a first order formula of the form where: (1) x1, . . . ,xk are thevariables in a; (2) the variables in b1, . . . ,bn are a subset of, or equal to, x1, . . . ,xk; (3) all ofa,b1, . . . ,bn are unquantified; and (4) the predicate symbol for a is not used as a predicate symbolfor any of b1, . . . ,bn.

8x1; . . . ; xk; a! b1 ^ � � � ^ bn

If there is a grounding set U s.t. Ground(a,U) is a report, then Ground(b1,U), and . . ., andGround(bn,U) are atoms that can be inferred from the report.

In the definition of an access rule, because a is ground by a report, a is a monadic predicate (i.e.it has exactly one argument). However, the b1, . . . ,bn predicates are of arbitrary arity. They do notneed to be restricted to monadic predicates.

Definition 3. Let D be a set of access rules and let q be a report. Let Access(D,q) be the set ofatoms obtained by exhaustively applying q to the access rules in D using generalized modusponens and conjunction elimination as defined below.

AccessðD; qÞ ¼ fGroundðb1;UÞ; . . . ;Groundðbn;UÞj8x1; . . . ; xk; a! b1 ^ � � � ^ bn 2 D

and Groundða;UÞ is the atom qg

Example 3. Let q be the following report.

reportðforecastðdateð25July01Þ;cityðMalagaÞ;todayðblizzardÞÞÞ
Suppose the set of access rules, D, contains the following.
8x;y;z; reportðforecastðdateðxÞ;cityðyÞ;todayðzÞÞÞ! dateðxÞ ^ locationðyÞ ^ todayðzÞ

So for q we get date(25July2001), location(Malaga), today(blizzard) is a member ofAccess(D,q). Suppose D also contains the following access rule.

8x;y; reportðforecastðdateðxÞ;cityðyÞ;todayðblizzardÞÞÞ! blizzardWarningðx;yÞ

So we also get blizzardWarning(25July2001,Malaga) 2 Access(D,q).

We assume that textentries in structured news reports are heterogeneous in format. For exam-ple, the format of date values is unconstrained (12/12/1974; 31st December 96; 12 November 2001etc.) as is the format of numbers and currency values (3 million; 3,000,000 GBP; $4, ¥500K etc.).Various kinds of heterogeneity in ontology can also arise, such as synonyms, hypernyms, etc., canbe used. Elsewhere, we have discussed how this heterogeneity can be handled in logic by variouskinds of equivalence axioms that can be used in conjunction with the access rules [2,8,3].


3. Four-valued models

We now start to consider how we can analyse inconsistent information. Our starting point is theanalysis of four-valued models. A four-valued model provides a natural characterization of incon-sistent information.

Definition 4. A four-valued model, denoted XD, is such that XD � {+aja 2 D} [ {�aja 2 D}where D is the domain of XD. An element of a four-valued model is called an object. A positiveobject is an object with a + prefix and a negative object is an object with a � prefix. A four-valuedmodel may contain both +a and �a for some a. If a four-valued model XD contains both +a and�a for some a, then we use the shorthand ±a for these objects, and hence ±a 2 XD iff +a 2 XD and�a 2 XD. Let K denote the set of four-valued models.

We can view four-valued models in terms of four truth values ‘‘true’’, ‘‘false’’, ‘‘both’’ or ‘‘nei-ther’’, which we denote by the symbols T, F, B, and N, respectively. Furthermore, we can relatethese truth values to a knowledge lattice (see Fig. 1(left)). As more ‘‘information’’ is obtainedabout a formula, the truth-value ‘‘increases’’, going upwards in the lattice. In other words, ifwe know nothing about a formula, it is N. Then as we gain some information, it becomes eitherT or F. Finally, if we gain too much information it becomes B. The knowledge lattice gives thefollowing ordering relation over truth values which we use to formalize the notion of a truthassignment for a four-valued model.

Definition 5. Let Pk be an ordering relation, called the knowledge ordering, such that B Pk B,B Pk T, B Pk F, B Pk N, T Pk T, T Pk N, F Pk F, F Pk N, and N Pk N.

Definition 6. Let XD be a four-valued model. A truth assignment for XD, denoted tX D , is an assign-ment tX D : D 7!fB; T ; F ;Ng, such that for all a 2 D,

tX DðaÞPkT iff þ a 2 X D

tX DðaÞPkF iff � a 2 X D

Hence, we get the following equivalences between truth assignments and four-valued models.

tX DðaÞ ¼ B iff þ a 2 X D and � a 2 X D

tX DðaÞ ¼ T iff þ a 2 X D and � a 62 X D

tX DðaÞ ¼ F iff þ a 62 X D and � a 2 X D

tX DðaÞ ¼ N iff þ a 62 X D and � a 62 X D

Fig. 1. The knowledge lattice (left) and the logical lattice (right).


Normally, four-valued models are presented directly in terms of a truth assignment. They werefirst presented as such by Belnap [9]. We have presented four-valued models using objects becausethey make it easier for us to present our definitions in subsequent sections.

Example 4. Let XD be a four-valued model where D = {a,b,c,d}, and XD = {+a,±b,�d}. So tX D

is such that tX DðaÞ ¼ T , tX DðbÞ ¼ B, tX DðcÞ ¼ N , and tX DðdÞ ¼ F .

The opinionbase of a four-valued model XD is the set of atomic beliefs (atoms) for which thereare reasons for the atom or its negation in XD, and the conflictbase of XD is the set of atomic be-liefs with reasons for the atom and its negation in XD.

Definition 7. Let XD be a four-valued model.

ConflictbaseðX DÞ ¼ fa 2 DjtX DðaÞ ¼ BgOpinionbaseðX DÞ ¼ fa 2 DjtX DðaÞ 6¼ Ng

If Opinionbase(XD) = ;, then XD has no arguments for/against any beliefs, and hence XD has noopinions. If Opinionbase(XD) = D, then XD is totally opinionated. If Conflictbase(XD) = ;, then XD

is a conflictfree four-valued model. If Opinionbase(XD) = D, and Conflictbase(XD) = ;, then we de-scribe XD as omniscient.

We now consider measures of inconsistency called coherence and concordance. Increasing thesize of the conflictbase, with respect to the size of the opinionbase, decreases the degree of coher-ence, as defined below. Similarly, increasing the size of the conflictbase, with respect to the size ofthe domain, decreases the degree of concordance.

Definition 8. The Coherence function from }(K) into [0,1], is defined below when Opinion-base(XD) 5 ;, and Coherence(XD) = 1 when Opinionbase(XD) = ;,

CoherenceðX DÞ ¼ 1� jConflictbaseðX DÞjjOpinionbaseðX DÞj

If Coherence(XD) = 1, then XD is a totally coherent, and if Coherence(XD) = 0, then XD is totallyincoherent, otherwise, XD is partially coherent (or equivalently partially incoherent).

Definition 9. The Concordance function from }(K) into [0,1], is defined below when D 5 ;, andConcordance(XD) = 1 when D = ;,

ConcordanceðX DÞ ¼ 1� jConflictbaseðX DÞjjDj

If Concordance(XD) = 1, then XD is a totally concordant, and if Concordance(XD) = 0, then XD

is totally discordant, otherwise, XD is partially concordant (or equivalently partially discordant).

Example 5. Let XD be a four-valued model such that D = {a,b,c}, and t(a) = T, t(b) = B, andt(c) = F. Hence, Coherence(XD) = 2/3 and Concordance(XD) = 2/3.

Example 6. Let XD be a four-valued such that D = {a,b,c,d}, and t(a) = T, t(b) = B, t(c) = F, andt(d) = N. Hence, Coherence(XD) = 2/3 and Concordance(XD) = 3/4.


Both concordance and coherence are potentially useful measures in applications. They providedifferent perspectives on a model. Obviously, the more of the domain covered by the opinionbase,the closer these measures become.

Proposition 1. For all XD 2 K, Concordance(XD) P Coherence(XD).

So far, we have assumed we have models to analyse. In the next section, we consider a logic foridentifying models that satisfy a set of formulae.

4. Analysing inconsistent knowledgebases

To extend concordance and coherence measures for formulae, and hence for inconsistentknowledgebases, we need a satisfaction relation for a set of formulae (i.e. a knowledgebase).For this, we use Belnap�s logic which provides a natural and intuitive form of paraconsistent rea-soning [9,10].

Definition 10. The language L for the logics that we will consider in this paper is the usualclassical language that can be formed from a set of atoms and the set of logical symbols f:;_;^g.We call any subset of L a knowledgebase. If C 2 }(L), then Atoms(C) returns the set of atomsymbols used in C.

The truth assignment (Definition 6) is extended to any formula in L by induction on the struc-ture of the formula using a distributive lattice called the logical lattice (see Fig. 1(right)). We alsoassume an involution operator • satisfying the conditions (1) a = a••, and (2) if a P b thenb• Pl a

•, where Pl is the ordering relation for the logical lattice. The truth assignment for formu-lae observes monotonicity and complementation, in the logical lattice, so x ^ y is the meet of{x,y} and x _ y is the join of {x,y}. The truth assignment for formulae is summarized in the truthtables (Tables 1–3) for the :;^, and _ connectives.

Table 1

Truth table for negation for Belnap semantics

a N F T B

:a N T F B

Table 2

Truth table for conjunction for Belnap semantics

^ N F T B

N N F N F

F F F F F

T N F T B

B F F B B

Table 3

Truth table for disjunction for Belnap semantics

_ N F T B

N N N T T

F N F T B

T T T T T

B T B T B


Example 7. Continuing Example 4, we have assignments using tX D including the following, wherea ^ :b ^ b; a _ c; b _ c; c _ :c 2 L.

X Di

X D1

X D2

X D3

X D4

X D5

X D6

tX Dða ^ :b ^ bÞ ¼ B tX Dðb _ cÞ ¼ T

tX Dða _ cÞ ¼ T tX Dðc _ :cÞ ¼ N

There is no a 2 L such that the semantic assignment function always assigns the value T. How-ever, there are formulae that never take the value F, for example a _ :a. Also, the set of formulaethat never take the value F is not closed under conjunction. For example, considerða _ :aÞ ^ ðb _ :bÞ when a is N and b is B.

In the following, we define B(D) as the set of four-valued models that satisfy D by the Belnapsatisfaction relation and the domain is exactly the set of atoms occurring in D.

Definition 11. The satisfaction relation for Belnap�s four-valued logic, denoted �b is defined asfollows, where XD is a four-valued model and a 2 L.

X D�ba iff tðaÞPkT

For D 2 }(L), let XD �b D denote that XD �b a for every a 2 D. Let B(D) be the set of Belnap mod-els for D defined as follows.

BðDÞ ¼ fX DjX D�bD and AtomsðDÞ ¼ Dg

Example 8. Let D ¼ f:a; a _ bg. So D = {a,b} and B(D) = {X D1 ; X D

2 ; X D3 ; X D

4 ; X D5 ; X D

6 g where themodels are defined as follows.

tDX iðaÞ tD

X iðbÞ Concordance Coherence

F T 1 1

F B 1/2 1/2

B N 1/2 0

B F 1/2 1/2

B T 1/2 1/2

B B 0 0

Example 9. Let D ¼ f:a; a _ b;:b; cg. So D = {a,b,c} and B(D)= fX D1 ; X D

2 ; X D3 ; X D

4 ; X D5 ; X D

6 gwhere the models are defined as follows.

X Di tD

X iðaÞ tD

X iðbÞ tD

X iðcÞ Concordance Coherence

X D1 B F T 2/3 2/3

X D2 F B T 2/3 2/3

X D3 B B T 1/3 1/3

X D4 B F B 1/3 1/3

X D5 F B B 1/3 1/3

X D6 B B B 0 0


From Examples 8 and 9, we see that it is not the case in general that all models in B(D) have thesame coherence or concordance. So we extend coherence and concordance to knowledgebases asfollows.

Definition 12. Let D 2 }(L).

CoherenceðDÞ ¼ MaxðfCoherenceðX DÞjX D 2 BðDÞgÞConcordanceðDÞ ¼ MaxðfConcordanceðX DÞjX D 2 BðDÞgÞ

Example 10. Continuing Example 8, we have D ¼ f:a; a _ bg, and so we get the following.

CoherenceðDÞ ¼ ConcordanceðDÞ ¼ 1

Example 11. Continuing Example 9, we have D ¼ f:a; a _ b;:b; cg, and so we get the following.

CoherenceðDÞ ¼ ConcordanceðDÞ ¼ 2=3

The definition of coherence (and similarly concordance) for knowledgebases can be consideredas an optimistic view on the nature of inconsistency in the knowledgebase, in contrast to a pessi-mistic view where coherence for a knowledgebase D would be the minimum in {Coher-ence(XD)jXD 2 B(D)}. Other possibilities include using a leximax function or an averagefunction. For example, using leximax can allow more fine grained discrimination between aknowledgebase that has all its models having coherence equal to 1, and a knowledgebase thathas some models equal to 1, and some less than 1.

For some applications, the coherence or concordance measures are useful to analyse the qualityof information. For example, sources of structured news reports can be monitored over time. If asource, has an average degree of coherence or concordance below a certain level, then someinvestigation of that source could be undertaken to see if the average quality could be improved.

However, in general for deciding how to act on structured news reports, we appear to needmore than just the coherence and concordance measures. As we saw in Section 1, it is clear thatsome inconsistencies in news reports are very significant whereas others are insignificant. If we areto consider, for example, rejecting news reports on the basis of the inconsistencies arising, we willwant to consider more than just the number of inconsistencies arising, we will want to take theirsignificance into account.


5. Evaluating significance of inconsistencies

We now evaluate the significance of inconsistencies in four-valued models, and thereby in setsof formulae. The approach is based on specifying the relative significance of incoherent modelsusing the notion of a mass assignment which is defined below.

Definition 13. Let XD be a four-valued model. If Coherence(XD) = 0, then XD is a frame. If X D1

and X D2 are frames, and X D

1 � X D2 , then X D

1 is a subframe of X D2 .

Definition 14. Let XD be a frame. A mass assignment m for XD is a function from }(XD) into [0,1]such that:

(1) If Y � X D and CoherenceðY Þ ¼ 1; then mðY Þ ¼ 0(2)P

Y�X DmðY Þ ¼ 1

In the definition for mass assignment, we have the constraint Coherence(XD) = 0 to ensure thatfor all a 2 D, we have ±a 2 XD. Condition 1 ensures mass is only assigned to models that containconflicts and condition 2 ensures the total mass distributed sums to 1.

Given a frame XD, a mass assignment can be localized on small subsets of XD, spread overmany subsets of XD, or limited to large subsets of XD. A mass assignment can be regarded as aform of metaknowledge, and so it needs to be specified for a domain, where the domain is char-acterized by XD, and so the possible models of the domain are subsets of XD.

Example 12. Let XD = {±a,±b}. A mass assignment m is given by m({±a}) = 0.2 andm({±b}) = 0.8. Another mass assignment m 0 is given by m 0({±a}) = 0.2, m 0({±a,�b}) = 0.6, andm 0({±a,±b}) = 0.2.

A significance function gives an evaluation of the significance of the conflicts in a QC model.This evaluation is in the range [0,1] with 0 as least significant and 1 as most significant.

Definition 15. Let m be a mass assignment for XD. A significance function for XD, denoted SX D , isa function from }(XD) into [0,1]. A mass-based significance function for m, denoted Sm

X D , is asignificance function defined as follows for each Y 2 }(XD).

SmX DðY Þ ¼

XZ�Y

mðZÞ

We will only consider mass-based significance functions in the rest of this paper. So to easereading in the following, we drop the superscript and subscript for significance functions.

Example 13. Let XD = {±a,±b,±c}, and let m be a mass assignment for XD, where m({±a}) = 0.4,m({±a,�b}) = 0.2, and m({�b,±c}) = 0.4. Below we consider the significance for some subsets ofXD.

Sðf�agÞ ¼ 0.4 Sðf�a;�bgÞ ¼ 0.6 Sðf�a;�bgÞ ¼ 0.6

Sðf�aþ cgÞ ¼ 0.4 Sðf�b;�cgÞ ¼ 0.4 Sðf�a;�b;�cgÞ ¼ 1.0


The definitions for mass assignment and mass-based significance correspond to mass assign-ment and belief functions (respectively) in Demspter–Shafer theory [11]. However, here theyare used to formalise significance rather than uncertainty.

In the following result, we see that mass-based significance is not additive. Also the remainingsignificance need not be for the complement of Z (i.e., Zc). Some may be assigned to models notdisjoint from Z.

Proposition 2. Let XD be a frame. If m is a mass assignment for XD, and S is a significance functionfor m, then for all Z,Y 2 }(XD), the following hold, where Yc is the set complement of Y (i.e.,Yc = XDnY).

(1) Z � Y implies SðZÞ 6 SðY Þ(2) SðZ [ Y ÞP ðSðZÞ þ SðY Þ � SðZ \ Y ÞÞ(3) SðY Þ þ SðY cÞ 6 1

We now extend the significance functions to knowledgebases. Since B(D) is not necessarily asingleton, the significance for a set of formulae D is the lowest significance obtained for anX 2 B(D). This means we treat the information in D as a ‘‘disjunction’’ of four-valued models,and we regard each of those models as equally acceptable, or equivalently we regard each of thosemodels as equally representative of the information in D.

Definition 16. Let D 2 }(L). We extend the definition for a significance function S toknowledgebases as follows.

SðDÞ ¼ minðfSðX ÞjX 2 BðDÞgÞ

Some knowledgebases have zero significance. If D 0 ?, and hence Coherence(D) = 1, thenS(D) = 0. Also significance is monotonic. So for D 2 }(L) and a 2 L, S(D) 6 S(D [ {a}).

Example 14. Let XD = {±a,±b,±c}. Let m({±a}) = 0.6, m({±a,+b}) = 0.3, and m({±b,+ c}) =0.1. So Sðfa ^ :a; b _ cgÞ ¼ 0.6.

In order to determine the frame for which a mass function is defined, we can use the delineationfunction as follows.

Definition 17. For D 2 }(L), Delineation(D) = {±aja 2 Atoms(D)}.

Example 15. Let D1 ¼ f:a; a ^ b;:bg, D2 ¼ fa _ b;:a ^ ag, and D3 ¼ fb;:a _ :bg. Let theframe XD = Delineation(D1 [ D2 [ D3) = {±a,±b}. Also let m({±a,±b}) = 0.2 and m({±a}) = 0.8.So S(D1) = 1, S(D2) = 0.8, and S(D3) = 0.

A mass assignment can be regarded as transforming the four-valued semantics of Belnap logicinto a many-valued logic where the value B has been split into a chain of truth values B0, . . . ,B1. Ifwe equate the truth values {N,T,F,B0} with the Belnap truth values {N,T,F,B}, respectively,then Belnap�s four-valued lattice is a sublattice of this lattice.


6. Modular mass assignments

In practice, we may need to consider a very large frame for monitoring coherence. This maytherefore involve considering many different subsets to which mass has to be assigned. One solu-tion is to take a modular approach by defining separate mass assignments to disjoint subsets of theframe, and then combining these mass assignments to give a mass assignment for the overallframe.

Definition 18. Let X D1

1 be a frame with mass m1, and let X D2

2 be a frame with mass m2. m1 isembedded in m2 iff (1) D1 � D2; (2) for all Y 2 }ðX D

1 Þ m1ðY ÞP m2ðY Þ; and (3) for all Z; Y 2 }ðX D1 Þ,

if m1(Z) P m1(Y), then m2(Z) P m2(Y).

Embedding is not necessarily unique as illustrated by the following example.

Example 16. Let X D1 ¼ f�ag and let X D

2 ¼ f�a;�bg. If m1({±a}) = 1.0, then two possibleembeddings are given by m2 and m 02 as follows:

m2ðf�agÞ ¼ 0.2 m2ðf�bgÞ ¼ 0.8

m02ðf�agÞ ¼ 0.7 m02ðf�bgÞ ¼ 0.3

The following definition gives a non-unique splitting of a frame into a set of subframes called aframesplit. A framesplit together with a mass assignment for each subframe in the framesplit iscalled a framebox.

Definition 19. Let XD be a frame and let P � }(XD). P ¼ fX D1

1 ; . . . ;X Dnn g is a framesplit for XD iff

(1) D = D1 [ � � � [ Dn; and (2) for all X Dii ;X

Djj 2 P, Di \ Dj = ;; and (3) for all X Di

i 2 P,CoherenceðX Di

i Þ ¼ 0. W is a massclass for P iff for each X Dii 2 P there is exactly one mass

assignment mi 2 W such that mi is a mass assignment for X Dii . A framebox for XD is a tuple hP,Wi

where P is a framesplit for XD and W is a massclass for P.

For a framebox, the following defines the embed function which gives the set of all mass assign-ments such that each is an embedding of all of the mass assignments in the framebox.

Definition 20. Let hP,Wi be a framebox for XD. The embed function, denoted Embed, is definedas follows:

EmbedðP;WÞ ¼ fmijfor all mj 2 W ðmj is embedded in miÞg
We adopt the following type of mass assignment which only assigns mass to disjoint models
that are totally incoherent.

Definition 21. Let m be a mass assignment for XD. m is acute iff (1) for all Y 2 }(XD) ifCoherence(Y) = 0 then m(Y) P 0; and (2) for all Y 2 }(XD) if Coherence(Y) > 0 then m(Y) = 0; and(3) for all Z,Y 2 }(XD) if m(Z) > 0 and m(Y) > 0 then Z \ Y = ;.

Significance is additive for totally incoherent models when the mass assignment is acute.

Proposition 3. Let m be an acute mass assignment for XD. Let S be a mass-based significancefunction for m and let Y 2 }(XD). If Coherence(Y) = 0, then S(Y) + S(Yc) = 1.


A useful feature of an acute mass-based significance function is that as the number of conflictsrises in a model, then the significance of the model rises. This is formalized by the following notionof conflict cumulativity. It does not hold in general (see Example 17).

Proposition 4. Let m be an acute mass assignment for XD. If S is a significance function for m, thenthe following property of conflict cumulativity holds for all ZD,YD 2 }(XD).

ConcordanceðZDÞP ConcordanceðY DÞ implies SðZDÞ 6 SðY DÞ

Example 17. Let XD = {±a,±b,±c}, ZD = {±a,+c} and YD = {±a,±b}. Let m be an mass assign-ment for XD such that m(ZD) = 0.6 and m(YD) = 0.4. So we have Concordance(ZD) P Concor-dance(YD), but S(ZD) i S(YD).

We now consider how we can combine the modular mass assignments in a framebox. For thiswe use the following definition of a fair assignment. It is perhaps the simplest and least biased ofthe possible ways to combine the mass assignments.

Definition 22. Let hP,Wi be a framebox for XD where each mi 2 W is an acute mass assignment. Afair assignment, denoted m, for hP,Wi is defined as follows: For each X Di

i 2 P, and for eachmi 2 W, such that mi is a mass assignment for X Di

i , if Y � X Dii , then

mðY Þ ¼ miðY Þ �jDijjDj

Essentially, the fair mass assignment is obtained from a normalized version of each of the massassignments in the framebox.

Example 18. Let hfX D1

1 ;XD2

2 g; fm1;m2gi be a framebox for X D3

3 where X D1

1 ¼ f�ag, X D2

2 ¼ f�bg,X D3

3 ¼ f�a;�bg, and m1 and m2 are defined below. For this framebox, the fair assignment is m3.



Example 19. Let hfX D1

1 ;XD2

2 gfm1;m2gi be a framebox for X D3

3 , where X D1

1 ¼ f�a;�cg,X D2

2 ¼ f�bg, and X D3

3 ¼ f�a;�b;�cg, and m1 and m2 are defined below. For this framebox,the fair assignment is m3.

m1ðf�agÞ ¼ 0.4 m1ðf�cgÞ ¼ 0.6 m2ðf�bgÞ ¼ 1.0

m3ðf�agÞ ¼ 0.27 m3ðf�cgÞ ¼ 0.4 m3ðf�bgÞ ¼ 0.33

Proposition 5. Let hP,Wi be a framebox for XD. If m is a fair assignment for hP,Wi, then m is amass assignment for XD.

Proof. Let m be a fair assignment. So Condition (1) of Definition 14 is trivially satisfied. For eachframe X Di

i 2 P, with mass assignment mi,
XY�X
Dii

miðY Þ ¼ 1


Hence, for each frame X Dii 2 P, with mass assignment mi,

XY�X

Dii

miðY Þ �jDijjDj

� �¼

XY�X

Dii

miðY Þ

0@

1A� jDij

jDj ¼jDijjDj

Since, the frames in P are disjoint, i.e. Conditions (1) and (2) of Definition 19 hold.

XY�X D

mðY Þ ¼X

i such that XDii 2P

jDijjDj ¼ 1

So Condition (2) of Definition 14 is satisfied. Hence, m is a mass assignment for XD. h

We can obtain a fair embedding using the embed function.

Proposition 6. Let hP,Wi be a framebox for XD. If m is a fair assignment for hP,Wi, thenm 2 Embed(P,W).

Proof. Let m be a fair assignment for hP,Wi. Let hP,Wi be a framebox for XD such thatP ¼ fX D1

1 ; . . . ;X Dnn g. From these assumptions, we get the following: (1) D = D1 [ � � � [ Dn, and

for each Di, Dj � D, Di \ Dj = ;; (2) for all Y 2 }ðX Di Þ mðY Þ ¼ miðY Þ � jDij

jDj , and so mi(Y) P m(Y);

and (3) for all Z; Y 2 }ðX Di Þ, if mi(Z) P mi(Y), then m(Z) P m(Y). Therefore, by Definition 18, (1),

(2), and (3) imply that for all i, where 1 P i P n, we have mi is embedded in m. Hence, by Defi-nition 20, m 2 Embed(P,W). h

As a direct consequence of Definition 22, a fair assignment is a unique embedding.

Proposition 7. Let hP,Wi be a framebox for XD. If m is a fair assignment for hP,Wi, and m 0 is a fairassignment for hP,Wi, then m = m 0.

A fair assignment is not necessarily an acute mass assignment. However, if a framebox onlycontains acute mass assignments, then the fair mass assignment for the framebox is acute.

Proposition 8. Let hP,Wi be a framebox for XD. If all mi 2 W are acute mass assignments, and m isa fair assignment for hP,Wi, then m is an acute mass assignment.

Proof. Let hP,Wi be a framebox for XD such that all mi 2 W are acute mass assignments. Also letm be a fair assignment for hP,Wi. (1) For all Y 2 }(XD), if Coherence(Y) = 0, then there is ami 2 P such that mi(Y) P 0, and hence m(Y) P 0; (2) For all Y 2 }(XD), if Coherence(Y) > 0, thenthere is not a mi 2 P such that mi(Y) > 0, and hence m(Y) = 0; (3) For all mi,mj 2 P, for allZ,Y 2 }(XD), if mi(Z) > 0, and mj(Y) > 0, then Z \ Y = ;, hence if m(Z) > 0, and m(Y) > 0, thenZ \ Y = ;. From (1), (2), and (3), by Definition 21, m is an acute mass assignment. h

So a framebox for a frame XD with a fair assignment provides a simple and effective modularapproach to mass assignment for larger frames.

Example 20. Consider a framebox hP,Wi where m1 and m2 are defined as follows. For thisexample, t and r are abbreviations for elements in the domain where t denotes temperatureand r denotes rainfall.


m1ð�tð1CÞÞ ¼ 0.02;m1ð�tð2CÞÞ ¼ 0.02; . . . ;m1ð�tð50CÞÞ ¼ 0.02

m2ð�rð1cmÞÞ ¼ 0.02;m2ð�rð1cmÞÞ ¼ 0.02; . . . ;m2ð�rð50cmÞÞ ¼ 0.02

A fair assignment gives m3 as follows.

m3ð�tð1CÞÞ ¼ 0.01;m3ð�tð2CÞÞ ¼ 0.01; . . . ;m3ð�tð50CÞÞ ¼ 0.01

m3ð�rð1cmÞÞ ¼ 0.01;m3ð�rð1cmÞÞ ¼ 0.01; . . . ;m3ð�rð50cmÞÞ ¼ 0.01

Hence, significance for some subsets are as follows.

S1ðf�tð1CÞ;�tð2CÞ; . . . ;�tð7CÞgÞ ¼ 0.14





S3ðf�tð1CÞ;�tð2CÞ; . . . ;�tð45CÞ;�rð5cmÞ;�tð6cmÞ; . . . ;�tð9cmÞgÞ ¼ 0.50

For domains with some numerical data, such as sport reports or weather reports, mass can beassigned to particular models, and then mass for further models is obtained by interpolation.

7. Classification of inconsistencies using thresholds

We can classify structured news reports on the basis of the significance of the inconsistenciesthat arise in them. For any set of formulae W, we can set a threshold T r for rejection of the for-mulae where T r 2 [0,1]. So if T r < S(W), then W should be rejected, whereas if T r P S(W), then Wshould be accepted, perhaps after some resolution of inconsistencies.

Further thresholds can be adopted for finer grained selection of actions. For example, we couldadopt a threshold T i for ignoring inconsistencies. So if S(W) < T i, then the inconsistencies in Ware ignored, whereas if S(W) P T i, then the inconsistencies in W should be amended. Intermediatethresholds between T i and T r could be used to select the resources committed to resolve theinconsistencies. In the rest of this paper we will just consider rejection thresholds. The definitionsand results can be easily extended to handle any number of thresholds.

In this section, we will consider thresholds for models, and in the next section for formulae. Athreshold should change with size/structure of frame/mass assignment. As the frame increases,and hence as the mass is distributed more widely, the action and rejection thresholds should fall.

Definition 23. Let T r1 be a rejection threshold for X D1

1 with mass m1, and let T r2 be a rejection

threshold for X D2

2 with mass m2, where D1 � D2 and m1 is embedded in m2. Let S1 be a mass-basedsignificance function for m1 and let S2 be a mass-based significance function for m2. If thecondition below holds for all Y 2 }ðX D1

1 Þ, then T r2 is careful relaxation of T r

1.

S1ðY Þ > T r1 iff S2ðY Þ > T r

2


If T r2 is careful relaxation of T r

1, then any model Y, classified for rejection using m1 and T r1, is

also classified for rejection using m2 and T r2.

Proposition 9. Let T r1 be a rejection threshold for X D

1 with mass m1, and T r2 be a rejection threshold

for X D2 with mass m2, and m1 is embedded in m2. If T r

2 is a careful relaxation of T r1 then T r

1 P T r2.

Proof. Let m1 be embedded in m2. Therefore, for all Y 2 XD, m1(Y) P m2(Y). Hence, for allY 2 XD, S1(Y) P S2(Y). So, if T r

2 is a careful relaxation of T r1, then T r

1 P T r2. h

Example 21. We continue Example 20. Let T r1 ¼ 0.2 be a rejection threshold for X D

1 with mass m1,and T r

3 ¼ 0.1 be a rejection threshold for X D3 with mass m3. Let S1 be a mass-based significance

function for m1 and let S3 be a mass-based significance function for m3. So for selected models,we have the following.

S1ðf�tð1CÞ;�tð2CÞ; . . . ;�tð7CÞgÞ < T r1

S1ðf�tð1CÞ;�tð2CÞ; . . . ;�tð15CÞgÞ > T r1

S3ðf�tð1CÞ;�tð2CÞ; . . . ;�tð7CÞgÞ < T r3

S3ðf�tð1CÞ;�tð2CÞ; . . . ;�tð15CÞgÞ > T r3

In effect, the absolute values for significance evaluations and thresholds are not importantper se. However the combination of them allows us to classify inconsistencies. Furthermore,the thresholds can be relaxed for larger frames whilst preserving the same classification givento inconsistencies in subframes.

8. Significance of inconsistencies in news

We now return to how we can analyse inconsistent news. We assume news is input in the formof structured news reports and that the corresponding atom representing the structured news re-port can be used with a set of access rules to give a set of inferences as explained in Section 2. As asimple illustration, some atoms representing structured news reports, denoted {q1, . . . ,q5} aregiven in Example 22.

Example 22. Each of q1–q5 is an atom representing a structured news report.

AccessðD;q1Þ ¼ ftempð30CÞ;pptnðsnowÞ;pollenðhighÞgAccessðD;q2Þ ¼ ftempð30CÞ;pptnðsnowÞ;pollenðlowÞgAccessðD;q3Þ ¼ ftempð10CÞ;pptnðsnowÞ;pollenðhighÞgAccessðD;q4Þ ¼ ftempð10CÞ;pptnðsnowÞ;pollenðlowÞgAccessðD;q5Þ ¼ ftempð0CÞ;pptnðsnowÞ;pollenðhighÞg

Given a structured news report, we want to consider its consistency with respect to domainknowledge that we may possess. We will represent this domain knowledge by a set of proposi-tional formulae. A potential inconsistency that can arise between the information obtained froma structured news report is any set of atoms that may be rebutted by the domain knowledge.


Definition 24. Let q be a report and let C be domain knowledge. For {/1, . . . ,/n} � Access(D,q),{/1, . . . ,/n} is rebutted by C iff C ‘ :ð/1 ^ � � � ^ /nÞ.

So, if {/1, . . . ,/n} is rebutted by C, then C [ {/1, . . . ,/n} is inconsistent.

Example 23. Domain knowledge for weather reports may include the following three clauses.

ðclause 1Þ:tempð30CÞ _ :pptnðsnowÞðclause 2Þ:tempð10CÞ _ :pptnðsnowÞðclause 3Þ:pollenðhighÞ _ :pptnðsnowÞ

So with the report q1 in Example 22, we get that {temp(30C),pptn(snow)} is rebutted byclause 1, and that {pollen(high),pptn(snow)} is rebutted by clause 3.

Now we need to consider how we can handle these inconsistencies. First, we apply our massassignment to evaluate the significance of the inconsistencies.

Example 24. Continuing Examples 22 and 23, we suppose we have a mass assignment as follows.

mðf�tempð30CÞ;�pptnðsnowÞgÞ ¼ 0.6

mðf�tempð10CÞ;�pptnðsnowÞgÞ ¼ 0.3

mðf�pollenðhighÞ;�pptnðsnowÞgÞ ¼ 0.1

So the reports q1 to q5 in Example 22, together with the domain knowledge in Example 23, de-noted here by P, gives the following significance evaluations. S(P [ q1) = 0.7, S(P [ q2) = 0.6,S(P [ q3) = 0.4, S(P [ q4) = 0.3, S(P [ q5) = 0.1. If we set the threshold of acceptability for anews report qi at a significance evaluation of 0.3, then only q4 and q5 would be acceptable, theothers would be rejected.

This example illustrates how we may find some inconsistencies acceptable and others unaccept-able, and thereby select some news reports in preference to others. In practice, to harness this ap-proach, we need to adopt more sophisticated (first-order) domain knowledge, and then groundsubsets of the domain knowledge (and so create sets of propositional formulae) that are inconsis-tent with the (propositional) news reports.

To illustrate, consider a news report on a football match, where an inconsistency in the finalscore of the match could be assigned significance 0.6, an inconsistency in the name of the playerwho scored the most goals in less significant and could be assigned a value say 0.3, and inconsis-tency in the name of the referee is least significant and could be assigned a value say 0.1. Here, wecould represent the domain knowledge by schema that would be instantiated by constant symbolsfrom a news report. For example, referee(X) ^ referee(Y)! X = Y is schema that wouldbe ground by constant symbols in a particular football news report. Inconsistency analysis wouldthen be undertaken on the ground propositional formulae rather than the original schema.

Similarly, for anything other than a very restricted domain, we are likely to want the massassignment to be done for schema. For example, instead of a mass assignment for the followingframe we adopt a schema representation of the form m({±referee(X)}) = 0.6 which says themass assigned to any possible grounding for X is 0.6.


f�refereeðMikeDeanÞ;�refereeðSteveBennettÞ;� refereeðMartinAtkinsonÞ;�refereeðMikeRileyÞ; . . . .g

To do this in practice, we would also need some simple integrity constraints to ensure that wemaintain some highly desirable conditions of mass assignments. We leave the development ofschema and associated integrity constraints to a subsequent paper.

9. Discussion

The traditional view in logic is that a set of formulae is either consistent or inconsistent.However, for artificial intelligence, we need more than this binary classification. We need tobe able to better describe the nature of inconsistency in a set of formulae. Some proposalsfor measuring inconsistency have been put forward (see for example [12,13,7,14]), but the fieldneeds further development including in terms of evaluating the significance of inconsistency[15].

In this paper, we have developed an approach to measuring of inconsistency based on four-valued logic. In order to evaluate the significance of inconsistency, we have adopted the only pro-posal (as far as we know) for this [7]. In [7], QC logic was used for finding the four-valued modelsfor a set of formulae. In this paper, we have adopted a simpler and better-known four-valued logicthat was proposed by Belnap [9,10].

In addition, we have gone beyond the proposal in [7] by considering in more detail how eval-uating inconsistency can be used to decide on a course of action for dealing with inconsistentstructured news reports. In this paper, we have introduced the notion of modular mass assign-ments, and the usage of thresholds for deciding courses of action.

As a result of the framework in this paper, we now have a potentially valuable way of filteringstructured news reports before making them available in a database or newsfeed, or processingthem further by for example merging potentially conflicting reports from heterogeneous sourcesto create more comprehensive, better confirmed, less redundant and less uncertain aggregatednews reports (see for example developments in fusion rule technology http://www.cs.ucl.ac.uk/staff/a.hunter/frt).

Another framework for merging potentially conflicting knowledge, by Cholvy and Garion [16],has been implemented in Prolog. It appears to be an ideal platform for future work for investigat-ing the role of measures of degree and significance of inconsistency in knowledge fusion. In this,mass assignments could be used as meta-knowledge for providing a finer grained approach toaggregation than those based on social choice theory.

Whilst the motivation and examples in this paper have been about news reports, we have al-ready suggested that we can take a broad view of news reports. Many different sources of infor-mation can be described as providing news reports. Furthermore, much information can berepresented in the form of structured news reports, including weather reports, business reports,summaries of scientific articles, results from clinical trials, market intelligence, operations reportswithin organisations, etc. We are therefore considering any kind of regular supply of informationthat conforms to some stereotypical format. This means that the import of inconsistency, and theassociated problems of inconsistency, are wide-spread. The framework in this paper, offers a

http://www.cs.ucl.ac.uk/staff/a.hunter/frt

http://www.cs.ucl.ac.uk/staff/a.hunter/frt


solution to some of these problems. Furthermore, it may indeed form a valuable part of inconsis-tency tolerance in data and knowledge systems, more generally.

References

[1] J. Cowie, W. Lehnert, Information extraction, Communications of the ACM 39 (1996) 81–91.

[2] A. Hunter, Logical fusion rules for merging structured news reports, Data and Knowledge Engineering 42 (2002)

23–56.

[3] A. Hunter, R. Summerton, Fusion rules for context-dependent aggregation of structured news reports, Journal of

Applied Non-classical Logic 14 (3) (2004) 329–366.

[4] E. Byrne, A. Hunter, Man bites dog: Looking for interesting inconsistencies in structured news reports, Data and

Knowledge Engineering 48 (3) (2004) 265–285.

[5] E. Byrne, A. Hunter, Evaluating violations of expectations to find exceptional information, Data and Knowledge

Engineering 54 (2) (2005) 97–120.

[6] P. Besnard, A. Hunter, A logic-based theory of deductive arguments, Artificial Intelligence 128 (2001) 203–235.

[7] A. Hunter, Evaluating the significance of inconsistency, in: Proceedings of the International Joint Conference on

AI (IJCAI�03), Morgan Kaufmann, 2003, pp. 468–473.

[8] A. Hunter, Merging structured text using temporal knowledge, Data and Knowledge Engineering 41 (2002) 29–66.

[9] N. Belnap, How a computer should think, in: G. Ryle (Ed.), Contemporary Aspects of Philosophy, Oriel Press,

1976, pp. 30–55.

[10] N. Belnap, A useful four-valued logic, in: G. Epstein (Ed.), Modern Uses of Multiple-valued Logic, Reidel, 1977,

pp. 8–37.

[11] G. Shafer, A Mathematical Theory of Evidence, Princeton University Press, 1976.

[12] E. Lozinskii, Information and evidence in logic systems, Journal of Experimental and Theoretical Artificial

Intelligence 6 (1994) 163–193.

[13] A. Hunter, Measuring inconsistency in knowledge via quasi-classical models, in: Proceedings of the 18th National

Conference on Artificial Intelligence (AAAI�2002), MIT Press, 2002, pp. 68–73.

[14] S. Konieczny, J. Lang, P. Marquis, Quantifying information and contradiction in propositional logic through

epistemic actions, in: Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI�03),

Morgan Kaufmann, 2003, pp. 106–111.

[15] A. Hunter, S. Konieczny, Approaches to measuring inconsistent information, in: Inconsistency Tolerance. Lecture

Notes in Computer Science, vol. 3300, Springer, 2004, pp. 189–234.

[16] L. Cholvy, C. Garion, Querying several conflicting databases, Journal of Applied Non-Classical Logics 14 (3)

(2004) 295–327.

Anthony Hunter received a B.Sc. (1984) from the University of Bristol and an M.Sc. (1987) and

Ph.D. (1992) from Imperial College, London. He is currently a Reader in the Department of

Computer Science at University College London. His main research interests are: Knowledge

representation and reasoning; Analysing inconsistency; Argumentation Systems; and Knowledge

Fusion.

how to act on inconsistent news: ignore, resolve, or reject

Documents