Theory and Practice in AI and Law: A Response to Branting

Katie Atkinson and Trevor Bench-Capon

Department of Computer ScienceThe University of Liverpool

Overview

• Over the years there has emerged something of an academic consensus in the AI and Law field

• Practitioners, however, often find it difficult to relate this theory to practice– Trenchant criticisms from Karl Branting

• This talk attempts to explain why these discrepancies exist

• Some remarks on categories of legal AI systems

Theoretical Consensus

• Models of reasoning with legal cases– CBR approaches of Rissland, Ashley,

Aleven, Brunighaus– Rule/Logic based approaches of Prakken,

Sartor, Hage, Roth, Bench-Capon– Theory (McCarty), Purpose (Berman and

Hafner) and Dialogue (Gordon)

• A range of work from which some agreement has emerged

Features of the Consensus

• Representation of cases using factors– Not facts, but patterns of facts with legal

significance• Reasoning based on argument: plaintiff and

defendant exchange arguments based on factors present in the case and precedents

• These arguments can be presented as dialectical exchanges

• Decisions are based on an evaluation of the competing arguments

• Strengths of arguments can be based on the purposes served by accepting them.

Questions for the Consensus

• Branting has argued that experience with real decisions deviates from this consensus

• He has six points of contention

Contentions

• The outcomes of disputes are highly predictable– It is typically hard to find plausible

arguments for both sides of the case• Uncertainty arises from

characterisation of facts. Not interpretation of precedents

• Decisions are justified by precedents, not by rules and construction

Contentions

• Judicial decisions are not frozen dialectic

• Decisions almost never make reference to purposes

• Factor based argumentation is rare

• So why does the practice theory not reflect the consensus theory?

Levels of Reasoning

• In previous work we identified three levels at which arguments can be presented in a case:– Reasoning about the world to determine

how the law should be– Reasoning about legal concepts and

their applicability to achieve this– Reasoning about the consequences of

decisions made at level two

Levels of Reasoning

Use of these Levels

• Often only Level Three is used:– Applicability is not disputed: no need to reason at level 2

• Where applicability is in dispute, Level 2 is needed. • But Level 1 is needed only if “new” law is required

– Gaps in understanding– Conflicts to resolve– Changes in response to social change

• Many decisions confine themselves to a report of the Level 3 argumentation.

• Precedents embody past reasoning from lower levels

Predictability

• At Level Three all is predictable: the consequences of applicable legal consequences are understood

• At Level 2, applicability of a concept may be disputed: but precedents are usually clear enough

• Hard cases are relatively rare: landmark cases producing new law are rarer

Characterisation of Facts

• Characterisation of facts takes place at the level 2, perhaps based on conclusions from Level 1.

• Level 1 is where there is little help from precedent, and is the least circumscribed, so new considerations can be introduced

• Level 1 is thus likely to be the source of uncertainty and unpredictability

Justification is by Precedent

• A precedent often embodies a rule, and the application of that rule.

• It acts as a summary of argumentation from a lower level

• If cited at, e.g. Level 3, therefore it can stand for arguments that were considered and evaluated previously, and so avoid the need to rehearse them again.

• Reasoning in a case need not be from first principles

Decisions are not Frozen Dialectic• It is important not to confuse the process

by which a decision is reached with the way in which the decision is reported.

• How we choose to summarise the conclusions from a dispute need not reflect the process by which we reached those conclusions

• If the decision confines itself to Level 3, the findings can be stated without rehearsing the arguments

Reference to Purpose is Rare• Level 1, which is where the teleological

factors are considered is required only when – “new” law is needed, – The law needs to be changed

• Very few such landmark cases arise in areas of settled law, and so this level is not invoked

• Conclusions as to purpose are embodied in precedents to be used at level 2

• Much more common in Supreme Court decisions: Branting focuses on lower level courts

Factor Based Argumentation is Rare• Again factor based argumentation

only becomes explicit when Level 1 is reached.

• Typically the precedent will encapsulate a rule originally produced in this way, but which can be cited to avoid restating such arguments

• Again used in reasoning from first principles

Theory Construction

• Bench-Capon and Sartor proposed seeing reasoning with legal cases as theory construction

• How does this fit with our levels?

Preferencesbetween sets

of values

Factors andoutcomes in

Decided Cases

Preferences between sets

of factors

Factors and outcomes inNew Cases

reveal

reveal explain/determine

determine

DomainAnalysis

input

Level 3Level 2

Level 1

Level 1

Preferencesbetween sets

of values

Factors andoutcomes in

Decided Cases

Preferences between sets

of factors

Factors and outcomes inNew Cases

reveal

reveal explain/determine

determine

DomainAnalysis

input

British Nationality Act/

Expert Systems

Preferencesbetween sets

of values

Factors andoutcomes in

Decided Cases

Preferences between sets

of factors

Factors and outcomes inNew Cases

reveal

reveal explain/determine

determine

DomainAnalysis

input

HYPO/

CATO

Preferencesbetween sets

of values

Factors andoutcomes in

Decided Cases

Preferences between sets

of factors

Factors and outcomes inNew Cases

reveal

reveal explain/determine

determine

DomainAnalysis

input

IBP

Preferencesbetween sets

of values

Factors andoutcomes in

Decided Cases

Preferences between sets

of factors

Factors and outcomes inNew Cases

reveal

reveal explain/determine

determine

DomainAnalysis

input

AGATHA

Berman/Hafner 93

Preferencesbetween sets

of values

Factors andoutcomes in

Decided Cases

Preferences between sets

of factors

Factors and outcomes inNew Cases

reveal

reveal explain/determine

determine

DomainAnalysis

inputUnaddressed?

Summary

• The consensus models reasoning from first principles

• In practice decisions do not recapitulate the whole story

• Precedents can be used to avoid the need to reason from first principles

• Decisions state conclusions, and some level of justification

• The consensus assumes a domain analysis, which avoids difficulties in characterising facts– This may be the hardest problem of them all