Common Sense Computing • MIT Media Lab

Interaction Challenges for Agents with Common Sense

Henry Lieberman

MIT Media Lab

Cambridge, Mass. USA

Http://www.media.mit.edu/~lieber

Common Sense Computing • MIT Media Lab

Agents with Common Sense

Some AI interfaces are now beginning to make use of large knowledge bases of Common Sense

Explicitly collected

Cyc, Open Mind, ThoughtTreasure

Distilled from other sources

Semantic Web, Wikipedia, Web resources

Web mining, other resource mining

Common Sense Computing • MIT Media Lab

Interaction Challenges

Finding opportunities for applying Common Sense in interfaces

Setting users' expectations

Making interfaces fail-soft

Taking advantage of user interaction

Common Sense Computing • MIT Media Lab

Interaction Challenges

Making better mistakes

Get lots of knowledge, but not too much

Common sense inference vs. mathematical inference

Debugging

Evaluating Common Sense interfaces

Common Sense Computing • MIT Media Lab

Interaction challenges for AI/Common Sense

Many interaction challenges for Common Sense interfaces are the same as for AI in general

But some are unique or critical for Common Sense…

Can't be sure what will be known

Reasonable, rather than right

Know a little about everything, not much about anything

Don't miss the obvious

Try not to make stupid mistakes

Common Sense Computing • MIT Media Lab

Open Mind Common Sense

Asks the Web community to contribute English sentences expressing Common Sense knowledge

"The Wikipedia version of Cyc" (#2 after Cyc)

Launched in 2001 by Push Singh

Now contains ~800 Kilofacts

Freely available / Open Source

Some versions in other languages/cultures

Brazilian Portuguese, Korean, Japanese, Chinese

Common Sense Computing • MIT Media Lab

Open Mind Common Sense

English sentences parsed by POS Tagger

Pattern-directed mining of 22 relations

(isA, PartOf, UsedFor…)

Strong focus on easy integration with applications

Semantic Net (ConceptNet: Liu, Eslick)

Natural Language toolkit (MontyLingua: Liu)

Tools for: Context, Analogy, Affect and more

Common Sense Computing • MIT Media Lab

What do we mean by “Common Sense”?

Simple statements about everyday lifeThings fall down, not upA wedding has a bride and a groomYou go to a restaurant to eat

And…the ability to use that knowledge when appropriate

Common Sense Computing • MIT Media Lab

Open Mind Common Sense- Push Singh & colleagues

Common Sense Computing • MIT Media Lab

ConceptNet - Liu, Singh, Eslick

Common Sense Computing • MIT Media Lab

Common Sense projects

Predictive typing

Speech recognition -- disambiguation & error correction

Storytelling with photo libraries

Searching social networks

Macro recording using Common Sense generalization

World construction for video games

Phrasebook for tourist information

Debugging problems in e-commerce interactions

Common Sense Computing • MIT Media Lab

Common Sense projects

Video editing based on story structures

Goal-oriented interfaces for consumer electronics

Mining Common Sense from the Web

Multi-lingual and multi-cultural Common Sense; translation

Games for acquiring, verifying and using Commonsense knowledge

Commonsense "Captchas"

Understanding imprecise qualities such as affect

ShapeNet and Expectation-driven Image Recognition

Understanding sensor data using Commonsense

Common Sense Computing • MIT Media Lab

Opportunities for using Common Sense

Find UI situations that are underconstrained

Ordinary system would either take no action or do something arbitrary

Then, give user some reasonable choices

Provide intelligent defaults

Make the most likely thing easiest to do

Common Sense Computing • MIT Media Lab

Opportunities for using Common Sense

Recognize users' likely goals

Help users map from goals to actions

Sanity checking

In the case of trouble, help users debug

Common Sense Computing • MIT Media Lab

Opportunities for using Common Sense

Find situations where every little bit helps

A little bit of knowledge is better than none

A little bit of knowledge about a lot of things can be more useful than a lot of knowledge about a few things

Common Sense Computing • MIT Media Lab

Setting users' expectations

Avoid direct question-answer interfaces

Right or wrong. Only one shot.

Better to cast system in role of advisor

Making suggestions, help

Adapting interface to most likely uses

Remove unnecessary steps in the interface

The user only expects intelligent behavior only once in a while

Common Sense Computing • MIT Media Lab

Take advantage of user interaction

Repurpose input that the user gives you for other reasons

Every time the user tells the interface something, they're telling you what their interests are -- learn from it

Common Sense Computing • MIT Media Lab

Make Common Sense interfaces "fail-soft"

There should no dire consequences of being wrong or not knowing what to do

Don't interfere if the user wants to use the application without interaction with the agent

If the relevant knowledge is missing, incomplete or wrong, the user is no worse off than without the agent

Common Sense Computing • MIT Media Lab

Make better mistakes

Common Sense approaches have the advantage that when they make mistakes, they tend to make plausible mistakes

Statistical approaches can make arbitrary mistakes

Better mistakes improve user trust in interfaces

Common Sense Computing • MIT Media Lab

Evaluation of Common Sense interfaces

Evaluation is tough because

Depends on what's in the knowledge base

Depends on limited-depth and other kinds of approximate inference

Standardized tasks don't test breadth of coverage

Try to relativize testing to coverage

Start with easier cases, then move to "typical" or "hard" cases

Intelligent User Interfaces 2008www.iuiconf.org

• Location: Canary Islands, Spain

• Dates: 13-16 January 2008

• Deadline: late September 2007