cs 561: artificial intelligence - university of southern...

1

CS 561: Artificial Intelligence

Instructor: Prof Hadi MoradiInstructor: Prof. Hadi Moradi, [email protected]: M-Th 09:00-10:40, OHE136Office hours: MW 2:30 – 4:00 pm, SAL310,

O b iOr by appointmentTAs: Jeong-Yoon Lee

SAL 112Office hours: TTH 1:00-2:30Email: [email protected]

CS 561: Artificial IntelligenceCourse web page:

http://www-scf.usc.edu/~csci561aUp to date information, lecture notes Relevant dates, links, etc.Also you may check http://den.usc.edu

Class format: two sections of 45 minutesCourse material:Course material:

[AIMA] Artificial Intelligence: A Modern Approach, by Stuart Russell and Peter Norvig. 2nd edition

2


Course overview: foundations of symbolicCourse overview: foundations of symbolic intelligent systems. Agents, search, problem solving, logic, representation, reasoning, symbolic programming, probabilistic reasoning, and robotics.Prerequisites: CS 455x, i.e.,

programming principles, discrete mathematics for computing, software design and software engineering concepts. Some knowledge of C/C++ for some programming assignments.


Grading:Grading:25% for midterm 25% for final 40% for homeworks and projects10% f Q i10% for Quizzes

3

Practical issues

Class list: use learn usc eduClass list: use learn.usc.edu

Login with your USC username and passwordIf CSCI561A is not listed as your courses, notify the TA.ot y t e

Submissions: See class web page under Assignmentssubmit -user csci561 -tag HW3 HW3.tar.gz

Administrative Issues

Midterm 1: 7/26/10 9:00 10:40pmMidterm 1: 7/26/10 9:00 - 10:40pm

Midterm 2: 8/10/10 9:00 - 10:40pm

See also the class web page:http://den usc edu/http://den.usc.edu/

4

Why study AI?

Search engines

Science

Labor

Medicine/Diagnosis

AppliancesWhat else?

Humanoid Robots: From Honda to Sony

Walk

Turn

Stairshttp://world.honda.com/robot/

5

Sony AIBO

movie1

http://www.aibo.com

Natural Language Question Answering

http://www.ai.mit.edu/projects/infolab/http://aimovie.warnerbros.com

6

Robot Teams

USC robotics Lab

Modular robots self re-assembly.

What is AI?The exciting new effort to make “The study of mental facultiesThe exciting new effort to make computers thinks … machine with minds, in the full and literal sense” (Haugeland 1985)“The art of creating machines that perform functions that require intelligence when

The study of mental faculties through the use of computational models” (Charniak et al. 1985)

A field of study that seeks to explain and emulate intelligent behavior in terms ofrequire intelligence when

performed by people” (Kurzweil, 1990)

behavior in terms of computational processes” (Schalkol, 1990)

7

AI – The Bigger Picture

Computer SciencePhilosophy

?

Artificial Intelligence

Cognitive Science(Psychology)

p y

Robotics(Engineering)

Neuroscience(Biology)

?

Acting Humanly: The Turing Test

Alan Turing's 1950 article Computing MachineryAlan Turing s 1950 article Computing Machinery and Intelligence discussed conditions for considering a machine to be intelligent

8

Acting Humanly: The Turing Test

What tasks require AI?

“AI is the science and engineering ofAI is the science and engineering of making intelligent machines which can perform tasks that require intelligence when performed by humans …”

What tasks require AI?

9

What tasks require AI?Tasks that require AI:q

Solving a differential equationBrain surgeryInventing stuffPlaying Jeopardy Playing Wheel of FortuneWhat about walking?What about grabbing stuff?What about pulling your hand away from fire?What about watching TV?What about day dreaming?

Acting Humanly: The Full Turing Test

• Problem:• Problem:

10

What would a computer need to pass the Turing test?

Communication:Communication:Memory:Reasoning:Learning:

What would a computer need to pass the Turing test?

Sensing:Sensing:

M t t l (t t l t t)Motor control (total test):

11

Thinking Humanly: Cognitive Science

1960 “Cognitive Revolution”:1960 Cognitive Revolution : information-processing psychology replaced behaviorism

Thinking Humanly: Cognitive Science

Cognitive science and modeling the activitiesCognitive science and modeling the activities of the brain

What level of abstraction? “Knowledge” or “Circuits”?How to validate models?

12

Thinking Rationally: Laws of Thought

Aristotle (~ 450 B.C.) attempted to codifyAristotle ( 450 B.C.) attempted to codify “right thinking”

What are correct arguments/thought processes?

Thinking Rationally: Laws of Thought

Problems:

13

Acting Rationally: The Rational AgentRational behavior: Doing the right thing!

Provides the most general view of AI because it includes:

Acting Rationally: The Rational Agent

Advantages:Advantages:

14

How to achieve AI?How is AI research done?

TheoreticalExperimental

How to achieve AI?There are two main lines of research:There are two main lines of research:

Biological, study humans and imitate their psychology or physiology. phenomenal, study and formalize common sense facts about the world and the problems that the world presents to the achievement of goals.world presents to the achievement of goals.

The two approaches interact to some extent, and both should eventually succeed. It is a race, but both racers seem to be walking. [John McCarthy]

15

Branches of AILogical AISearchNatural language processingpattern recognition Knowledge representationInference From some facts, others can be inferred. ,Automated reasoning Learning from experiencePlanning To generate a strategy for achieving some goal

AI Prehistory

16

Brief History of AITh

ough

t Ancient Times 384 B.C. ----

AristotleLogic: The science of knowing.

Middle Age 1200 Ramon Lull

Ars Magnus: a rule-based device to model man's behavior and nature

Renaissance Empiricism

Next time implement links

tion

ally

:Law

s of

T Renaissance - EmpiricismExplanation of processes

17th Century ---

Gottfried Leibniz 1st system of formal logic

18th Century Rene Descartes

Dualism

19th Century 1845 -

--

Charles Babbage Analytical Engine

-

-George Boole Formalization of the Laws of Logic

Thin

king

Rat

--

Formalization of the Laws of Logic

1879-1903 -

--

Gottlob Frege First-order predicate calculus

Early 20th Century

1910-1912 ---

Russel-Whitehead Principia Mathematica

Bertrand Russel

1931 ---

Kurt Godel Incompleteness Theorem of Logic

Roots of AI in Science:Aristotle(b.384-): syllogism – formal reasoningRamon Lull (b.1235): Ars Magna – a machine capable of answering all questionsRene Descartes (1596): mind / body separation (dualism); "cogito ergo sum“Wilhelm Liebniz (1646-1716): a mechanical concept generator; "materialism"g ;Charles Babbage(1792-1871), Ada Lovelace (1815-1860): Analytical Engine – a general-purpose calculatorGeorge Boole(1815-1864): logic algebras - logical encoding and calculation of thoughtsGottlob Frege(1848-1925): predicate calculus

17

Birth of Artificial Intelligence1940-1956 - 1942 - ENIAC :First digital computer

Grea

--

1943 ---

Mc Culloch and Pitts Artificial neural network

1945 ----

J. Von Neumman Modern computer architecture

1949 ---

Claude Shannon Use of heuristics to solve complex problems at E

xpectations

1950 ----

Alan M.Turing Computing Machinery and Intelligence:

Turing Test

1955 ---

Herbert Simon,Alan Newell 1st AI program:Logic Theorist

Herbert Simon

1956 ---

Dartmouth Conference

The Beginning of AIMcCulloch & Pitts

developed theory of artificial neurons (precursor to ANN's) – 1943

Alan Turing – "Can Machines Think?"the turing test (1950)the turing machine

Marvin Minsky & Dean Edmondsfirst ANN constructed, 1951

John McCarthyconvened the Dartmouth conference that coined the term artificial intelligence (AI) (1956) and set the research agenda

symbolic AIconnectionism

LISP (list processing) 1958 1st AI language

18

The Rise of AI1957- 1960’s - 1958 -

--

John McCarthy. LISP

Grow

ing

1960 ---

Marvin Minsky Theory of Frames

1961 ----

Herbert Simon,Alan Newell GPS:General Problem Solver

Herbert Simon

1962 ---

Frank Rosenblatt Perceptron: Learning in Neural Networks

1965 L tfi A Z d h

g Disenchantm

ent

1965 ----

Lotfi A. ZadehFuzyy Logic Fuzzy Sets

1968 Joseph Weizenbaum ELIZA: simulates diagnosis by a psychiatrist.

1969 ---

Marvin Minsky,Seymour Papert Limitations of Perceptrons

S. Papert

An Optimistic StartIn the 50's, 60's and early 70's, much exciting progress was being made in AI:Chess

Claude Shannon, 1950The Logic Theorist

Alan Newell, Cliff Shaw, Herb Simon, 1957Checkers (Machine Learning)

Arthur Samuels, 1959Eliza - NLP

Joseph Weizenbaum, 1966DENDRAL – Knowledge-Based System

Feigenbaum, Buchanan, Lederberg, 1969SHRDLU – NLP (Blocks World)

Terry Winnograd, 1972GPS (General Problem Solver)

Alan Newell & Herb Simon, 1972

19

The 70’s

Bi th d Ri f E t S tBirth and Rise of Expert Systems

1970-mid 1980’s - 1973 -

--

Alain Colmerauer PROLOG

1974 ---

Paul Werbos Neural Networks Back Propagation Law

1975 E. Feigenbaum, R. Lindsay. Dendral

E.FeigenBaum

Edward Shortliffe MYCIN

1976-1980

R. Duda, P.Hart, P. Barnett PROSPECTOR: The first commercial Expert System

P.Hart

1982 John McDermott XCON – "Expert Configurer

The PlateauIn the 70's, AI researchers began to discover that the problem wasn't as easy as it looked!

The Frame ProblemL k f C S R iLack of Common Sense ReasoningCombinatorial ExplosionThe Gap – "Toy" vs. "Real" worldsPerceptrons, by Minsky & Papert (1969) – proved limitations of perceptron networks and acted to limit significant research in the 70'ssignificant research in the 70 sLighthill Report – 1973: curtailed research funding in British Universities

AI developed a reputation as "over-hyped" and unrealistic

20

The 80’s 1982 -

---

John HopfieldHopfield Networks

1982 Teuvo Kohonen

self-organising feature maps for speech recognitizion

1986 T S j kite

ms

work

s 1986 -

--

Terrence SejnowskiNETTalk

Rumerhalt,McMelland

Neural Networks Rediscovering of Back-Propagation Learning

1987 ----

Marvin Minsky The Society of Minds

atio

n of

Exp

ert

Syst

tifi

cial

Neu

ral N

etw

Fuzzy Appliances

1989 ---

Dean Pomerleau ALVINN

Com

mer

cial

iza

Rebi

rth

of A

rt

Commercial Success Despite it's reputation as "over hyped" certainDespite it s reputation as over-hyped , certain AI applications became very successful during the 70's – 80's:

• Expert SystemsIndustrial Robotics• Industrial Robotics

• Planning & Scheduling Applications

AI became a $2,000,000,000 industry by 1988

21

Nowadays…Early 90’s -

-Major advances in all areas of AI, with significant demonstrations

--

significant demonstrations

1995 Birth of Intelligent Systems

1997 The Deep Blue chess program beats Garry Kasparov

Late 90’s - Web crawlers --

AI-based information extraction programs

Intelligent Room and Emotional Agents at MIT's AI Lab

2000- Interactive robot pets The Nomad robot

The Gartner Hype Curve

Interest in AI followed this patternInterest in AI followed this pattern, typical of the hype surrounding new technologies

22

AI State of the artHave the following been achieved by AI?

World-class chess playingPlaying table tennisCross-country drivingSolving mathematical problemsDiscover and prove mathematical theoriesDiscover and prove mathematical theoriesEngage in a meaningful conversationUnderstand spoken languageObserve and understand human emotions…

Types of expertise (with examples)Deepcognitive

Judgmentalskills

High-levelsocial skillscognitive

skillsskills social skills

Highlycreative

Musician Seniormanager

Author, poet

Analytical Mathemat-i i

Economist, Sociali ti tician programmer scientist

Strictlyprocedural

Typist Driver Socialworker

23

A driving example: Grand Challenge

Goal:

Artificial Intelligence Applications

CognitiveScience

ArtificialIntelligence

Robotics NaturalInterfaceScience

Applications Applications InterfaceApplications

•Expert Systems•Fuzzy Logic•Genetic Algorithms•Neural Networks

•Visual Perceptions•Locomotion•Navigation•Tactility

•Natural Language•Speech Recognition•Multisensory Interface•Virtual Reality

24

AI Application Areas in Business

Neural Networks

Fuzzy Logic Systems

Virtual Reality

Genetic Algorithms

y

Expert Systems

AI ApplicationAreas inBusiness

Intelligent Agents

Components of Expert SystemsThe Expert System

K l d

ExpertAdvice User

I t fInferenceE i Knowledge

Base

User Workstation

InterfacePrograms

EngineProgram

Expert System Development

Workstation

KnowledgeEngineering

KnowledgeAcquisition

ProgramExpert and/or

Knowledge Engineer

25

Expert System ApplicationsDecision Management

Diagnostic/TroubleshootingDiagnostic/Troubleshooting

Maintenance/Scheduling

Design/Configuration

MajorSelection/Classification

Major ApplicationCategoriesof Expert Systems

Process Monitoring/Control

General Introduction

Course Overview


Introduction. [AIMA Ch 1] Course Schedule. Homeworks, exams and grading. Course material, TAs and office hours. Why study AI? What is AI? The Turing test. Rationality. Branches of AI. Research disciplines connected to and at the foundation of AI. Brief history of AI. Challenges for the future. Overview of class syllabus.

26


Course Overview

sensors

effectors

Agent

General IntroductionIntelligent Agents. [AIMA Ch 2] What is an intelligent agent? Examples. Doing the right thing (rational action). Performance measure. Autonomy. Environment and agent design. Structure of agents Agent types Reflex agentsStructure of agents. Agent types. Reflex agents.Reactive agents. Reflex agents with state. Goal-based agents. Utility-based agents. Mobile agents. Information agents.

Course Overview (cont.)9 l

Problem solving and search. [AIMA Ch 3]

measuring problem. Types of problems. More examples. Basic idea behind search algorithms.

3 l 5 l9 l

Using these 3 buckets,measure 7 liters of water.

Complexity. Combinatorial explosion and NP completeness.Polynomial hierarchy.

Traveling salesperson problem

27

Course Overview (cont.)How can we solve complex problems?

Uninformed search. [AIMA Ch 3]

Depth-first. Breadth-first. Uniform-cost.

3 l 5 l9 l

Using these 3 buckets,measure 7 liters of water.

Depth-limited. Iterative deepening. Examples. Properties.


Course Overview (cont.)How can we solve complex problems?

Informed search. [AIMA Ch 4] Best-first. A* search. Heuristics.

p p

Hill climbing. Problem of local extrema. Simulated annealing.


28

Course Overview (cont.)Practical applications

of search

Constraint Satisfaction[AIMA Ch 5]

BacktrackinggLocal search

Course Overview (cont.)Practical applications

of search

Game playing [AIMA Ch 6]

The minimax algorithm.gResource limitations. Aplha-beta pruning. Elements of chance and non-deterministic games.

tic-tac-toe

29

Course Overview (cont.)Towards intelligent agents

Agents that reason logically 1

[AIMA Ch 7]Knowledge-based

Towards intelligent agents

agents. Logic and representation.Propositional (boolean) logic.

wumpus world

Course Overview (cont.)Towards intelligent agentsAgents that reason logically 2.

[AIMA Ch 7]Inference in

i i l l i

Towards intelligent agents

propositional logic. Syntax.Semantics. Examples. wumpus world

30

Course Overview (cont.)Building knowledge-based u d g o edge ased

agents: 1st Order LogicFirst-order logic 1. [AIMA Ch 8]

Syntax. Semantics. Atomic sentencesAtomic sentences. Complex sentences. Quantifiers. FOL knowledge base. Situation calculus.

Course Overview (cont.)

Building knowledgeBuilding knowledge-based agents: 1st

Order Logic

First-order logic 2.[AIMA Ch 9]

Describing actions. Planning. Action sequences.

31


Reasoning LogicallyReasoning LogicallyInference in first-order logic.

[AIMA Ch 9]Proofs. UnificationUnification. Generalized modus ponens. Forward and backward chaining.

Example ofbackward chaining

Course Overview (cont.)Representing and OrganizingRepresenting and Organizing

KnowledgeBuilding a knowledge base.

[AIMA Ch 10]Knowledge bases.Knowledge bases. Vocabulary and rules.OntologiesOrganizing knowledge.

An ontology for the sports domain

32

Course Overview (cont.)Systems that can Plan y

Future BehaviorPlanning.[AIMA Ch 11]

Definition and goals. Basic representations for l iplanning.

Situation space and plan space. Examples.

Course Overview (cont.)Learning from Observationg

Decision Trees[AIMA 18]

Introduction to decision trees. Information theory.Constructing DT.Examples.

33

Course Overview (cont.)Expert Systemsp y

Probabilities + Bayesian Networks[AIMA 13 + 14]

Basics of probability theory Bayesian rule.

d lConditional reasoning. Bayesian Networks. Reasoning under uncertainty

Course Overview (cont.)Statistical Learning Methodsg

Neural Networks.[AIMA 20]

Human brain structureNeuron and activation function.Forward and backward propagations.Examples.

34

Course Overview (cont.)Logical Reasoning in the g gPresence of Uncertainty

Fuzzy logic [Handout]

Introduction to fuzzy logic. Center of gravity

y gLinguistic Hedges.Fuzzy inference.Examples.

Center of largest area

Course Overview (cont.)Machine Learningg

Genetic Algorithms [Handout + AIMA 4]

Genetic algorithm approach. Mutation, Crossover, Fitness function.Examples.

35

Course Overview (cont.)What challenges remain?g

Towards intelligent machines.[AIMA Ch 25]

The challenge of robots: with what we have learned, what hard problems remain to be solved?what hard problems remain to be solved? Different types of robots. Tasks that robots are for. Parts of robots. Architectures.Configuration spaces.

robotics@USC


What challenges remain?What challenges remain?

Overview and summary. [all of the above]

What have we learnedWhat have we learned. Where do we go from here?

robotics@USC

36

Outlook

AI is a very exciting area right nowAI is a very exciting area right now.This course will teach you the foundations.

cs 561: artificial intelligence - university of southern...

Documents