exam preparation exam review

20
1 Exam Review GENERAL Exam Preparation Look through • Assignments • Notes • Flakey-example • Logic-Test Know Terminology of the field Concepts and Methods (abstract level) How to apply methods (some) Review & Exam Preparation You should know • the terminology (key words etc.) e.g. – KR, description logics: concept, role, TELL & ASK, classifier – NN: neuron/unit, threshold, competitive learning, SOM, ... – NL Parsing: grammar, parse tree, ambiguity, metaphor, pragmatics, ... Review & Exam Preparation You should know • the main ideas, concepts, and methods e.g. – KR, description logics: inheritance hierarchies, concept definition language, constructive semantics, can be related to FOPL – NN: learning through weight adjustment; supervised vs. unsupervised learning, backprop uses delta-rule; general I/O processing e.g. in classification – NL Parsing: parsing, problems (ambiguity), chart, Earley-Algorithm (why is it good?) Review & Exam Preparation You should be able to transfer and apply knowledge e.g. – KR, description logics: use different formalisms (FOPL, DL, Sets, Frames) to describe KB – NN: map problem onto NN representation; calculate small network – NL Parsing: inpret and develop a chart (graphic, Earley-ALgorithm) Introduction to Agents

Upload: others

Post on 21-Apr-2022

53 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Exam Preparation Exam Review

1

Exam Review

GENERAL

Exam Preparation

Look through• Assignments• Notes• Flakey-example• Logic-TestKnow• Terminology of the field• Concepts and Methods (abstract level) • How to apply methods (some)

Review & Exam Preparation

You should know• the terminology (key words etc.) e.g.

– KR, description logics: concept, role, TELL & ASK, classifier

– NN: neuron/unit, threshold, competitive learning, SOM, ...

– NL Parsing: grammar, parse tree, ambiguity, metaphor, pragmatics, ...

Review & Exam Preparation

You should know• the main ideas, concepts, and methods e.g.

– KR, description logics: inheritance hierarchies, concept definition language, constructive semantics, can be related to FOPL

– NN: learning through weight adjustment; supervised vs. unsupervised learning, backpropuses delta-rule; general I/O processing e.g. in classification

– NL Parsing: parsing, problems (ambiguity), chart, Earley-Algorithm (why is it good?)

Review & Exam Preparation

You should be able to• transfer and apply knowledge e.g.

– KR, description logics: use different formalisms (FOPL, DL, Sets, Frames) to describe KB

– NN: map problem onto NN representation; calculate small network

– NL Parsing: inpret and develop a chart (graphic, Earley-ALgorithm)

Introduction to Agents

Page 2: Exam Preparation Exam Review

2

• General Agent Architectures • From sensory input to motoric output • Task Environments• Examples of Agent Types• Flakey and ShakeyBased on Norvig, Ch. 2 and Nilsson, Ch. 1, 2, 25

Intro Agents Agent Architecture (Norvig)

Agent Processingsensor data speech signal, image, ...perception phonemes, visual objects, ...cognition concepts (language or visual) reasoning conclusions, generalizationgoal setting & evaluation priorities, utility functionplanning from goal to set of actions action selection & execution control action performance & motor control transform

high-level actions into low-level robot actionslearning perceptual, conceptual, plan level

Example 1: Mother hears her Baby cry.

sensor data - soundwave, auditory inputperception - some squeaky noise; baby screamcognition - “my baby cries”reasoning - “I hope she is okay.” “She is hungry.”goal setting, evaluation - “I have to see the doctor with her.” “We have to move to another city.” ...action / plan selection - go feed herplanning - drop laundry, walk upstairs, feed her action selection - drop laundry action performance - open hand motor control - move fingers in certain position

Example 2: Taxi Driver sees Stop sign.

sensor data - light waves, visual inputperception - red sign with some letterscognition - “STOP sign” reasoning - “I have to stop.” “I will be late.” goal setting, evaluation – “Stop the car” “Next time I’ll take the other route.” “I quit my job.”action / plan selection - stop and wait; watch trafficaction selection - hit the brakes, ... action performance - move right foot on brake pedal motor control - move foot along a trajectory until it rests on the brake pedal; apply certain force

Robotic Sensors• (digital) camera - vision• infrared sensor - vision• range finders, (radar, sonar) - distance• GPS - location• tactile (whiskers, bump panels)• proprioceptive sensors, e.g. shaft decoders• force sensors - haptic• torque sensors - haptic

Page 3: Exam Preparation Exam Review

3

Robotic Effectors

• ‘limbs’ connected through joints; • degrees of freedom= #directions in which

limb can move (incl. rotation axis) • drives : wheels (land), propellers, turbines

(air, water)• driven through electric motors , pneumatic

(gas), or hydraulic (fluids) actuaction• statically stable, dynamically stable

Task Environment Classifications (Norvig)Depends on task, environment, and sensors

§ fully observable vs. partially observable§ video camera in bright room vs. infrared camera

§ deterministic vs. stochastic vs. non-deterministic§ assembly line vs. weather vs. “odds & gods”

§ episodic vs. non-episodic § assembly line vs. diagnostic repair robot, Flakey

§ static vs. dynamic§ room without vs. with other agents

§ discrete vs. continuous§ chess game vs. autonomous vehicle

Examples of Agents (Norvig) Describe Flakey

Sensor Equipment?Action Repertoire?Task Environment?Perceptions and Cognition?Goals? Intentions?Type of Agent?

Knowledge Representation

First-Order Predicate LogicDescription LogicsExotic LogicsOntologyConcepts, Frames etc.

First-Order Predicate Logic

Ø Syntax Ø SemanticsØ Inference System Ø Axioms (facts and rules)Look at 'Logic-at-a-glance' first.Look at Assignment1 and Logic-Exercise!

Page 4: Exam Preparation Exam Review

4

Syntax

• well formed formula• open and closed formulas

quantifiers; bound and unbound variables

Decide whether a formula is wff or not.Transform statement into wff.Do tasks in Logic-Test

Semantics

• interpretation, valuation• structure, domain, universe• constructive semantics • model (of formula or theory)• logical consequence α |= βLook at the tasks in Assignment 1 and

the Logic-Test.

Logic System, Inference

• inference syntactic manipulation of formulasfamous inference rules (MP, UI, EG)

• soundness and completeness of FOPL• theorem and theory

theorem is derived or concluded formula theory is set of axioms asumed to be true

Syntax and Semantics

• model (of formula or theory) • axioms (FOPL, Proper) • logical consequence |= • formal inference |−• soundness and completeness

Description Logics

Description Logics

• Background and Motivation for DL• Relation between DL and FOPL (extensional Semantics)

• Example Definitions• The horrible truth of DL• T-Box, A-Box• LOOM

Page 5: Exam Preparation Exam Review

5

Description Logics - Background

• Prevent flaws of Semantic Networks ; provide clear semantics of nodes and links

• Derived from Inheritance Hierarchies (IS-A hierarchies; taxonomies; terminologies) and Frame Systems → Concepts, Roles, Features

• Series of KR Languages starting with KL-ONE, later KL-TWO, LOOM, CLASSIC, ...

• Relate to FOPL via extensional semantics

• Applied e.g. in Semantic Web Languages(DAML+OIL)

Description Logics – Basic Elements• Concepts• Roles• Features / Attributes• Definition of concepts

(defconcept Father (AND (Parent Man)), (defconcept Biped (AND (Animal (exactly 2 has-legs Legs)))

• Primitive / Atomic Concepts and Roles (not completely defined)

• Individual Concepts (define singletons)• Instances (entities)

Basic Architecture and Functions

• T-Box – terminological knowledge (definition of concepts)

• A-Box – assertional knowledge (statements about instances, concrete objects)

• Classification - insert concept into hierarchy based on it's description / definition

• TELL – insert knowledge into KB, e.g. define new concept or assert instance info

• ASK – querying KB, e.g. ask about existence of instance

Exotic Logics

Non-Classical Logics

Non-Classical Logics• Specific Language Constructs added to classic

FOPL• Different Types of Logics• Modal Logics most popular ones, e.g.

• Deontic Logic (allowed and forbidden; ethics; law)

• Epistemic Logic (Knowledge) and Doxastic(Belief) Logic

• Possible World Semantics

Non-Classical Logics 1

• (many-) sorted logic individuals pre-arranged in sets = sorts

• many-valued logicmore than two truth values (e.g. Lukasiewicz“intermediate truth” = I; "don't know" status)

• fuzzy logicdegree of truth between 0 and 1 (predicate corresponds to fuzzy set; membership in set to a certain degree, not just yes or no)

Page 6: Exam Preparation Exam Review

6

Non-Classical Logics 2

• non-monotonic logicbelief modeling; defaults; set of true formulae can change (non-monotonicity); TMS

• higher-order logicquantification over predicates (as variables), like ∀P: ..., or statements about statements, e.g. “This sentence is false.”

• modal logics (see later slides)describe “subjunctive” statements in addition to assertional statements, using Modal Operators, i.e. "possible P" and "necessary P"

Non-Classical Logics 3• time logics

• time as temporal modality

• time logic based on time points and relations between them (like “t1 earlier than t2”)

• Allen’s model of time intervals• situational logic; situation calculus (McCarthy)

• situation as additional parameter to predicate expressions, for describing change due to events

• additional axioms describe transformations of ssituations due to actions

• used for reasoning with time and planning

Modal Logics 1Uses additional operators and axioms to describe logic. Includes FOPL assertions, and in addition statements using Modal Operators. Different Modalities express different types of statements, e.g.• alethic modality

“necessary” and “possible” as additional operators• temporal modality

with necessary ≡ “always” and possible ≡“sometimes”

• deontic modality“permissible” (allowed) and “obligatory” (must)

• epistemic modality“knows” and “beliefs” as operators

Ontology

Ontology

• Ontology Languages• Semantic Web• "Real" Ontologies

Ontology Languages

LOOM – DL based language by Bob McGregor, ISI, USC, Los Angeles

Classic – DL based language by Deborah McGuiness, Alex Borgida et al., ATT Bell Labs

Ontolingua – KB development tool (with predefined Ontology), Stanford U.

Semantic Web – Ontology languages for Internet organization and search (DAML, OIL, OWL), see W3C

Page 7: Exam Preparation Exam Review

7

Ontologies

Wordnet – based on linguistic descriptions

Ontosaurus – Ontology used with LOOMCyc, OpenCyc – Knowledge Base organization system Ontolingua – KB development tool with predefined Ontologies, Stanford U.Microcosmos – Ontology developed for Computational Linguistics, CRL

OWL, OIL, and DAML

• DAML - DARPA Agent Markup Language• OIL - Ontology Inference Engine• DAML+OIL - "semantic markup language for

Web resources" • integrate Ontology Language with the

Resource Description Framework (RDF) • OWL -Web Ontology Language (follower of

DAML and OIL; based on DL)• Developed by the WWW Consortium: W3C.

"Real" Ontology

Ontology• looks for semantic and ontological primitives

and concepts to describe aspects or parts of “the world”

• aim of research is to develop KBs which can be shared and commonly used in various contexts, i.e. for different applications and different fields (e.g. for NLP, reasoning, planning etc.)

• relates to psychology and philosophy, as well as data bases and object-oriented programming systems

Basic Concepts in Ontologies

• Physical Objects ("Things")– Stuff and Things; Composite Objects– Fluids– Substances

• Abstract Objects ("Ideas")– abstract concepts (ideas) like 'freedom'– difficult to define and describe– also mathematical concepts– intermediate concepts (model properties)

Basic Concepts in Ontologies

• Actions and Events– have time dimension– agent (actions) or natural cause or no cause

(events)– modeling of dynamic aspect problematic

• Physical Structures– Time

– Space

• Mathematical Structures– Numbers– Sets, Groups, ...

Basic Concepts in Ontologies

• Physical Properties (visual, haptic,auditory...)– weight– spatial measures– colour– size– volume– shape– texture– temperature– hardiness

Page 8: Exam Preparation Exam Review

8

Basic Concepts in Ontologies

• Concepts representing Property Values– (qualitative) weight values– colours– (qualitative) size values– (qualitative) volume values– shapes– textures– (qualitative) temperatures– (qualitative) hardiness

Basic Concepts in Ontologies

• More Concepts and Properties– emotional states– social attributes– social behaviours (difficult) → deontic logic

Planning

Planning Methods 1

STRIPS

• actions specified by preconditions and effects stated as formulae in (restricted) First-Order Predicate Logic

• planning as search in space of (world) states• plan is sequence of actions from start to goal statePartial Order Planning• planning through plan refinement• parallel expansion to satisfy preconditions• causal links (effect of a used in precondition of a') • threats (effect of a negates precondition of a'; a'<a)

Planning Methods 2

Hierarchical Task Networks / Plan DecompositionPlan schemata are organized in a hierarchy. Links between nodes at different levels in this hierarchy denote a decomposition of a complex plan/action.

Situation Calculus Planning as Theorem Proving. Describe situations (world states) using a situation parameter in action descriptions.

Planning: STRIPS

STRIPS (Nils J. Nilsson)• actions are specified by preconditions and

effects stated as restricted FOPL formulae

• planning is search in space of world states

• means-ends analysis guides planning

Page 9: Exam Preparation Exam Review

9

STRIPS – Example

Example:

move (x, y, z)precondition: on (x, y) ∧ clear (x) ∧ clear (z) delete-list: clear (z), on (x, y)

add-list: on (x, z), clear (y), clear (Table)

Planningfind action-operator which achieves goal-stateinitial world state: on (A, B) ∧ clear (A) ∧ clear (C)goal state: on (A, C)

ADL - Action Definition Language

ADL Can be seen as extension of the STRIPS language.Contains typing of paramaters (sorts).Allows explicit expression of negation.Allows equality of terms in precondition formula.

Example:

Fly (p: plane; from: airport; to: airport; c: cargo)precondition: at(p,to)∧at(c,to)∧in(c,p)∧to≠fromeffect: at(p,to)∧at(c,to)∧¬at(p,from)∧¬at(c,from)

Planning: Partially Ordered Plans

Partially Ordered Plans - or: "How Do You Put Your Shoes On?"

Partially Ordered Plans:• no strict sequence

• partly parallel

• observe threats

FINISH

START

True

Nil

Goal-cond. 1 Goal-cond.

n

Goal-cond. n

Start-cond. 1

Start-cond. n

POP - Initial Plan

Partial Order Planning - Overall

Use plan transformation operators to refine the partial plan and construct a complete plan: • add an action (operator),

• reorder actions (operators), • instantiate actions (operators).

A partial order plan consists of a set of action sequences (partial plans; action strings) which together achieve the complete set of goal literals.

Partial Order Planning 2

Add causal links to connect effects from actions to matching preconditions for plan.

Causal links specify a partial order.Recognize threats - the effect of an action A

negates the precondition of another action B.

Add threats as partial order to plan: B<A (do B before A ).

Page 10: Exam Preparation Exam Review

10

Planning - Situation Calculus

Situation Calculus (John McCarthy)• models actions and events in First-Order

Predicate Logic• situation as additional parameter for some

formulae (fluents) allows to specify change due to events

• action applied in situation: Result-function• effect (changes) and frame (remain) of an

action specified through axioms• planning as theorem-proving

Situations - Blocks World Example

Situation s0

s0 = {on(A,B),on(B,Fl),clear(A),clear(Fl)}

on(A,B,s0),on(B,Fl,s0),clear(A,s0),

clear(Fl,s0)

Action: move (A, B, Fl)

Situation s1

s1 = {on(A,Fl), on(B,Fl), clear(A),clear(B),clear(Fl)}

on(A,F,s1),on(B,Fl,s1),clear(A,s1),clear(B,s1),clear(Fl,s1)

AB

A B

Result Function

Result (or: do) is a function from actions and situations into situations.

Example

s' = Result (move (x, y, z), s)

specifies a new situation s' which is the result of performing a move-action in situation s.

General

s’ = Result (a, s) for action a and situations s, s ’

FluentsFormulas affected by actions have situations as parameters.

Predicates and functions in these formulae which change due to actions are called fluents.

Integrate situation parameter into these fluent formulas.

on(A,B,s), on(B,Fl,s), clear(C,s)

Note: Block(A), Block(B), ... without s

Situation Calculus

Effect axioms describe how an action changes a situation when the action is performed.

Frame axioms describe what remains unchanged between situations.

Successor-state axioms combine effect and frame axioms for an action.

Situation Calculus and Planning

Planning starts with a specified start situationand the specification of a goal situation.

Planning comprises of finding a proof which infers the goal situation from the start situation using successor-state and other axioms.

A Plan is a sequence of actions which specifies a sequence of transformations of situations from the initial situation to the final situation.

Page 11: Exam Preparation Exam Review

11

initial situation s = {on(A,B), on(B,Fl), clear(C)}goal sitation s' = {on(A,C), on(B,Fl), clear (B)}

Example

AB C

AB C

action a = move (A,B,C)on (x, y, s) ∧ clear (z, s) ∧ x ? z ⇒on (x, z, Result (move (x, y, z), s))

Instantiate: x → A, y → B, z → C

Hierarchical Planning

Hierarchical Planning / Plan DecompositionPlans are organized in a hierarchy. Links between nodes at different levels in the hierarchy denote a decomposition of a “complex action” into more primitive actions (operator expansion).Example:

move (x, y, z)

operatorexpansion pickup (x, y) putdown (x, z)

The lowest level corresponds to executable actions of the agent.

Planning - Problems

Frame-Problemspecify everything which remains stablecomputationally very expensiveApproach: successor-state axioms ; STRIPS

Qualification-Problemspecify all preconditions to an actiondifficult to include every possible preconditionApproach: non-monotonic reasoning with defaults

Ramification-Problemconsidering side-effects of actionsconflict between effects and framesApproach: integrate TMS

Resource Constraints in Planning• Resources

– physical quantities, e.g. money, fluids etc.– time

• Integrate Measures into Action Description and Planning– representation of physical quantities and

reasoning / calculation, e.g. for buy-action: effect: cash := cash – price (x)

– time system / time logic, e.g. go-to-action: effect: time := time + 30 (Minutes)

• Backtracking on Constraint Violation

Other Issues in Planning

Disjunctive Preconditions

Conditional Effects– change is due to specific condition– integrate into partial planning with threats

Disjunctive Effects– parallel future worlds to consider

All-Quantified Variables (in preconditions and effects)

– only for finite, static Universe of objects

Natural Language Processing

Page 12: Exam Preparation Exam Review

12

Natural Language Processing

Presentations / NotesAssignment 3Look back at the NLP-OverviewLook at the Flakey-example

Areas in Natural Language Processing

• Morphology (word stem + ending)• Syntax, Grammar & Parsing (syntactic

description & analysis)• Semantics & Pragmatics (meaning;

constructive; context-dependent; references; ambiguity)

• Intentions • Pragmatic Theory of Language (Communication

as Action)• Discourse / Dialogue / Text• Spoken Language Understanding• Language Learning

MorphologicalAnalyzer

Lexicon

Part-of-Speech(POS)

Tagging

GrammarRules

Parser

the the – determiner Det NP ? Det Noun NP recognizedNP

Det Noun

parse treeLinguistic Background Knowledge

NLP Syntax Analysis -Processes Bottom-up and Top-down Parsing

Bottom-up – from word-nodes to sentence-symbol Top-down Parsing – from sentence-symbol to words

S

Aux NP VP

Det Nominal Verb NP

Noun Det Nominal

does this flight include a meal

Problems in Parsing - Ambiguity

Ambiguity“One morning, I shot an elephant in my pajamas. How he got into my pajamas, I don’t know.” Groucho

Marx

syntactical/structural ambiguity – several parse trees are possible e.g. above sentence (attachment or binding problem)

semantic/lexical ambiguity – several word meanings e.g. bank (financial) and (river) bank

different word categories – “Fruit flies from the balcony”vs. “Fruit flies are on the balcony.”

Problems in Parsing - Left Recursion

Problems with left -recursive rules like NP ? NP PP: don’t know how many times recursion is needed

Combine top-down and bottom-up approach:Start with sentence; use rules top-down (look-ahead); read input; try to find shortest path from input to highest unparsed constituent (from left to right).

? Chart -Parsing / Earley-Parser

Page 13: Exam Preparation Exam Review

13

Chart-ParsingEarley-Algorithm

Chart Parsing / Early Algorithm

Earley-Parser based on Chart -ParsingEssence: Integrate top-down and bottom-up parsing.

Keep recognized sub-structures (sub-trees) for shared use during parsing.

Top-down: Start with S-symbol. Generate all applicable rules for S. Go further down with left-most constituent in rules and add rules for these constituents until you encounter a left-most node on the RHS which is a word category (POS).

Bottom-up: Read input word and compare. If word matches, mark as recognized and move parsing on to the next category in the rule(s).

Chart - 3c

VP→ V NP .

V

Book this flight

NP→ Det Nom .

DetNom → Noun .

Noun

S → . VP

Chart 0 with two S-Rules

Book this flight

S → . VP VP→ . V NP

S → . VP NP

Earley Algorithm

Start with initial chart ; generate charts:scannerif word category (POS) is found right of the • , the Scanner reads the next input word and adds a rule for it to the chart (read input)completerif rule is completely recognized (the • is far right), the recognition state of earlier rules in the chart advances: the • is moved over the recognized constituent (bottom-up recognition).predictorgenerates new rules for partly recognized RHS with constituent right of • (top-down generation)

Page 14: Exam Preparation Exam Review

14

From Syntax to Semantics

From Syntax to Semantics

• Grammatical Extensions• Sentence Structures• Noun Phrase - Modifications• Verb Phrase - Subcategorization

• Feature Structures

• λ-expressions

Grammar – Sentence Level Constructs

Sentence Level Constructs– declarative S ? NP VP

“This flight leaves at 9 am.”– imperative S ? VP

“Book this flight for me.”– yes-no-question S ? Aux NP VP“Does this flight leave at 9 am?”– wh-question S ? Wh-NP Aux NP VP“When does this flight leave Winnipeg?”

Grammar – Noun Phrase Modification 1Noun Phrase Modifiershead = the central noun of the NP (+ modifiers)

• modifiers before the head noun (prenominal)• determiner the, a, this, some, ...

• predeterminer all the flights• cardinal numbers, ordinal numbers one flight,

the first flight, ...

• quantifiers much, little• adjectives a first-class flight, a long flight

• adjective phrase the least expensive flight

NP ? (Det) (Card) ( Ord) (Quant) (AP) Nominal

Grammar – Noun Phrase Modification 2Noun Phrase Modifiers (continued)• modifiers after the head noun (post-nominal)

– prepositional phrase PP all flights from Chicago

Nominal ? Nominal PP (PP) (PP)

– non-finite clause, gerundive postmodifers all flightsarriving after 7 pm

Nominal ? GerundVP

GerundVP ? GerundV NP | GerundV PP | ...

– relative clause a flight that serves breakfast

Nominal ? Nominal RelClause

RelClause ? (who | that) VP

Grammar – Verb Subcategorization

Verb Phrase and SubategorizationVP = Verb + other constituents. Different verbs accept or need

different constituents ? Verb Subcategorization ; captured inverb frames.

– sentential complement VP → Verb inf -sentenceI want to fly from Boston to Chicago.

– NP complement VP → Verb NPI want this flight.

– no complement VP → Verb I sleep.

– more forms VP → Verb PP PPI fly from Boston to Chicago.

Grammar – Feature Structures 1Feature Structures

• describe additional syntactic-semantic information, like category, person, number, e.g. goes ≡ <verb, 3 rd, singular>

• specify feature structure constraints (agreements) as part of the grammar rules

• during parsing, check agreements of feature structures (unification)

example

S ? NP VP <NP number> = <VP number>

or S ? NP VP <NP agreement> = <VP agreement>

Page 15: Exam Preparation Exam Review

15

SemanticsDistinguish between • surface structure (syntactic structure) and • deep structure (semantic structure) of sentences.

Remember Steve Helmreich's Class:

Different forms of Semantic Representation• logic based• ontology based / semantic language / interlingua

– Case Frame structures– DL and similar KR languages– linguistics based Ontologies

John-n1syn-struc

root johncat noun-proper

sem-struchuman

name johngender male

tool-n1syn-struc

root toolcat n

sem-structool

Relevant parts of the (appropriate senses of the)lexicon entries for John and tool

Semantics - Lambda Calculus 3

Lambda Expressions can be constructed from central expression, inserting semantic representations for complement phrases

Verb → serves {λxλy ∃e IS-A(e, Serving) ∧ Server(e,y) ∧

Served(e,x)}represents general semantics for the verb 'serveFill in appropriate expressions for x, y, for example

'meat' for y derived from Noun in NP as complement to Verb.

Semantics - Lambda Calculus 2

Transform sentence into lambda-expression:

“AI Caramba is close to ICSI.”

specific: close-to (AI Caramba, ICSI)general: ∃x,y: close-to (x, y) ∧ x=AI Caramba ∧ y=ICSILambda Conversion:

λ-expr: λxλy: close-to (x, y) (AI Caramba) Lambda Reduction:

λy: close-to (AI Caramba, y) close-to (AI Caramba, ICSI)

Speech & Language

Speech & Natural Language Processing

• Natural Language Processing• written text as input• sentences (well-formed)

• Speech Recognition• acoustic signal as input

• conversion into written words

• Spoken Language Understanding• analysis of spoken language (transcribed speech)

Page 16: Exam Preparation Exam Review

16

Speech Processing -Important Types and Characteristics

single word vs. continuous speechunlimited vs. large vs. small vocabularyspeaker-dependent vs. speaker-independenttrainingSpeech Recognition vs.Speaker Identification

Phoneme Recognition:HMM, Neural Networks

Phonemes

Acoustic / sound waveFiltering, Sampling Spectral Analysis; FFT

Frequency Spectrum

Features (Phonemes; Context)

Grammar or Statistics Phoneme Sequences / Words

Grammar or Statistics for likely word sequences

Word Sequence / Sentence

Speech Recognition

Signal Processing / Analysis

Speech Signal Analysis

Analog-Digital Conversion of Acoustic SignalSampling in Time Frames (“ windows”)

Ø frequency = 0-crossings per time frame → e.g. 2 crossings/second is 1 Hz (1 wave)→ e.g. 10kHz needs sampling rate 20kHz

Ø measure amplitudes of signal in time frame → digitized wave form

Ø separate different frequency components→ FFT (Fast Fourier Transform) → spectrogram

Ø other frequency based representations → LPC (linear predictive coding), → Cepstrum

Phoneme Recognition

Recognition Process based on• features extracted from spectral analysis• phonological rules• statistical properties of language/ pronunciation

Recognition Methods • Hidden Markov Models • Neural Networks• Pattern Classification in general

Page 17: Exam Preparation Exam Review

17

Pronunciation Networks / Word Models as Probabilistic FAs (fig 5.12) Speech Recognizer Architecture (fig. 7.2)

Word Recognition with Probabilistic FA / Markov Chain (fig 5.14)

NETtalk - Network Architecture

NETtalk - Some Articulatory Features (Output)

Neural Networks

Page 18: Exam Preparation Exam Review

18

Neural Networks

Neural Networks (NN)

• basic processing units

• general network architectures

• learning

• qualities and problems of NNs

Anatomy of a Neuron

Diagram of an Action Potential

From: Ana Adelstein , Introduction to the Nervous System, Part Ihttp://www.ualberta.ca/~anaa/PSYCHO377/PSYCH377Lectures/L02Psych377/

Neural Networks – Central Concepts

biologically inspired– McCulloch-Pitts Neuron (automata theory), Perceptron

basic architecture– units with activation state, – directed weighted connections between units – "activation spreading", output used as input to connected units

basic processing in unit– integrated input: sum of weighted outputs of connected “pre-

units”– activation of unit = function of integrated input– output depends on input/activation state– activation function or output function often threshold dependent,

also sigmoid (differentiable for backprop!) or linear

Page 19: Exam Preparation Exam Review

19

NN Architectures + FunctionFeedforward, layered networksØ simple pattern classification, function estimating

Recurrent networks

Ø for space/time-variant input (e.g. natural language)Completely connected networks

Ø Boltzman Machine, Hopfield NetworkØ optimization; constraint satisfaction

Self -Organizing NetworksØ SOMs, Kohonen networks, winner-take-all (WTA)

networksØ unsupervised development of classification

Ø best-fitting weight vector slowly adapted to input vector

Neural Networks - Learning

Learning = change connection weightsadjust connection weights in network, changes input -

output behaviour, make it react “properly” to input pattern– supervised = network is told about “correct”

answer = teaching input; e.g. backpropagation, reinforcement learning

– unsupervised = network has to find correct output (usually classification of input patterns) on it’s own; e.g. competitive learning, winner-take-all networks, self-organizing or Kohonenmaps

Backpropagation LearningBackpropagation Learning is supervised

Correct input -output relation is known for some pattern samples; take some of these patterns for training: calculate error between produced output and correct output; propagate error back from output to input units and adjust weights. After training perform tests with known I/O patterns. Then use with unknown input patterns.

Idea behind the Backpropagation Rule (next slides):

Determine error for output units (compare produced output with 'teaching input' = correct or wanted output). Adjust weights based on error, activation state, and current weights. Determine error for internal units based on the derivation of activation function. Adjust weights for internal units using the error function, using an adapted delta-rule.

Competitive Learning 2Units specialize to recognize pattern classes

Unit which responds strongest (among all units) to the current input, moves it's weight vector towards the input vector (use e.g. Euclidean distance):§ reduce weight on inactive lines, raise weight on

active lines§ all other unitskeep or reduce their weights (often a

Gaussian curve used to determine which units change their weights and how)

Winning units(their weight vectors) represent a prototype of the class they recognize.

from Haykin, Simon: Neural Networks, Prentice-Hall, 1999, p. 60

Competitive Learning - Figure NN-Learning as OptimizationLearning: adjust network in order to adapt its input-output

behaviour so that it reacts “properly” to input patternsLearning as optimization process : find parameter setting for

network (in particular weights) which determines network that produces best -fitting behaviour (input-output relation)

→ minimize error in I/O behaviour→ optimize weight setting w.r.t error function→ find minimum in error surface for different weight settingsBackpropagation implements a gradient descent search for

correct weight setting (method not optimal)Statistical models (include a stochastic parameter) allow for

“jumps” out of local minima (cf. Hopfield Neuron with probabilistic activation function, Thermodynamic Modelswith temperature parameter, Simulated Annealing)

Genetic Algorithms can be used to determine parameter setting of Neural Network.

Page 20: Exam Preparation Exam Review

20

Evolutionary Algorithms

• Principles of Evolutionary Algorithms

• Demos (PBS Archives, ‘Life’s really Big Questions, Dec 2000) featuring Karl Sims and Jordan Pollack

Evolution Processes I• Selection determines, which individuals are

chosen for mating (recombination) and how many offspring each selected individual produces.

• In order to determine the new population (generation), each individual of the current generation is objected to an evaluation based on a fitness function.

• This fitness is used for the actual selection step, in which the individuals producing offspring are chosen (mating pool).

Evolution Process II• Recombination produces new individuals in

combining the information contained in the parents, e.g. cross-over.

• Mutations are determined by small perturbations of parameters describing the individuals, which yield new offspring individuals.

• Re-iterate Evolution Process until system satisfies optimization demands.

Evolutionary Algorithm - Structure