ACT-R 6.0(Cognitive Science Prosem)
Wayne D. GrayRensselaer Polytechnic Institute
CogWorks [email protected]
Mike SchoellesRensselaer Polytechnic Institute
CogWorks [email protected]
Tutorial Overview
Cognitive Architecture/Modeling OverviewACT-R Theory - Symbolic levelAddition,counting and letter models
ACT-R Theory Sub-symbolic levelSternberg and Building Sticks models
Production Compilation
What is a Cognitive Architecture?
Infrastructure for an intelligent system
Cognitive functions that are constant over time and across different task domains
Analogous to a building, car, or computer
Unified Theories of Cognition
Account of intelligent behavior at the system-level
Newell’s claim“You can’t play 20 questions with nature and win”
Integrated Cognitive Architecture
Cognition does not function in isolation
Interaction with perception, motor, auditory, etc. systems
Embodied cognitionRepresents a shift from• “mind as an abstract information processing system”
• Perceptual and motor are merely input and output systems
Must consider the role of the environmentOther body processes• Effects of caffeine, stress and other moderators
Motivations for a Cognitive Architecture *
1. Philosophy: Provide a unified understanding of the mind.
2. Psychology: Account for experimental data.
3. Education: Provide cognitive models for intelligent tutoring systems and other learning environments.
4. Human Computer Interaction: Evaluate artifacts and help in their design.
5. Computer Generated Forces: Provide cognitive agents to inhabit training environments and games.
6. Neuroscience: Provide a framework for interpreting data from brain imaging.
7. All of the above
Requirements for Cognitive Architectures*
1. Integration, not just of different aspects of higher level cognition but of cognition, perception, and action.
2. Systems that run in real time.
3. Robust behavior in the face of error, the unexpected, and the unknown.
4. Parameter-free predictions of behavior.
5. Learning.
Newell’s Time Scale of Human Activity (amended)
TaxonomyComputational
Cognitive Models
Connectionist
Cognitive Architectures
Home grown -- one-off codeSymbolic
Other AI
other Production System
ACT-R 6.0
HybridSymbolic only
SOAR EPIC
Model Name Acronym/ Abbreviation
Pew and Mavor (1998)
Ritter, Shadbolt et al. (2001)
Present IDA Study
Atomic Components of Thought ACT Yes No Yes
Adaptive Resonance Theory ART No No Yes
Architecture for Procedure Execution APEX No Yes Yes
Artificial Neural Networks ANNs Yes No Noa
Business Redesign Agent-Based Holistic Modeling System
Brahms No No Yes
Cognition and Affect Project CogAff No Yes Yes
COGnition As a NEtwork Of Tasks COGNET Yes No Yes
Cognitive Complexity Theory CCT No No Yes
Cognitive Objects within a Graphical EnviroNmenT COGENT No Yes Yes
Concurrent Activation-Based Production System CAPS No No Yes
Construction-Integration Theory C-I Theory No No Yes
Distributed Cognition DCOG No No Yes
Elementary Perceiver And Memorizer EPAM No Yes Nob
Executive Process/Interactive Control EPIC Yes No Yes
Human Operator Simulator HOS Yes No Yes
Belief-Desire-Intention architecture BDI No Yes Noc
Man-machine Integrated Design and Analysis System MIDAS Yes No Yes
Micro Systems Analysis of Integrated Network of Tasks
Micro SAINT Yes No Yes
MIDAS Redesign Yes No Yesd
Operator Model ARchitecture OMAR Yes No Yes
PSI PSI No Yes Yes
Situation Awareness Model for Pilot-in-the-Loop Evaluation
SAMPLE Yes No Yes
Sparse Distributed Memory SDM No Yes Nob
State, Operator, And Result Soar Yes No Yes
History of the ACT-framework*
Predecessor HAM (Anderson & Bower 1973)
Theory versions ACT-E (Anderson, 1976)ACT* (Anderson, 1978)ACT-R (Anderson, 1993)ACT-R 4.0 (Anderson &
Lebiere, 1998)ACT-R 5.0 (Anderson &
Lebiere, 2001)
ImplementationsGRAPES (Sauers & Farrell, 1982)PUPS (Anderson & Thompson,
1989)ACT-R 2.0 (Lebiere &
Kushmerick, 1993)ACT-R 3.0 (Lebiere, 1995)ACT-R 4.0 (Lebiere, 1998)ACT-R/PM (Byrne, 1998)ACT-R 5.0 (Lebiere, 2001)Windows Environment (Bothell, 2001)Macintosh Environment (Fincham, 2001)ACT-R 6.0 (Bothell, 2004??)
Other Cognitive Architectures
SoarProduction rule system• Organized in terms of operators associated with problem spaces
• Goal oriented– Sub-goaling
• Learning mechanism - Chunking
EPIC -- Executive Processes in Cognition
Parallel firing of production rules (in principle)Well developed visual and motor systemEmphasis on executive processes
MetaIssues
The divide between high-level architectures and low-level (primarily connectionist ones) is mainly a levels issueModeling in high-level architectures reflects a concern with the task structure of behavior and can be considered a form of task analysis
MetaIssues
The problems tackled by the two sets of approaches tends to reflect this differenceFrom this perspective ACT-R is a major success in “re-use” -- low level parameters are reused in most ACT-R models. The higher level production rules differ in part because they reflect the task analysis for the different tasks being modeled
MetaIssues
From this perspective the focus on reusability should focus onLow-level productions that control the interleaving of cognitive, perceptual, and action at the 1/3 sec level of analysis
Not onHigh-level productions that implement task-specific executive control
MetaIssuesLow-Level Productions Implement Interactive Routines
MetaIssuesInteractive Routines
Neurocognitive evidence increasing stresses the modular nature of human cognitionBut -- these modules are constrained by their need to work together to survive in the worldInteractive routines can be viewed as the basic level elements of embodied cognitionProvide constraints on the input/output functions for perception, attention, memory, and motor systemsNotion is that the use of cognitive, perceptual, and motor resources is optimized via the selection of one set of interactive routines over another
ACT-R Overview
Modules (buffers) Knowledge RepresentationSymbolic/Sub-symbolicPerformance/Learning
I. ACT-R Theory IV. Problem Solving and Decision Making
II. Perception and Attention Tower of Hanoi
Psychophysical Judgement Choice and Strategy Selection
Visual Search Mathematical Problem Solving
Eye Movements Spatial Reasoning and Navigation
MultiTasking Dynamic Systems
Task Switching Use and Design of Artifacts
Subitizing Game Playing
Stroop Insight and Scientific Discovery
Driving and Flying Behavior Programming
Situational Awareness and Embedded Cognition Reasoning
Graphical User Interfaces Errors
Time Perception V. Language Processing
III. Learning and Memory Parsing
List Memory Analogy and Metaphor
Interference Language Learning
Implicit Learning Sentence Memory
Skill Acquisition Lexical Processing
Cognitive Arithmetic VI. Other
Category Learning Cognitive Development
Learning by Exploration and Demonstration Individual Differences
Updating Memory and Prospective Memory Motivation, Emotion, & Cognitive Moderators
Causal Learning Cognitive Workload
Working Memory Computer Generated Forces, Video Games, and Agents
Practice and Retention fMRI
Communication, Negotiation, and Group Decision Making
Instructional Materials
User Modeling
Intelligent tutoring systems
Information Search
Tools
Comparative
ACT-R Applications559 Papers Listed on
http://act-r.psy.cmu.edu/publications/index.php
In the following areas as of 2006-01-24
Interactive Session
Load and run Addition model
Addition model exerciseIn this exercise you will load a simple model and run it to see
how a model runs. You will also get some experience with the interface.
1. Open model
Click on the "Open Model" button on the Environment Control Pane, and select the Addition model. This will open up the model so that you can see it and its parts.
You should be able to see the working memory elements in the model (window "Chunk"), the productions (Production window). There are three further windows, Chunk Type, Command, and Miscellaneous, that we will cover later.
You should briefly examine the chunk and production contents. You may note that there about 11 pieces of working memory, and just 4 rules in this system.
2. Run the model
You can run the model using the Lisp command line, but we will use the environment because it provides a recognition-based interface rather than a recall-based interface.
You should first click on "Reset"; this will reset the model and make it ready to run. You can do this to a model that has run as well, or has been stopped in the middle of a run.
You can run the model by clicking on the "Run" button. A trace of the model will appear in the (Lisp) "Listener" window. You can see how the order that rules are selected and fired, as well as when chunks are retrieved from memory by the rules.
3. Inspect the model
Click on "Declarative viewer" in the Control Pane to bring up an inspector window for the declarative memory elements. If you scroll, you can find the chunks a-j and second-goal. Pay most attention to their structure, and note that they have several parameters. These parameters are used to compute how fast they are used and if they can be retrieved. With learning and use, the activation, for example, goes up. These are covered later in this tutorial.
The Procedural viewer provides a view onto the rules.
End of First Tuesday
HomeworkAnderson, J. R., Bothell, D., Byrne, M. D., Douglas, S., Lebiere, C., & Quin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036-1060.
Change addition model to subtraction model
ACT-R 6.0 Architecture
MotorModules
Current Goal
PerceptualModules
DeclarativeMemory
Pattern MatchingAnd
Production Selection
Check
RetrieveModify
Test
Check State Schedule
Action
IdentifyObject
MoveAttention
ACT-R 6.0
Environment
ACT-R 6.0 Mapping to the Brain*
Environment
Pro
duct
ions
(Bas
al G
angl
ia)
Retrieval Buffer (VLPFC)
Matching (Striatum)
Selection (Pallidum)
Execution (Thalamus)
Goal Buffer (DLPFC)
Visual Buffer (Parietal)
Manual Buffer (Motor)
Manual Module (Motor/Cerebellum)
Visual Module (Occipital/etc)
Intentional Module (not identified)
Declarative Module (Temporal/Hippocampus)
PerformanceDeclarative Procedural
SymbolicRetrieval of
chunksApplication of
production rules
SubsymbolicNoisy activationscontrol speed and
accuracy
Noisy utilitiescontrol choice
LearningDeclarative Procedural
SymbolicEncoding environment
andcaching goals
Productioncompilation
SubsymbolicBayesianlearning
Bayesianlearning
ACT-R: Assumption Space*
Addition-FactThree Seven
Four
addend1 sum
addend2
Declarative-Procedural Distinction
Procedural Knowledge: Production Rules
for retri eving chunks to solve problems.
336
+848
4
IF the goal is to add the numbers in a column
and n1 + n2 are in the column
THEN retrieve the sum of n1 and n2.
Productions serve to coordinate the retrieval of
information from declarative memory and the enviroment
to produce transformations in the goal state.
Declarative Knowledge: Chunks
Configurations of small numbers of elements
ACT-R: Knowledge Representation*
goal buffer visual buffer retrieval buffer
Declarative Memory Syntax
(chunk-type count-order first second)
(chunk-type add arg1 arg2 sum count)
Chunk type : ((chunk-type type-name slot-name1 …(slot-namek init-val)..slot-namen)
(add-dm (a ISA count-order first 0 second 1)
(b ISA count-order first 1 second 2)
(second-goal ISA add arg1 5 arg2 2) ) ;;sum and count are nil
Chunk Instantiation: (add-dm (chunk-name isa type-name slot-name slot-value) …)
Declarative Memory
Chunks that are added explicitlyAdd-dm
Chunks merge into DM from buffersAll buffers’ chunks go to DM when cleared
Mergings are the references for BLLNot the LHS usage as in ACT-R 5
Because buffers hold copies, DM chunks can’t be changed from within a production
Previously it was a recommendation
ADDITION-FACT
AADDENDDDEND11 TTHREEHREE
AADDENDDDEND22 FFOUROUR
SSUM UM
FACT3+4(
SSEVENEVEN )
isa
Addition Fact Example
(Chunk-type addition-fact addend1 addend2 sum)
Addition Example(CLEAR-ALL)(DEFINE-MODEL addition)(CHUNK-TYPE addition-fact addend1 addend2 sum)(CHUNK-TYPE integer value)(ADD-DM (fact3+4
isa addition-fact addend1 three addend2 four sum seven) (three
isa integer value 3) (four
isa integer value 4) (seven
isa integer value 7)
ADDITION-FACT
FACT3+4ADDEND1 SUM
ADDEND2
THREE
FOUR
SEVEN
isa
isa
INTEGER
isa
VALUE VALUE
3 7
isa
Addition Fact Example
VALUE4
A Production is*
1. The greatest idea in cognitive science.
2. The least appreciated construct in cognitive science.
3. A 50 millisecond step of cognition.
4. The source of the serial bottleneck in otherwise parallel system.
5. A condition-action data structure with “variables”.
6. A formal specification of the flow of information from cortex to basal ganglia and back again.
Key Properties • • modularitymodularity• • abstractionabstraction• • goal/buffer factoringgoal/buffer factoring• • conditional asymmetryconditional asymmetry
Productions*
( p
==>
)
Specification ofBuffer Transformations
condition part
delimiter
action part
name
Specification ofBuffer Tests
Structure of productions
Productions LHSOnly four possible conditions available=buffer>
•Test the chunk in the buffer just like in 5
!eval! or !safe-eval!!bind! or !safe-bind!
•Same as in ACT-R 5•Safe- versions accepted by production compilation
?buffer>•Query the buffer or its module•Come back to queries later
Possible RHS actions
=buffer>-buffer>+buffer>!eval! and !safe-eval!!bind! and !safe-bind!!output!!stop!
RHS actions
=buffer>!eval! and !safe-eval!!bind! and !safe-bind!!output!
All the same as in ACT-R 5The safe- versions do not inhibit the production compilation mechanism
!stop!Not actually new, but does work nowGenerates a break event in the schedulerTerminates the current “run” command
RHS +buffer>+buffer> isa chunk-type {{modifier} [slot | request parameter] value}*
or
+buffer> chunk-reference
Sends a request to the moduleAlways clears the buffer implicitlyEssentially the same as ACT-R 5
Buffer queries
Replaces the *-state buffersSyntax
?buffer>{ {-} query value}+
Either true or false No bindings
Must all be true for production to match
Examples
?retrieval> ?visual> state busy - state error buffer empty buffer =check
Queries continued
Every buffer/module must respond toState •Values: busy, free, or error
Buffer •Values: full, empty, requested or unrequested
Others can be added by a module writer•Modality for the current PM modules for example
Production Syntax(P initialize-addition =goal> ISA add arg1 =num1 arg2 =num2 sum nil==> =goal> sum =num1 count 0 +retrieval> isa count-order first =num1 ?retrieval> state free)
(P increment-sum =goal> ISA add sum =sum count =count =retrieval> ISA count-order first =sum second =newsum==> =goal> sum =newsum +retrieval> isa count-order first =count)
(p got-number =goal> isa make-a-call who =person =retrieval>
isa phone-number who =person where =office ph-num =num==> !output! (The phone number for =person is =num) +goal> isa dial-number who =person ph-num =num current-digit 1)
(p dial-a-digit =goal> isa dial-number ph-num =num current-digit =digit state nil==> !bind! =d (get-digit =digit =num) +visual-location> isa visual-location value =d =goal> state dialing)
(P increment =goal> ISA count-from start =num1 - end =num1 step counting =retrieval> ISA count-order first =num1 second =num2==> =goal> start =num2 +retrieval> ISA count-order first =num2 !output! (=num1))
(p read-choose =goal> isa read-letters state verify-choose =visual> isa text value "choose" ==> =goal> state find-letter )
(P rotate-counter-clockwise =goal> ISA translate&rotate step counter-clockwise reference-y =y axis =axis form =form !bind! =y1 (+ =y 15)) =retrieval> ISA point-i > screen-y =y1 ;;no longer can do (!eval! (+ =y 15))==> !eval! (rotate-counter-clockwise =form =axis))
(P get-direction =goal> ISA make-report step find-point =retrieval> ISA point-i screen-x =x screen-y =y==> =goal> step find-direction !bind! =x1 (+ =x 15) +retrieval> ISA direction < screen-x =x1 ;;;no longer (!eval! (+ =x 15)) )
Interactive Session
Load and run Counting model
Count model
This model works much like the previous model, but prints out its count.
1. Open the model
Either quit and restart your Lisp, or else click on "Close Model".
Open the Count model by clicking on "Open Model" and then selecting the Count model.
Run the model to see its trace, and examine its rules and chunks.
2. Using the Stepper
Click on "Stepper", and a stepper window should appear.
Reset the model, and then click on the run button. This starts the stepper. You can now step through the model by clicking on the "Step" button on the Stepper.
As you step through the model, you should be able to see most of the mechanisms in ACT-R now, the productions, how they are matched, the chunks, and how they are retrieved, and the buffers (click on Buffer Viewer to see the buffers and their contents).
3. Checking on a rule that does not fire.
After you have run the model a few steps, click on the Procedural Viewer. Select a rule in the dialogue box, and see why it does not fire.
4. Edit the model
Look at the model and consider how to have it count backwards.
You can change the production rules in the Production window. After you make changes, save the model (it will automatically increment). Close the model and reopen it to try your new model.
The Modules(reprise)
CognitionMemoryVisionMotorAuditionSpeech
ACT-R 6.0 Buffers*
1. Goal Buffer (=goal, +goal)-represents where one is in the task-preserves information across production cycles
2. Retrieval Buffer (=retrieval, +retrieval)-holds information retrieval from declarative memory-seat of activation computations
3. Visual Buffers-location (=visual-location, +visual-location)-visual objects (=visual, +visual)-attention switch corresponds to buffer transformation
4. Auditory Buffers (=aural, +aural)-analogous to visual
5. Manual Buffers (=manual, +manual)-elaborate theory of manual movement include feature preparation, Fitts law, and device properties
6. Vocal Buffers (=vocal, +vocal)-analogous to manual buffers but less well developed
Cognition
Executive Control - Production SystemSerialParallel at sub-symbolic levelUtility selects production to fireUtility = benefit - cost•Benefit = probability of success * value of achieving goal
Production System Cycle
Match conditions of all rules to buffers
Those that match enter the conflict set
Conflict resolution selects a rule to fire
Action side of rule initiates changes to one or more buffers
If no production can match and no action is in progress then quit else repeat
Goal directed
Represents what you are trying to doA declarative memory element that is the focus of “internal” attention(goal-focus second-goal)
Memory Module
Activation basedFrequency and recencyContextual cues
CognitionRequests retrieval• Specifies constraints• Partial matching
Memory Parallel search of memory to match constraintsCalculates activation of matching chunksReturns most active chunk
Vision Module
ACT-R’s “eyes”Dorsal “where” systemVentral “what” system
“Where” SystemCognition
Requests “pre-attentive” visual searchSpecifies a set of constraintsAttribute/value pairs • Properties or spatial location
– e.g. color red, screen-x greater-than 150
“Where” system Returns a “location” chunk• Specifies location of an object whose features satisfy the constraints
OnsetsFeatures are held in vision module’s memory
Visual location
(P find-unattended-letter =goal> ISA read-letters state start ==> +visual-location> ISA visual-location :attended nil =goal> state find-location)
(chunk-type visual-location screen-x screen-y distance kind color value size nearest)
(P attend-letter =goal> ISA read-letters state find-location =visual-location> ISA visual-location ?visual> state free==> +visual> ISA move-attention screen-pos =visual-location =goal> state attend)
“What” System
CognitionRequests “move attention” Provides “location” chunk
“What” System Shifts visual attention to that locationEncodes object at that location• Added to Declarative Memory• Episodic representation of visual scene
Places encoding in “visual” bufferCalculates latency• EMMA
Visual Object
(P encode-letter =goal> ISA read-letters state attend =visual> ISA text value =letter==> =goal> letter =letter state respond)
(P attend-letter =goal> ISA read-letters state find-location =visual-location> ISA visual-location ?visual> state free==> +visual> ISA move-attention screen-pos =visual-location =goal> state attend)
(chunk-type visual-object screen-pos value status color height width)
Visual State
(P attend-letter =goal> ISA read-letters state find-location =visual-location> ISA visual-location ?visual> state free ==> +visual> ISA move-attention screen-pos =visual-location =goal> state attend)
Vision Module Memory
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Motor Module
ACT-R’s HandsBased on EPIC’s Manual Motor ProcessorMovement StylesPhased Processing
Motor Syntax(P respond =goal> ISA read-letters letter =letter state respond ?manual> state free==> +manual> ISA press-key key =letter =goal> state stop)
Movement Styles
Style
Punch (hand finger)
HFRT(hand finger r theta)
PeckPeck-recoilPly (hand r theta)
Hand Cursor
Movement Styles
Ply - moves a device (e.g. move-mouse) to a given locationPunch - pressing a key below finger or click-mousePeck - directed movement of finger to a location followed by keystrokePeck-recoil - same as peck but finger moves back
Phased Processing (1)
Preparation PhaseHierarchical feature preparation•Style->hand->finger
Prep time depends on•Complexity of movement•Number of features
State buffer set to prep busy
Phased Processing (2)
Initiation (fixed 50 ms)Execution• Time depends on
– Type of movement– Minimum execution time
– Distance– Fitt’s Law
Allow overlapping of preparation and execution
Audition Module
Simulated perception of audioMemory of featuresTemporal-extent - sound events
Tones, digits, and speechAttributesOnset, duration, delay, recode time
Audition Module Syntax
(defp alpha-task/listen =goal>
isa alpha-task step listen==> +aural-location> isa audio-event onset highest :attended NIL =goal> step check-feature )
(defp alpha-task/check-feature =goal>
isa alpha-task step check-feature =aural-location> isa audio-event ?aural> state free
==> +aural> isa sound event =aural-location =goal> step attend-sound)
(chunk-type audio-event onset offset pitch kind location) (chunk-type sound kind content event)
Audition Module Processing
Parallels vision systemCognition
Specifies a set of constraintsAttribute/value pairs
AuditionReturns a “location” chunk
CognitionRequests shift of auditory attention providing the “location”chunk
AuditionEncodes the sound
Device InterfaceSimulated device with which ACT-R interacts
Contains graphical objects
Typically a WindowCan be entire screen
InteractionConstructing vision system’s iconic memory (sets of features) from graphical objectsHandle mouse and keyboard actions
(defclassic button-
feature (icon-
feature) feature-slot)
(defclassic my-button-dialog-item (button-dialog-item) dialog-item-slot)
(defmethod build-features-for (self my-button-dialog-item )
(defclassic button-feature (icon-feature) feature-slot)
Visual Icon (visicon)
button-feature
+visual-location
(defmethod featinfo->loc (icon-feature)
(CHUNK-TYPE visual-location slota, slotb, slotc)
Declarative Memory
visual-location DMEs
Locs
+visual
(CHUNK-TYPE visual-object slota, slotb, slotc)
Declarative Memory
visual-objectt DMES
(defmethod feat-to-dmo ( button-feature )
linked
Visual Icon (vi...
THESE are Lisp elements
-- but are into vision -- pre
move location
(defmethod featinfo->loc (icon-featu...
Where button-feature is subclass of icon-feature
This builds a visual-location DME using the features -- puts
them into the slots of the chunk-type.
(defclassic my-button-dialog-item (button-dialog-item) dialog-item-slot)
simply a new screen object that inherits from predefined screen object (button-dialog-item)
I. (defmethod build-features-for (self my-button-dialog-item)
Makes an instance of "button-feature"
Is called by pm-proc-display
A. Visual Icon (visicon)
button-feature
THESE are Lisp elements -- but are into vision -- pre move location
1. +visual-location
a. (defmethod featinfo->loc (icon-feature)
Where button-feature is subclass of icon-feature
This builds a visual-location DME using the features -- puts them into the slots of the
chunk-type.
(1) Declarative Memory
visual-location DMEs
Locs
These are part of ACTR --- so these "locs" are DMEs
(a) +visual
i) (defmethod feat-to-dmo (button-feature)
(1) Declarative Memory
visual-objectt DMES
2. linked
(defclassic button-feature (icon-feature) feature-slot)
This inherits slots from icon-feature. Can also have new slots as indicated by "feature-slot" in the above.
(CHUNK-TYPE visual-location slota, slotb, slotc)
(CHUNK-TYPE visual-object slota, slotb, slotc)
Handy commands(dm)(sdm slot value)(sdp chunk-name)(get-chunk chunk-name) ;;returns chunk structure(get-chunk-type 'name) ;;gets type structure from type name(get-module module-name)(my-name (get-module :vision))(print-module-state (get-module :vision))(current-mp)(current-device)(current-device-interface)(buffers)(buffer-chunk buffer-name)(buffer-read 'buffer-name)(chunk-slot-value chunk-name slot-name)(pprint-a-chunk chunk-name)(sdp-fct (no-output (dm)))(sdp-fct (no-output (sdm isa ...)))(gethash :vision (act-r-modules-table *modules-lookup*))(act-r-chunk-type-name (act-r-chunk-chunk-type (get-chunk (buffer-read 'goal))))
Interactive Session
Load and run Letter model
Sub-symbolic level
Sub-symbolic learning allow the system to adapt to the statistical structure of the environment
Production Utilities are responsible for determining which productions get selected when there is a conflict.
Chunk Activations are responsible for determining which (if any chunks) get retrieved and how long it takes to retrieve them.
Chunk Activations have been simplified in ACT-R 5.0 and a major step has been taken towards the goal of parameter-free predictions by fixing a number of the parameters.
Parameters
NoiseUtility and activation
LearningActivation - frequency and recencyUtility - probability and cost
ThresholdsUtility and activation
Sub-symbolic ACT-R theory
Activation equationProduction Utility equationProduction Compilation
Chunk i
Seven
Three FourAddend1 Addend2
Sum
=Goal> isa write
relation sum arg1 Three arg2 Four
+Conditions
+Retrieval> isa addition-fact
addend1 Three addend2 Four
+Actions
Sji
Simkl
Bi
Activation*A i =Bi + Wj ⋅ Sji
j∑ + MPk ⋅Simkl +N(0,s)
k∑
Chunk Activation
Activation = Base Level + Associative + Partial Matching + Noise
Reflects the degree that a chunk will be useful in the current context based upon past experiences:
Base level - general past usefulness
Associative - relevance to current context
Partial Matching - relevance to the specific match
Noise - stochastic, non-deterministic behavior
A i =Bi + Wj ⋅ Sjij∑ + MPk ⋅Simkl +N(0,s)
k∑
Base-level Activation
The base level activation Bi of chunk Ci reflects a context-independent estimation of how likely Ci is to match a production, i.e. Bi is an estimate of the log odds that Ci will be used.
Two factors determine Bi:
• frequency of using Ci
• recency with which Ci was used
Bi is set globally using the “set-all-base-levels” functionBi is set for individual chunks using (set-base-level (get-wme 'set-dow) '(100 -100)) ;;if learning on(set-base-level (get-wme 'set-dow) 50.0) ;if learning off
Ai = Bi
baseactivati
on
activation
=
Base-level Learning
Base-Level Activation reflects the log-odds that a chunk will be needed.
The odds that a fact will be needed decays as a power function of how long it has been since it has been used.
The effects of multiple uses sum in determining the odds of being used.
Base-level Learning Equations
Base Level Learning (BLL) equation (approximation) is
Bi = ln(∑nj=1t-dj) (computationall y expensive learning version tha t is
used if :ol parameter is se t t o n)il
n = number o f presentations for chunkit j= t hetime since t hejth presentationd = decay rate, controlled by base level learni ng paramete (r :bll)
defaul t= 0.5 (s gp:bll 0.5)
Bi = log(∑t-dj) ≈ lo (g n / (1 – d)) – d*lo (g L) (optimize d learning versi ontha t is used if :ol parameter is se t t o )t
L = The lifetime of chunk I (t hetime since its creati )on
Chunk “Presentation”
CreationInitialization (add-dm)Encode from environment (+visual)New goal (+goal> isa …)
MergingWhen goal is cleared (-goal or +goal)Merged if matches a current chunk
Harvest of a retrieval=retrieval Could get multiple presentations from one +retrieval
Source Activation*
The source activations Wj reflect the amount of attention given to elements of the current goal. ACT-R assumes a fixed capacity for source activation
W= Wj reflects an individual difference parameter.
associativestrength
sourceactivation( )+ *
+ Wj * Sjij
Associative Strengths*
The association strength Sji between chunks Cj and Ci is a measure of how often Ci was needed (retrieved) when Cj was element of the goal, i.e. Sji estimates the log likelihood ratio of Cj being a source of activation if Ci was retrieved.
associativestrength
sourceactivation( )+ *
+ Wj * Sji
Sji = S - ln(FANj)FANj = # of chunks in which chunk j is the
value of a slot + 1 for chunk j being associated with itself
S = maximum associative strength- a constant (:mas)
S11
W1
W2
S12
S13
S21
S22
S23
Slot 1
Slot 2
Source1
Source 2
chunk1
chunk2
chunk3
Goal
B1
B2
B3
A1
= B1
+ W1
S11
+ W2
S21
A2
= B2
+ W1
S12
+ W2
S22
A3
= B3
+ W1
S13
+ W2
S23
Partial Matching*
• The mismatch penalty is a measure of the amount of control over memory retrieval: MP = 0 is free association; MP very large means perfect matching; intermediate values allow some mismatching in search of a memory match.
• Similarity values between desired value k specified by the production and actual value l present in the retrieved chunk. This provides generalization properties similar to those in neural networks; the similarity value is essentially equivalent to the dot-product between distributed representations.
+ MPk ⋅Simklk∑
similarityvalue
mismatchpenalty( )*+
Noise
•Noise provides the essential stochasticity of human behavior
•Noise also provides a powerful way of exploring the world
•Activation noise is composed of two noises:
•A permanent noise accounting for encoding variability
•A transient noise for moment-to-moment variation
σ
2
=
π
2
3
s
2
• Generated according to a logistic distribution characterized by parameter s. The mean the distribution is 0 and variance σ2.
Latency
• Retrieval time for a chunk is a negative exponential function of its activation:
A = activation of chunk being retrievedF = latency scale factor (set globally with the :lf parameter)
• If no chunk matches or no chunk is above retrieval threshold
Timei =F ⋅e−Ai
€
Time i = F ⋅e−t
t = retrieval threshold
Probability of Retrieval
• Probability of retrieval of a chunk follows the Boltzmann (softmax) distribution:
€
Pi =eA it
eA jt
j
∑t= 2⋅s=
6⋅σπ
•The chunk with the highest activation is retrieved provided that it reaches the retrieval threshold
•For purposes of latency and probability, the threshold can be considered as a virtual chunk
Interactive Session
Load and run Sternberg model
Making Choices: Conflict Resolution
Expected Gain = E = PG-C + noise
Probability of choosing i = e
E
i
/ t
e
E
j
/ t
j
∑
P =Successes
Successe s + Failures
P is expected probability of successG is value of goalC is expected costNoise = σ2 2 3 s2
where s is set globally by the :egs parameter
t reflects noise in evaluation and is like temperature in the Bolztman equation
Production Utility*
P & C
P =
C =
SuccessesSuccesses + Failures
EffortsSuccesses + Failures
Interactive Session
Load and run Building Sticks model
Production Compilation: The Basic Idea* (p read-stimulus =goal> isa goal step attending state test =visual> isa text value =val==> +retrieval> isa goal relation associate arg1 =val arg2 =ans =goal> relation associate arg1 =val step testing)
(p recall =goal> isa goal relation associate arg1 =val step testing =retrieval> isa goal
relation associate arg1 =val arg2 =ans==> +manual> isa press-key key =ans =goal> step waiting)
(p recall-vanilla =goal> isa goal step attending state test =visual> isa text value "vanilla==> +manual> isa press-key key "7" =goal> relation associate arg1 "vanilla" step waiting)
Production Compilation: The Principles*
1. Perceptual-Motor Buffers: Avoid compositions that will result in jamming when one tries to build two operations on the same buffer into the same production.
2. Retrieval Buffer: Except for failure tests proceduralize out and build more specific productions.
3. Goal Buffers: Complex Rules describing merging.
4. Safe Productions: Production will not produce any result that the original productions did not produce.
5. Parameter Setting: Successes = P*initial-experience*Failures = (1-P) *initial-experience*Efforts = (Successes + Efforts)(C + *cost-penalty*)
References
Introduction to ACT 5.0 Tutorial by Christian Lebiere,http://act-r.psy.cmu.edu/tutorials/
ACTR 5.0 Equations, Variables and Parameters by Jerry Ball
Base Level Activation - Bi
The base level activation Bi of a chunk Ci reflects a context-independent estimation of how likely Ci is to match a production, i.e. Bi is an estimate of the log odds that Ci will be used.
(set-base-level (get-wme 'set-dow) '(100 -100)) ;;if learning on
(set-base-level (get-wme 'set-dow) 50.0) ;if learning off
Associative LearningNot ACT-R 6.0
Sji ≈ ln[ (P i|)/j P( )]i(P i|j) = probability i is needed given j( )P i = base probability ofi
(per Christia n LeBiere if P(i|) j < (P i) then inhibition results)this equation applies in a learning context
Sji = ln(Rji)(aka Posterior Strength Equation 4.3, p. 130 – used in
learning and turne d on bysetting the :al parameter tot)
Rji = assoc*R* ji + F(Cj) Eji / asso c+ F(Cj)assoc* = estimate of we ighting of prior strengthR* ji (fan) = (1/n)/(1/m) = /m n (default prior strengths
tha t reflec tguesses as towha t t he strengths should be)F = frequenc ycount used toestimate Pe(Ni|Cj) a nd Pe(Ni)Cj = the eve ntt hat j is in a goal slot
Eji = Pe(Ni|Cj)/Pe(Ni) = probability ratioP e= Probability estimate
Ni = the eve ntt hat i is neededassoc = weighting of prior strength