a software agent controlling 2 robot arms in co-operating concurrent tasks
TRANSCRIPT
Background slides for invited challenge demo at Rule-ML 2017
A software agent controlling 2 robot arms in co-operating concurrent tasks
Robotic agent programming in QuLog and TeleoR
Keith Clark Imperial College & University of Queensland
Joint work with Peter Robinson
University of Queensland
1
QuLog - LP+FP+AR Language • Flexibly typed, multi-threaded, higher order • Relation and function defining rules
• Function calls and set expression arguments in relation calls • Function rule guards can query relations • Relation rules more declarative than Prolog, closer to predicate
logic • Relations are typed and moded
• Dynamic relations defined only by facts • Used for the agent’s Belief Store
• Top layer of action rules • Update dynamic relations, fork threads, do I/O, communicate with
other Qulog, C, Python or Java processes • Inter-process comms uses our Pedro pub/sub and addressed
message router • Threads execute actions calling funs and rels as needed • Rel and fun rules cannot call actions
2
TeleoR • Development of Nilsson’s T-R robotic agent language of guard ~> action rules sequenced in parameterized procedures. • guard is a QuLog deductive query to the BS • action is a tuple of robotic actions executed in parallel or
a T-R procedure call, maybe a recursive call • TeleoR has forms of rules and actions not in T-R
– Extra rules for more fine grained behaviour control – Can combine robotic action with QuLog action sequence: action ++ qact1 ;...; qact1 – This allows communication and updates of BS as parallel non-robotic actions
• Compile time guarantee: – all rule actions are correctly typed and fully instantiated when a rule is fired
• The concept of task atomic procedures: – Used for multi-tasking sharing one or more robotic resources – No interference, no starvation, deadlock free – Compiler generates code that uses BS for co-ordination 3
Deductive Belief Store Dynamic facts
Relation and function rules
? Percepts Handler
Message Handler ag@host
Percept fact Interpretations of Sensor data
Mid-level Action control messages
Incoming Messages and replies
Control Thread
?Atomic re-evaluation of rule guards after each atomic update
Atomic Updates
Multi-threaded Agent Architecture
Only outgoing Messages
4 pedro Other Agents
Pub/Sub and Addressed Message Router
TeleoR + QuLog
QuLog
One arm block tower building
5
Task Ontology
6
def block::= 1..9
durative pickup(block), put_on_block(block), put_on_table()
percept on(block,block), on_table(block), holding(block)
rel sub_tower(list(block)), tower(list(block))
sub_tower([B]) <= on_table(B) sub_tower([B1,B2,..Blocks]) <= on(B1,B2) & sub_tower([B2,..Blocks])
tower([B,..Blocks]) <= not on(_,B) & sub_tower([B,..Blocks])
Nilsson’s universal conditional top-level plan for block tower building
7
tel makeTower(list(block))
makeTower(Blocks){
tower(Blocks) ~> () % Goal is achieved
sub_tower(Blocks) & Blocks=[B,..] ~> make_clear(B)
Blocks=[B] ~> move_to_table(B)
Blocks=[B1,B2,..Bs] & tower([B2,..Bs]) ~> move_to_block(B1,B2)
Blocks=[B,..Bs] ~> makeTower(Bs) }
Action achieves
Mutually recursive procedures
8
tel make_clear(block) make_clear(Block){
not on(_,Block) ~> () % Goal is achieved
on(OnBlock,Block) ~> move_to_table(OnBlock) } tel move_to_table(block) move_to_table(OnBlock){
on_table(OnBlock) ~> () % Goal is achieved
holding(OnBlock) ~> put_on_table()
not on(_, OnBlock) & not holding(_) ~> pickup(OnBlock)
not holding(_) ~> make_clear(OnBlock)
holding(_) ~> put_on_table() }
Interleaving the building of several towers
9
Just need to add:
task_start makeTower
task_atomic move_to_table, move_to_block
Baxter concurrent tower building
10
TR procedures
Sensor data
Control actions for different robotic resources
BeliefStore
Dynamic Facts
Percepts Handler
TR Evaluation Threads All
incoming messages
Message Handler Task1
Using R1,R2
Task2 Using
R3
Outgoing messages
Multi-tasking architecture
Task3 Waiting for
R1,R3
Task4 Waiting for
R1,R5 Fixed Facts & Rules
?
Elements of the two arm tower builder program
12
def table ::= table1 | shared | table2 def block ::= ... def arm ::= arm1 | arm2 def resource::= arm || table % resource a reserved type name
durative pickup(arm, block, table), put_on_block(arm, block, table), put_on_table(arm, table)
tel makeTower(arm, list(block), table) makeTower(Arm,Blks,Tbl){ % Tbl is where tower must be built, tower(Blks, Tbl) ~> {} sub_tower(Blks, Tbl) & Blocks=[B,..] ~>
makeClear(Arm, B, Tbl) Blks=[B,.. Bs] & tower(Bs, Tbl) ~>
moveAcrossToBlock(Arm, B, firstOf(Bs), Tbl) Blks=[B] ~> moveAcrossToTable(Arm, B, Tbl) Blks=[B,.. Bs] ~> makeTower(Arm, Bs, Tbl) }
Entire control program 40 TeleoR rules clustered into 6 procedures.
Auxiliary procedures
13
tel moveAcrossToTable(arm, block, table) % Will fetch a block from wherever it is and put it onto the table % May need two calls to proc below using different arms to first % move the block to shared then to destination table
task_atomic oneArmMoveToTable(arm, block, table, table) % Has 3 resource args. an arm and two tables which might be the same oneArmMoveToTable(Arm, Blk, FromTab, ToTab){
on(Blk, ToTab) ~> ()
clear(Blk) ~> clearOneArmMoveToTable(Arm, Blk, FromTab, ToTab)
not holding(Arm,_) ~> makeClear(Arm, Blk, FromTab)
holding(Arm,_) ~> put_on_table(Arm, FromTab) }
TeleoR semantics and implementation
14
• Formal State Transition Semantics • Optimized Reference Implementation
• Runtime system that implements state transition semantics • Compile time analysis ensures rule guards are not re-evaluated on BS update if no relevant change made
• Currently compiled to multi-threaded Qu-Prolog • Will soon be compiled to specialized Abstract Machine Code similar to but simpler than Warren’s Prolog AMC
Sources and software
15
Clark & Robinson, Robotic Agent Programming in TeleoR, ICRA 2015, IEEE
Clark & Robinson, Multi-tasking Robotic Agent Programming in TeleoR, Research Report, on www.doc.ic.ac.uk/~klc
Clark et al, A Framework for Integrating Symbolic and Sub-symbolic Representations, IJCAI 2016, AAAI Press
Programming Communicating Multi-tasking Robotic Agents: A Teleo-Reactive Rule Based Approach, Springer, Early 2018 first 5 chapters at teleoreactiveprograms.net including a formal operational semantics of Nilsson’s TR lang.
Clark & Robinson, Engineering Agent Applications in QuLog, Springer, 2017/8,
TeleoR and QuLog Software for Unix, Linux and OS X at http://staff.itee.uq.edu.au/pjr/HomePages/QulogHome.html
Collaboration and users welcomed