achieving next-level robotic intelligence with...
TRANSCRIPT
![Page 1: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/1.jpg)
Achieving Next-Level Robotic Intelligence with SoS Implications
14th International Conference onSystem of Systems Engineering
Sheraton Anchorage20 May 2019
Associate Director, Robotics
Dr. Edward Tunstel, [email protected]
President
![Page 2: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/2.jpg)
2 E. Tunstel
Robots are here
![Page 3: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/3.jpg)
3 E. Tunstel
High-Tech
![Page 4: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/4.jpg)
4 E. Tunstel
Transdisciplinary
![Page 5: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/5.jpg)
5 E. Tunstel
Robotic intelligence pipeline
§ Advances to date are impacting individual pipeline elements§ How to effectively integrate the pipes to realize practical
intelligent robots and robotic systems is unclear.
§ I do not have all of the answers…§ Will pull the thread on the issue as food for thought:
Ø Some select robotics research topics and applications Ø Some next-level intelligence considerations for each
SENSE
ACTPLAN
INTEGRATED INTELLIGENCE
PER
CEI
VE
PLAN
ACT
CO
NTR
OL
LEARN INTERACT
REASON
![Page 6: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/6.jpg)
6 E. Tunstel
Early Automata at APL (1960s)
HopkinsBeast
APL News, March 1964
Ferdinand
E. Tunstel
![Page 7: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/7.jpg)
7 E. Tunstel
Fast forward…2018Got robot? …and Internet?
§ RoboEarth, WWW for robots: giant repository where robots can share information; learn from each other about their behavior & environment –similarly, Robo Brain
§ Cloud Robotics – cloud computing centered around shared robot computing resources (central to robotics within Industry 4.0)
§ Robot-App Store – an apps marketplace intent on enabling apps sharing among robots
http://www.robotappstore.com/
![Page 8: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/8.jpg)
8 E. Tunstel
Robots and IoT
§ Robots as systems connected to IoT in and around intelligent homes and buildings (including hotels, universities and the workplace) – delivery and other services
StarshipTechnologies (UK)
Amazon Scout
PepsiCo & Robby Technologies
Alibaba
Singapore
![Page 9: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/9.jpg)
9 E. Tunstel
Robots and IoT
§ Robots as systems connected to IoT in manufacturing and logistics environments, future (smart) factories, precision agriculture, etc
§ Robots as the truly physical or apparently tangible nodes in Cyber-Physical Systems
IoT-enabled smart robot: KUKA Connect
![Page 10: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/10.jpg)
10 E. Tunstel
Robotics Research Context
Modeling &Simulation
Hardware+ Software
Theoreticalfoundation
Practicalexperience
Mobility AutonomousNavigation Manipulation Sensing &
Perception
Learning, Adaptation, Deliberative planning, Robust field operation, Systems-thinking,
Transdisciplinary ideas…
...
Breakthroughs
I N N
O V
A T
I O
N
RoboticsResearch
InnovationSpace
![Page 11: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/11.jpg)
11 E. Tunstel
Command Sequencing
Engineering Assessment
Uplink
CommandSequences
Downlink
Telemetry
Science Team
scienceactivities
autonomousexecution
• Best health knowledge
•Recommendations• engineering & image data• science data
Spirit / Opportunity
Semi-autonomous operations from Earth
Intelligence and Autonomy• Mission intelligence (science/exploration) is largely human while remote autonomy
is necessarily robotic• Sequencing and analysis teams plan and assess robotic activities using their
perception of the rover surroundings and knowledge of rover state and behavior
![Page 12: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/12.jpg)
12 E. Tunstel
Mars Rover Technology & Robotic Intelligence
§ Autonomous navigation§ Local/global waypoint planning§ Dense stereo vision§ Autonomous terrain assessment§ Visual odometry§ Goal-driven visual servoing§ Robotic arm motion planning§ Precision arm placement§ Terrain sampling and handling§ Autonomous fault response§ Cmd sequencing & visualization
M. Maimone, J. Morrison, JPL
![Page 13: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/13.jpg)
13 E. Tunstel
Next up…https://mars.nasa.gov/mars2020/
§ Faster navigation§ More modern path planning
(RRT, particle filters)
![Page 14: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/14.jpg)
14 E. Tunstel
Next-Level rover intelligence: Mars Sample Return
§ Richer set of capabilitiesØ Sample acquisition & handlingØ Sample fetch & retrievalØ Lander detection & rendezvous
§ Mobility & manipulation capability maturation for sample caching andfetch rovers (prototypes demonstrated in field tests > a decade ago *)
§ Mars 2020 rover is representative of the sample-caching rover in Mars sample return mission concepts
§ Extreme terrain access (cliffs, caves, subsurface lava tubes, etc)
JPL
Schenker, Huntsberger, Pirjanian, Baumgartner and Tunstel, "Planetary Rover Developments Supporting Mars Exploration, Sample Return and Future Human-Robotic Colonization," Autonomous Robots, Vol. 14, 2003.
*
![Page 15: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/15.jpg)
15 E. Tunstel
Intelligent Co-Robots
Beyond mobile sensing and toward environment manipulation and intuitive & physical HRI
Background imgae: http://gp-email.brtapp.com
![Page 16: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/16.jpg)
16 E. Tunstel
Intelligent Co-Robots – Comms. network topology§ UGVs, UAVs, and a ground
link to the user are connected using a wireless mesh Mobile Ad hoc NETwork (MANET)
§ Mini-MACSS uses a COTS comms module that cannot directly link to the Wave Relay mesho One UAV assigned to act as
bridge to mesh (Mini-MACSS is outside of MANET)
![Page 17: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/17.jpg)
17 E. Tunstel
DARPA Robotics Challenge Tech Exposition 2015
MULTI-ROBOT SEARCH & SAMPLINGIN INCREASINGLY CONSTRAINED ENVIRONMENTS
“Russian Doll” scenario
UGV à UAV à micro-UGV
§ A unique demonstration scenario that focused our development of underlying capabilities in key IRAD areasØ Autonomous UAV and UGV mobility/navigationØ Intelligent co-robots and human-robot teamingØ Dexterous manipulationØ Robot vision and perceptionØ Data fusion, distribution, and display
RoboSally
Pelican
Mini-MACCS
Moore, J., Wolfe, K.C., Johannes, M.S., Katyal, K.D., Para, M.P., Murphy, R.J., Hatch, J., Taylor, C.J., Bamberger Jr., R.J. and Tunstel, E., "Nested Marsupial Robotic System for Search and Sampling in Increasingly Constrained Environments," 2016 IEEE Intl. Conf. on Systems, Man, and Cybernetics, Budapest, Hungary, pp. 2279-2286. Oct. 2016.
![Page 18: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/18.jpg)
18 E. Tunstel
Mini-MACSSMiniature Autonomous Crawling Surveillance System
![Page 19: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/19.jpg)
19 E. Tunstel
Human Interface
Operator Console
Keyboard and Mouse
Multi-monitor Display
State Info. & Situational Awareness: maps, state estimates, video, point clouds
User Intent: motion primitives, desired poses
User Confirmation and/or Refinement of Plan
Planned Behavior
Remote Robot Team
UGV: Robo-Sally
UAV: Modified AscTec Pelican
Miniature Ground Vehicle Joystick
![Page 20: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/20.jpg)
20 E. Tunstel
DRC Tech. Expo. demo scenario
![Page 21: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/21.jpg)
21 E. Tunstel
Demo video
https://www.youtube.com/watch?v=Hvh20ySwgPw
![Page 22: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/22.jpg)
22 E. Tunstel
Next-Level manipulation intelligence§ Enhancing perception capabilities beyond
the visual modality
§ Moving beyond object recognition and grasping to knowledge and reasoning about object properties
§ Dense tactile arrays / e-skin, and sensor processing (e.g., neuromorphic)
CEA LIST, FranceKing’s College, London
![Page 23: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/23.jpg)
23 E. Tunstel
Beyond ISR toward environment manipulationand intuitive & physical HRI
Autonomy for Marsupial Robot Team
![Page 24: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/24.jpg)
24 E. Tunstel
Autonomous Team
Planning
Collision-free motion planning
Marsupial-team motion planning
Whole-body planning
Collaborative mapping
Autonomous Team Planning
Vision and Object Recognition
https://www.youtube.com/watch?v=7Gz7yjYqEEM
P.G. Stankiewicz, S. Jenkins, G.E. Mullins, K.C. Wolfe, M.S. Johannes and J.L. Moore, "A Motion Planning Approach for Marsupial Robotic Systems," 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sept. 2018.
![Page 25: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/25.jpg)
25 E. Tunstel
Next-Level multi-robot system intelligence
§ Realizing smart behavior for not only singular robots but for multi-robot systems
§ Tactical behaviors with greater, situational intelligence
§ Coupling w/high-level reasoning systems/ architectures
§ Move beyond swarm intelligence to enable multi-functional swarms that do more than distributed sensing – e.g., manipulation of their environment
![Page 26: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/26.jpg)
26 E. Tunstel
User Interaction & Interfaces
Next Generation First ResponderDHS
![Page 27: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/27.jpg)
27 E. Tunstel
Head-Up, Hands-Off HRI
Current robots require oversight from a trained operator, resulting in reduced
alertness and higher cognitive workload for one or more human teammates. This results in a less effective team.
A conversational human-robot interface using standard, unambiguous military language would allow more intuitive robot C2 and would reduce
the human cognitive burden. This results in more efficient and safer operations.
“Head-Down, Hands-On” “Head-Up, Hands-Off”
![Page 28: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/28.jpg)
28 E. Tunstel
Motivating Use Cases
![Page 29: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/29.jpg)
29 E. Tunstel
Multi-modal and Robot agnostic
{Voice Cmds} {Gestures} {BMI} {AR}
{Behaviors}
BRUCE
Set of common high-level
commands
Other Robots
Multi-modal Head-Up, Hands-Off HRI
BMI ARVoice Text
Gesture
![Page 30: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/30.jpg)
30 E. Tunstel
EWT-30
DARPA RP-2009 (to present):Modular Prosthetic Limb§ New approach to limb systems closely
mimicking human natural form and movements (developed for amputees)
§ Control options range from non-invasive or minimally-invasive techniques to use of sets of wireless implants (targeted muscle reinnervationto thought control)
§ Modular design (hand only, hand+ forearm, whole arm); all power, actuation, and control self contained
§ APL served as lead & systems integrator for a large multi-institutional team
![Page 31: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/31.jpg)
31 E. Tunstel
EWT-31
Thought control…
§ “Breakthrough: Robotic limbs moved by the mind”
§ Scott Pelley reports on movement of robotic limbs using human thought; reception of sensory feedback from a robotic hand
§ Subject, Jan Scheuermann, suffers from a genetic disease that severs the brain-body connection (spinocerebellar degeneration)
http://www.jhuapl.edu/newscenter/stories/
CBS’60 MinutesDec. 2012
![Page 32: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/32.jpg)
32 E. Tunstel
EWT-32
Targeted muscle reinnervation…
§ Breakthrough: Robotic prosthetic attachment to implant in bone
§ Subject, John Matheny, lost arm to cancer in 2008§ Implant inserted in marrow space of bone replaces constricting and
potentially uncomfortable harnesses§ Prosthetic guided by brain signals via nerve reassignment surgery
https://hub.jhu.edu/2016/01/12/prosthetic-limb-more-mobility-apl/
![Page 33: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/33.jpg)
33 E. Tunstel
Robotic Exoskeletons
§ Human Augmentation (able bodied users)Ø Provides user augmentation or assistance to
accomplish generalized tasking
§ Technical ChallengesØ Other than engineering (power, actuation,
controls, ergonomics, physical constraints)
Ø Control software that can assist operators in highly dynamic behaviors.
Ø Exoskeleton control to seamlessly sense and interpret the operator’s intended behavior and then introduce assistive power that coordinates with the operator's motions.
![Page 34: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/34.jpg)
34 E. Tunstel
Applications
§ Health CareØ Functional RestorationØ AssistanceØ Rehabilitation
§ IndustrialØ LogisticsØ Load CarriageØ Heavy Machine Operations
§ Military & Law EnforcementØ Operator Assist Carrying Heavy LoadØ Improve Operator Performance Under
Stressed ConditionsØ Tactical ProtectionØ Heavy Load Carrying
![Page 35: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/35.jpg)
35 E. Tunstel
Next-Level interaction / interface intelligence
§Smart human-collaborative robots that are responsive to intuitive, physical, and brain-interfaced interaction
§Human-robot fusion – Neural interfaces, BMI, etc
§ Tighter feedback loops with brain, EMG signals, etc enabling the robotics to make smart decisions and take appropriate shared control actions leveraging the human as a sensor & supervisory controller
![Page 36: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/36.jpg)
36 E. Tunstel
Modular Open Systems Architecture:Example: AEODRS Standards-based Common Open Architecture
Autonomous BehaviorsVisual SensorsManipulator End Effector Master Comm Link Power
System
InfrastructureUGV
DismountedUGV
TacticalUGV
AEODRS standardized wireless link
Handheld OCU Common OCU
AEODRSElectrical Interface
AEODRSPhysical Interface
AEODRSLogical Interface
AEODRS standardized
interfaces
System Capabilities
Interchangeable and Interoperable
![Page 37: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/37.jpg)
37 E. Tunstel
AEODRS – A Modular Open System
![Page 38: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/38.jpg)
38 E. Tunstel
AEODRS – A Modular Open System
![Page 39: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/39.jpg)
39
AEODRS Capability Modules and Distributed Architecture
Implication for next-level robotic intelligence…but intelligence is non-trivial to modularize in a similar manner
Softweare/Hardware System Test Bed
![Page 40: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/40.jpg)
40 E. Tunstel
Next-Level systems intelligence
§ MOSA provides interoperability, but that is not enough; need “conversability” or dialog as well, enabling next-level robotic intelligence to interoperate with humans & IoTØperhaps borrowing techniques from MAS wherein
agents communicate, not solely to pass data and messages, but particularly in order to achieve their goals
§ Greater leverage and improved adaptation of cognitive architectures (ACT-R, Soar, DUAL, CLARION, etc)
§ Better ways to modularize intelligence
![Page 41: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/41.jpg)
41 E. Tunstel
Robot Learning – Next-Level
§ Need to advance from learning for X (perception, control, etc) Ø autonomous/developmental learningØ knowledge/skill transfer
Next-level intelligence for robot learning moves toward:
§ Autonomous learning (e.g. book by P Angelov)
§ Empirical machine learning (new book. P. Angelov)
§ Broad learning (P. Chen)
§ Robot memetics
PER
CEI
VE
PLAN
ACT
CO
NTR
OL
LEARN INTERACT
REASON
![Page 42: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/42.jpg)
42 E. Tunstel
Robot Memetics
Forthcoming book (colleagues and I):
§ Concept of memes and memetics as elements of collective robotic intelligence and implications for emergence of a hybrid community of humans and intelligent robots
§ Illustration of robot memetics ideas in the context of a space exploration scenario (development and operation of a human-robot settlement on Mars)
§ New ways of thinking about how to realize higher levels of intelligence and learning in robots and robotic communities
![Page 43: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/43.jpg)
43 E. Tunstel
![Page 44: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/44.jpg)
44 E. Tunstel
National GeographicIllustration by Jason Treat, NGM Staff, and Dylan ColeSources: Robert Braun, Georgia Institute of Technology; NASA/JPL
![Page 45: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/45.jpg)
45 E. Tunstel
NASA
![Page 46: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/46.jpg)
46 E. Tunstel
Achieving Next-Level robotic intelligence
Sharp focus on
§ Increased robustness
§Cognitive facilities
§More sophisticated behavior
§System-/colony-level situational awareness
![Page 47: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/47.jpg)
47 E. Tunstel
Need for Basic Robotic-SE
§ Systems engineering (SE) for research products should serve to bridge the research-applications gap
§ Like systems engineering, robotics is inter-disciplinary; thus can benefit from structured approach SE offers
§ A more structured development stage, during or following fundamental research, will allow us to field robots and expect them to be reliable for extended periods
SE – Building robotic systems right
![Page 48: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/48.jpg)
48 E. Tunstel
Some Robotic-SE Considerations
Robotics practice needs more attention to:§Requirements definition/analysis
Ø Customer needs and objectivesØ Requirements V&V (Does design meet requirements? Can it perform
the required mission?)
§Capability characterizationØ What are the capabilities?Ø How reliable/risky are they, and under what conditions?Ø Can the customer trust the system? (Big issue for autonomy!)Ø What specific investment(s) will make it better?
§SpecificationsØ What is the customer really getting? What’s on the spec sheet?Ø Is it supported by statistically significant test results?
![Page 49: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/49.jpg)
49 E. Tunstel
More Robotic-SE Considerations
Robotic-systems development also needs:§Performance evaluation/benchmarking
Ø Metrics/measures of performance over life-cycleØ How does the system measure up to the state of the art?Ø Methods to evaluate performance to facilitate system comparisons
§Test planning/logisticsØ System tests in relevant operating environmentsØ Why is the chosen test environment relevant? To what degree?Ø Support equipment? Costs? Environment regulations?
§People-oriented collaborationØ Complex/large systems call for more personnel from different groups
or organizations (“people skills” – communication, social, etc)Ø Beyond the documents, people facilitate subsystem interface and
configuration control, their negotiation, and cross-system transparency
![Page 50: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/50.jpg)
50 E. Tunstel
SoSE Implications / Challenges
§ Robots (like people) will fail...how to engineer the capacity to self-recover, recover with human assistance without re-programmingØ autonomicity, Ø conversational co-troubleshooting (explainable AI) Ø real-time re-directioning / re-purposing
§ How to evolve from moving data across interfaces to moving information, intelligence, and contextual dialog
§ How to construct design and concept exploration environments that enable non-experts to configure, use and re-use robotic systems as needed
SoSE – Right systems & Interactions
![Page 51: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/51.jpg)
51 E. Tunstel
Robotics & Systems Foci in the IEEE SMC SocietySELECTED ROBOTICS & SYSTEMS TECHNICAL COMMITTEES
§ Autonomous Bionic Robotic Aircraft
§ Bio-mechatronics and Bio-robotics Systems
§ Brain-Machine Interface Systems
§ Computational Cybernetics
§ Cyber-Medical Systems
§ Intelligent Learning in Control Systems
§ Model-Based Systems Engineering
§ Robotics and Intelligent Sensing
§ Shared Control
§ Systems of Systems
§ Unmanned Maritime Systems Engineering
http://ieeesmc.org/
IEEE SMCis where it all
comes together!
Join Us!
![Page 52: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/52.jpg)
52 E. Tunstel
Acknowledgments
Research and real-world applications discussed are based
on past work as PI or contributor, or owing to influence on
or exposure to work by fellow group members at NASA JPL
(Advanced Robotic Controls Group; Robotic Intelligence
Group, Mars rover project teams) and Johns Hopkins APL
(Intelligent Systems Center).
![Page 53: Achieving Next-Level Robotic Intelligence with SoSImplicationssosengineering.org/2019/wp-content/uploads/2019/07/Keynote-Edward-Tunstel.pdf5 E. Tunstel Robotic intelligence pipeline](https://reader034.vdocuments.mx/reader034/viewer/2022042911/5f426a71156264115d6c33df/html5/thumbnails/53.jpg)
53
5353
Thank You!
Sunset as imaged by the Spirit rover from a hilltop on the surface of Mars
Q U E S T I O N S ?