ccc/robotics ‐ roadmap f. service robotics › reports › ccc-serv-rob-v16.pdf · service...
TRANSCRIPT
1
CCC/Robotics‐Roadmapf.ServiceRobotics1.Introduction........................................................................................................................ 2
2.StrategicFindings................................................................................................................ 32.1PrincipalMarketsandDrivers..................................................................................................... 42.2NearTermOpportunitiesandFactorsEffectingCommercialization ............................................ 52.3ScientificandTechnicalChallenges ............................................................................................. 6
Mobility ................................................................................................................................................6Manipulation ........................................................................................................................................7Planning ................................................................................................................................................8SensingandPerception ........................................................................................................................9Architectures,Cognition,andProgrammingParadigms.......................................................................9HumanRobotInteraction(HRI) ..........................................................................................................10ResearchInfrastructure ......................................................................................................................10MechanicalHardware.........................................................................................................................11
3.KeyChallenges/Capabilities ............................................................................................ 123.1MotivatingScenarios................................................................................................................ 123.2CapabilitiesRoadmap............................................................................................................... 15
Human‐likeDexterousManipulation .................................................................................................16Real‐World3DPlanningandNavigation ............................................................................................17Cognition ............................................................................................................................................18RobustPerception ..............................................................................................................................19Physical,intuitiveHRIandinterfaces .................................................................................................19SkillAcquisition...................................................................................................................................20Saferobots..........................................................................................................................................20
4.BasicResearchandTechnologies ...................................................................................... 21ArchitectureandRepresentations.................................................................................................. 21ControlandPlanning...................................................................................................................... 21Perception ..................................................................................................................................... 22Robust,High‐fidelitySensors.......................................................................................................... 22NovelMechanismsandHigh‐PerformanceActuators ..................................................................... 22LearningandAdaptation................................................................................................................ 22PhysicalHuman‐RobotInteraction ................................................................................................. 23SociallyInteractiveRobots ............................................................................................................. 23
5.Contributors ..................................................................................................................... 23
2
1.IntroductionServiceRoboticsisdefinedasthoseroboticssystemsthatassistpeopleintheirdailylivesatwork,intheirhouses,forleisure,andaspartofassistancetohandicappedandelderly.Inindustrialroboticsthetaskistypicallytoautomatetaskstoachieveahomogenousqualityofproductionorahighspeedofexecution.In
contrast,serviceroboticstasksareperformedinspacesoccupiedbyhumansandtypicallyindirectcollaborationwithpeople.Serviceroboticsisnormallydividedintoprofessionalandpersonalservices.
Professionalserviceroboticsincludesagriculture,emergencyresponse,pipelinesandthenationalinfrastructure,forestry,transportation,professionalcleaning,andvariousotherdisciplines.
[Professionalservicerobotsarealsousedformilitarypurposesbuttheirapplicationinthisareaisnotincludedinthisreport.]Thesesystemstypicallyaugmentpeopleforexecutionoftasksintheworkplace.AccordingtotheIFR/VDMAWorldRoboticsmorethan38,000professionalrobotsareinusetodayand
themarketisgrowingrapidlyeveryyear.Severaltypicalprofessionalrobotsareshowninfigure1.
Figure1:Typicalservicerobotsforprofessionalapplications.
Personalservicerobotsontheotherhandaredeployedforassistancetopeopleintheirdailylivesin
theirhomesorasassistantstothemforcompensationformentalandphysicallimitations.Thebyfarlargestgroupofpersonalservicerobotsconsistsofdomesticvacuumcleaners;over3millioniRobotRoomba’salonehavebeensoldworldwideandthemarketisgrowing60%+/year.Inaddition,alarge
numberofrobotshavebeendeployedforleisureapplicationssuchasartificialpets(AIBO),dolls,etc.Withmorethan2millionunitssoldoverthelast5years,themarketforsuchleisurerobotsisexperiencingexponentialgrowthandisexpectedtoremainoneofthemostpromisinginrobotics.A
numberoftypicalpersonalservicerobotsystemsareshowninfigure2.
Figure2:Typicalservicerobotsforpersonalapplications.
TALON®Hazmatrobot
EnvirobotTMPaintStrippingRobotPackBotTMTacticalRobot ResponderTMPipelineRobot
RoombaTMVacuumCleaningRobot ATRSTMRoboticWheelchairSystemStrippingRobot
VerroTMPoolCleaningRobot LEGO®MindstormsTMEducationalRobot
3
Theservicerobotspanelincludedbothprofessionalandpersonalservicesandassuchcoveredahighlydiversesetofapplicationsandproblems.
2.StrategicFindingsAftermuchdiscussion,therewasgeneralagreementamongthosepresentatthemeetingthatwearestill
10to15yearsawayfromawidevarietyofapplicationsandsolutionsincorporatingfull‐scale,generalautonomousfunctionality.Someofthekeytechnologyissuesthatneedtobeaddressedtoreachthatpointarediscussedinalatersectionofthisreport.Therewasfurtheragreementamongthosepresent,
however,thatthetechnologyhassufficientlyprogressedtoenableanincreasingnumberoflimitedscaleand/orsemi‐autonomoussolutionsthatarepragmatic,affordable,andproviderealvalue.Commercialproductsandapplicationsbasedonexistingtechnologyhavealreadybeguntoemergeandmoreare
expectedasentrepreneursandinvestorsrealizetheirpotential.Theparticipantsidentifiedseveralmarketswheretheseearlycommercialsolutionsareappearingandwhereserviceroboticsislikelytohavethegreatestimpact.Amongtheareasidentifiedarehealthcare,nationalinfrastructureandresource
management,energyandtheenvironment,security,transportationandlogistics,andeducationandentertainment.
Figure3.ThechangesindemographicsinUSAandJapanrespectively.
4
Oneofthekeyfactorscontributingtotheidentifiedtrendsisouragingpopulation.Thisimpactsserviceroboticsbothintermsoftheneedtoaddressashrinkingworkforceaswellastheopportunitytodevelop
solutionsthatwillmeettheirhealthcareneeds.Asshowninfigure3,theUnitedStatesisonthethresholdofa20yeartrendthatwillseeaneardoublingofthenumberofretireworkersasapercentageofthecurrentworkforce;fromjustover2retireesforevery10workerstodaytojustover4retireesforevery10
workersin2030.InJapanthesituationisevenworseandhasfueledamajornationalinitiativetodeveloptheroboticstechnologyneededtohelpcarefortheirrapidlyagingpopulation.Generallyspeaking,professionalserviceroboticsisexpectedtoserveasaworkforcemultiplierforincreasedeconomicgrowth,
whiledomesticserviceroboticsisexpectedtoenablesustainedpersonalautonomy.
Whileincreasingproductivityandreducingcostsarethecommondenominatorofservicerobotics,eachsystemisexpectedtouniquelyprovideacompellingsolutiontocertain,criticalmarketspecificissuesorneeds.Forexample,akey,primarydriverinusingroboticstechnologytoautomatetheautomobile
factorieswasthedesiretoobtainconsistent,day‐to‐dayqualityandavoidthe“builtonMonday”syndrome.
2.1PrincipalMarketsandDriversHealthcare&QualityofLife–thecurrentapplicationofroboticstechnologytoprovidetele‐operatedsolutionssuchasIntuitiveSurgical’sdaVincisurgicalsystemrepresentsthetipofthe
iceberg.Roboticstechnologyholdsenormouspotentialtohelpcontrolcosts,empowerhealthcareworkers,andenableagingcitizenstolivelongerintheirhomes.
Energy&Environment–theattendeesidentifiedthesetwocloselylinkedissuesasbothcriticalto
thefutureofourcountryandripefortheemergenceofroboticstechnologyapplications,especiallyintheareasofautomatingtheacquisitionofenergyandmonitoringtheenvironment.
Manufacturing&Logistics–beyondthetraditionalapplicationofroboticstechnologytoautomatecertainassemblylinefunctions,themeetingparticipantsagreedthatthereistremendouspotential
tofurtherautomatethemanufactureandmovementofgoods;asfullyexploredintheparallelroadmappingeffortinthisarea.Inparticular,roboticstechnologypromisestotransformsmallscale,or“micro”,manufacturingoperationsandintheprocesshelpacceleratethetransitionof
manufacturingbacktoAmerica.Thisbeliefhassincebeensubstantiatedbytheformationofanewstart‐uproboticscompany,HeartlandRobotics,organizedspecificallyforthatpurpose.
Automotive&Transportation–althoughwearestilldecadesawayfromthefullyautonomousautomobile,roboticstechnologyisalreadyappearingintheformofadvanceddriverassistanceand
collisionavoidancesystems.Publictransportationisanotherareathatisexpectedtobecomeincreasinglyautomated.Asroboticstechnologycontinuestoimproveandmature,unmannedtransportationsystemsandsolutionsdevelopedforlimitedscaleenvironmentssuchasairportswill
beadaptedforimplementationinurbancentersandothergeneralpurposeenvironments.
HomelandSecurity&InfrastructureProtection–participantsinthemeetingagreedthatroboticstechnologyofferstremendouspotentialforapplicationsinborderprotection,searchandrescue,portinspectionandsecurity,andotherrelatedareas.Inaddition,roboticstechnologyisexpectedto
5
beincreasinglyusedtoautomatetheinspection,maintenance,andsafeguardingofournation’sbridges,highways,waterandsewersystems,energypipelinesandfacilities,andothercritical
componentsofournation’sinfrastructure.
Entertainment&Education–thisarea,perhapsmorethananyotherhasseentheearlyemergenceofroboticstechnologyenabledproducts.Inparticular,roboticshasthepotentialtosignificantlyaddressthescience,technology,engineering,andmath(“STEM”)crisisfacingthenationandto
becometheveritable“fourthr”ofeducation.ThisisevidencedbythetremendoussuccessofFIRST,anon‐profitorganizationfoundedin1999thatrunsnationalroboticscompetitionstoinspireyoungpeopletobescienceandtechnologyleaders,andotherroboticsinspirededucationalinitiatives.
Roboticsprovideskidswithacompellingandtactileavenuetolearnandapplyboththeunderlyingkeymathematicsandsciencefundamentalsandtheengineeringandsystemintegrationprinciplesrequiredtoproduceintelligentmachinestoaccomplishcertainmissions.
2.2NearTermOpportunitiesandFactorsEffectingCommercializationSignificantinvestmentisrequiredforexpandedresearchanddevelopmentofroboticstechnologyifthefull
promiseofwhatcanbeachievedineachoftheaboveareasistoberealized.Asnotedabove,wearestillalongwayfromthefullyautonomousroboticstechnologyrequiredtoautomateprocessestotheextentthatnohumanattentionorinterventionisrequired.Thatsaid,itwasthecollectiveopinionofthosein
attendancethatenoughprogressinroboticstechnologyhasbeenmadetoenablethedevelopmentandmarketingofawidevarietyofinitialapplicationsandproductsineachoftheseareastoachievesignificant
levelsof“humanaugmentation”.
Suchsolutionswillbecapabletovaryingdegreesofautomaticallyperformingthefollowingtypesoffunctions:monitoringdefined,yetdynamicphysicalenvironments,identifyingobjects,detectingchanges,orotherwiseperceivingthestatusoftheirassignedenvironments,analyzingandrecommendingactionsthat
shouldbetakeninresponsetodetectedconditions,takingsuchactionsinresponsetohumancommands,and/orautomaticallyperformingsuchactionswithincertainpre‐authorizedboundariesnotover‐riddenbyhumanoperators.
Examplesofsuchroboticssolutionstodayincludetele‐operatedsystemssuchasthedaVincisurgicalsystem
andautonomous,specializedproductivitytoolssuchastheRoomba.AstheInternetcontinuestoevolve,itwillinspireanaturalprogressionfromsensingatadistancetotakingactionatadistance.ThisextensionoftheInternetintothephysicalworldwillservetofurtherblurtheboundariesamongcommunity,
communication,computing,andservicesandinspirenewdimensionsintelecommutingandtelepresenceapplications.Hybridsolutionsarelikelytoemergethatenabledistributedhumancognitionandenabletheefficientuseofhumanintelligence.Suchsolutionswillcombinetherobotics‐enabledcapabilitytoremotely
andautonomouslyperceivesituationsrequiringinterventionwiththeInternet‐enabledcapabilityforhumanoperatorstotakeactionfromadistanceonanas‐neededonlybasis.
6
Asreferencedabove,ouragingpopulationwillresultinafuturelaborshortage.Asworkersseektomoveupthejobhierarchy,therewillbeagrowingneedtoaugmentandincreasinglyautomatejobsatthebottom
becausetheworkerstoperformthemmaynotbereadilyavailableandeventuallymaynotexist.Whilethechallengeofachievingfullyautonomoussolutionsinthelongrunremainsprimarilytechnological,thechallengeintheneartermisoneofinvestinginthescienceofdevelopingrequirementsandotherwise
determininghowtobest“crossthechasm”;itisoneofidentifyingtherightvaluepropositions,drivingdowncosts,developingefficient,effectivesystemsengineeringprocesses,determininghowtobestintegratesuchsolutionsintocurrentoradaptedprocesses,andotherwiseaddressingtheknow‐howgapof
transitioningtechnologyintoproducts.
2.3ScientificandTechnicalChallengesWorkshopparticipantsworkedinthreebreak‐outgroupstoidentifytechnicalandscientificchallengespertinenttotheapplicationsandbusinessdriversdescribedintheprevioussection.Thefirstbreak‐outgroupfocusedonapplicationandsystemsdesign;thesecondgroupdiscussedaction,cognition,
planning,andotherelementsofroboticintelligence;andthefinalgroupidentifiedchallengesinhumanrobotinteraction.Thissectionsummarizestheirfindings.Becausethechallengesidentifiedbythethreegroupsspantheboundariesbetweentherespectivetopicareas,wewillpresentthetechnicaland
scientificchallengesidentifiedbythebreak‐outgroupsinanintegratedmanner.Theemphasisofthissectionisondescribingthechallenges,notonlayingoutaroadmaptowardsaddressingthesechallenges—sucharoadmapwillbeoutlinedinthenextsection.
MobilityMobilityhasbeenoneofthesuccessstoriesofroboticsresearch.Thissuccessisexemplifiedbya
numberofsystemswithdemonstratedperformanceinrealworldenvironments,includingmuseumtourguidesandautonomouslydrivingcars,asintheDARPAGrandChallengeandUrbanChallenge.Nevertheless,workshopparticipantsagreedthatanumberofimportantopenproblemsremain.Finding
solutionstotheseproblemsintheareaofmobilitywillbenecessarytoachievethelevelofautonomyandversatilityrequiredfortheidentifiedapplicationareas.
Participantsidentified3Dnavigationasoneofthemostimportantchallengesintheareaofmobility.Currently,mostmapping,localization,andnavigationsystemsrelyontwo‐dimensionalrepresentations
oftheworld,suchasstreetmapsorfloorplans.Asroboticapplicationsincreaseincomplexityandaredeployedineveryday,populatedenvironmentsthataremoreunstructuredandlesscontrolled,however,these2Drepresentationswillnotbesufficienttocaptureallaspectsoftheworldnecessaryfor
commontasks.Ifwillthereforebeimportanttoenabletheacquisitionofthree‐dimensionalworldmodelsinsupportofnavigationandmanipulation(seenextsection).These3Drepresentationsshouldnotonlycontainthegeometrylayoutoftheworld;instead,mapsmustcontaintask‐relevantsemantic
informationaboutobjectsandfeaturesoftheenvironment.Currentrobotsaregoodatunderstandingwherethingsareintheworld,buttheyhavelittleornounderstandingofwhatthingsare.Whenmobilityisperformedinservicetomanipulation,environmentalrepresentationsshouldalsoinclude
objectaffordances,i.e.knowledgeofwhattherobotcanuseanobjectforAchievingsemantic3Dnavigationwillrequirenovelmethodsforsensing,perception,mapping,localization,objectrecognition,
7
affordancerecognition,andplanning.Someoftheserequirementsarediscussedinmoredetaillaterinthissection.
Oneofthepromisingtechnologiestowardssemantic3Dmapping,asidentifiedbytheparticipants,is
usingdifferentkindsofsensorsforbuildingmaps.Currently,robotsrelyonveryhighprecisionlaser‐basedmeasurementsystemsforlearningabouttheirenvironment,usingmappingalgorithmsknownas"SLAM"algorithms.Theparticipantsidentifiedadesiretomoveawayfromlaserstocameras,todevelop
anewfieldof"visualSLAM"(VSLAM).Thistechnologyreliesoncameras,whicharerobust,cheap,andreadilyavailablesensors,tomapandlocalizeinathree‐dimensionalworld.Alreadytoday,VSLAMsystemsexhibitimpressivereal‐timeperformance.ParticipantsthereforebelievedthatVSLAMwill
likelyplayaroleinthedevelopmentofadequateandmoreaffordable3Dnavigationcapabilities.
Participantsidentifiedadditionalrequirementsfor3Dnavigationthatwillbecriticaltomeettherequirementsoftargetedapplications.Outdoor3Dnavigationposesanumberofimportantchallengesthathavetobeaddressedexplicitly.Amongthemisthefactthatcurrent2Denvironmental
representationscannotcapturethecomplexityofoutdoorenvironmentsnorthechanginglightingconditionsthatcausesubstantialvariabilityintheperformanceofsensormodalities.Participantsalsoidentifiedrobustnavigationincrowdsasanimportantmobilitychallenge.
ManipulationSubstantialprogressinmanipulationisneededforalmostalloftheserviceroboticsapplications
identifiedintheprevioussection.Theseapplicationsrequirearobottointeractphysicallywithitsenvironmentbyopeningdoors,pickingupobjects,operatingmachinesanddevices,etc.Currently,autonomousmanipulationsystemsfunctionwellincarefullyengineeredandhighlycontrolled
environments,suchasfactoryfloorsandassemblycells,butcannothandletheenvironmentalvariabilityanduncertaintyassociatedwithopen,dynamic,andunstructuredenvironments.Asaresult,participantsfromallthreebreak‐outgroupsidentifiedautonomousmanipulationasacriticalareaof
scientificinvestigation.Whilenospecificdirectionsforprogresswereidentified,thediscussionsrevealedthatthebasicassumptionsofmostexistingmanipulationalgorithmswouldnotbesatisfiedintheapplicationareastargetedbythiseffort.Graspingandmanipulationsuitableforapplicationsin
open,dynamic,andunstructuredenvironmentsshouldleveragepriorknowledgeandmodelsoftheenvironmentwheneverpossible,butshouldnotfailcatastrophicallywhensuchpriorknowledgeisnotavailable.Asacorollary,trulyautonomousmanipulationwilldependontherobot’sabilitytoacquire
adequate,task‐relevantenvironmentalmodelswhentheyarenotavailable.Thisimpliesthat—incontrasttomostexistingmethodswhichemphasizeplanningandcontrol—perceptionbecomesanimportantcomponentoftheresearchagendatowardsautonomousmanipulation.
Participantsidentifiednovelrobotichands(discussedinthesubsectiononHardware),tactilesensing(seeSensingandPerception),andhighly‐accurate,physicallyrealisticsimulatorsasimportantenablersforautonomousmanipulation.Participantsuggestedthatcompetent“pickandplace”operationsmayprovideasufficientfunctional
basisforthemanipulationrequirementsofamanyofthetargetedapplications.Itwastherefore
8
suggestedthatpickandplaceoperationsofincreasingcomplexityandgeneralitycouldprovidearoadmapandbenchmarkforresearcheffortsinautonomousmanipulation.
PlanningResearchintheareaofmotionplanninghasmadenotableprogressoverthelastdecade.Theresultingalgorithmsandtechniqueshaveimpactedmanydifferentapplicationareas.Nevertheless,participants
agreedthatrobustdynamic3Dpathplanningremainsanopenproblem.Animportantaspectofthisproblemisthenotionofarobot’ssituationalawareness,i.e.therobot’sabilitytoautonomously
combine,interleave,andintegratetheplanningofactionswithappropriatesensingandmodelingoftheenvironment.Theterm“appropriate”alludestothefactthatcompleteandexactmodelsoftheenvironmentcannotbeacquiredbytherobotinrealtime.Instead,itwillbenecessarytoreasonabout
theobjectives,theenvironment,andtheavailablesensingandmotoractionsavailabletotherobot.Asaresult,theboundarybetweenplanningandmotionplanningisblurred.Toplanamotion,theplannerhastocoordinatesensingandmotionundertheconstraintsimposedbythetask.Toachievetask
objectivesrobustlyandreliably,planninghastoconsiderenvironmentalaffordances.Thismeansthattheplannerhastoconsiderinteractionswiththeenvironmentandobjectsinitaspartoftheplanningprocess.Forexample:topickupanobject,itmaybecomenecessarytoopenadoortomoveintoa
differentroom,topushawayachairtobeabletoreachtoacabinet,toopenthecabinetdoor,andtopushanobstructingobjectoutoftheway.Inthisnewparadigmofplanning,thetaskandconstraintsimposedbythetaskandtheenvironmentarethefocus;the“motion”of“motionplanning”isameans
toanend.Constraintsconsideredduringplanningcanarisefromobjectmanipulation,locomotion(e.g.footstepplanning),kinematicanddynamicconstraintsofthemechanism,postureconstraints,orobstacleavoidance.Planningundertheseconstraintsmustoccurinrealtime.
Someoftheconstraintsontherobot’smotionaremosteasilyenforcedbyleveragingsensorfeedback.
Obviousexamplesarecontactconstraintsandobstacleavoidance.Theareaoffeedbackplanningandtheintegrationofcontrolandplanningarethereforeimportantareasofresearchtowardssatisfyingtheplanningrequirementsidentifiedbytheparticipants.Afeedbackplannergeneratesapolicythat
directlymapsstatestoactions,ratherthangeneratingaspecificpathortrajectory.Thisensuresthatsensor,actuation,andmodelinguncertaintiescanadequatelybeaddressedusingsensoryfeedback.
Theincreasedcomplexityofplanninginthiscontextwillalsorequirenovelwaysofcapturingtaskdescriptions.Whileinclassicalmotionplanningthespecificationoftwoconfigurationsfullyspecifieda
planningtask,theviewofplanningdescribedherehastohandlemuchrichertaskrepresentationstoaddresstherichnessofmanipulationtasksandintermediateinteractionswiththeenvironment.
Participantsalsoperceivedtheneedforformalmethodstoperformverificationandvalidationoftheresultsofplanners.Suchguaranteesmayberequiredtoensuresafeoperationofrobotsin
environmentspopulatedwithhumans.
9
SensingandPerceptionSensingandperceptionareofcentralimportancetoallaspectsofrobotics,includingmobility,manipulation,andhuman‐robotinteraction.Participantswereconvincedthatinnovationinsensingandperceptionwillhaveprofoundimpactontherateofprogressinrobotics.
Participantsbelievedthatnewsensingmodalitiesaswellasmoreadvanced,higher‐resolution,lower‐
costversionsofexistingmodalitieswouldbeareasofimportantprogress.Forexample,participantsexpectimportantadvancesinmanipulationandmobilityalikefromdense3Drangesensing,possiblyby
LIDAR.Advancesindexterousmanipulationarelikelytorequireskin‐liketactilesensorsforrobotichands.Butalsospecializedsensors,forexampleforsafety,termedsafetysensors,werediscussedbytheparticipants.Thesesensorscouldtakevariousforms,suchasrangeorheatsensingtodetectthe
presenceofhumans,orcouldbeimplementedbyspecialtorquesensorsaspartoftheactuationmechanism,capableofdetectingunexpectedcontactbetweentherobotanditsenvironment.Skin‐likesensorsfortheentireroboticmechanismwouldalsofallintothiscategory.
Thedatadeliveredbysensormodalitiesmustbeprocessedandanalyzedbyalgorithmsforperceptionin
complexandhighlydynamicenvironmentsundervaryingconditions,includingdifferencesbetweendayandnightandobscurantslikefog,haze,brightsunlight,andthelike.Participantsidentifiedtheneedforprogressinhigh‐levelobjectmodeling,detection,andrecognition,inimprovedsceneunderstanding,
andintheimprovedabilitytodetectactivitiesandintent.Novelalgorithmsforaffordancerecognitionarerequiredtosupportthetypeofplanningdescribedintheprevioussubsection.Participantsalsodiscussedtheneedforaccuratesensormodelsinsupportofperceptualalgorithms.
Architectures,Cognition,andProgrammingParadigmsThediscussionsonthetopicsofmobility,manipulation,planning,andperceptionrevealedthattheseissuescannotbeviewedinisolationbutareintricatelylinkedtoeachother.Thequestionofhowto
engineerasystemtoeffectivelyintegratespecificskillsfromthoseareastoachievesafe,robust,task‐directed,orevenintelligentbehaviorremainsanopenquestionoffundamentalimportanceinrobotics.Researchtowardsthisobjectivehasbeenconductedunderthenameofarchitectures,cognition,and
programmingparadigms.Thisdiversityinapproachesorevenphilosophicalviewpointsmayreflectthelackofunderstandinginthecommunityonhowtoadequatelytacklethischallenge.Thisdiversityofviewpointsisalsoreflectedinthediversityoftoolscurrentlybroughttobearonthisissue:theyrange
fromimitationlearningtoexplicitprogrammingofso‐calledcognitivearchitectures.Someparticipantsfeltthatamixtureofthesewouldprobablyberequiredtoachievethedesiredoutcome.
Oneoftheclassicalapproachestowardstheoverarchingissueofgeneratingrobust,autonomousbehavioristhesense/plan/actloopusuallyemployedbymoderncontrolsystems.While
sense/plan/acthasbeenaconstantinroboticsresearchoverthelastseveraldecades,someparticipantsfeltthatnovelapproacheswouldlikelydeviatefromthisapproachinitssimplestform.Possible
alternativesaremultiplenestedorhierarchicalloops,thebehavior‐basedapproach,combinationsofthetwo,orpossiblyevencompletelynovelapproaches.
10
Allparticipantsagreedthatthisareaofinvestigationwillrequiresubstantialattentionandprogressonthepathtowardsautonomousroboticsystems.
HumanRobotInteraction(HRI)Giventheultimategoalofdeployingmobileanddexterousrobotsinhumanenvironmentstoenablecoexistenceandcooperation,substantialprogresswillberequiredintheareaofhumanrobotinteraction.Theseinteractionscouldalsobecomeanimportantcomponentinanoverarchingapproach
torobustrobotbehavior,asdiscussedintheprevioussubsection.Robotmightlearnnovelskillsfromtheirinteractionswithhumansbutunderallcircumstancesshouldbecognizantofthecharacteristicsandrequirementsoftheircommunicationwithhumans.
Inadditiontothemodesofcommunication(verbal,nonverbal,gesture,facialexpression,etc.),
participantsidentifiedanumberofimportantresearchtopics,includingsocialrelationships,emotions(recognition,presentation,socialemotionalcognition/modeling),engagement,andtrust.Anunderstandingoftheseaspectsofhumanrobotcommunicationshouldleadtoanautomaticstructuring
oftheinteractionsbetweenhumansandrobotswhereroboticsystems’abilitytooperateindependentlyrisesorfallsautomaticallyasboththetaskandthehumansupervisor'sinteractionwiththesystemchange.
Progresstowardstheseobjectiveswilldependoneffectiveinputdevicesandintuitiveuserinterfaces.
ParticipantsalsoadvocatedthedevelopmentofavarietyofplatformstostudyHRI,includinghumanoidrobots,mobilemanipulationplatforms,wheelchairs,exoskeletons,andvehicles.Participantsidentifiedadesign/build/deploycycleinwhichHRIresearchshouldprogress.Thedesignprocessshouldconsider
inputfromanumberofrelevantcommunities,includingthebasicresearchcommunityandendusers.Thebuildprocessintegratesnumerouscomponentsandresearchthreadsintoasinglesystem;herethereisanopportunityforindustrycollaborationsandtechnologytransfer.Finally,theintegrated
systemisdeployedinareal‐worldcontext.ParticipantssuggestedthenotionofaRobotCity(seenextsubsection)asapromisingideatoevaluateHRIinareal‐worldcontext.Thecycleisclosedbyincorporatingenduserfeedbackintotheexperimentaldesignofthenextiterationofthe
design/build/deploycycle.
ResearchInfrastructureWorkshopparticipantsfeltstronglythatrapidprogresstowardstheidentifiedscientificobjectiveswillcriticallydependonthebroadavailabilityofadequateresearchinfrastructure,includinghardwareandsoftware.Toaddresstheresearchchallengesgivenabove,itwillbenecessarytoconstructrobotic
platformsthatcombinemanyadvancedandinteractingmechanicalcomponents,providingadequatecapabilitiesformobility,manipulation,andsensing.Thesesplatformswillbecontrolledbyamultitudeofindependentlydeveloped,yetinterdependentlyoperatingsoftwarecomponents.Asaresult,these
integratedroboticplatformsexhibitadegreeofcomplexitythatisbeyondwhatcaneasilybedesigned,developed,tested,andmaintainedbymanyindependentlyoperatingresearchgroups.Thelackofstandardizationofhardwareandsoftwareplatformsmayalsoresultinafragmentationoftheresearch
11
community,difficultiesinassessingthevalidityandgeneralityofpublishedresults,andthereplicationofmuchunnecessaryengineeringandintegrationeffort.
Toovercomethesechallenges,workshopparticipantsadvocatedcoordinatedcommunityeffortsforthe
developmentofhardwareandsoftwaresystems.Theseeffortsshouldincludethedevelopmentofanopenexperimentalplatformthatwould—preferablyatlowcost—supportabroadrangeofresearcheffortsontheonehand,whileenablingtechnologyandsoftwarereuseacrossresearchgroupsonthe
otherhand.OneexampleofsuchanopenplatformisROS,arobotoperatingsystembeingdevelopedbyWillowGaragethatenablescodereuseandprovidestheservicesonewouldexpectfromanoperatingsystem,suchaslow‐leveldevicecontrol,implementationofcommonly‐usedfunctionality,and
message‐passingbetweenprocesses.Ideally,suchplatformswouldbecomplementedbyphysicalsimulationsoftwaretosupportearlydevelopmentandtestingofalgorithmswithoutcompromisingthesafetyofresearchersandhardware.Developmenteffortscouldalsobenefitfromroboticintegrated
developmentenvironments(IDEs);theseIDEsenforcedmodularityinsoftwaredevelopmenttherebyfacilitatingreuseanddocumentation.
Participantsnotedthatresearchinroboticsisrarelythoroughlyevaluatedandtestedinwell‐defined,repeatableexperiments.Otherfields,suchascomputervision,havegreatlybenefitedfrompublicly
availabledatasets,whichenabledanobjectivecomparisonbetweenmultiplealgorithmsandsystems.Theparticipantsthereforesuggestedthecreationandexpansionofrepositoriesofexperimentaldata,whichcouldthenserveascommunity‐widebenchmarks.However,asmuchoftheresearchinrobotics
isfocusedonthephysicalinteractionbetweentherobotanditsenvironment,electronicdatasetsarenotsufficient.Theyshouldbecomplementedbyskill‐specificbenchmarksconsistingofphysicalobjects.Forexample,anumberofreadilyavailableobjectscanbeselectedasabenchmarkforgrasping
research.Furthermore,entirebenchmarkenvironmentsweresuggestedtodevelop,evaluate,andcomparetheperformancewithrespecttoaparticularapplicationorimplementation.Such
environmentscouldrangeinsizeandcomplexityfromasimpleworkspace(anofficedeskorakitchencounter)toanentireroom,ahouse,oranentirecityblock.Inthiscontext,thenotionofaRobotCitywasmentioned:aregularurbanenvironmentinwhichallinhabitantsarepartoftheexperimentand
helpintheevaluationprocessaswellaswiththedefinitionofadequaterequirementsforeverydayapplicationenvironments.
Manyoftheproposedefforts—andinparticularhardwareorsoftwareintegrationefforts—falloutsideofthescopeofexistingfundingprograms.Participantsnotedthatapolicychangeinthisregardwould
benecessarytoensurethattheavailabilityofresearchinfrastructuredoesnotrepresentabottleneckintheprogresstowardsautonomousroboticsystemsineverydayenvironments.
MechanicalHardwareSafetyisacriticalfactorforthedeploymentofroboticsystemsinhumanenvironments.Inherentlysaferobotswouldalsoenablemodesofhumanrobotinteractionthatcanincreaseacceptanceofrobotictechnologyineverydaylife.Participantsthereforefeltthatinherentlysafermotorsandmechanisms
withincreasedstrengthtoweightratiowouldrepresentanimportantenablingtechnology.Insuchmechanismsvariablecompliancewouldbeadesirableproperty.Theconceptofvariablecompliance
12
referstoamechanismsabilitytoadjustitsbehaviortoreactionforceswhencontactingtheenvironment.Thesereactionforcescanbevariedfordifferenttasks.Suchmechanismsenablesafe
operation,especiallywheninteractingwithhumans,aswellasflexible,robust,andcompetentmotionwhenincontactwiththeenvironment.Furthermore,energyefficiencywasidentifiedasacriticalconcernformanyapplications,asrobotswillhavetooperatewithouttethersforextendedperiodsof
time.Finally,novelorimprovedmodesoflocomotionbeyondwheelsareneededtoenablesafeandreliableoperationinindoorandoutdoorenvironments.Outdoorenvironmentsoftentimesexhibithighlyvariableterrainpropertieswhileoutdoormaycontainstairs,ladders,ramps,escalators,or
elevators.
Participantsidentifiedhighlydexterousandeasilycontrollablerobotichandsasanimportantareaforresearch.Progressinroboticgraspingandmanipulationverylikelywillgohandinhandwiththedevelopmentofnovelhandmechanisms.Atthesametime,participantsfeltthatthepotentialof
currenthandtechnologywerenotfullyleveragedbyexistinggraspingandmanipulationalgorithms.Itisthereforeconceivablethatmanyinterestingandrelevantapplicationscanbeaddressedwithavailablegraspingandmanipulationhardware.
3.KeyChallenges/Capabilities
3.1MotivatingScenarios
QualityofLifeRoboticstechnologyisexpectedtomakeatremendouscontributiontothelivesoftheelderlyanddisabled.Onesuchexampleofanexistingapplicationisarevolutionarytransportationmobilitysolutionthatenables
thosewithlimitedmobilitywhousewheelchairstoindependentlygetintoandoutoftheirvehiclesandremotelyloadandunloadtheirwheelchairsfromawiderangeofvehicles.Thissystemmakesitpossibleforthosedependent
onwheelchairstotransporttheirwheelchairusinganordinarypassengervanandtoaccessitwheneverneededwithoutassistancefromothersofferingthemadegreeoffreedomandindependenceheretoforeunavailable.Thissystemprovidessignificant
benefitsoverexistingtransportationmobilitysolutions,includinglowercostofownership,abilitytousestandardcrash‐testedautomotiveseats,greaterchoiceofvehicles,norequiredstructuralmodifications,andabilitytore‐installonsubsequentvehicles.
Agriculture
Roboticstechnologyisexpectedtoimpactamyriadofapplicationsinagricultureandaddressfarmers’constantstruggletokeepcostsdownandproductivityup.Mechanicalharvestersandmanyotheragriculturalmachinesrequireexpertdriverstoworkeffectively,whilefactors
suchaslaborcostsandoperatorfatigueincreaseexpensesandlimittheproductivityofthesemachines.Automatingoperationssuchascropspraying,harvesting,andpickingofferthepromiseofreducedcosts,increasedsafety,
greateryields,increasedoperationalflexibility,includingnighttime
ATRSTMRoboticWheelchair
AutonomousTractor
13
operations,andreduceduseofchemicals.Anumberofsuchprototypesystemsandapplications,includingautomatedfruitcropsprayingandfieldcropharvesting,havebeendevelopedandthetechnologyhasnow
maturedtothepointwhereitisreadytobetransitionedforfurthercommercializationandfielddeploymentwithinthenextfewyears.
InfrastructureRoboticstechnologyhastremendouspotentialtoautomatetheinspectionandmaintenanceofournation’sbridges,highways,pipelines,andotherinfrastructure.Already,thetechnologyhasbeenadaptedtodevelopautomatedpipelineinspectionsystemsthatreducemaintenanceandrehabilitationcostsbyproviding
accurate,detailedpipeconditioninformation.Suchsystems,basedonadvancedmulti‐sensorandotherroboticstechnology,aredesignedforundergroundstructuresandconditionsthatareotherwisedifficulttoinspect,includinglargediameterpipes,longhaulstretches,inverts,crowns,culverts,andmanholes,andin‐serviceinspections.Theseroboticplatformsnavigatethiscriticalwastewaterinfrastructuretoinspectsewerpipeunreachablebytraditionalmeansandproduceveryaccurate3Dimagesofthepipeinsidesurface.The
inspectioninformation,capturedindigitalform,servesasabaselineforfutureinspectionsandasaresultcanautomaticallycalculatedefectfeaturechangesovertime.
MiningRoboticstechnologyisalreadystartingtohaveadramaticimpactonboththeundergroundandsurfaceminingindustries.Aninnovativebeltinspectionsystemthatusesahigh‐speed"machinevision"systemandsoftwarealgorithmstomonitortheconditionofconveyorbeltsandhelpoperatorsdetectdefects,forexample,isineverydayuseatseveralundergroundcoalmines.Thepatentedsystemisdesignedtoreducecostlydowntimecausedbythedegradationandeventualruptureofconveyorbeltsplices.Onalargerscaleroboticstechnologyisbeingusedtodevelopautonomousversionsoflargehaultrucksusedinminingoperations.Caterpillarrecentlyannouncedthatitisdevelopinganautonomousmininghaulagesystemwithplanstointegrateautonomoushaultrucks,eachwithpayloadcapacitiesof240tonsormore,intosomeminesitesby2010.Theautonomoustechnologyisdesignedtoprovideproductivitygainsthroughmoreconsistencyinprocessesandminimizeenvironmentalimpactbybothimprovedefficiencyandoverallminesafety.
TransportationRoboticstechnologywillsignificantlyaffecteveryaspectofhowwetransportpeopleandgoodsinthecomingdecades;frompersonaltransportationsystemstointelligenthighwaystoautonomouspublictransportationsystems.CompaniessuchasSegwayandToyotahaveintroducedpersonaltransportationrobotsthatareriddeninstandingpositionandcontrolledbyinternalsensorsthatconstantlymonitortherider’spositionandautomaticallymaketheaccordingadjustments.Meanwhile,carmakersanddevicemanufacturersarecreating“smartcars”byinstallingmorepowerfulcomputersandsensors,givingdriversabetterideaoftheirenvironmentandcarperformance.
ResponderTMPipelineRobot
AutonomousHaulTruck
14
AlthoughAmericandriverslognearlytwiceasmanymiles(1.33trillionperyear)astheydid25yearsago,theroadstheyaredrivingonhaveincreasedincapacitybyonly5percent,resultingin3.7billionhoursofdriverdelaysand2.3billiongallonsofwastedfuel.Toaddressthisissuehighwayagenciesareattemptingtocreate“smartroads”byinstallingsensors,camerasandautomatictollreadersandapublic‐privatenationalinitiativecalledVehicleInfrastructureIntegration(VII)hasbeenlaunchedtomergesmartcarsandsmartroadstocreateavirtualtrafficinformationnetworkandbustupgridlock.Masstransportationsystemsarealsoexpectedtoadoptroboticstechnologytoprovideoperatorswithgreatersituationalawarenessandnavigationassistanceincrowdedurbancorridorstherebyhelpingtocontrolcostsandincreasesafety.
EducationRoboticshasalreadycommencedtransformingtheAmericanclassroom.RoboticsputsacademicconceptsincontextandisbeingusedatalllevelsinK‐12andcollegeeducation.Roboticsprovidesstudentswitha
tactileandintegratedmeanstoinvestigatebasicconceptsinmath,physics,computerscienceandotherSTEMdisciplines,whileenablingteachersatthesametimetointroduceconceptsaboutdesign,innovation,problemsolving,andteamwork.Roboticscurriculumshavebeendeveloped,teachershavebeentrained,andscoresofcompetitionsareheldeveryyearacrossthecountry.PerhapsthebestknownroboticscompetitionprogramsareoperatedbyFIRST,anon‐profitorganizationfoundedin1999toinspireyoungpeopletobescienceandtechnologyleaders.Asameasureofthe
growingpopularityofroboticscompetitions,FIRSTisexpectingover195,000studentstoparticipateinitscompetitionsinthecomingyear.Evenmoresignificantly,arecentBrandeisUniversitysurveyfoundthatFIRSTparticipantsaremorethantwiceaslikelytopursueacareerinscienceandtechnologyasnon‐FIRSTstudentswithsimilarbackgroundsandacademicexperiences.Althoughmuchprogresshasbeenmade,thesurfacehasonlybeenscratchedintermsofthepotentialimpactofroboticsineducation.Tomorefullyrealizethispotential,robotsneedtobemademoreaccessible,affordableandeasytouseforbothstudentsandteachers.
HomelandSecurityandDefenseTheuseofroboticstechnologyforhomelandsecurityanddefensecontinuestogrowasinnovativetechnologyhasimprovedthefunctionalityandviabilityofsearchandrescueefforts,surveillance,explosivescountermeasures,firedetection,andotherapplications.Unmannedsurveillance,detection
andresponsesystemswillbeabletomakeuseofroboticplatforms,fixedsensors,andcommandandcontrolnetworkstopotentiallymonitorandpatrolhundredsofmilesofroughborder
TopThreeFinishersinthe2008DARPAUrbanGrandChallenge
FIRSTLegoLeagueTMParticipants
DisasterSiteApplication
15
terrain,tosniffoutandlocatechemical/biological/radioactive/nuclear/explosivethreats,andsurveylargeperimetersassociatedwithborders,powerplantsorairports.Suchsystemswillenablesecuritypersonneltoautomaticallydetectpotentialthreats,totakeaclose‐infirstlookfromasafedistance,andtoprovideinitialdisruptionandinterdictionatthepointofintrusionifnecessary.Whileother“man‐packable”robotsequippedwithinstrumentsincludinginfraredcameras,nightvisionsensorsandmillimeter‐waveradarhavebeenusedatdisastersites,includingtheWorldTradeCenter,tosearchforvictims.
3.2CapabilitiesRoadmapInthefollowing,weidentifythekeychallengesthathavetobemetandthekeycapabilitiesthathaveto
bedevelopedinordertodeliverservicerobotscapableofaddressingtheaforementionedmotivatingscenarios.Figure4providesanoverviewoftheproposedroadmapandtheremainderofthisdocument.Therightcolumninthefigurelaysouttheapplicationareas,manyofwhicharedescribedinthe
motivatingexamplescenariosabove.High‐impactadvancesintheseapplicationareascanonlybeenabledifanumberofcapabilitiesforautonomousservicerobotsbecomeavailable.ThesecapabilitiesarelistedinthemiddleofthefigureanddescribedinmoredetailinSection3.Toachievetherequired
levelofcompetencyinthoseareas,sustainedinvestmentinresearchanddevelopmentsinanumberofbasicresearchareasandtechnologiesisrequired.Figure4showstheseresearchareasandtechnologiesintheleftcolumn;theyaredescribedinmoredetailinSection4.
16
Figure4.Overviewoftheroadmapfordomesticandindustrialservicerobotics:Sustainedresearchanddevelopmentinthebasicresearchareasintherightmostcolumnofthefigurewillenableanumberofelementarycapabilities,showninthemiddlecolumnofthefigure.Thesecapabilitiesinturnenable
progressintheapplicationareasontheright.
HumanlikeDexterousManipulationEvensimpletasks,suchaspickingupunknownobjects,stillrepresentmajorresearchchallenges.The
levelofdexterityandcapabilitiesinphysicalreasoningrequiredforautonomousmanipulationinthecontextofprofessionalanddomesticserviceroboticsseemsfaroutofreach.Pressingproblemsinthisareaincludeadequatesensorsandassociatedperceptualcapabilities,dexteroushandsandsafe
manipulators,planningunderuncertainty,advancedcontrol,skilllearningandtransfer,andmodelingandsimulation.
Someparticipantsbelievedthattherequiredcompetencyinmanipulationcanonlybeachievedwhenthesedifferentareasareadvancedinacoordinatedfashionratherthaninisolation.Forexample,novel,
skin‐liketactilesensorsholdgreatpromisefordexterousin‐handmanipulation.However,welackthealgorithmstoprocessthedatafromsuchsensors.Itisconceivablethattechniquesfromcomputer
17
visioncouldinterpretthetactileinformationasanimageandthereforeareabletocomputeusefulabstractionsofthehigh‐dimensionaltactiledata.Atthesametime,inspirationfromcomputervision
algorithmsmayenablethedesignofsimplertactilesensorsthatcontainsimplelocalpre‐processingtailoredtothespecificalgorithmstheysupport.
In5,10,and15yearsthefollowinggoalsarepossiblewithsustainedresearchanddevelopment:
• 5years:Robotsperformlimitedpickandplacetaskinthehomeandinindustrialsettings;robotsareabletoreliablyopendoorsandcabinets.Thesemanipulationtasksareaccomplished
partiallybyengineeringtheenvironment,partiallybyequippingrobotswithspecialized(oratleastnotverygeneralpurpose)end‐effectors,andbymakingsimplifyingassumptionsregardingtheenvironment.
• 10years:Robotsrobustlymanipulatelarge,graspable,rigid,possiblyarticulatedobjectsandtoolswithoutpossessingapriorimodels.Robotsimprovetherobustnessandapplicabilityofmanipulationandgraspingskillswithexperience.Robotsacquiregeneralizedmanipulation
knowledgetogivetheminformationabouttheuseofobjectsandtools,eveniftheyhavenotencounteredthembefore.
• 15years:Robotspossesshandswithnearlyhumanlevelsofmechanicaldexterity.Handsare
coveredwithhigh‐resolutiontactileskin.Robotsareabletoperformrobust,sensor‐based,prehensileandnon‐prehensilemanipulationofobjects.Theypossessrudimentarycapabilitiesofmanipulatingflexibleobjects.
RealWorld3DPlanningandNavigationAutonomousservicerobotsaccomplishtasksbymovingabouttheirenvironmentandbyinteractingwith
theirenvironment.Thesemotionsandinteractionsneedtoachieveagiventaskbychangingtherobot’sposeandbymovingobjectsintheenvironment.Theaccomplishmentofataskmayrequirecomplexsequencesofmotionsandinteractions;therobotmayhavetomovefromoneroomtoanotheroritmay
havetoopendoors,clearobstaclesoutofitspath,removeobstructions,orusetools.Toachievethislevelofcompetency,substantialadvancesattheintersectionofmotionplanning,taskplanning,andcontrolhavetobemade.Historically,theseareashaveprogressedinisolation.Theproblemsposedby
servicerobotics,however,canonlybeaddressedthroughatightintegrationofthesetechniques.
Considerthetaskofpickingupacuptowhichaccessisobstructedbyabox.Toreasonaboutpushingtheboxtothesidetopickupthecup,therobothastoreasonaboutitsowncapabilities,thegeometryofthescene,constraintsimposedbyactuationandjointlimits,thecontactdynamicsandfrictionthat
arisewhenpushingthebox,etc.
Toreasonabouttheworldinsuchawaythattheappropriatesequenceofactionsandmotionscanbedetermined,therobothastobeawareofitsenvironment.Notalloftherequiredinformationcanbeprovidedtotherobotbeforehand,asservicerobotsoperateinunstructuredanddynamic
environments.Therobotthereforehastopossesscapabilitiestoperceiveandmapitsenvironment.“Semanticmapping”providestherobotwithinformationabouttheenvironmentthatisrequiredto
18
achieveatask.Objectdetectionandrecognitionandrelatedperceptualskillsprovideinformationforsemanticmappingandforobjectmanipulation.
In5,10,and15yearsthefollowinggoalsarepossiblewithsustainedresearchanddevelopment:
• 5years:Robotsinresearchlaboratoriescannavigatesafelyandrobustlyinunstructured2D
environmentsandperformsimplepickandplacetasks.Relevantobjectsareeitherfromaverylimitedsetorpossessspecificproperties.Robotslearnsemanticmapsabouttheirenvironmentthroughexplorationandinteractionbutalsothroughinstructionfromhumans.Theyareableto
reasonabouttasksofmoderatecomplexity,suchasremovingobstructions,openingcabinets,etc.toobtainaccesstootherobjects.
• 10years:Givenanapproximateandpossiblyincompletemodelofthestaticpartofthe
environment(possiblygivenaprioriorobtainedfromdatabasesvietheInternet,etc.),servicerobotsareabletoreliablyplanandexecuteatask‐directedmotioninserviceofamobilityormanipulationtask.Therobotbuildsadeepunderstandingoftheenvironmentfromperception,
interaction,andinstruction.Therobotmodifiesitsenvironmenttoincreasethechancesofachievingitstask(removeobstructions,clearobstacles,turnonlights),anditcandetectandrecoverfromsomefailures.
• 15years:Servicerobotscanperformhigh‐speed,collision‐free,mobilemanipulationincompletelynovel,unstructured,dynamicenvironments.Theyperceivetheirenvironment,translatetheirperceptionsintoappropriate,possiblytask‐specificlocalandglobal/short‐and
long‐termenvironmentalrepresentations(semanticmaps)andusethemtocontinuouslyplanfortheachievementofglobaltaskobjectives.Theyrespondtodynamicchangesintheenvironmentinawaythatisconsistentwiththeglobalobjective.Theyareabletointerleave
exploratorybehaviorwhennecessarywithtask‐directedbehavior.Theyinteractwiththeirenvironmentandareabletomodifyitinintelligentwayssoastoensureandfacilitatetask
completion.Thisincludesreasoningaboutphysicalpropertiesofinteractionsbetweenobjectsandtheenvironments(sliding,pushing,throwing,etc.)andtheuseoftoolsandotherobjects.
CognitionInserviceroboticsthereisaneedtooperateinnon‐engineeredenvironments,toacquirenewskills
fromdemonstrationbyusers,andtointeractwithusersfortaskingandstatusreporting.Cognitivesystemsenableacquisitionofnewmodelsoftheenvironmentandtrainingofnewskillsthatcanbeusedforfutureactions.Cognitionisessentialforfluentinteractionwithusersanddeploymentindomains
wherethereislimitedopportunitiesforusertraining.Inadditionanaddeddegreeofintelligenceforcopingwithnon‐engineeredenvironmentisessentialtoensuresystemrobustness.
In5,10,and15yearsthefollowinggoalsarepossiblewithsustainedresearchanddevelopment:
• 5years:Demonstrationofarobotthatcanlearnskillsfromapersonthroughgestureand
speechinteraction.Inadditionacquisitionofmodelsofanon‐modeledin‐doorenvironment.
19
• 10years:Arobotthatinteractswithuserstoacquiresequencesofnewskillstoperformcomplexassemblyoractions.Therobothasfacilitiesforrecoveryfromsimpleerrors
encountered.• 15years:Acompanionrobotthatcanassistinavarietyofservicetasksthroughadaptationof
skillstoassisttheuser.Theinteractionisbasedonrecognitionofhumanintentandre‐planning
toassisttheoperator.
RobustPerceptionServicerobotsoperateinrelativeunconstrainedenvironmentsandassuchthereisaneedtoproviderobustperceptualfunctionalitytocopewiththeenvironmentalvariation.Perceptioniscriticaltonavigationandinteractionwiththeenvironmentandforinteractionwithusersandobjectsintheproximityofthesystem.Todayperceptionistypicallyusedforrecognizingandinteractingwithsingle,knownobjects.Toenablescalabilitythereisaneedtohavefacilitiesforcategorizationofperceptsandgeneralizationacrossscenes,eventandactivities.Alreadytodaytherearemethodsformappingandinterpretationofscenesandactivitiesandthemainchallengeisinscalabilityandrobustnessforoperationinunconstrainedenvironments.In5,10,and15yearsthefollowinggoalsarepossiblewithsustainedresearchanddevelopment:
• 5years:Demonstrationofarobotsystemthatcancategorizespacesandautomaticallyassociatesemanticswithparticularplaces.Thesensingwillbeintegratedovertimeforrobust
operationinlargescalescalessuchasmallorabuildingstructure.Therobotwillbeabletorecognizehundredsofobjects.
• 10years:Demonstrationofarobotsystemthatcanperceiveeventandactivitiesinthe
environmenttoenableittooperateoverextendedperiodsoftime.• 15years:DemonstrationofarobotthatintegratesmultiplesensorymodalitiessuchasGPS,
visionandinertialtoacquiremodelsoftheenvironmentandusethemodelsfornavigationand
interactionwithnovelobjectsandevents.
Physical,intuitiveHRIandinterfacesDeploymentofservicerobotsbothinprofessionalanddomesticsettingsrequirestheuseofinterfacesthatmakesthesystemseasilyaccessiblefortheusers.Diffusionofroboticstoabroadercommunityrequires
interfacesthatcanbeusedwithnoorminimaltraining.Therearetwoaspectstointerfaces:physicalinteractionwithusersandpeopleinthevicinityandthecommandinterfacefortaskingandcontroloftherobot.Thephysicalinteractionincludesbodymotiontomove/nudgeobjectsandpeopleandnon‐contact
interactionsuchaschangeofmotionbehaviortocommunicateintentorstate.Theinterfaceaspectisessentialtotaskingandstatusreportingforoperatorstounderstandtheactionsoftherobot.
In5,10,and15yearsthefollowinggoalsarepossiblewithsustainedresearchanddevelopment:
• 5years:Demonstrationofarobotwheretaskinstructionisfacilitatedbymulti‐modaldialogfor
simpleactions/missionsandrobotsthatcancommunicateintentofactionsbythebodylanguage.• 10years:Demonstrationofarobotwhereprogrammingbydemonstrationcanbeusedfor
complextasklearningsuchasmealpreparationinaregularhome.
20
• 15years:Demonstrationofarobotthatcanbeprogrammedbyanoperatorforcomplexmissionatatimescalesimilartotheactualtaskduration.
SkillAcquisitionServicerobotsmustpossesstheabilitytosolvenoveltaskswithcontinuouslyimprovingperformance.Thisrequiresthatservicerobotsbeabletoacquirenovelskillsautonomously.Skillscanbeacquiredin
manyways:theycanbeobtainedfromskilllibrariesthatcontainskillsacquiredbyotherrobots;skillscanbelearnedfromscratchorbycomposingotherskillsthroughtrialanderror;skillscanalsobe
learnedthroughobservationofotherrobotsorhumans;furthermore,theycanbetaughttoarobotbyahumanorroboticinstructor.Butskillacquisitionalsorequirestherobottoidentifythosesituationsinwhichaskillcanbebroughttobearsuccessfully.Skillscanbeparameterized;learningandselecting
appropriateparametersforavarietyofsituationsisalsoincludedinthecapabilityofskillacquisition.Theabilitytotransferskillsfromonedomaintoanotherortotransferexperienceacquiredwithoneskilltoanotherskillcanbeexpectedtoprovidesubstantialadvancesinskillacquisition.Adequate
capabilitiesinskilllearningwillbeenabledbyadvancesinperception,representation,machinelearning,cognition,planning,control,activityrecognition,andotherrelatedareas.
In5,10,and15yearsthefollowinggoalsarepossiblewithsustainedresearchanddevelopment:
• 5years:Robotscanlearnavarietyofbasicskillsthroughobservation,trialanderror,andfrom
demonstration.Theseskillscanbeappliedsuccessfullyunderconditionsthatvaryslightlyfromtheonesunderwhichtheskillwaslearned.Robotscanautonomouslyperformminoradaptationsofacquiredskillstoadaptthemtoperceiveddifferencefromtheoriginalsetting.
• 10years:Asperceptualcapabilitiesimprove,robotscanacquiremorecomplexskillsanddifferentiatespecificsituationsinwhichskillsareappropriate.Multipleskillscanbecombinedintomorecomplexskillsautonomously.Therobotisabletoidentifyandreasonaboutthetype
ofsituationinwhichskillsmaybeappliedsuccessfully.Therobothasasufficientunderstandingofthefactorsthataffectthesuccesssoastodirecttheplanningprocessinsuchawaythatchancesofsuccessaremaximized.
• 15years:Therobotcontinuouslyacquiresnewskillsandimprovestheeffectivenessofknownskills.Itcanacquireskill‐independentknowledgethatpermitsthetransferofsingleskillsacrossdifferenttasksanddifferentsituationsandthetransferofskillstonoveltasks.Therobotisable
toidentifypatternsofgeneralizationfortheparameterizationofsingleskillsandacrossskills.
SaferobotsTodaysafetyforrobotsisachievedthroughaclearseparationoftheworkspacesforhumansandrobotsor
throughoperationatspeedsthatdonotrepresentarisktohumansintheproximityofthesystem.Astheoperationofhumansandrobotsbecomemoreandmoreintertwinedtherewillbeaneedtoexplicitly
consideroperationathigherspeedswhileoperatingindirectproximitytopeople.Thereisaneedtoconsiderstandardsforsafetytoenablecertification.Whiletechnologically,safetyinvolvesseveralaspectsincludingtheneedfor:advancedperceptioncapabilitiestodetectobjectsandpersonsandpredict
possiblesafetyhazards,controlsystemsthatreacttopossibledangeroussituations,andinherentlysafeactuationmechanismstoensurethatcontactwithapersonorobjectscauseslittleornodamage.
21
In5,10,and15yearsthefollowinggoalsarepossiblewithsustainedresearchanddevelopment:
• 5years:Asafetystandardforserviceroboticshasbeendefinedandacceptedworldwide,which
specifiesallowimpactsandenergytransfers.Basicmanipulationsystemshavefirstversionsofsafetystandardimplemented.
• 10years:Aninherentlysaferobotforoperationinproximityofhumansisdemonstratedfor
industrialapplicationscenarios.• 15years:Arobotsystemthatdoesmobilemanipulationincooperationwithhumansis
demonstratedandthesafetyisdemonstratedbothforhardwareandsoftwarecomponents.
4.BasicResearchandTechnologies
ArchitectureandRepresentationsOverthelast20yearsanumberofestablishedmodelsforsystemorganizationhaveemerged.Characteristically,however,noagreementoroverallframeworkforsystemorganizationhas
materialized.Forautonomousnavigation,mobility,andmanipulationtherearesomeestablishedmethodssuchas4D/RCSandHybridDeliberativeArchitectures,butonceinteractioncomponentsareaddedsuchasHuman‐RobotInteraction(HRI)thereislittleagreementonacommonmodel.Overthe
lastfewyearstheareaofcognitivesystemshasattemptedtostudythisproblem,butsofarwithoutaunifiedmodel.Forwideradoptionofrobotsystemsitwillbeessentialtoestablisharchitecturalframeworksthatfacilitatesystemsintegration,componentmodeling,andformaldesign.Appropriate
architecturalframeworksmayinitiallyorinherentlydependonthetask,theapplicationdomain,therobot,oravarietyofotherfactors.Nevertheless,adeeperunderstandingoftheconceptsunderlyingcognitioncanbeexpectedfromanincrementalunificationofmultipleframeworksintomoreless
problem‐orrobot‐specificarchitectures.Anyoftheaforementionedarchitecturalframeworkswillbeintricatelylinkedtoasetofappropriaterepresentationsthatcaptureaspectsoftheenvironmentand
theobjectscontainedinit,therobot’scapabilities,domaininformation,aswellasadescriptionoftherobot’stask
ControlandPlanningAsservicerobotsaddressreal‐worldproblemsindynamic,unstructured,andopenenvironments,novelchallengesariseintheareasofrobotcontrolalgorithmsandmotionplanning.Thesechallengesstem
fromanincreasedneedforautonomyandflexibilityinrobotmotionandtaskexecution.Adequatealgorithmsforcontrolandmotionplanningwillhavetocapturehigh‐levelmotionstrategiesthatadapttosensorfeedback.Researchchallengesincludetheconsiderationofsensingmodalitiesanduncertainty
inplanningandcontrolalgorithms;thedevelopmentofrepresentationsandmotionstrategiescapableofincorporatingfeedbacksignals;motionsubjecttoconstraints,arisingfromkinematics,dynamics,andnonholonomicsystems;addressingthecharacteristicsofdynamicenvironments;developingcontrol
andplanningalgorithmsforhybridsystems;andunderstandingthecomplexityofthesealgorithmicproblemsincontrolandmotionplanning.
22
PerceptionOverthelastfewdecadestremendousprogresshasbeenachievedinperceptionandsensoryprocessingasisseenforexampleinwebbasedsearchessuchasGoogleimagesandfacerecognitioninsecurity
applications.Mappingandlocalizationinnaturalenvironmentsisalsopossibleforengineeredenvironments.OverthelastdecadeinparticularuseoflaserscannersandGPShaschangedhownavigationsystemsaredesignedandenabledanewgenerationofsolutions.Nonetheless,localization
andplanninginGPS‐deniedenvironmentswhicharequitecommonremainsaveryimportantresearcharea.Inadditiontherehasbeentremendousprogressonimagerecognitionwithscalingtolargedatabases.Inthefuturealargenumberofrobotswillrelyonsensoryfeedbackfortheiroperationand
theapplicationdomainwillgobeyondpriormodeledsettings.Thereisthereforeaneedforrelianceonmultiplesensorsandfusionofsensoryinformationtoproviderobustness.Itisexpectedthattheuseofimage‐basedinformationinparticularwillplayamajorrole.Visionwillplayacrucialroleinnew
mappingmethods,infacilitatingthegraspingofnovelobjects,inthecategorizationofobjectsandplacesbeyondinstancebasedrecognition,andinthedesignofflexibleuserinterfaces.
Robust,HighfidelitySensorsAdvancesinmicroelectronicsandpackaginghaveresultedinarevolutioninsensorysystemsoverthe
lastdecade.Imagesensorshavemovedbeyondbroadcastqualitytoprovidemega‐pixelimages.MEMStechnologyhasenabledanewgenerationofinertialsensorpackagesandRFIDhasenabledmoreefficienttrackingofpackagesandpeople.Sensorshaveenabledsolidprogressindomainswithgood
signalquality.Asthedomainsofoperationarewidenedtherewillbetheneedfornewtypesofsensorsthatallowrobustoperation.Thisrequiresbothnewmethodsinrobustcontrol,butmoreimportantlysensorsthatproviderobustdatainthepresenceofsignificantdynamicvariationsandadomainwith
poordataresolution.NewmethodsinsiliconmanufacturingandMEMSopenopportunitiesforanewgenerationofsensorsthatwillbeakeyaspectoffutureprogressinrobotics.
NovelMechanismsandHighPerformanceActuatorsThereisanintricateinterplaybetweenprogressinmechanicaldevicesandactuationandthealgorithmiccomplexityrequiredtousetheminaccordancewiththeirfunction.Somealgorithmicproblemscanbe
solvedortheirsolutiongreatlyfacilitatedbyintelligentmechanicaldesign.Advancesinmechanismdesignandhigh‐performanceactuatorscouldthereforecriticallyenableground‐breakinginnovationsinotherbasicresearchareasaswellasenableseveralofthecapabilitieslistedintheroadmap.Importantresearch
areasincludethedesignanddevelopmentofmechanismswithcomplianceandvariablecompliance,highlydexteroushands,inherentlycomplianthands,energy‐efficient,safe,high‐performanceactuators,energy‐efficientdynamicwalkers,andmanymore.Ofparticularinterestare“intelligent”mechanical
designsthatcansubsume—throughtheirdesign—afunctionthatotherwisehadtobeaccomplishedthroughexplicitcontrol.Examplesincludeself‐stabilizingmechanismsorhandswithspecialprovisionstoachieveformclosurewithoutexplicitcontrol.
LearningandAdaptationManyofthebasicresearchareasdescribedinthissectioncanbenefitfromadvancesinandapplicationof
learningandadaptation.Servicerobotsoccupycomplexenvironmentandliveinhigh‐dimensionalstate
23
spaces.Knowledgeoftheenvironmentandoftherobot’sstateisinherentlyuncertain.Therobot’sactionsmostoftenarestochasticinnatureandtheirresultcanbestbedescribedbyadistribution.Many
ofthephenomenathatdeterminetheoutcomeofanactionaredifficultorevenimpossibletomodel.Techniquesfrommachinelearningprovideapromisingtooltoaddresstheseaforementioneddifficulties.Thesetechniquescanbeusefulforlearningmodelsofrobots,taskorenvironments;learningdeep
hierarchiesorlevelsofrepresentationsfromsensorandmotorrepresentationstotaskabstractions;learningofplansandcontrolpoliciesbyimitationandreinforcementlearning;integratinglearningwithcontrolarchitectures;methodsforprobabilisticinferencefrommulti‐modalsensoryinformation(e.g.,
proprioceptive,tactile,vision);structuredspatio‐temporalrepresentationsdesignedforrobotlearningsuchaslow‐dimensionalembeddingofmovements.
PhysicalHumanRobotInteractionGraduallythesafetybarriersthathavebeencommoninindustrialroboticsareremovedandrobotswilltoalargerdegreeengagewithpeopleforcooperativetaskexecutionandforprogrammingby
demonstration.Aspartofthis,robotswillhavedirectphysicalcontactwiththeuser.Thisrequiresfirstofallcarefulconsiderationofsafetyaspects.Inadditionthereisaneedtoconsiderhowtheserobotscanbedesignedtoprovideinteractionpatternsthatareperceivedasnaturalbyusers.Thisspansallaspectsof
interactionfromphysicalmotionoftherobottodirectphysicalinteractionwithaperceptionofminimuminertiaandfluidcontrol.Inadditionthereisaneedheretoconsidertheinteractionbetweendesignandcontroltooptimizefunctionality.
SociallyInteractiveRobotsAsrobotsengagewithpeoplethereisaneedtoendowthesystemswithfacilitiesforcooperativeinteractionwithhumans.Thisinteractionisneededfortaskingofasystem,forteachingofnewskillsandtasksandforcooperativetaskexecution.Thecurrentmodelsforsocialinteractionincludegestures,
speech/sound,bodymotion/pose,andphysicalposition.Thereishereaneedtointegrateskillandtaskmodelswithinterpretationofhumanintenttoenableinterpretationofnewandexistingactivities.Inserviceroboticsthereisabroadneedforsocialinteractionfromencounterswithnoviceusersto
cooperativetaskingwithanexpertoperator.Thefullspanofcapabilitiesisrequiredtoprovideengagingandlong‐termadoptionofrobotics.
5.ContributorsThisreportdocumentstheresultofbrainstormingsessionthattookplace7‐8August2008inSanFrancisco,CA.ThereportispartoftheCCCstudyonRobotics.TheComputingCommunityConsortium
(CCC)isaprojectmanagedbytheComputingResearchAssociation(CRA)andissponsoredbytheNationalScienceFoundation(NSF).ThepresentreporthasbeenauthoredbytheworkshoporganizersanddoesnotreflecttheoptionofCRA,CCCorNSF.Theresponsibilityofthereportliesentirelywiththe
authors.
TheCCCworkshoponserviceroboticswasorganizedbyOliverBrock,UniversityofMassachusetts,BillThomasmeyer,TheTechnologyCollaborative,Inc,andHenrikIChristensen,GeorgiaInstituteofTechnology.Theworkshopwasattendedbythefollowingpeoplefromacademiaandindustry:
24
ChadJenkins BrownNicholasRoy MITAaronDollar MITStefanoCarpin UCMercedJanaKosecka GeorgeMasonAndrewNg StanfordAndreaThomaz GeorgiaTechJingXiao UNCCharlotteCharlesRich WPICandaceSidner WPIStewartTansley MicrosoftResearchJoshuaSmith IntelEricBerger WillowGarageMartinBuehler iRobotPaoloPirjanian EvolutionRoboticsBillTownsend BarrettTechnologyScottThayer RedZoneChrisUrmson CMU/GMCynthiaBreazeal MITMichaelO’Connor NovariantPaulJames AdeptEricWhinnem Boeing‐MfrTechCharlieKemp GeorgiaTechTrevorBlackwell AnybotsDanMiller AnybotsBrianCarlisle PreciseAutomationParagBatavia Foster‐MillerAndreasHoffman VecnaJamesKuffner CMUAlexFoessel DeereOliverBrock UMassAmherstBillThomasmeyer TechCollaborativeHenrikChristensen GeorgiaTechJakeHuckaby GeorgiaTech