oberseminar 2014 - tams activities in robotera · oberseminar 2014 tams activities in robotera...

University of Hamburg

MIN Faculty

Department of Informatics

3. June 2014

Oberseminar 2014TAMS activities in RobotEra

Hannes Bistry

University of HamburgFaculty of Mathematics, Informatics and Natural SciencesDepartment of Informatics

Technical Aspects of Multimodal Systems

3.June 2014

TAMS 1


MIN Faculty


3. June 2014

Outline

1. Introduction2. ROS Overview3. Example of a ROS-based robot4. Camera Integration5. MoveIt Integration

TAMS 2


MIN Faculty


2 ROS Overview 3. June 2014

General problem of research in robotics

(from willowgarage.com)TAMS 3


MIN Faculty



ROS

Robot Operating System

I component-oriented robotics framework

I Inter Process Communication middleware

I development suite

I language support (C++, Python, Java,...)

I debugging functions

I package management system

I active community

TAMS 4


MIN Faculty



ROS - basic concepts

Main components:I roscore (directory service and parameter storage)

I usually one in an environment

I Nodes (each component is called a node)I Communication mechanisms

I topicsI servicesI (actions)

I Parameter ServerI part of the “roscore“I configure nodes / share files and information

TAMS 5


MIN Faculty



ROS - Topics

Each Node can listen or publish on arbitrary topics:

I a “named“ datastream of a certain type (e.g.“/front camera/image raw“)

I N:N communicationI every node can listen to a topic

I example: multiple nodes need data from the camera

I multiple nodes can publish on a topicI example: all nodes may generate debug output (collected)

I Problem: dysfunctional nodes may break the whole systemI trusted zoneI ROS offers good debugging options

The topic names are used to match clients and servers.

TAMS 6


MIN Faculty



ROS - Topics

Every topic is restricted to one Message type:

I (first come - first serve to announce the message type for a topic)

I Built-in types string, float (32 or 64 bits), integer, booleans andHeader

I defined in other own and third-party packages“MyOtherPackage/CustomMessage“

Definition of new message formats in a .msg file:

float64 myDouble

string myString

float64 [ ] myArrayOfDouble

The message definition can also contain full messages defined in other

packages (nested definition).TAMS 7


MIN Faculty



ROS - Topics - Properties

I “Fire-and-Forget“ style of communication

I comparable to UDP-packetsI messages may be dropped due to:

I limited network bandwidthI processing of last message lasted too long

Behavior can be adjusted:

I message queue length

I whether to latch last message for new subscribers

TAMS 8


MIN Faculty



ROS - Topics - Examples

I joint values

I camera images

I laser scanner data

I point cloud (Microsoft Kinect)

I system health status (temperature, battery, ...)

TAMS 9


MIN Faculty



ROS - Topics - Examples

Publishing from the command line:

rostopic pub /topic_name std_msgs/String ” H e l l oWorld ”

Receiving the Topic:

rostopic echo /topic_namedata : Hello World

−−−

TAMS 10


MIN Faculty



ROS - Topics - Example N:N communication

Publishing repeatedly from the command line 1:

rostopic pub −r 1 /topic_name std_msgs/String ” H e l l o ”


rostopic pub −r 1 /topic_name std_msgs/String ” World ”

Receiving the Topic:

rostopic echo /topic_namedata : Hello

−−−data : World

−−−data : Hello

−−−data : World

TAMS 11


MIN Faculty



ROS - Topics - Invalid message types


rostopic pub −r 1 /topic_name std_msgs/String ” H e l l o ”

Listen to message with command line 2:

rostopic echo /topic_name


rostopic pub −r 1 /topic_name std_msgs/Int16 42

Error Message:

[ WARN ] [ WallTime : 1403186454 .670754] Could not process inbound connection : topic

types do not match : [ std_msgs/String ] vs . [ std_msgs/Int16 ]{ ’message_definition ’ : ’ string data\n\n ’ , ’ callerid ’ : ’/rostopic_12368_1403186454023 ’ , ’ tcp_nodelay ’ : ’ 0 ’ , ’ md5sum ’ : ’992ce8a1687cec8c8bd883ec73ca41d1 ’ , ’ topic ’ : ’/ topic_name ’ , ’ type ’ : ’ std_msgs/String ’}

TAMS 12


MIN Faculty



ROS - Services

Nodes can provide and call so called Services.

I asynchronous communication (RPC)

I Service request / replyI each part has its of datatype

I definition similar to messages in a .srv file (request / reply areseparated by dashes)

float64 query_x

float64 query_y

−−−float64 result

TAMS 13


MIN Faculty



ROS - Actions

Interface for preemptable tasks.

I for longer lasting tasksI goal / result /feedback

I client calls service with goal messageI (server sends one/many feedback messages)I server sends result

I definition similar to messages in a .action file (example fromros.org)

# Def i n e the goa luint32 dishwasher_id # Spe c i f y which d i s hwa she r we want to use−−−# Def i n e the r e s u l tuint32 total_dishes_cleaned

−−−# Def i n e a f e edback messagefloat32 percent_complete

TAMS 14


MIN Faculty



ROS - Actions

TAMS 15


MIN Faculty



ROS - Filesystem Structure

ROS workspace:I the ”root”directory where all your packages are inI announce this to the ROS system by

export ROS PACKAGE PATH=$ROS PACKAGE PATH:/path/to/your/workspace

ROS package:I directories under one directory defined in your

ROS PACKAGE PATH variableI may contain executables, libraries, configuration files, robot

models, message/service/actions definition filesI anything that logically constitutes a useful entity can be part of

the packageI may depend on other packagesI description / dependencies defined in Manifest.xml

TAMS 16


MIN Faculty



ROS - Filesystem Structure

ROS Nodes:

I are usually executables within one package

I instead of “path/to/your/workspace/package name/bin/node name“

you can simply type “rosrun package name node name“ in ANY directory

Launch-files:

I comparable to shell-scripts

I .launch files in the“path/to/your/workspace/package name/launch“ folder

I start roscore if no one is running

I start one or multiple nodes, also from other packages

I define parameters

I start with “roslaunch package name launchfile name.launch“

TAMS 17


MIN Faculty



Gazebo Simulator

ROS Nodes:

I Physical simulation of robots

I ROS integrationI simulate sensors

I laserI camerasI Kinect

I simulate actuatorsI robot armsI drive system

Note: RViz can only visualize, not simulate.

TAMS 18


MIN Faculty


3 Example of a ROS-based robot 3. June 2014

System Architecture of DORO

I Scitos G5 platformI ROS Hydro (Robot Operating System)I Core i7 Industrial PCI Software Subsystems:

I MoveIt (IK, Manipulation)I Mira / Cognidrive (navigation)I PEIS (high level task planning)I GStreamer / OpenCV for vision

TAMS 19


MIN Faculty



The Robot Description File

One essential part of running a Robot under ROS is a URDFdescription of a Robot, that is needed for:

I specification of coordinate frames

I calculation of frame transformations

I visualizationI simulation

I needs additional files

Usage:

I The file is uploaded to the ROS parameter server.

I several ROS-nodes fetch the robot-description file from there

I In practice, the URDF file is often generated by a .xacro file,that offers comfortable features like macros.

TAMS 20


MIN Faculty



The Robot Description File

...TAMS 21


MIN Faculty



The Robot Description File - Xacro

In practice, the URDF file is often generated by a .xacro file, thathas several advantage over pure URDF

I define macros (i.e. sensor or actuator type)

I algebraic expressions

I define constants

I run ROS-commands

Example: usage of a macro:

<xacro : include filename=”$ ( f i n d d o r o d e s c r i p t i o n ) / u r d f / x t i o n . u r d f . x a c r o ” />. . .<xacro : xtion parent=”${p a r e n t} l i n k ”>

<origin xyz=”${0.045} 0 . 0 3 5 ${0.2101 − 0.03912 + 0.02} ” rpy=”0 0 0” />

TAMS 22


MIN Faculty



The Robot State Publisher

The “robot state publisher“ is a ROS node, that reads out thestate of all joints, and publishes frame transformations:

I subscribes to ROS-topic “/joint states“ and publishes “/tf“

TAMS 23


MIN Faculty


4 Camera Integration 3. June 2014

Implementation of Camera Systems

ROS GStreamer interface:

I using camera with GStreamer drivers in ROS

I use features of GStreamer (compression, network data transfer)

I existing ROS-GStreamer interface “gscam“

I implemented as a standalone ROS-Node

I GStreamer Pipeline is configures in the environment variable“GSCAM CONFIG“

TAMS 24


MIN Faculty



Drawbacks of the current integration:

I restricted to RGB, 24 bpp format

I fixated to ROS-topic (needs remapping)

I inefficient due to unnecessary copy operations

I no timestamps exchanged

I supports only one direction: GStreamer → ROS

I run-time control of parameters not possible

Thus the ROS-GStreamer integration was redesigned from thescratch.

TAMS 25


MIN Faculty



New ROS Integration - Concept

TAMS 26


MIN Faculty



ROSSink

Publish arbitrary GStreamer video streams in ROS.

gst−launch videotestsrc ! capsfilter caps=” v i d e o /x−raw−rgb , bpp=24” ! rossink topic=testvideo

Will generate the following topics:

/testvideo/camera_info/testvideo/image_raw/testvideo/image_raw/compressed/testvideo/image_raw/compressed/parameter_descriptions/testvideo/image_raw/compressed/parameter_updates/testvideo/image_raw/compressedDepth/testvideo/image_raw/compressedDepth/parameter_descriptions/testvideo/image_raw/compressedDepth/parameter_updates/testvideo/image_raw/theora/testvideo/image_raw/theora/parameter_descriptions/testvideo/image_raw/theora/parameter_updates

TAMS 27


MIN Faculty



ROSSink (cont.)

Use cases:

I use arbitrary GStreamer compliant camera in ROS

I integration of smart camera

I publish preprocessed video

I use camera on remote system, using encoders from GStreamer(h264)

Format support: RGB, YUV,gray,jpegTimestamp and metadata conversion.

TAMS 28


MIN Faculty



ROSSrc

Subscribe to a ROS Topic and “publish“ images inside GStreamer:

gst−launch rossrc topic=/testvideo/image_raw !ffmpegcolorspace ! ximagesink sync=0

Use cases:

I use ROS compliant cameras in GStreamer (including virtualcameras)

I record ROS image topics (with compression)I drop-in replacement for Image View (ROS)

I Display VGA 60 Hz on Core i5:I GStreamer: 19 % CPU-loadI Image View: 57 % CPU-load

TAMS 29


MIN Faculty



Sift based Object detection

TAMS 30


MIN Faculty



ROSSiftfolder

Implementation of SIFT-based object detection

I algorithm can determine the 6D pose of an object based on a2D camera image

I previously known:I camera calibrationI dimensions of the objectI sample image from the object

I algorithm detects corresponding points

I calculates camera position to the object

I currently cuboids with sufficient optical features are supported

TAMS 31


MIN Faculty



ROSSiftfolder

SIFT-based Object detection, publishes transformations in ROS

I all image files in a folder are detected

I Input: feature-vectorI currently, one dimension is encoded in the filename (like

box100.jpg )I planned feature: database access

I if > 4 matches are found, the 3D pose is calculated

Published topics:

I ROS Vizualization Markers

I tf-Messages from camera frame to object frame

TAMS 32


MIN Faculty



Example: Displaying Objects with RViz

TAMS 33


MIN Faculty



Application: ROS-GStreamer Interface

Application: Visualization of grasp planning in a Gazebo Demo

I virtual camera in Gazebo publishing images in ROS

I virtual screen displaying images send to another ROS topicI one GStreamer Pipeline, featuring an input switcher with 3

ports:I one sample videoI the simulated Gazebo camera via the GstROSSrcI the RViz window via the screen capturing plugin of GStreamer

I the resulting video is sent to the virtual screen in Gazebo

Currently only a demo, but can be used for next generation userinterfaces.

TAMS 34


MIN Faculty



Application: ROS-GStreamer Interface

TAMS 35


MIN Faculty


5 MoveIt Integration 3. June 2014

ROS-MoveIt

I software for mobile manipulation

I inverse kinematics implementation (IK)I integration of OMPL (Open Motion Planning Library)

I collection of randomized motion planners

I collision checking library (FCL)I integration of sensors

I robot stateI obstacles (any kind of Point Cloud generating sensor)

I World Geometry Monitor

TAMS 36


MIN Faculty



MoveIt Concept

Simplified function of MoveIt trajectory generation.

TAMS 37


MIN Faculty



MoveIt Integration: Stumbling Blocks:

I incompatible meshes for robot-model:I self collisions were not prevented for parts of the robotI solution: import and export files in MeshLab

I MoveIt reports self-collisions for adjacent links:I motion planning fails as collision checks are positiveI add adjacent links to whitelist

I Malfunction on Intel GPUs:I Robot Model is displayed incorrectly, body parts have wrong

pose, collision checks are positive, trajectory generation failsI (needs testing: updated drivers from the xorg-edgers repository)I stick to NVidia GPUs

TAMS 38


MIN Faculty



MoveIt Integration: Stumbling Blocks (cont.):

I wrong order of joints in trajectory message:I robot arm executes wrong trajectoryI MoveIt ignores user-provided joint sequenceI Jaco-Node ignored actual sequence in the messageI solution: Jaco-Node fixed

I MoveIt chooses arbitrary arm-configuration for goal poseI long trajectories with high risk of collisionsI examples:

I unnecessary rotation of first joint (shoulder yaw) by 180 degreesI change from elbow up to elbow down configuration

I due to wide range of Jaco jointsI (intermediate) solution: limited range of joints in the URDF filesI side-effect: planning time decreased from 5 sec to 0.5 sec

TAMS 39


MIN Faculty



Navigation

Task: we want to drive the robot to a certain point in the map.I Input:

I goal positionI mapI angular velocity of the wheelsI robot geometry (URDF)I sensors (laser range finders, point cloud)

I Output:I geometry msgs/Twist to topic /cmd vel (desired velocity /

angular velocity of the robot base)I feedback of action server

TAMS 40


MIN Faculty



Navigation (cont.)

TAMS 41


MIN Faculty



Navigation

Alternative implementation on the service robot Doro:

I Scitos G5 is using a different robot framework (Mira)

I Mira-ROS interface

I comercial navigation software Cognidrive

I exchange/translate certain topics / messages betweenframeworks

This is mostly a “political“ decision. (EU project partner isresponsible for navigation) According to the partner, it hasadvantages over the ROS AMCL based navigation stack:

I reliability

I industrial proven

I support for precise dockingTAMS 42


MIN Faculty



Diagnostic Aggregator - Motivation

Problem: A robot system has MANY potential points of failure:I a distributed system, i.e. multiple networked PCs that may be

I crashedI unavailable due to network conditionsI forgotten to switch onI not properly working due to several reasons (temperature,

CPU-load, disk space, etc.)

I many peripheral devices, that may..I be disconnected or have no powerI cable may be broken / loose (mechanical stress)I be broken itselfI have driver problems (that in practise sometimes need a reboot)

TAMS 43


MIN Faculty



Diagnostic Aggregator - Motivation (cont)

Problem: A robot system has MANY potential points of failure(cont):I need multiple software components (in ROS nodes), that

I may be of questionable qualityI and therefore crashed or show other malfunctionsI have not been started in the right order and exitedI have exited due to malfunctions in other components

I the users/ developers from other labs, that...I may provide input that no developer thinks of...I close software componentsI modify the startup scripts

How to debug this?

TAMS 44


MIN Faculty



Diagnostic Aggregator

A ROS node collection and analysing all messages on the“/diagnostics“ topic.

I every Node can report problems on this topic

I every Node can report periodic status messages

I Diagnostic aggregator is configured via a .yaml file (check forseveral conditions)

I configurable timeout fot each condition (watchdog likeoperation)

I conditions may be grouped

I additional aggregators may be implemented

TAMS 45


MIN Faculty



Diagnostic Aggregator

The result of the Aggregator is published on the“/diagnostics agg“ topic and may be displayed with a GUI.

TAMS 46


MIN Faculty



Software Architecture of DORO

handoverobject-

services supervisor and sequencer

BringObjectServiceGraspObjectService

MoveToService CameraImageService

OMPLOpenCV

Kinova.DLLgstreamerOpenNI

Orocos

rospy

roscore

ROS

MIRA-ROS PCL

SCITOS

PEIS

. . .

rviz

CenterMIRA-

MIRA / CogniDrive

CamerasKinectlaser-scanners Jaco

MoveIt!roscpp

navigation_stack object_perception

environment_perception

messages

doro-GUI

HRI

TAMS 47

oberseminar 2014 - tams activities in robotera · oberseminar 2014 tams activities in robotera...

Documents