on experimental research in sampling-based motion planning roland geraerts workshop on benchmarks in...
Post on 20-Dec-2015
216 views
TRANSCRIPT
On Experimental Research in Sampling-based Motion Planning
Roland GeraertsWorkshop on Benchmarks in Robotics Research
IROS 2006
Probabilistic Roadmap Method
Construction (G =V,E )Loop
c a free sample
add c to the vertices V
Nc a set of nodes
for all c’ in Nc in increasing distance
if c’ and c are not connected in G then
if local path between c and c’ exists then
add the edge c’c to E
Forbidden spaceFree space
Sample
c
Colliding path
c
c’
c
Local path
c’c’c
c
Probabilistic Roadmap Method
Construction (G =V,E )Loop
c a free sample
add c to the vertices V
Nc a set of nodes
for all c’ in Nc in increasing distance
if c’ and c are not connected in G then
if local path between c and c’ exists then
add the edge c’c to E
Query connect sample s and g to roadmap
Dijkstra’s shortest path
Forbidden spaceFree space
Sample
Start / goalLocal path
Shortest path
Methods
• General setup– SAMPLE
• Implemented in C++ using VS.NET 2003• Easy API to add techniques• GUI: easily set up experiments• Repeatability: load/save an experiment• Easily comparing different techniques• Easily examining parameter of a technique• Automatically collect/process data of experiment
– Demo
Methods
• Test problems– Conclusions were often too general due to
limited set of problems– Also choose worst-case problems
Methods
• Interchangeability– Libraries taking take of common functionality
• Collision checking, visualizationCallisto: http://www.cs.uu.nl/dennis/callisto/callisto.html [Nieuwenhuisen]
• Graph utilitiesAtlas: http://www.cs.uu.nl/dennis/atlas/atlas.html [Nieuwenhuisen]
• Nearest neighborMPNN: http://msl.cs.uiuc.edu/~yershova/mpnn/mpnn.htm [Yershova, Lavalle]
• Deterministic sampling methodshttp://msl.cs.uiuc.edu/~yershova/so3sampling/so3sampling.htm [Yershova]
• Rotation in 3Dhttp://www.kuffner.org/james/software [Kuffner]
Methods
• Interchangeability– Source code of motion planning framework
• Motion planning kitMPK: http://ai.stanford.edu/~mitul/mpk [Latombe]
• Move3Dhttp://www.laas.fr/~nic/Move3D [Siméon]
• Motion strategy libraryMSL: http://msl.cs.uiuc.edu/msl [Lavalle]
– Unfortunately, code is often not up-to-date
Methods
• Interchangeability– Sources
• Geometry of environment/robot: VRML
• Problem descriptions: XML
– Advantages of using existing languages• Well documented• Parsers/type checkers are available for all platforms• Existing programs for creating/editing the files
Methods
• Interchangeability– Sources of geometry files and benchmarks
• http://www.give-lab.cs.uu.nl/movie/moviemodels [MOVIE]
• http://faculty.cs.tamu.edu/amato/dsmft/benchmarks [Amato]
• http://mpb.ce.unipr.it/ [Reggiani]
– Problems should be put online when article is published
Results
• Evaluation of solution– Compare new technique with existing ones
• Pitfall: parameter tuning only for the new technique
– Compare against optimal solution• Often only known for trivial cases• Approximate optimal solution by many runs
– User studies
Results
• Statistics– Large variances in running times
• Complicates statistical analysis• Makes analysis unreliable• Is undesirable from a user’s point of view
– Perform large number of runs– Provide more statistical info, e.g. box plots– Deterministic versus randomized techniques
• Deterministic techniques can respond sensitively to small changes in the problem setting
Conclusion
• Automate conducting experiments as much as possible
• Choose test problems carefully
• Source code, software components and problem data should be made available
• Use standard file formats (VRML, XML)
• Provide an extensive statistical analysis