1

Computer Science Undergraduate ProjectEPFL, Logic Systems Laboratory

June 21, 2003

AdamA Modular Robot Evolution

and Simulation Tool

Author: Daniel MarbachSupervisor: Prof. Auke Ijspeert

2

Contents

1 INTRODUCTION ...................................................................................................................................................3

1.1 MODULAR ROBOTS..............................................................................................................................................31.2 CO-EVOLUTION ....................................................................................................................................................31.3 ROBOT SIMULATION ............................................................................................................................................4

2 STATE-OF-THE-ART............................................................................................................................................4

2.1 KARL SIMS BLOCK CREATURES ...........................................................................................................................42.2 MODULAR TRANSFORMER...................................................................................................................................42.3 POLYBOT..............................................................................................................................................................52.4 CONRO ...............................................................................................................................................................62.5 FRAMSTICKS.........................................................................................................................................................6

3 INTRODUCTION TO ADAM...............................................................................................................................7

3.1 MODULES .............................................................................................................................................................73.2 THE ADAM SCRIPT ...............................................................................................................................................73.3 STRUCTURES ........................................................................................................................................................83.4 PARAMETERS .....................................................................................................................................................113.5 BODY PARTS.......................................................................................................................................................123.6 FORMATTING SCRIPTS........................................................................................................................................123.7 EXAMPLE: A QUADRUPED ROBOT .....................................................................................................................12

4 GENETIC ALGORITHM....................................................................................................................................14

4.1 PHENOTYPE SPACE.............................................................................................................................................144.2 GENETIC ENCODING ..........................................................................................................................................144.3 GENETIC OPERATORS.........................................................................................................................................154.4 INITIALIZATION..................................................................................................................................................164.5 SELECTION AND REPLACEMENT.........................................................................................................................164.6 ALGORITHM .......................................................................................................................................................16

5 IMPLEMENTATION...........................................................................................................................................17

5.1 INTERNAL REPRESENTATION OF ROBOTS ..........................................................................................................175.2 CONTROL OF POWERED HINGES.........................................................................................................................17

6 RESULTS ...............................................................................................................................................................18

7 CONCLUSIONS....................................................................................................................................................19

3

1 IntroductionThis chapter is a broad overview of the three mainconcepts of this work: Modular robotics, co-evolutionary algorithms and robot simulation. Chap-ter 2. focuses on the state-of-the-art in modular ro-bot design and shows some current researchprojects.

1.1 Modular RobotsModular robots (figure 1) are built by connectingmodules together, much like a child’s Lego bricks.Current research is done with robots built up fromtens to hundreds of modules. In the future one canimagine robots with thousands, potentially millions ofmodules.

Modular reconfigurable robots have further theability to change their structure by reconnecting themodules in different ways [3]. They offer many in-teresting qualities such as versatility, robustness andlow cost.

They are versatile due to the many differentways modules can be connected together. Hundredsof modules allow usually millions of configurations,which can be applied to many diverse tasks. If therobot is self-reconfigurable, he can even adaptautonomously to different situations. Such a robotcould for example move through a small hole in a‘snake configuration’ and then change its structure toa four-legged creature to climb up a staircase.

Robustness comes from redundancy. Modularsystems typically use a small number of moduletypes but they are built of many modules. If somemodules are damaged, overall function degrades butthe system still works. Furthermore, self-repairmechanisms can be implemented, for example byadding spare modules that replace the faulty elementor by reconfiguring the robot around it.

Figure 1: Some possible configurations of modularrobots using simple hinge modules.

Potentially, the modules can be produced at a lowprice because they are small, relatively simple andused in large numbers. Another aspect is reuse:Suppose a robot is working on an assembly line. Ifthe production process is changed, he can be recon-figured to suit his new task with the same modules.

All this promises to make modular reconfiguringrobots very interesting for a wide range of tasks.Versatility and reliability make them especially wellsuited to harsh and unpredictable environmentsfound in applications such as urban search and res-cue operations after natural catastrophes or terroristattacks, space exploration or battlefield reconnais-sance.

1.2 Co-evolutionTraditionally, biologically inspired learning and evo-lutionary algorithms (EAs) only affect (if they’re ap-plied at all) the controller of the robot; the hardwareis still totally designed by humans. Aspects of mor-phology like the shape, size, method of propulsion ortype and position of sensors are all decisions the en-gineers make.

This unbalance in the use of EAs regarding mor-phology and control probably has practical reasons:Usually the controllers are designed for specific ro-bots that already exist, engineering a completely newrobot type is too expensive. On the other handbuilding and testing evolved hardware is much morecomplicated than doing the same for software. Whileit is possible to test an evolved controller immedi-ately just by loading it onto the robot or even by do-ing the EA online (on the robot itself), one first hasto build the evolved hardware to test it.

Nevertheless, if we have a look at nature, we seeco-evolution on all levels. There’s no such thing as abody and a brain god, each one doing his job oneafter the other. Evolution wouldn’t suddenly come upwith a leg and then slowly evolve the required sen-sors and controller for it. The powerful solutions na-ture came up with to allow creatures to survive intheir complex environments were found with coop-erative, co-evolutionary processes. There is co-evolution on several levels: The body is cooperativelyco-evolving with the sensors and the brain andthere’s co-evolution between a species and its envi-ronment (for example predator-prey relationships).

There are many arguments if favor of body-brainco-evolution: Biologically inspired engineering tech-niques have proven to be effective and often result inbetter solutions than traditional approaches. If na-ture uses co-evolution, why shouldn’t we do thesame in robot engineering?

If the EA is applied only to the controller, it workson a static fitness landscape. Even if it finds the op-tima, the resulting robot is certainly not the bestpossible solution for his task. Co-evolution results inchanging fitness landscapes over time and expandsthe search space to find solutions that are morepowerful [4].

4

Modular robots are especially well suited to co-evolution. The complexity of designing the structureand programming the controller of the robot growsexponentially with the number of used modules. Itmight be impossible to configure and program a ro-bot made up of thousands of modules by hand. Co-evolutionary algorithms allow us to evolve morphol-ogy and controllers simultaneously to suit a particu-lar task.

1.3 Robot SimulationSimulation plays an important role in evolutionaryrobotics. In general, it is impossible to calculate thefitness analytically. Therefore, the fitness has to bemeasured either on the real robot (online) or insimulation (offline).

The advantage of online EAs is the ‘what-you-see-is-what-you-get’ effect. Suppose we want toevolve a robot that is able to push a button. If a ro-bot succeeds the EA could be stopped, naturally ifthe robot were tested now, it would succeed againprovided the task is identical to the one in the EA.The problem with online EAs is that they are ex-tremely slow.

Using a simulation to measure fitness values ismuch faster than testing the robots in the real worldeach time but they might have problems closing the‘reality gap’. A robot that perfectly accomplishes atask in simulation can be completely useless in thereal world. To ensure a good transfer from simulationto real world the body of the robot and the environ-ment must be carefully (not accurately) reproduced.There are a number of different approaches:

Adding noise from a uniform distribution centeredabout zero to the precise values produced by analyti-cal models is the most simple and common way tofacilitate transfer from simulation to real world. Un-fortunately, the noise in the real world is not uniformand robots still might have problems closing the re-ality gap.

Figure 2: The famous block creatures of Karl Simscompeting over control of the box in the middle.

Sampling guarantees a much better transfer. Valuesreturned by the robot sensors for given objects andby actuators at given speeds are measured andstored in a look-up table. The simulation accessesthese values and adds some noise.

Jakobi proposed another approach: Minimalsimulations only try to model the base set features.These are the characteristics of the interaction be-tween robot and environment that are relevant forthe expected behavior. The remaining features areconsidered implementation-specific and therefore aresimplified and varied randomly from one trial to thenext so that evolution does not rely on them.Minimal simulations speed up significantly computingtime and transfer well to the real world but requirethe programmer to know in advance, what the rele-vant features will be that must be accurately mod-eled [6].

2 State-of-the-art2.1 Karl Sims block creaturesKarl Sims co-evolved body and brain of block crea-tures (figure 2) quite similar to our Adam robots al-ready in 1994. The morphology and the neuralsystems for controlling their muscle forces are bothgenetically determined, and the morphology and be-havior can adapt to each other as they evolve si-multaneously.

The genotypes are structured as directed graphsof nodes and connections, and they can efficientlybut flexibly describe instructions for the developmentof creatures’ bodies and control systems with re-peating or recursive components (figure 3). Whensimulated evolutions are performed with populationsof competing creatures, interesting and diversestrategies and counter-strategies emerge [7] [4].

2.2 Modular TransformerIn 2002, the Distributed System Design ResearchGroup of the National Institute of Advanced Indus-trial Science and Technology of Japan developed aself-reconfigurable modular robot (Modular Trans-former). It successfully realized multi-mode roboticmotion by changing its shape smoothly from acrawler to a four-legged walking robot (figure 4).

The used modules are very similar to the Adamhinge module. They consist of two semi-cylindricalparts that are connected by a hinge. Each semi-cylindrical part can rotate 180 degrees and has threeconnection surfaces, which can connect to othermodules by magnetic force. In addition, electrodesare placed on each connection surface for powersupply and communication with the host computer[8].

5

Figure 3: The genotype of Karl Sims block crea-tures is a directed graph.

Figure 4: The Modular Transformer of the AIST iscapable of changing its shape from a crawler to afour-legged walking robot.

2.3 PolyBotThe Palo Alto Research Center (PARC), a subsidiaryof Xerox Corporation, developed PolyBot (figure 5).The modules of the first generation were manuallyscrewed together, so they did not self-reconfigure.The G1 modules showed the first instance of simplereconfiguration for locomotion in 1997. The modulesof the second generation have the ability to auto-matically attach and detach from each other makingthem self-reconfigurable. PolyBot is the first robot todemonstrate sequentially two topologically distinctlocomotion modes by self-reconfiguration. The thirdgeneration of PolyBot is currently under construction.The goal is to demonstrate reconfiguration and loco-motion with large configurations. PARC uses twomodule types: Hinges and connection modules. Onboth sides of the hinge, only one module can be at-tached. Connection modules are cubes; other mod-ules can be fixed on all six faces.

PolyBot has shown versatility, by demonstratinglocomotion over a variety of terrain and manipulatinga variety of objects. It has successfully completedobstacle courses including ramps, steps, pipes andchicken wire [9].

Figure 5: The second generation of PolyBot is ableto self-reconfigure itself to spider, crawler, loop andsnake configurations.

6

2.4 CONROThe CONRO Project of the USC Information SciencesInstitute has a goal of providing the Warfighter witha miniature reconfigurable robot that can be used toperform reconnaissance and search and identificationtasks in urban, seashore and other field environ-ments. CONRO (figure 6) is homogenous (made fromidentical modules) that can be programmed to alterits topology in order to respond to environmentalchallenges such as obstacles.

The base topology is simply connected, as in asnake, but the system can reconfigure itself in orderto grow a set of legs or other specialized append-ages. Each module consists of a CPU, some memory,a battery, and a micro-motor plus a variety of othersensors and functionality, including vision and wire-less connection and docking sensors. The controlmechanism is distributed and hormone-based [10].

Figure 6: CONRO self-reconfiguring itself from asnake to a two armed crawler configuration.

2.5 FramsticksFramsticks is a three-dimensional life simulationproject. Both physical structures (“bodies”) and con-trol systems (“brains”) of creatures are modeled. Itis possible to design various kinds of experiments,including simple optimization (by evolutionary algo-rithms), co-evolution, open-ended and spontaneousevolution, distinct gene pools and populations, di-verse genotype/phenotype mappings, and spe-cies/ecosystems modeling.

Framsticks creatures are built of sticks. Modifi-cators can be applied to change the properties of thesticks. The genotype encodes all parameters of acreature, including morphology, sensors and actua-tors placement as well as control network. There aregyroscope, touch and smell sensors. Actuators canrotate or bend sticks.

Adam and Framsticks have very similar philoso-phies, the main difference being that Adam focuseson robotics while Framsticks focuses on artificial life.Both systems use simple modules to construct crea-tures and the genotype encodes both structure andcontrol. Currently there’s no notion of energy used inAdam. Framsticks creatures consume energy and dieif they don’t find a ‘food’ source. A creature can alsokill another one and steal its energy [11].

Figure 7: A Framsticks creature with a sense oftouch (on the top) and a sense of equilibrium (glass-like cell).

Figure 8: Framsticks user interface allows editingthe genotype and watch the structure and controlnetwork change in real time.

7

3 Introduction to AdamAdam is a system to evolve and simulate modularrobots. Currently the robots are not reconfigurableand loop configurations like the one of PolyBot in fig-ure 5 are not allowed. The simulation accuratelymodels rigid body dynamics (kinematics, gravity,friction, collisions, etc) in a world that consists simplyof an infinite plane. In future versions of Adam re-configuration and cycles will be supported and thesimulation might offer environments that are morecomplex.

3.1 ModulesAdam robots are built with a number of different,well-defined module types. Modules are like LEGOblocks: Each type has certain qualities and positionswhere it is possible to put another building stone.Adam is built to make it easy to add new types ofmodules. One could define modules that correspondto real hardware and then actually build the evolvedrobots but for now, we test the system with only asingle element type.

The hinge module (figure 9) consists of twocubes, fixed together with a hinge. Other elements

Parameter DefinitionInitialangle

A hinge has an initial angle between–150.0 and 150.0 degrees. High andlow stops as well as the oscillation ofthe desired angle are all relative tothe initial angle.

Low stop The maximal negative deviation of thehinge with respect to the initial angle.The low stop must be set between 0.0and 150.0 degrees.

High stop The maximal positive deviation of thehinge with respect to the initial angle.The high stop must be set between0.0 and 150.0 degrees.

Powered A Boolean indicating if the hinge ispowered with a motor or rigid.

Maximalforce

The maximal force of the motor (onlyrelevant for powered hinges).

AmplitudeFrequencyPhase

Amplitude, frequency and phase ofthe oscillation of the desired angle(only relevant for powered hinges).

Elastic A Boolean indicating if the hinge iselastic or not. Note that elasticitydoes not have any effect on poweredhinges.

Spring The spring constant of an elastichinge.

Damping The damping constant of an elastichinge.

Table 1: Definition of hinge parameters

Figure 9: A rigid Adam hinge module with an initialangle of 30 degrees in the simulation world.

can be attached at each one of the ten free faces.The hinge can be rigid or powered by a motor. Eachpowered hinge has a PD controller on its angle. Fur-thermore, the joint can be elastic or not.

The desired angle a of a powered hinge is definedwith the amplitude A, frequency f and phase j of asinus oscillation:

a = A sin(2pft + j)

Naturally, if a gets outside the bounds set by the lowand high stop, the hinge can’t follow the desired tra-jectory and will wait at the stop until a reenters theinterval. The spectrum of movements of the robots isenriched by the option of setting the low and highstops purposely smaller than the amplitude.

Unpowered hinges can be elastic. In this case,the joint applies a spring force defined by the springand damping constants.Refer to chapter 5 for more detailed informationabout the implementation of these features.

3.2 The Adam scriptThe user can define his own Adam robots with a sim-ple script. Furthermore, this allows evolved robots tobe saved in plain text format. They can then be in-spected and edited by the user with the text editor ofhis choice.

As mentioned above, Adam robots are built withmodules that can be fixed together like LEGO blocks .Each module type defines the characteristics of theelement such as how it can be fixed with other mod-ules. An Adam robot is valid if the same structurecould be built in the real world with correspondingelements.

8

Figure 10: Syntactic grammar in EBNF

Figure 8 shows the syntactic grammar of the Adamscript in Extended Backus Naur Form (EBNF). Don’tworry if you are not familiar with EBNF, just read thenext two sections and study table 2, which showsbasic structures with their corresponding script ex-pressions. If you have Adam installed you can buildand simulate robots defined with scripts by typing:Adam –open file

The first part of the Adam script defines thestructure of the robot (how the modules are con-nected to each other). Each module in the structurepart must be given a unique identifier. The first letterof the identifier indicates which type it is. Hingeidentifiers must always begin with a capital ‘H’. Theseidentifiers are used in the second part of the script toset the parameters of the modules.

3.3 StructuresIn this section I will explain how Adam structures canbe defined using the script. As mentioned above,scripts must have a structure as well as a parametersection. The examples in this section represent onlythe structure expression as defined in the grammar.If you want to build them with Adam, begin your filewith the keyword ‘STRUCTURE’ followed by such anexpression. The structure expression must be fol-lowed by the keyword ‘PARAMETERS’. As you can seein the grammar, the section following this keywordcan be left empty. In that case, all the modules willhave default parameter values.

The most natural way for humans to write orread a description of a configuration consisting ofmany elements is a sequential building plan. Such abuilding plan is usually structured as a series of pic-tures showing which elements have to be added atwhich position to the part that has already beenbuilt. The Adam script follows this philosophy withthe only difference that we can’t use pictures andthat always only one new module is added to thestructure at a time.

Figure 11: The hinge module has 10 positionswhere another module can be attached. Each posi-tion can be identified with respect to the local coordi-nate system of the hinge by its number as indicatedin the figure.

P0: First cube back faceP1: First cube top faceP2: First cube right faceP3: First cube bottom faceP4: First cube left faceP5: Second cube top faceP6: Second cube right faceP7: Second cube bottom faceP8: Second cube left faceP9: Second cube front face

script = { ident '{' bodypart '}' } [headOrient] bodypartheadOrient = 'ORIENTATION' '(' real ',' real ',' real ',' real ')'bodypart = 'STRUCTURE' structure 'PARAMETERS' parameters

structure = ( [pos] [orient] module { pos limb | [pos] [orient] module } ) | [pos] [orient] identlimb = ‘(‘ structure ‘)’module = hingehinge = 'H' 'identifier'orient = 'N' | 'E' | 'S' | 'W'pos = ‘P0’ | ‘P1’ | ‘P2’ | ‘P3’ | ‘P4’ | ‘P5’ | ‘P6’ | ‘P7’ | ‘P8’ | ‘P9’

parameters = { module function }function = '.' 'identifier' '(' real arguments ')'arguments = [ ',' 'real' ]

real is a real valued number with the usual syntax(scientific notation is supported). There's one re-striction: The number shouldn’t start with a point,so write '0.4' instead of '.4'

identifier is a string with lower or uppercase let-ters. Underscores are allowed as well.

Quick reminder of grammars and EBNF:

A syntactic (non-contextual) grammar is definedby a set of terminal and non-terminal symbols, aset of productions and an initial symbol. It definesa language as the set of finite sequences of termi-nal symbols that can be derived from the initialsymbol by successively applying productions.Repetitions can be expressed with {x} (zero ormultiple occurrences of x), options with [x] (zeroor one occurrence of x).

P1P4

P5 P8

P9

P0

P2

P6P3

P7

9

The key to understanding structure expressions andthe rest of this section is the difference between ‘at-tached AT’, ‘attached TO’ and ‘attached WITH’. Sup-pose that in our building plan, a new hinge Hb has tobe attached to another hinge Ha that is already partof the unfinished structure. We say that Hb is at-tached TO Ha WITH a certain position and orienta-tion. With respect to Ha, we could say that Hb wasattached AT a certain position. The first element ofthe body expression is the head of the robot. It is theonly module that is not attached TO another one.Naturally, other elements can be attached AT variouspositions. All other elements are attached TO oneand only one other module WITH a specific position.

I will now explain how a new hinge Hb can beattached to another hinge Ha. In the following exam-ples, Ha is always the head of the robot. Naturally,the same syntax is used when attaching a hinge toan arbitrary module of a partially built robot. Buildthe examples with Adam if you have the possibility.In table 2 you can find a more exhaustive list of sim-ple expressions and their corresponding phenotype.In this section, colors have no specific meaning. Theyare only used to distinguish the modules better.

The simplest way to attach the two hinges to-gether is in a straight row. ‘Ha Hb’ tells Adam to at-tach Hb to Ha using default positions. Hb can beattached at any free position of Ha. This is done byspecifying the position and putting the new modulein parenthesis. ‘Ha P5 (Hb)’ attaches Hb at the topface of the second cube (position 5).

Ha Hb Ha P5 (Hb)

The part within the parenthesis is called a limb of Ha.In the previous example, the limb consists of onlyone module but in general, it can be any legal bodyexpression. You can attach as many limbs at a hingeas there are free positions. The example on the leftattaches two limbs at the downside of the first andsecond cube of Ha (positions 3 and 7) and a thirdhinge at the default position. On the right, the sec-ond limb is attached at Hd instead of Ha.

Ha P3(Hb) P7(Hc) Hd Ha P3(Hb) Hd P7(Hc)

Now that we have seen how to attach limbs at vari-ous positions of a module, let’s see how a hinge canbe attached with other positions than the defaultone. ‘Ha P4 Hb’ attaches Hb with the left side of thefirst cube (position 4) to Ha. Naturally, it is still pos-sible to specify where Hb should be attached to Ha.The example below on the right attaches Hb with po-sition 4 to position 5 of Ha. Note that the positionbefore the parenthesis is a face of Ha while the posi-tion within the parenthesis is one of Hb.

Ha P4 Hb Ha P5 (P4 Hb)

Finally, once the faces that will be fixed together arespecified, there’s the possibility to set four differentorientations for the second hinge (N, E, S and W forNorth, East, West and South respectively). Let’smodify the previous examples to change the orienta-tion of Hb from default to East. The hinges are stillattached together with the same positions, but Hbhas been rotated 90 degrees around an axis perpen-dicular to the faces of contact.

Ha P4 E Hb Ha P5 (P4 E Hb)

I mentioned default positions several times already.Every Adam module type has to define a default po-sition to attach limbs as well as a default positionand orientation with which it gets attached to othermodules. This makes the scripts shorter and easierto read.

For hinges, these default values are the frontface of the second cube (position 9) for limbs to beattached at and the back face of the first cube (posi-tion 0) with orientation ‘north’ to attach it to anotherelement. This allows us to put two hinges in astraight row, the most natural and often used con-figuration, with the simple expression ‘Ha Hb’. Thesame structure can also be built by explicitly namingthe positions and orientation:

Ha Hb= Ha P9 (Hb)= Ha P0 Hb= Ha N Hb= Ha P9 (P0 N Hb)

10

Ha Hb = Ha P9(Hb)

Attach Hb at the defaultposition, which is thefront face of the secondcube of Ha.

Ha P5(Hb)

Attach Hb at the topface of the second cubeof Ha.

Ha P6(Hb)

Attach Hb at the rightside of the second cubeof Ha.

Ha P7(Hb)

Attach Hb at the bot-tom face of the secondcube of Ha. This pictureis the first frame of thesimulation, naturallythis structure will fall

over immediately.

Ha P8(Hb)

Attach Hb at the leftside of the second cubeof Ha.

Table 2: The five ways to attach a hinge AT thesecond cube of another hinge. Use positions zero tofour to attach it to the first cube.

Hb has been given an initial angle of 30 degreesto show its orientation better. The various positionsare obtained by rotating Hb in the default position(first example) plus or minus 90 degrees around thex-, y- or z-axis.

Ha Hb = Ha P0 Hb

Attach Hb with the de-fault position, which isthe back face of its firstcube.

Ha P1 Hb

Attach Hb with the topface of its first cube.

Ha P2 Hb

Attach Hb with the rightside of its first cube.

Ha P3 Hb

Attach Hb with the bot-tom face of its firstcube. Naturally thisstructure falls over im-mediately after simula-tion starts.

Ha P4 Hb

Attach Hb with the leftside of its first cube. Ifthe initial angle of Hb isset to 30 degrees, thisstructure is invalid be-cause of self-collision

(See chapter X).

Table 3: The five ways to attach Hb WITH differentpositions of its first cube to Ha. Use positions five tonine to attach Hb with its second cube to Ha.

11

Ha Hb = Ha N Hb

Attach Hb with the de-fault orientation North.The z-axis of Hb is fac-ing up.

Ha E Hb

Attach Hb with orienta-tion East. The z-axis ofHb is now directed to-wards the viewer.

Ha S Hb

Attach Hb with orienta-tion South. The z-axisof Hb is now facingdown.

Ha W Hb

Attach Hb with orienta-tion West. The z-axis ofHb is now directed awayfrom the viewer.

Table 4: Two hinges can be fixed together with twospecific faces in four ways. Orientations are labeledNorth, East, South and West. They correspond torotations of the default orientation North by 90, 180and 270 degrees respectively around the x-axis ofthe second hinge.

3.4 ParametersNow that we have seen how the structure of amodular robot can be defined, let’s see how the pa-rameters of the modules can be set. Remember thateach module has been given a unique identifier inthe structure part. These identifiers can now be usedto set the parameters of the module with the follow-ing notation:

identifier.function(arguments)

All hinge parameter-setting functions currently avail-able are defined in Table 5. The arguments are realnumbers separated with commas. You can use sci-entific notation, but you shouldn’t start the numberswith a point (write ‘0.5’ instead of ‘.5’)

Parameters of various modules can be set in any or-der. You can, for example, group all setters for aspecific module together or order them by functiontype. The parameter setting functions can be sepa-rated by spaces, but I recommend using a new linefor each one.

For internal reasons (see chapter X) it is not pos-sible for a powered hinge to be elastic. If a hinge isset to be powered and elastic at the same time, theelasticity and damping constant will be ignored andthe hinge behaves exactly the same way as if it werejust powered and not elastic.

It is only necessary to set parameters that aredifferent from the default value. Hinges with defaultparameters are rigid and have an initial angle of zerodegrees. It is also possible to redefine the defaultparameter values using the same syntax as for set-ting parameters of a specific module. Instead of theidentifier of a module, the indicator of the moduletype is used. For hinges, this is a capital ‘H’. For ex-ample, you could set all hinges except Ha to be elas-tic with a spring constant of 10 and a dampingconstant of 1 like this:

PARAMETERSH.soft(true, 10, 1)Ha.soft(false, 0, 0)

As mentioned above, the values of the elasticity anddamping constant of a rigid hinge do not have anyeffect. In the previous example they were set to 0,we might as well have chosen 1 or any other value.

initAngle(a)

Sets the initial angle of the hinge to a.

powered(isPowered, loStop, hiStop, Fmax, A, f, j)

Sets all parameters that are relevant for poweredhinges. isPowered indicates if the hinge has amotor or not. If this argument is equal false, thevalues of the other arguments have no influenceon the behavior of the hinge. LoStop and hiStopare the low and high stops of the hinge, Fmax isthe maximal force of the motor and A, f and j arethe amplitude, frequency and phase of the desiredoscillation.

soft(isSoft, elast, damp)

Sets all parameters that are relevant for elastichinges. isElastic determines if the hinge will berigid or elastic, elast and damp are the elasticityand the damping constant of the hinge.

Table 5: The three parameter-setting functions ofthe hinge module. Refer to chapter 3.1 for exactdefinitions of all hinge parameters.

12

However, currently it is mandatory to specify all ar-guments. Opening a script with the following pa-rameter section results in an error:

PARAMETERSH.soft(true, 10, 1)Ha.soft(false) // ILLEGAL

The parser prints out error messages if the syntax ofthe script is not correct. Usually the error messageindicates what was expected and what was found.Naturally, this holds for the whole script and not justfor the parameter section. If you open a file with theprevious illegal statement, the following error mes-sage will be displayed in the terminal:

Expected ',' found ')'

Currently the parser does not count lines and wordsand can’t indicate exactly where the error occurred.If you don’t find it, try again with the –debug option.Now you can see how far the parser got and wherethe error is:

PARAMETERS--> parameters H.soft( true, 10, 1) Ha.soft( true--> Expected ',' found ')'

3.5 Body partsThe Adam script offers the possibility to define bodyparts that can be reused several times in the mainstructure. For example, you can define a leg or anarm at the beginning of the script and then attachthese predefined structures at different positions withdifferent orientations to the robot. The syntax is thesame as for the main bodypart. To declare a body-part, start with its identifier, open a left brace fol-lowed by the structure and parameter section asdescribed in the previous chapters and finish with aright brace:

ident { STRUCTURE … PARAMETERS …}

The bodypart can now be attached with various posi-tions and orientations using the same syntax as if itwere a simple module. Remember that what you’reactually doing is attaching the head of the bodypart,which is a hinge. Default values can therefore beused as described in the previous chapters:

// attach with default position and orientationHa P1(ident)// attach with position 3 and orientation EastHa P1( P3 E ident)

3.6 Formatting scriptsAs in every script or programming language, it ispossible with the Adam script to write the same thingin many different ways. Some of them are easy toread, others not.

The Adam parser ignores white spaces, tabs andnew lines; you are therefore free to format yourscripts as you want. In this chapter, I propose onegood way to do it. This is also, how the examples inthe following chapters are formatted and how Adamsaves evolved robots to text files. Formatting is cru-cial for the structure part, it’s not so important forthe rest of the script. The parameter section is for-matted, quite naturally by beginning a new line forevery parameter setting function. Formatting of thestructural part can be done with the following threerules:

1. Begin a new line for every limb.2. Every time you begin a new line, indent by i.

At the beginning, i is zero.3. Increase i by one for the limbs of a module.

Remember that only structures in parenthesis areconsidered limbs. Just go on reading the followingchapters if this seems somewhatcryptic and you willget used to writing nice scripts even without knowingthese rules by heart. The advantage of writingstructure expressions like this is that one immedi-ately sees where a limb begins, ends and where it isfixed. Furthermore, the tree structure of the robot(see chapter 4.2) is quite visible.

3.7 Example: A quadruped robotWe will now apply the learned concepts to build awalking quadruped robot. Unfortunately, Adam can’tyet show the robot growing as the script is writtenlike Framsticks. Therefore, we always have to write apart of the script, save it to a file and then build itwith Adam to check if the structure is correct. Let’sbegin with the body of the robot, a rectangle of rigidhinges (figure 12). Since Adam doesn’t yet supportcycles, the first and the last hinge can’t be fixed to-gether.

Let’s now attach a leg at the right face of the firstcube of H_body0 (figure 13). Three more legs couldbe added in the same way but it is much nicer to de-fine the leg as a modular bodypart and then attach itfour times at the desired positions (figure 14).

If you simulate this robot, you will see that itdoes push-ups and doesn’t walk because all fourpowered hinges of the legs have the same oscillation.In future versions of Adam it will be possible to seedpopulations and to evolve only certain parameters ofthe robots while leaving the others untouched (seechapter X). We could now seed a population with thisrobot and hope that evolution will somehow take ad-vantage of it, or we could evolve only the phases ofthe powered hinges to get this specific structure towalk.

13

Figure 12: This rectangle of rigid hinges will be thebody of the quadruped robot. Note that Hbody0 andHbody5 are not fixed together.

Figure 13: A leg has been attached at the rightface of the first cube of Hbody0. The ‘foot’ is set tobe elastic. Note that usually legs have two poweredjoints, for simplicity we didn’t add a knee to this ro-bot.

Figure 14: The final script for the quadruped robot.

If you want to test your Adam scripting skills, finishthe script of figure 13 by adding the three restinglegs explicitly. Try to get the robot to walk. Set thephase of the first leg to zero and play around withthe other three phases. Can you get it to jump byalso modifying the other parameters of the poweredhinges?

///////////////////////////////////// A simple quadruped robot///////////////////////////////////

//////////// LEG /////////////////leg { STRUCTURE H_leg H_foot

PARAMETERS // default for hinges of the leg H.initAngle(-60)

H_leg.powered(true, -60, 60, 100, 60, 0.2, 0) H_foot.soft(true, 50, 0)}

/////////// BODY ////////////////STRUCTURE

H_body0 P2(E leg)H_body1 P4 H_body2 P0(W leg)P4 H_body3 P2(E leg)H_body4 P4 H_body5 P0(W leg)

PARAMETERS

STRUCTURE STRUCTUREHbody0 Hbody1 Hbody0 Hbody1P4 Hbody2 P4 Hbody2 P4 Hbody3PARAMETERS Hbody4 P4 Hbody5

PARAMETERS

STRUCTURE

H_body0 P2( E H_leg H_foot )H_body1 P4 H_body2 P4 H_body3 H_body4P4 H_body5

PARAMETERS

H_leg.initAngle(-60)H_foot.initAngle(-60)

H_leg.powered(true, -60, 60, 100, 60, 0.2, 0)H_foot.soft(true, 50, 0)

14

4 Genetic algorithmThis chapter describes the genetic algorithm (GA)that Adam uses to co-evolve morphology and controlof robots. The primary goal of this project is thesimulator of modular robots and not the evolution ofa specific robot type. Consequently, the GA is ratherprimitive and probably not optimal, even though ithas proved to evolve powerful solutions (see chapterX). Some of its basic parameters like the populationsize or the mutation rate, can be changed but a lot ofother choices, for example how individuals are se-lected for reproduction and deletion, are hard codedand need further testing to know if they really per-form well.

The implementation of this first version of a GAfor Adam has two purposes. It should be the startingpoint for a more in depth analysis of co-evolution ofmodular, potentially even self-reconfigurable, robots.The second purpose is to test the stability of thesimulation environment.

4.1 Phenotype spaceThe phenotype space of Adam robots is huge. Sup-pose you have a hinge Ha and you want to attach asecond hinge Hb to it. If the two hinges are not yetattached to any other modules, there are a priori 400ways to fix them together. Hb can be attached onany of the 10 free faces of Ha with any of its 10faces, plus there are 4 possible ways of attachingtwo hinges together with two specific faces. A thirdhinge could be attached to any of the 18 free facesand this can be done, as before, in 10*4 differentways. Theoretically, only three hinges therefore offeralready 1’120 different structures. This is only an es-timate and not the exact number of structural con-figurations because some of them might befunctionally identical, just built another way and italso has to be considered if the space that the at-tached hinges occupy is free. Nevertheless, this givesus an idea of the vast phenotype space we are deal-ing with. Note that we only considered how thehinges could be attached together; the parameters ofthe different modules are also part of the searchspace.

The precise definition of the Adam phenotypespace relies on the following hypothesis:

1. There’s a finite number N of module typesused (usually few).

2. There are an infinite number of modulesavailable of each type.

3. There’s infinite space available to build therobot (there are no limitations on size andform of the robot)

The Adam phenotype space consists of all robotsin the simulated world that could be built theo-retically in the real world with correspondinghardware modules under hypothesis 1-3.

Note that phenotypes are simulated robots. The pre-vious definition allows robots to have cycles butmodules must not intersect because such a structurecould not be built in the real world (it works perfectlywell in simulation). For pratical reasons, , the secondhypothesis can obviously not hold because thememory of the computer is limited and simulationgets very slow for robots with more than 50 hinges.Evolved robots usually had up to 30 modules.

On the other hand, hypothesis 3 must alwayshold: First, the robot is built in empty space, andthen it is put into the simulation world (so there willnever be such a thing as a leg entering into theground).

4.2 Genetic EncodingThe type of encoding plays an important role in theevolution process, at the level of genetic operators,fitness landscape and, finally, speed and quality ofevolution. The more complex a phenotype space is,the more difficult it is to find a good genotype-to-phenotype mapping, or genetic encoding. In nature,billions of genes are necessary to define organismsable to survive. Even though we have succeeded indecoding many genomes, we don’t know much aboutthe corresponding genotype-to-phenotype mappings.

There’s an infinity of possible genetic encodingsfor a given phenotype space. An encoding and thecorresponding genotype are complete if the genotypespace is mapped onto the phenotype space. Recentstudies on different genotype encodings for simu-lated 3D agents have found that completeness is notnecessarily a good thing to have [1]. An incompletegenotype does not fully cover the phenotype space.It limits and structures the search space and there-fore speeds up evolution, the drawback is obviouslythat some optima’s might be lost because they’re notcovered anymore.

Figure 15: The Adam genotype is a tree. The headof the robot is the root; the nodes represent moduleinstances with all their parameters.

15

A phenotype space has an inherent topology: phe-notypes of similar fitness are ‘close’ to each other.The genotype encoding imposes another topology, bydefining which phenotypes are close genetically(measured as the number of mutations separatingthe corresponding genotypes) and which ones have ahigher probability of being visited. Under a good en-coding, these two topologies are more highly corre-lated [5]. Unfortunately, this correlation is impossibleto measure directly, because of the vast complexityof the phenotype space and its fitness landscape.Our judgments of encodings are thus subjective,based on various theoretical and experimental obser-vations.

It is possible to apply the genetic operators di-rectly to the phenotype itself without encoding it.This is usually done with simple, parameter optimiz-ing GAs. For complex phenotype spaces, such directencodings have proven to perform worse thanhigher-level encodings. The two most currently usedgenotypes for co-evolution of morphology and con-trol of artificial creatures or robots are directedgraphs and developmental encodings. The latter onesare based on L-systems and tend to result in morecomplex, better structured but not necessarily fittersolutions [1].

One could imagine using the script as genotypefor Adam robots. However, the script has been de-signed to be easy to read and manipulated by hu-mans and is not well suited for the GA. For example,the decomposition in a structure and a parameterpart is convenient for users but very inefficient for aGA. To delete modules or to do crossover it wouldhave to gather all the parameter-setting functions ofthe involved hinges in the second part of the script.

Adam uses trees, a subset of directed graphs, asgenotype (figure 15). The main advantages of thisrepresentation are that it is compact, robust in theface of genetic operators and identical to the internaldata type of Adam robots (see chapter 5.1), allowingthe genotype-to-phenotype mapping to directly buildthe robot in the simulation world. With a develop-mental encoding this step would be much more com-plicated and time consuming. Unfortunately, it isimpossible to express cycles and to reuse predefinedbody parts with trees alone. It would therefore bedesirable to generalize the current genotype to di-rected graphs like the ones Karl Sims used (figure3).

The genetic encoding structures the phenotypespace. As mentioned above, the GA is suspected toperform better if this topology is highly correlatedwith the one induced by fitness values. The Adamgenotype clearly fulfills this criterion because geneti-cally neighboring individuals have similar phenotypesand therefore usually also similar fitness values. Thisis not true for developmental encodings where a sin-gle mutation can result in a completely differentphenotype.

4.3 Genetic operatorsThe mutation operator acts on parameters as well ason the structure of the robot. The mutation rate de-termines the chance of a value being mutated. Itsdefault value is 0.01 but it can also be set with theoption –mut when Adam is launched. Random valuesare always selected with a uniform distribution.

The effect of mutation depends on the type ofthe value being mutated. Booleans are set to theiropposite value while numerical parameters are resetto a random value, selected with a uniform distribu-tion from an appropriate interval. The position andorientation that the module is fixed with can also bemutated and are set to another random posi-tion/orientation. Furthermore, each position where alimb can be attached has a chance of being mutated.If mutation occurs, a new, randomly initialized mod-ule is added if the position was free. In the case thatthere is already a limb attached at this position, it isdeleted.

Figure 16: Mother (top left), father (top right) andchild (bottom) after crossover. A randomly selectedsub tree of the mother (Hb) has been swapped withone of the father (H3).

16

Thanks to the tree structure of the genotype, the im-plementation of crossover is straightforward. First amother and a father are selected as described inchapter 4.5. The child is then formed by copying themother and swapping one of its sub trees with a subtree of the father (figure 16). The sub trees are se-lected by randomly choosing a node from the look-uptable of the robot. Each module has an equal chanceof being selected.

Obviously, crossover and mutation can generateinvalid robots with intersecting modules. If this hap-pens, the concerned robot is deleted and replacedwith a new, randomly initialized individual. This isdone in the hope of increasing diversity in the genepool. In advanced populations, it might be better toreplace such individuals with another child.

4.4 InitializationAt the beginning of the GA the population is initial-ized with randomly created robots. Individuals withself-collision are deleted and recreated until thewhole population consists of valid robots. The size ofthe population can be specified when Adam islaunched with the option –pop. It has a default valueof 100 robots.

In order to increase the chance of creating validstructures, default positions have a higher probabilityof being retained. Experience has shown that betterresults can be achieved by not initializing with com-pletely random values. For example, hinges have ahigher probability of being rigid or powered thanelastic, the frequency of powered hinges is alwaysthe same and the phase is a multiple of p/6. A hingecan only have up to two limbs and the depth of thetree representing the robot is limited at 3.

This initialization algorithm has proven to con-struct a big variety of mostly valid robots. It wassurprising to see that very simple but efficient solu-tions were already found with the initialization, justby coincidence (figure X).

4.5 Selection and replacementAfter evaluating a generation and assigning a fitnessvalue to every individual, parents have to be selectedto produce offspring. Fitter individuals have a betterchance of being parents. This gives us selectionpressure and drives the population forward. Oneshould be careful that selection pressure is not toohigh; otherwise, the population diversity might suf-fer. There are many different strategies to selectparents from the population.

Fitness proportionate selection (also known asthe roulette wheel method) gives each individual achance proportional to how good its fitness is withrespect to the population fitness. Fitness proportion-ate selection does not work if all individuals of thepopulation have roughly the same fitness. In thatcase, the chance of being a parent is almost thesame for all individuals and selection pressure is toolow. The other extreme occurs if an individual has a

very high fitness compared to the rest of the popula-tion. Its probability of producing offspring is close to1 and in a few generations, the whole population willconsist of equal copies. For these reasons, fitnessvalues should be scaled when using the roulettewheel method.

Tournament selection randomly selects k indi-viduals. k is the size of the tournament. The best in-dividual of the tournament is taken to makeoffspring.

Rank-based selection sorts individuals on theirfitness values from best to worse. The place in thissorted list is called rank. Adam uses a rank-proportional roulette wheel method. The probabilityps(i) for an individual i to be a parent is proportionalto its rank r(i). N is the population size.

ps(i) = (N + 1 - r(i)) / Âr(i)

For every new child, another individual has to be de-leted if the population size is to stay constant. Torandomly choose an individual for replacement, thesame methods as for selection of parents can beused. Now fitter individuals will have a smallerchance of being selected. Adam uses the same strat-egy for replacement as for selection. The probabilitypr(i) of an individual to be deleted is:

pr(i) = r(i) / Âr(i)

4.6 AlgorithmAdam uses a standard genetic algorithm. For everygeneration, a number of individuals are replaced withoffspring. Afterwards the mutation operator is ap-plied to all robots of the population except for thefittest one (steady-state evolution) and finally thefitness values of the modified robots are updated insimulation.

Figure 17: Diagram of the GA implemented inAdam. k is the number of replaced individuals pergeneration, i is the total number of generations.

17

5 Implementation5.1 Internal representation of robotsInternally, a robot is represented as a tree (figure16). Each module has pointers to the heads of itslimbs. It is not necessary for the limbs to know towhich module they are attached to so the tree is notdoubly linked.

This tree structure has been proven extremelyefficient, flexible and well adapted to represent Adamrobots. The structural parts of the script are them-selves representations of such trees (figure 15). Theimplementation of a recursive parser constructing arobot is therefore straightforward and efficient(chapter X). Furthermore, this representation canalso be used as genotype. This simplifies the geno-type-to-phenotype mapping a lot and therefore savestime in the EA (chapter 4.3). To speed up randomaccess robots also have a lookup table with all theirmodules listed.

Note that in the script it is possible to point topredefined trees (body parts) from another one. Theinternal representation (data type) doesn’t allow this.The body parts from the script are copied and addedto the tree as many times as needed because cur-rently one module represents exactly one object inthe simulated world. This might change in the nextversion of Adam (see chapter X).

Figure 18: The structural parts of a script, in thiscase the quadruped robot of chapter X, describe theconfiguration as a set of trees. It is possible to pointto previously defined trees (body parts).

5.2 Control of powered hingesThe desired angle q of powered hinges is defined bya sinus oscillation as described in chapter 3.1:

q = A sin(2pft + j)

But how can we achieve the desired behavior insimulation? The simplest solution is a PD controller. Acouple of torques is applied to get a rotation of thehinge that depends on the difference between thedesired angle qd, the actual angle qa and also on theangular rate w (the derivative of the actual angle):

T = a(qd - qa) - bw

However, ODE offers motors to control certain joints.These motors can be set at a specific speed (angularrate). This rate is applied immediately, in one simu-lation step. Even though it might seem weird to con-trol the angular rate and not the angle itself, oneshould still use the motors and not apply torques di-rectly because with motors ODE takes care that thesimulation is stable. Therefore, control can be doneby setting at each time step the angular rate of themotor to:

w = a(qd - qa)

Figure 19: The tree of the quadruped robot fromchapter X. This internal representation (data type) isalso the genotype of Adam robots.

18

6 ResultsLocomotion has been evolved with a very simple buteffective fitness function. The fitness of a robot isdefined to as its distance after a constant time ofsimulation from the origin. To calculate the fitnessone only needs to know the x- and y-coordinates ofthe robot when simulation ends because the startingposition is always in the origin. The position of therobot is defined to be the one of its head.

f = sqrt(x2 + y2)

This simple fitness function has proved to evolvebetter locomotion than more complex ones, consid-ering the center of mass or the instant velocity ofrobots. An advantage is also, that robots are evolvedto move less in a straight line and not just for exam-ple, turn in circles.

Experience showed that the time of simulation iscrucial to achieve good results. If it is too long, theGA gets very slow but if it is too short, we don’treally evaluate the ability of locomotion but only afast jump at the beginning of simulation. Figures 20to 23 show some examples of evolved robots.

Figure 20: Simple but sometimes very efficientsolutions were usually found in early generations.Later, they had no chance against more complexstructures.

Figure 21: A jumping worm. This is (as well as thetwo following examples) a highly specialized individ-ual that is difficult to improve on. Evolution tookabout three hours.

Figure 22: A two legged jumper. He has a long‘horn’ on the front side of his head that effectivelyprevents him from falling over.

19

Figure 23: This dragon is one of the most fearedcreatures in the Adam world… It walks on two smalllegs keeping its balance with the arms.

7 ConclusionsThe main goal of this project was to build a simulatorfor modular robots. By applying the genetic algo-rithm, we have tested this simulation environmentrepeatedly. I estimate the number of robots built andsimulated by Adam well over a million – without asingle crash. Furthermore simulation seems to beaccurate and without bugs. Otherwise, evolved ro-bots would have taken advantage of such irregulari-ties. I also expected the simulation to perform wellbecause ODE is a well-recognized simulation tool.

Clearly future work with Adam has to focus onthe genetic algorithm and the control of the robots.It would be interesting to generalize the Adam treegenotype to directed graphs and to test develop-mental encodings. Furthermore, robots could beequipped with sensors and neural networks as con-trollers.

Another feature that should be implemented isself-reconfiguration. One could even somehow try toevolve self-reconfiguration. Obviously, cycles withinthe structure of the robot should be allowed as well.

20

References[1] M. Komosinsky, A. Rotaru-Varga, Comparison of Different Genotype Encodings for Simulated 3D Agents. Arti-ficial Life Journal, 7:395-418, 2001

[2] Hornby, Gregory. S. and Pollack, Jordan. B. (2001). Body-Brain Co-evolution Using L-systems as a GenerativeEncoding. Genetic and Evolutionary Computation Conference.

[3] M. Yim, D. Goldberg, A. Casal, Proc. of SPIE, Connectivity Planning for Closed-Chain Reconfiguration, SensorFusion and Decentralized control in Robotic Systems III, Volume 4196, Nov. 2000

[4] Karl Sims, Evolving 3D Morphology and Behavior by Competition. Artificial Life IV Proceedings, ed. by R.Brooks & P. Maes, MIT Press, 1994, pp28-39.

[5] W. Hordijk, Population flow on fitness landscapes.

[6] Jesper Rasmussen, A General Robot Simulator for Evolutionary Robotics. Master’s thesis, 2001

[7] Biota website, http://www.biota.org/ksims/

[8] Modular Transformer website, http://staff.aist.go.jp/e.yoshida/test/top-e.htm

[9] Modular robotics at PARC website, http://www2.parc.com/spl/projects/modrobots/

[10] CONRO website, http://www.isi.edu/conro/

[11] Framsticks website, http://www.frams.alife.pl