Design and Implementation of a Central

Pattern Generator (CPG) Control System

for a Biped Robot

Prepared by:

Alexander F. Russell

Department of Electrical Engineering

University of Cape Town

South Africa

Supervisor:

Dr Ralph Etienne-Cummings

Department of Electrical and Computer Engineering

Johns Hopkins University

United States of America

This thesis is submitted in partial fulfilment of the requirements for the

degree of Bachelor of Science in Engineering (BSc Eng) in Mechatronics

October 23, 2006

ii

Homunculus: An exceedingly minute body that according to

medical scientists of the 16th and 17th centuries was

contained in a sex cell and whose preformed structure

formed the basis for the human body.

Stedmans Medical Dictionary

A possible Homunculus controlling walking (figure from Inman et al 1981)

iii

Plagiarism Declaration I, Alexander F. Russell, declare that the work on which this thesis is based is my own original

work (except where the acknowledgements indicate otherwise) and that neither the whole, nor

any part of it, has been or will be submitted for any other degree at this or any other university.

___________________________________

Signed

___________________________________

Date

iv

ACKNOWLEDGEMENTS Thank you to:

Dr Ralph Etienne-Cummings for supervising the project.

Samuel Ginsberg for an endless supply of Microchip products and C advice.

Prof. John Greene for help with Genetic Algorithms.

Dr Kit Vaughan for help in obtaining human walking data.

v

SYNOPSIS In the near future it is envisaged that robots will be a part of our everyday lives. Their

involvement will include helping with everyday chores, caring for the sick or elderly, and

performing hazardous tasks.

Currently, the biggest obstacle robots face in achieving this vision is a reliable way to navigate

complex terrains. Legged robots provide the solution to this navigation problem, however, much

research is still needed before they can become a reality.

By building the control systems for the robots using biological principles, such as Central Pattern

Generators and spiking neurons, the systems may be more easily compatible with other biological

principles and biomedical applications such as neuroprosthesis (Lewis et al, 2005).

The aim of this thesis was to design and implement a software based Central Pattern Generator,

using spiking neurons, on the RedBot biped robot. The scope of the project included the design of

the controlling algorithm as well as the necessary hardware and software to implement it.

Literature review A Central pattern generator (CPG) is a collection of neurons in the spinal chord that are able to

endogenously (i.e. without external input) produce sustained rhythmic or oscillatory, patterned

outputs (Hooper, 2000). The outputs of central pattern generators are responsible for most

rhythmic motor patterns (Marder and Calabrese, 1996; Stein et al, 1997) such as walking, flying,

swimming and breathing.

Izhikevich (2004) shows that as the biological plausibility of a neuron model increases so does its

computation cost. It follows, that simple neuron models are better suited to CPG design where

there is limited processing power available. Lewis et al (2005) showed that it is possible to use

simple Integrate and Fire with Spike Frequency Adaptation neurons to build a CPG capable of

producing joint angle profiles that allowed the RedBot robot to walk. This network is called the

Canonical network.

vi

One of the problems of using spiking neural networks is that they are not amendable to

mathematical analysis and it is a difficult process to find the parameters for these networks

through trial and error.

Lewis et al (1992) showed that it was possible to solve the parameters for these networks using

Genetic Algorithms (GAs). More recently, Inda and Ishii (2004) used a GA to evolve a CPG

controller for biped walking using recorded human movements as the CPGs target output.

The development and simulation environment A simulation of the RedBot was programmed in MATLAB. The simulation accepts a vector of

joint angle profiles and displays a simple visualisation of the RedBot walking with them. The

simulation calculates the torques, velocities and accelerations required to implement the walking

pattern.

The hardware The Microchip dsPIC30F4011 microcontroller was chosen to implement the CPG

algorithms and to interface the RedBot to the host PC.

A simple connection board was built to power the RedBot and to interface its various

signal wires to the dsPIC30F4011.

Adapting the Canonical Network The Canonical Network proposed by Lewis et al (2005) produced a simplified knee angle profile

to that of a human knee angle profile. The Canonical Network was adapted to produce a more

realistic knee angle profile. The resulting network was called the Double Bump Network. Values

were found, through trial and error, for the Double Bump Network to cause joint angle profiles

for walking at a frequency of 1.5Hz.

The Double Bump Network was implemented on the dsPIC30F4011 and the RedBot walked

successfully, however a small amount of slip was observed between the RedBots feet and the

ground.

vii

The frequency of walking of the RedBot was not at the desired 1.5Hz but instead at 0.75Hz. This

was a result of the dsPIC30F4011 not being able to process the Double Bump Network in real

time.

Automating the search for parameters An automated method, using a Genetic Algorithm called FBGAMIN, was designed to find the

necessary parameters to cause oscillations of a CPG, over a frequency range of 0.5Hz to 3.5Hz,

with prescribed phase shifts between specific neurons within the CPG.

The principles of Behavioural Memory (de Garis, 1990) and staged evolution (Lewis et al, 1992)

were used to increase the rate of convergence of FBGAMIN.

Parameters were found to cause oscillations at all the desired frequencies and phase shifts were

accurate to within 1. The convergence rate of the search was extremely fast with a maximum of

four generations needed to find any solution.

The parameters found, using this method, were applied to the Canonical Network. Visual

inspection of the resultant joint angle trajectories suggested that the RedBot could walk with

them. However, insufficient time was available to physically test these joint angle trajectories on

the RedBot.

Evolution of a CPG controller Human joint angles, recorded during normal walking, were used as target joint angle profiles for

a CPG whose parameters were evolved using FBGAMIN. The resultant joint angle profiles

closely approximated the human joint angle profiles. FBGAMIN was then used to make the CPG

as efficient as possible by removing any redundant neurons within the network.

The evolved network was too large to be simulated on the dsPIC30F4011 and thus a single

periods worth of walking patterns was simulated in MATLAB and then uploaded to the

microcontroller for testing on the RedBot.

The RedBot walked extremely well using the evolved CPG, however a small amount of slip

between the RedBots feet and the ground was observed.

viii

Conclusions Two Central Pattern Generators, using different network architectures, were designed and both

caused successful walking of the RedBot robot. However, the algorithms proved too complex to

be implemented on the dsPIC30F4011 microcontroller in real time. Both the CPGs designed

outputted more physiologically accurate joint angle profiles than Lewis et als Canonical

Network, with the joint angle profiles generated by the evolved controller being very accurate to

the joint angle profiles of a walking human.

In addition to this, an automated search method was developed to find parameters for the

Canonical Network to generate joint angle profiles for the RedBot at any frequency between

0.5Hz and 3Hz.

The following paper was a direct result of this work and has been submitted to the International

Symposium on Circuits and Systems(ISCAS) 2007 for publishing.

Russell A, Orchard G and Etienne-Cummings R, Configuring of Spiking Central Pattern

Generator Networks for Bipedal Walking Using Genetic Algorithms.

Recommendations The methods used to implement the CPGs on the microcontroller should be changed so

that the CPGs can run in real time. This could be achieved in many ways, however, the

first thing to be redesigned should be the filter.

The feet of the RedBot should be altered to provide better traction.

The method used in the automated parameter search should be applied to hardware CPG networks instead of software networks. This would allow the GA to optimise the network

in real time (possibly even faster) allowing for on-line reconfiguration of the network

parameters to adjust for any desired change in gait.

ix

TABLE OF CONTENTS

PLAGIARISM DECLARATION III

ACKNOWLEDGEMENTS IV

SYNOPSIS V

TABLE OF CONTENTS IX

LIST OF FIGURES XIV

LIST OF TABLES XIX

1. INTRODUCTION 1

1.1. BACKGROUND TO STUDY 1 1.2. PROJECT DESCRIPTION 2 1.2.1. OBJECTIVES OF THIS THESIS 2 1.2.2. WORK BREAKDOWN 2 1.3. THESIS OUTLINE 3 1.4. LIMITATIONS 5 1.5. INTRODUCTION TO EQUIPMENT 5 1.6. CONVENTIONS USED 6 1.7. GLOSSARY 7 1.7.1. TERMS RELATED TO GAIT 7 1.7.2. TERMS RELATED TO NEURONS 8 1.8. ACRONYMS 8

2. BACKGROUND TO WALKING ROBOT CONTROL 9

2.1. WALKING ROBOT CLASSIFICATION METHODS 9 2.2. METHODS OF CONTROLLING WALKING ROBOTS 10

x

2.2.1. TRAJECTORY BASED METHODS 10 2.2.2. RULE BASED METHODS 10 2.2.3. BIOLOGICALLY INSPIRED METHODS 10

3. CENTRAL PATTERN GENERATOR LITERATURE REVIEW 11

3.1. INTRODUCTION AND SCOPE OF REVIEW 11 3.2. WHAT IS A CENTRAL PATTERN GENERATOR? 11 3.3. A BRIEF HISTORY OF CENTRAL PATTERN GENERATORS 11 3.4. SUCCESSFUL IMPLEMENTATIONS OF CPGS IN LEGGED ROBOTS 12 3.5. CONSTRUCTION OF CENTRAL PATTERN GENERATORS USING NEURONS 13 3.5.1. SPIKING NEURON MODELS AND THEIR AFFECT ON CPG DESIGN 13 3.5.2. CPG DESIGN USING SPIKING NEURONS 14 3.5.3. CPG DESIGN USING NON-SPIKING NEURON MODELS 16 3.5.4. PROBLEMS WITH CPG DESIGNS USING BI-STABLE OSCILLATORS 16 3.5.5. ALTERNATIVE APPROACH TO CPG DESIGN 16 3.6. ENDOGENOUS OSCILLATOR NEURON APPROACH 16 3.7. CONSTRUCTION OF CPGS USING NON-NEURAL OSCILLATORS 17 3.8. CONCLUSIONS 17

4. BACKGROUND TO GENETIC ALGORITHMS 18

4.1. WHAT IS A GENETIC ALGORITHM? 18 4.2. INCREASING THE RATE OF CONVERGENCE 19 4.3. TYPES OF GENETIC ALGORITHMS 19 4.3.1. NSGA-II 19 4.3.2. FBGAMIN 19 4.4. APPLICATIONS OF GENETIC ALGORITHMS TO CPG NETWORKS 19 4.5. CONCLUSIONS 20

5. THE DEVELOPMENT AND SIMULATION ENVIRONMENT 21

5.1. CONVENTIONS AND ASSUMPTIONS 21 5.2. THE SIMULATION PROCESS AND CALCULATIONS 23

xi

5.3. THE FINAL DEVELOPMENT GUI 25

6. HARDWARE 27

6.1. INTRODUCTION TO THE REDBOT 27 6.2. THE MICROCONTROLLER 28 6.3. REDBOT TO DSPICDEM2 CONNECTION BOARD 29

7. PRELIMINARY METHODS 30

7.1. EXTRACTING HUMAN WALKING DATA FROM GAITLAB 30 7.2. 4TH ORDER RUNGE-KUTTA METHOD 30 7.3. SIMULATION OF A SINGLE NEURON 31

8. PRELIMANRY RESULTS 35

8.1. EXTRACTION OF HUMAN WALKING DATA 35 8.2. 4TH ORDER RUNGE-KUTTA RESULTS 36 8.3. SIMULATION OF A SINGLE NEURON RESULTS 37

9. TRIAL AND ERROR CPG METHODS 39

9.1. CONSTRUCTION OF A HALF-CENTRED OSCILLATOR 39 9.2. REFINING THE NEURON MODEL 40 9.3. USING GENETIC ALGORITHMS TO AID TRIAL AND ERROR 43

10. TRIAL AND ERROR CPG RESULTS 45

10.1. HALF-CENTRED OSCILLATOR RESULTS 45 10.2. REFINING THE NEURON MODEL RESULTS 46 10.3. USING GENETIC ALGORITHMS TO AID TRIAL AND ERROR RESULTS 48

11. FILTER AND SCALING DESIGN METHOD 50

12. FILTER AND SCALING RESULTS 52

xii

13. DIFFERENT CPG NETWORK ARCHITECTURES 54

13.1. CANONICAL NETWORK 55 13.2. DOUBLE BUMP NETWORK 56 13.3. MODIFICATIONS OF NETWORKS TO REDUCE COUPLING WEIGHTS 58

14. C IMPLEMENTATION 61

14.1. BASIC CODE STRUCTURE 61 14.2. REDUCING THE CPG NETWORKS 62 14.3. IMPLEMENTATION OF THE DOUBLE BUMP NETWORK ON THE REDBOT 63

15. EVOLUTION OF CPG PARAMETERS METHOD 66

15.1. AUTOMATING THE SEARCH FOR PARAMETERS 66 15.1.1. STAGE 1: EVOLVING OSCILLATOR PARAMETERS 68 15.1.2. STAGE 2: EVOLVING PHASE SHIFT PARAMETERS 72 15.2. EVOLUTION OF A CONTROLLER TO SIMULATE HUMAN WALKING 77

16. EVOLUTION OF CPG PARAMETER RESULTS 80

16.1. AUTOMATING THE PARAMETER SEARCH: STAGE 1 RESULTS 80 16.2. AUTOMATING THE PARAMETER SEARCH: STAGE 2 RESULTS 82 16.2.1. RESULTS OF A SINGLE PHASE SHIFT AT VARIABLE FREQUENCIES 82 16.2.2. RESULTS OF MULTIPLE PHASE SHIFTS AT A SINGLE FREQUENCY 83 16.2.3 DISCUSSION OF RESULTS 84 16.3. APPLICATION OF GA RESULTS TO CANONICAL NETWORK 84 16.4. EVOLUTION OF A HUMAN CONTROLLER RESULT 86 16.5. IMPLEMENTATION OF THE EVOLVED CPG ON THE REDBOT 90

17. CONCLUSIONS AND RECOMMENDATIONS 93

17.1. CONCLUSIONS 93 17.2. RECOMMENDATIONS 94

xiii

18. REFERENCES 96

19. BIBLIOGRAPHY 100

20. APPENDICES 101

20.1. APPENDIX 1 101 20.2. APPENDIX 2 102 20.3. APPENDIX 3 104 20.4. APPENDIX 4 107 20.5. APPENDIX 5 108 20.6. APPENDIX 6 (SEE ATTACHED CD) 109 20.7. APPENDIX 7 109

xiv

LIST OF FIGURES Figure 1 The different planes of the human body Figure 2 Comparison of Neuron Model Implementation Cost and Biological Plausibility Figure 3 A half-centred oscillator Figure 4 Canonical network design (Lewis et al, 2005) Figure 5 Naming conventions for RedBot simulation Figure 6 RedBot weights and lengths used in the simulation Figure 7 Torque calculation flow diagram Figure 8 The development and simulation GUI Figure 9 The RedBot Figure 10 Algorithm for CPGFRAMEWORK1 Figure 11 Algorithm for the IFA function Figure 12 Hip and Knee Angle Trajectories during normal human walking Figure 13 Runge-Kutta results vs. actual solution Figure 14 Results of Neuron Simulation experiment to illustrate tonic spiking Figure 15 Results of Neuron Simulation Experiment Showing the Neurons Spike

Frequency Adaptation Characteristic

Figure 16 the half-centred oscillator layout Figure 17 Algorithm for CPGFRAMEWORK2 Figure 18 Membrane Potentials of a half-centred oscillator with an initial frequency of

0.5Hz

Figure 19 Firing patterns of a half-centred oscillator with a starting frequency of 0.5Hz Figure 20 Potassium concentrations of a half-centred oscillator with a starting frequency

of 0.5Hz

Figure 21 Membrane potential of a half-centred oscillator, using IF&A2 neurons, at 0.5Hz

Figure 22 Firing patterns of a half-centred oscillator, using IF&A2 neurons, at 0.5Hz Figure 23 Potassium concentrations of a half-centred oscillator, using IFA2 neurons, at

0.5Hz

Figure 24 Results of NSGA-II first trial

xv

Figure 25 Results of NSGA-II second trial Figure 26 Results of NSGA-II third trial Figure 27 Results of NSGA-II fourth trial Figure 28 Half-centred oscillator at 1.5Hz using parameters found through NSGAII and

trial and error

Figure 29 algorithm for LPF (Low Pass Filter) Figure 30 Input spike train to filter Figure 31 Power Spectral Density of input signal Figure 32 scaled filtered signal with fc1=15Hz, fc2=1Hz, A = 0 and B = 0 Figure 33 Enlarged view of ripple and sampling on scaled signal Figure 34 scaled filtered signal with fc1=15Hz, fc2=1Hz, A = 140 and B =15 Figure 35 Canonical Network with summing nodes and filtering Figure 36 Joint angle trajectories from Canonical network Figure 37 Double Bump Network with summing nodes and filtering Figure 38 Joint Angle profiles from the Double Bump Network Figure 39 Canonical Network 2 Figure 40 Double Bump Network 2 Figure 41 Joint angle profiles generated from Double Bump Network 2 Figure 42 MAIN algorithm Figure 43 reduced CPG network architecture Figure 44 RedBot walking at 0.75Hz (1.5Hz desired) using Double Bump 2 Network Figure 45 Joint angle profiles obtained whilst the RedBot was walking Figure 46 CPG network layout used in the automated search for parameters Figure 47 Algorithm for automated parameter search to cause oscillations of CPG1 and

CPG2 at different frequencies

Figure 48 Overview of OSCILLATIONS Figure 49 Detailed algorithm of OSCILLATIONS Figure 50 Algorithm for automated search for parameters to cause a single phase shift

between CPG1 and CPG3 at different frequencies

Figure 51 Algorithm for automation of parameter search to cause different phase shifts between CPG1 and CPG3 at a single frequency.

xvi

Figure 52 Overview of PHASES Figure 53 A detailed algorithm of PHASES Figure 54 Initial neural network layout used to evolve both the hip and knee controllers Figure 55 Algorithm to evolve human hip or knee controller Figure 56 Actual frequencies of oscillation vs. desired frequencies of oscillation using

parameters found by FBGAMIN

Figure 57 Absolute values of parameters found by FBGAMIN vs. the frequency of oscillation they cause

Figure 58 Number of generations FBGAMIN required to find the parameters to cause oscillations at each frequency

Figure 59 Actual phase shift vs. desired 90 phase shift at different frequencies using parameters found by FBGAMIN

Figure 60 Actual phase shift vs. desired 45 phase shift at different frequencies using parameters found by FBGAMIN

Figure 61 Absolute values of parameters found by FBGAMIN to cause a phase shift of 90 at different frequencies of oscillation

Figure 62 Absolute values of parameters found by FBGAMIN to cause a phase shift of 45 at different frequencies of oscillation

Figure 63 Number of generations FBGAMIN required to find the parameters to cause a phase shift of 90 at each frequency.

Figure 64 Number of generations FBGAMIN needed to find the parameters to cause a phase shift of 45 at each frequency.

Figure 65 Actual phase shifts vs. desired phase shifts at oscillations of 1Hz using parameters found by FBGAMIN

Figure 66 Actual phase shifts vs. desired phase shifts at oscillations of 2Hz using parameters found by FBGAMIN

Figure 67 Absolute value of parameters found using FBGAMIN vs. the phase shifts they cause at 1Hz

Figure 68 Absolute value of parameters found using FBGAMIN vs. the phase shifts they cause at 2Hz

Figure 69 Number of generations FBGAMIN required to find the parameters to cause a particular phase shift at 1Hz

xvii

Figure 70 Number of generations FBGAMIN required to find the parameters to cause a particular phase shift at 2Hz

Figure 71 Hip and knee angles for 0.5Hz walking Figure 72 Hip and knee angle for 1Hz walking Figure 73 Hip and knee angles for 1.5Hz walking Figure 74 Hip and knee angles for 2Hz walking Figure 75 Hip and knee angles for 2.5Hz walking Figure 76 Hip and knee angles for 3Hz walking Figure 77 Neuron network to simulate human hip trajectory Figure 78 Neuron network to simulate human knee trajectory Figure 79 Hip simulation results Figure 80 Knee simulation results Figure 81 Spike trains of the surviving neurons for the hip controller Figure 82 The weighted sum of CPG3s, CPG5s and CPG8s outputs Figure 83 Spike trains of surviving neurons for the knee controller Figure 84 The weighted sum of CPG3s, CPG6s, CPG7s and CPG8s outputs Figure 85 RedBot walking using evolved controller Figure 86 Recorded hip and knee joint angle profiles during walking Figure 87 Actual phase shift vs. desired 90phase shift at different frequencies using

parameters found by FBGAMIN, with I2-3 scaled by 1000

Figure 88 Number of generations FBGAMIN needed to find parameters causing a phase shift of 90 at each frequency, where I2-3 was scaled by 1000

Figure 89 Actual phase shift vs. desired 45phase shift at different frequencies using parameters found by FBGAMIN, with I2-3 scaled by 1000

Figure 90 Number of generations FBGAMIN needed to find parameters causing a phase shift of 45 at each frequency, where I2-3 was scaled by 1000

Figure 91 Actual phase shifts vs. desired phase shifts at oscillations of 1Hz using parameters found by FBGAMIN, where I2-3 was scaled by 1000

Figure 92 Number of generations FBGAMIN required to find the parameters to cause a particular phase shift at 1Hz, where I2-3 was scaled by 1000

Figure 93 Actual phase shifts vs. desired phase shifts at oscillations of 2Hz using parameters found by FBGAMIN, where I2-3 was scaled by 1000

xviii

Figure 94 Number of generations FBGAMIN required to find parameters to cause a particular phase shift at 2Hz, where I2-3 was scaled by 1000

xix

LIST OF TABLES Table 1 Joint Servo Motor Names and Characteristics Table 2 Requirements for microcontroller used to control the RedBot Table 3 dsPIC30F4011 pin layouts Table 4 Parameters to test the IFA2 neuron model Table 5 Maximum and minimum parameter values for nsga_2 Function Table 6 Parameters to cause oscillations of a half-centred oscillator at a steady state

frequency of 0.63Hz

Table 7 parameters found using NSGAII for oscillations at 1.5Hz Table 8 parameters for Canonical network to generate walking patterns at 1.5Hz Table 9 parameters to cause 1.5Hz oscillations and 90 shifts in Canonical Network 2 and

Double Bump Network 2

Table 10 Trial solution for 0.5Hz oscillations Table 11 showing the neuron parameters to illustrate tonic spiking and spike frequency

adaptation

Table 12 parameters to cause oscillations of CPG1 and CPG2 at different frequencies Table 13 parameters to cause 45 and 90 phase shifts between CPG1 and CPG3 at

frequencies ranging from 0.5Hz to 3.5Hz

Table 14 parameters to cause phase shifts of 90 and 45 for different frequencies of oscillation

Table 15 Filter parameters used in applying the parameters found by FBGAMIN to Lewis et als Canonical Network

Table 16 Trial and final solutions for the evolution of hip and knee controllers

1

1. INTRODUCTION 1.1. Background to study In the near future it is envisaged that robots will be a part of our everyday lives. Their

involvement will include helping with everyday chores, caring for the sick or elderly, and

performing hazardous tasks.

Currently, robots, such as autonomous vacuum cleaners, already exist to perform some of these

functions. However, these robots are generally wheel based and are thus limited in the type of

terrain they can navigate. Walking robots, being able to traverse varying terrains, would be far

more effective in performing these tasks. However, walking robots are much more complicated

than wheeled ones and much research is required before they can be produced for mass use.

The research around bipedal walking also has application in the biomedical field. Spinal cord

injuries are often debilitating as lesions to the spinal cord disrupt communications between the

brain and the central and peripheral nervous systems below the lesion, leaving the affected

individual paralysed. This paralysis may be partial paralysis of a limb or in the worst case all

motor control below the lesion may be destroyed. The damaged nerve cells do not grow back,

disabling the affected individual for life. Stem cell research shows promising results in repairing

these damaged cells but the research is a long way away from being feasible. In addition to this,

ethical issues with this research have resulted in it being illegal in many countries.

Mushahwar et al (2000) showed that by microstimulating the spinal cord of cats it is possible to

activate certain muscles in the cats legs, and that if the microstimulation was applied to specific

areas, coordinated limb movements could be generated. With further research, it may be possible

to activate the CPGs of humans using microstimulation, or to use an artificial CPG to activate the

necessary muscles and thus artificially control human walking. For this to happen it is crucial to

have a firm understanding of how Central Pattern Generators work, and to have thoroughly tested

and researched them on robots, before testing is done on humans.

Furthermore, by building the CPG using spiking neurons the CPG may be more easily interfaced

with biological networks, which also operate using spiking neurons.

2

1.2. Project description This project was to be completed by a team of two. The aim of the project was to implement two

different walking algorithms (a Central Pattern Generator algorithm and an If-Then Rules

algorithm) on the RedBot biped robot. The Central Pattern Generator algorithm was to use

spiking neurons as its building blocks. Each member of the team was to be responsible for

implementing a different algorithm.

The implementation process was to include the design of the walking algorithms, simulating the

RedBot in MATLAB, and building the necessary hardware to interface the RedBot to the host PC

and to control the RedBots servo motors.

The following specifications were given for the project

Control of the RedBot must be achieved using a PIC microcontroller to interface the PC to the RedBot.

The final deliverables were to be a written report and evidence (e.g. video) of the RedBot walking using the designed controller.

Work on this thesis was to start on 31 July 2006 and the final report and any evidence of the RedBot walking were to be submitted on 23 October 2006.

1.2.1. Objectives of this thesis

The author chose to implement the Central Pattern Generator based algorithm and thus the

objective of this thesis was to build a software implemented Central Pattern Generator, using

spiking neurons to control the walking of the RedBot biped robot.

1.2.2. Work breakdown

As this was a joint thesis, certain sections of it were conducted together with Garrick Orchard, the

authors thesis partner. These sections included:

The initial research into the two different walking algorithms Sourcing the necessary hardware components Simulating the RedBot in MATLAB

Garrick Orchard was responsible for setting up the skeleton code for the PIC microcontroller and

the author was responsible for building the connection board to supply the RedBot with power

3

and to interface the RedBot to the PIC microcontroller. Garrick orchard was, however,

responsible for adding the necessary hardware to the connection board to allow for foot and shin

switches to be installed on the RedBot.

1.3. Thesis outline Chapter 2 provides an introduction to walking robot control including different methods of

classifying walking robots and the methods used to control them.

Chapter 3 is the Central Pattern Generator literature review. It provides a brief history of Central

Pattern Generators as well as providing insights into the different methods of implementing

CPGs. The focus of this chapter, however, is on central pattern generators using spiking neurons.

Chapter 4 is the Genetic Algorithm literature review. It provides a very brief introduction into

how Genetic Algorithms work and the terminology associated with them. Examples of previous

applications of Genetic Algorithms to CPG design have been included.

Chapter 5 gives the details of the simulation of the RedBot in MATLAB.

Chapter 6 provides a short description of the different hardware elements used in this thesis.

Chapter 7 describes the methods of the preliminary tasks that had to be performed before the

design of the Central Pattern Generator could commence. This includes giving a detailed

algorithm for the MATLAB code used to simulate a network of n neurons.

Chapter 8 describes the results of the preliminary methods.

Chapter 9 describes how trial and error can be applied to find parameters for a CPG. It also

explains how the Integrate and Fire with Spike Frequency Adaptation neuron model was adapted

to ease the search for parameters.

Chapter 10 gives and discusses the results of a trial and error approach to the design of a Central

Pattern Generator.

4

Chapter 11 shows the details of the digital filter used to filter the resultant spike trains from the

Central Pattern Generator and how the filtered signals are scaled to lie between the desired

angles. It discusses the method used to find filter parameters for different spike trains.

Chapter 12 gives results of the filtering and scaling of a specific spike train and shows how the

method discussed in Chapter 11 is used to find parameters.

Chapter 13 describes different CPG architectures. The CPG network proposed by Lewis et al

(2005) is first discussed and then adaptations of this network, designed by the author, are given.

The joint angle profiles that result from these different networks are shown.

Chapter 14 gives a brief overview of how the CPG code was implemented on the dsPIC30F4011

microcontroller. It also shows the results of implementing the Double-Bump 2 Network

(discussed in Chapter 13) on the RedBot.

Chapter 15 provides a method, to automate the search for parameters, to cause different

frequencies of oscillation of a CPG and phase shifts between the neurons within the CPG. This

chapter also provides the method followed to evolve a CPG controller that mimics human hip and

knee joint angle profiles.

Chapter 16 gives and discusses the results for the automated parameter search. It shows the

results of applying these parameters to Lewis et als Canonical network. The results of the

evolution of a controller are presented and the results of implementing this controller on the

RedBot are shown and discussed.

Chapter 17 This chapter gives conclusions on the thesis and provides recommendations for

further research.

5

1.4. Limitations The following limitations were encountered during this thesis:

Only 12 weeks were allowed for this project and hence time played a critical factor. The computers available for this thesis only had 256MB of RAM. This prevented the

Central Pattern Generator algorithms from being run on the computers in real time.

Only versions 19 or higher of the MPLAB ICD2 (The debugger/programmer module for the PIC) are compatible with the dsPIC30F series of microcontroller. This is not

documented in the Microchip documentation. Initially an incompatible version of the

ICD2 was being used with the dsPIC30F4011 microcontroller, however, as there was no

documentation explaining this incompatibility problem, it was a time consuming process

to discover it.

The ICD2 is not robust and the Microchip documentation regarding the order in which power is to be supplied to the ICD2 and target PIC is incorrect. The documentation states

that power should first be applied to the ICD2 before it is applied to the target PIC,

however, this permanently damages the ICD2. Power should actually be applied to the

microcontroller first. As a result of the error in the documentation two ICD2s were

damaged and as there are only a limited number of ICD2s available for use at UCT,

delays were encountered whilst waiting for new ICD2s.

Replacement servo motors for the actuation of the RedBot joints are not locally available in Cape Town. One of the gears in the Futaba s4092 servo motor used to control the

RedBots right hip broke. The RedBot was unusable until a new servo motor could be

sourced from the United States of America.

1.5. Introduction to equipment The following hardware was used in this thesis:

A 2.4Ghz Pentium 4 with 256MB RAM PC was used for all simulations and testing of algorithms

A 1.7Ghz Celeron with 256MB RAM PC was used to program and interface with the dsPIC30F4011 microcontroller

A MPLAB ICD2 programmer and debugger module A dsPIC30F4011 microcontroller

6

A dsPICDEM2 development board was used. It contains all the peripheral circuitry needed by the dsPIC30F4011 microcontroller to run and to interface with the host PC.

A RedBot to dsPICDEM2 connection board A RedBot biped robot

The RedBot, dsPIC30F4011 and connection board are discussed in more depth in Chapter 6.

The following software was used in this thesis

MATLAB 7 Release 14 was used to develop the walking algorithms, simulate the RedBot and communicate with the PIC microcontroller.

MPLAB IDE 7.41 is the development environment for the PIC microcontroller and provides support for the MPLAB ICD2 debugger/programmer. All C code was written in

this environment.

MPLAB C30 Compiler (Student edition) is the C compiler for the dsPIC range of Microchip products.

Microsoft Windows Hyper Terminal was used to display data sent from the PIC to the PC.

Gaitlab is gait analysis software designed for use on a home PC. Gaitlab provides examples and analysis of different gaits, where each gait was captured using video-based

kinematics methods. Gaitlab allows the user to extract information such as the joint angle

profiles from this captured data.

1.6. Conventions used The following conventions have been used:

Names of functions written for use in MATLAB or C are in uppercase italics. Variable names are in italics. The term spiking is analogous to a neuron firing. E.g. if a neuron spikes at 1s intervals

then the neuron fires at 1s intervals

Extension of the hip was defined to be a movement of the hip to move the thigh towards the front of the RedBot

Flexion of the hip was defined to be a movement of the hip to move the thigh towards the back of the RedBot.

Extension of the knee was defined to be a straightening movement of the knee.

7

Flexion of the knee was defined to be a bending movement of the knee. In the figures describing the neural network architectures, there is at least a 180 phase

difference between the firing patterns of the white coloured neurons and the gold

coloured neurons. The blue coloured neurons denote that the amount of phase shift of the

blue neurons firing pattern from a white or gold neurons firing pattern is unknown. If

the neuron is green then the neuron fires for more than 50% of the oscillatory period.

It must be noted that while attempts were made to make the Central Pattern Generator

biologically compatible through the use of spiking neurons, no attempt was made to make the

system biologically plausible. The focus was instead on what the resulting neuron spike trains

looked like and whether or not they were useful for generating joint angle profiles for walking.

As a result of this, no restrictions were placed on the parameters to cause the neuron membrane

potentials to lie within any particular range and no scale was set for the values of the neuron

parameters (be it a voltage, current or impedance). E.g. if a parameter was a voltage it was never

specified whether the units of this voltage were megavolts, volts or millivolts. Likewise, for those

parameters that are currents, it was never specified whether the units of current were mega-amps,

amps or milliamps. Consequently it does not make sense to put units on the graphs and tables

showing these parameters as the units are meaningless. However, when the neuron model that is

used is discussed, the parameter types (e.g. voltage, current or impedance) are specified.

It was, however, decided that the spike trains that emerge from the neurons should have the units

of volts, and that a value of 1v would represent the neuron firing and a value of 0v would

represent the neuron not firing.

1.7. Glossary

1.7.1. Terms related to gait

The foot that is in contact with the ground is called the stance foot. The leg whose foot is the stance foot is called the stance leg. The foot that is not in contact with the ground is called the swing foot. The leg whose foot is the swing foot is called the swing leg.

8

Figure 1 The different planes of the human body

(Inman et al, 1981)

1.7.2. Terms related to neurons

The input current to a neuron is called the tonic input. When a neuron has an applied tonic input it will fire a train of spikes. This behaviour is

known as tonic spiking.

The firing rate of tonic spiking of certain neurons decreases over time. This is known as spike frequency adaptation

1.8. Acronyms ADC - Analogue to Digital Converter

COG - Centre of Gravity

COM - Centre of Mass

CPG - Central Pattern Generator

DSP - Digital Signal Processing

GA Genetic Algorithm

IO - Input Output

PIC - Generic name for microcontrollers manufactured by Microchip

PWM Pulse Width Modulation

UART - Universal Asynchronous Receiver Transmitter

UCT University of Cape Town

ZMP - Zero Moment Point

Transverse plane

Sagittal plane Frontal plane

9

2. BACKGROUND TO WALKING ROBOT CONTROL

2.1. Walking robot classification methods There are many methods to classify walking robots, the most obvious being by the number of legs

the robot has. Other methods include classifying according to the type of joint actuation used or

according to the position of the Centre of Mass (COM) whilst walking.

There are three types of actuated robots:

Passive walkers have no joint actuation and rely on gravitational forces to drive the robot. As a result they are only able to walk down inclined surfaces. These robots are

very efficient as they require no input power to cause walking. Mcgeer (1990) showed

that it was possible to realise stable walking motion through mechanical design only (i.e.

without control input).

Ballistic walkers use joint actuation only during the stance phase of the walking cycle. The walker uses gravity and inertia to provide the motion of the leg in the swing phase.

Powered walkers use actuation during both the swing and stance phases.

Classification according to the robots position of the Centre of Mass (COM) results in two

classes:

Static walkers ensure that their COM is within the support area of the walkers feet at all times. As a result they are stable at any position of their feet and at all points during their

walking cycle. However, in order to achieve this stability, the speed and step size of the

walker is limited.

Dynamic walkers use velocity or inertia to achieve stability. The COM is not constrained to lie within the support are of the feet and as a result the walker is able to

move at higher speeds and with larger step sizes.

The robot used in this thesis falls into the powered, dynamic walker class of robots.

10

2.2. Methods of controlling walking robots

2.2.1. Trajectory based methods

These methods use mathematical equations to calculate the position of each limb and the angle of

each joint of the robot at every moment. There are two main methods based on trajectory:

Zero Moment Point (ZMP) methods specify the movement of the robot such that the sum of the moments from the active forces sum to zero at a particular point on the ground

(the Zero Moment Point). If the ZMP falls within the area generated by all possible

contact points between the feet and the ground then it is possible for a biped robot to

walk (Huang et al, 2001). Kanniah et al (2004) used the ZMP method to achieve stable

walking by calculating the compensatory motion required by the hips for a given swing

leg trajectory. Kajita et al (2003) specify the placement of the feet and trajectory of the

COM such that the ZMP always coincides with the stance foot.

Inverse Kinematics involves the computation of the required joint angles from pre-defined joint trajectories

2.2.2. Rule based methods

Rule base methods use predefined rules to determine the next action to be taken by the robot,

given its current state (obtained through sensory feedback) and objective. The rule based method

for walking is commonly called the If-Then rules approach.

In some circumstances it is hard to define an exact line between the different states and in this

case fuzzy logic can be used to determine which rule should be applied (Prochazka, 1996)

2.2.3. Biologically inspired methods

These methods are based on biological principles such as the Central Pattern Generator (CPG).

By basing the control of movement on biological principles it may make the robot more

compatible with other biologically inspired aspects of intelligence (Lewis et al, 2005). Central

pattern generators are discussed in depth in the next chapter.

11

3. CENTRAL PATTERN GENERATOR LITERATURE REVIEW

3.1. Introduction and scope of review Central Pattern Generators have successfully been used in controlling a number of different

robots. These CPGs have been implemented either through software or hardware. Both

implementations share many characteristics and thus, although this project involves a software

implemented CPG, papers concerning both implementations have been researched.

3.2. What is a Central Pattern Generator? A Central pattern generator (CPG) is a collection of neurons in the spinal chord that are able to

endogenously (i.e. without external input) produce sustained rhythmic or oscillatory, patterned

outputs (Hooper, 2000). The outputs of central pattern generators are responsible for most

rhythmic motor patterns (Marder and Calabrese, 1996; Stein et al, 1997) such as walking, flying,

swimming and breathing.

Central Pattern Generators are more than just rhythm control centres responsible for creating

rhythmic, patterned motor control signals. Central Pattern Generators help coordinate the brain

and the spinal chord by providing phase information to the cerebellum (Cohen and Boothe, 1999).

They are also involved in the modulation of sensory information from the periphery (Arshavsky

et al, 1983), as well as providing the additional state variable over sensory information which is

necessary to understand occurrences such as reflex reversal (Forssberg et al, 1975).

3.3. A brief history of Central Pattern Generators Towards the end of the 19th century scientists started studying the neural mechanics responsible

for locomotion and the first evidence of a central pattern generator was discovered at the

beginning of the 20th century by T. G. Brown. He was experimenting with dogs and discovered

that locomotor-like patterns persisted after sensory input was destroyed by extensive surgical

transection of sensory roots entering the spinal chord. Brown concluded the reason for this was an

intrinsic factor within the spinal cord, which autonomously produced the rhythm and that the

role of sensory input was to control this intrinsic factor to manage variations and abnormalities

in the external environment. After 1920 there was a lull in the research of complex movement

12

control; however a renewed interest in this field occurred in the 1960s and the intrinsic factor

described by Brown was named the central pattern generator or CPG (Prochazka, 2002).

Wilson (1961) demonstrated that when the locust nervous system is removed from the animal, it

is able to produce a rhythmic output similar to that observed during flight, and thus provided the

first modern evidence that rhythmic motor patterns are centrally generated. Additional research

has shown similar results in a wide variety of animals, further demonstrating that rhythmic

pattern generation is not dependant on the nervous system acting as a whole, but rather on small

autonomous systems of neural networks (Hooper, 2000). Various studies have provided evidence

of the existence of a central pattern generator in humans (Bussel et al, 1996; Duysens, Van der

Crommert, 1998; Dimitijevic et al, 1998).

3.4. Successful implementations of CPGs in legged robots Central Pattern Generators have successfully been implemented, in both hardware and software,

to control a variety of legged robots. Inagaki et al. (2006) used a software implemented wave

Central Pattern Generator model to provide decentralized control of gait generation and walking

speed of a hexapod robot. Hata et al (2006) used a pulse-type hardware model to successfully

control a quadruped robot, achieving the following gaits: walk, pace, trot, bound and gallop. Pinto

and Golubitsky (2006) used a software implementation of a Central Pattern Generator to achieve

the five gaits described above, as well as eight additional gaits. Pinto and Golubitsky (2006) also

used the software implementation of a Central Pattern Generator for biped gait control. They

achieved the following gaits: walk, run, two-legged hop, two-legged jump, skip, gallop,

asymmetric hop and one-legged hop. Tenore et al (2005) used a hardware implemented Central

Pattern Generator to control the RedBot biped robot.

The above references are not meant to be a listing of the only successful implementations of

Central Pattern Generators in robot control, instead they are meant to show the variety of legged

robots they have been controlled using central pattern generators.

13

3.5. Construction of Central Pattern Generators using neurons

3.5.1. Spiking neuron models and their affect on CPG design

There are many different spiking neuron models available ranging from the extremely accurate

and biologically plausible Hodgkin-Huxley model to the very basic leaky integrate and fire (IF)

model. Izhikevich (2004) provides a detailed discussion of the different modes of spiking neurons

and discusses the neuron models available. He shows that with an increase in biological accuracy

comes an increase in the computational time needed to process the neuron model. Figure 2 shows

the available spiking neuron models and their computation times when computed using a fixed,

single-step Euler integration method.

Figure 2 Comparison of Neuron Model Implementation Cost and Biological Plausibility

(Izhikevich , 2004)

Having such a wide range of models leads itself to two approaches when designing a CPG. One

can use as biologically plausible a model as possible and discover what functionality and control

comes with such a model (examples of this approach include Rasche et al (1998) using the

Hodgkin-Huxley model and Simoni and DeWeerth (1999) using the Morris-Lecar model) or one

can use as simple a model as possible and design the extra functionality into the system as needed

(An example of this approach is of Lewis et al (2005) using Integrate and Fire with Spike-

Frequency Adaptation neurons).

The advantages of using the second approach are twofold. Firstly, each neuron model is based on

a differential equation with multiple parameters. As the complexity of the model and the network

14

increases so does the parameter count making it more and more difficult to solve the required

parameters. Secondly, if the CPG is software based and is to be implemented with limited

processing power, having a computationally efficient neuron model means that the equations

governing the central pattern generator can be solved faster allowing more time for other

processing applications or for the processing of a more complicated network.

3.5.2. CPG design using spiking neurons

The simplest CPG that can be built using spiking neurons is the half-centred oscillator (Tenore et

al, 2005), which consists of two neurons driven by the same tonic input and mutually coupled

through inhibition. By themselves these neurons have no rhythmogenic abilities, but when

coupled, they are capable of producing rhythmic outputs. If the inhibition between the two

neurons is equal (i.e. the inhibition signal from neuron one to neuron two is equal to the inhibition

signal from neuron two to neuron one) it is possible to cause a 180 phase difference between

neural outputs, with each neuron firing for 50% of the period. If unbalanced inhibition coupling

strengths are used, it is possible to cause different phase differences and one neuron will fire for a

longer portion of the period than the other neuron.

Figure 3 A half-centred oscillator

Figure 3 shows the layout of a typical half-centred oscillator. Neurons CPG1 and CPG2 have the

same parameters with the same constant tonic input and are reciprocally coupled through

inhibition; that is when one neuron fires, it inhibits the other neuron causing a negative

hyperpolarisation of its membrane potential. Assume that one neuron, say CPG1, will reach its

threshold voltage first. CPG1 will fire, inhibiting CPG2 and causing a negative hyperpolarization

I1-2 = Inhibitory coupling from CPG1 to CPG2

I2-1 = Inhibitory coupling from CPG2 to CPG1

I1-2 I2-1

CPG1

CPG2

15

of CPG2s membrane potential Both CPG1 and CPG2 will start charging towards their

thresholds, however, CPG1 will reach its threshold first as CPG2 has to recover from its

inhibition. CPG1 will once again inhibit CPG2. This process would continue forever if there was

no spike-frequency adaptation. However, if the neuron model used features spike-frequency

adaptation, eventually the firing rate of CPG1 will decrease such that CPG2 has sufficient time to

charge to its threshold voltage before the next inhibition signal from CPG1. When CPG2 reaches

its threshold voltage an inhibition signal is sent to CPG1 and the cycle repeats itself.

More complex CPGs can be built up by using half-centred oscillators as their building blocks.

This is based on the concept of using one half-centred oscillator per limb or segment to be

controlled and then coupling the different oscillators together to introduce the required phase

shifting between the oscillators.

Lewis et al (2005) used 2 half-centred oscillators coupled together to produce the basic pattern

responsible for coordinating locomotion of a biped robot. This network was then coupled, via

inhibition, to an eight neuron network responsible for driving the patterned movement of the

bipeds hips and knees. Walking patterns were generated by using the following phase

relationships. The left and right hips were 180 out of phase with each other. The hip flexor and

extensor neurons were 180 out of phase with each other and the knee flexor and extensor

neurons were 90 out of phase with the hips flexor and extensor neurons. Lewis et al called this

network the Canonical network and it is shown in Figure 4.

Figure 4 Canonical network design (Lewis et al, 2005)

16

3.5.3. CPG design using non-spiking neuron models

An alternative to CPG design using spiking-neuron models, is to use models that approximate the

mean firing rate of neurons, such as the Matsuoka neuron model. In a network of n Matsuoka

type neurons the ith neuron is described as follows

Where, ui is the membrane potential of the ith neuron, vi is used to describe the degree of

adaptation, Tu, Tv and are used to specify the time constant for the adaptation, wij describes the

strength of the inhibitory connection between the neurons and u0 is a constant input signal (Inada

and Ishii, 2004).

3.5.4. Problems with CPG designs using bi-stable oscillators

Marder et al (1997) have shown that CPG networks constructed using bi-stable oscillators, of

which half-centred oscillators are an example of, do not guarantee oscillations and may produce

synchrony.

3.5.5. Alternative approach to CPG design

Eliasmith and Anderson (2000) used an alternative approach in the construction of a Central

Pattern Generator to model Lamprey swimming. This approach avoided using bi-stable oscillators

and thus the problems associated with them. They applied an iterative five step process that

involved creating a system of dynamical differential equations defining the forces to be generated

by the neural system and then redefining the system until it was defined in the space of neural

activities. The resulting network avoided the problems described by Marder et al (1997), The

method proposed by Eliasmith and Anderson (2000), however, has not been tested in designing

controllers for more complicated movements such as bipedal walking.

3.6. Endogenous oscillator neuron approach Hooper (2000) briefly mentions that an alternative approach to creating rhythmic neural output

through a network of interconnecting neurons is to use endogenous oscillator neurons, however,

no further reading on this concept could be found.

),0max(,

10

iiiiiv

n

jjijii

iu

uyyvdtdvT

uywvudtduT

==

+= =

17

3.7. Construction of CPGs using non-neural oscillators Neuron based oscillators are not the only means of constructing central pattern generators. Other

implementations include the use of Phase Oscillators (Cohen et al, 1982; Lewis and Bekey, 2002)

or Van der Pol oscillators. Using these oscillators has the advantage in that they are more open to

mathematical analysis than neuron oscillators. The Van der Pol equation describes self-sustaining

oscillations and has application in the analysis of circuits containing vacuum tubes (Weisstein,

1999 from MathWorld). It is defined as follows

0)1( 2 =+ xxxx &&&

3.8. Conclusions The following conclusions were drawn from the literature survey:

Simple neuron models are well suited to the design of CPGs to be implemented with limited processing power.

The Integrate and Fire with Spike Frequency Adaptation is most suited for use in this thesis as it is computationally efficient, yet its spike-frequency adaptation characteristic allows it to

form a half-centred oscillator and hence a CPG.

It is possible to use a simple network like the Canonical Network shown in Figure 4 to create the joint angle profiles necessary to realise bipedal walking.

18

4. BACKGROUND TO GENETIC ALGORITHMS 4.1. What is a Genetic Algorithm? A Genetic Algorithm (GA) is an adaptive stochastic optimisation algorithm involving search and

optimisation that is based on the ideas of evolutionary biology and genetics. These ideas include:

natural selection, inheritance, mutation and crossover. Genetic algorithms were first used by

Holland in 1975 and provide better performance than typical optimisation methods (e.g. linear

programming) in searching large parameter spaces or with fitness functions that are not smooth.

Genetic algorithms use randomization to search very large parameter spaces; however, the

searching done by the algorithm is by no means random in its method. GAs use past

performances to direct the search towards better performing areas of the parameter space, and in

doing so are able to search enormous solution spaces effectively

The algorithm starts by creating a random population of solutions for the problem, however, if an

area in the parameter space is thought to give optimal solutions then the initial population may be

seeded with solutions in that area. Seeding of the initial population may help the algorithm

converge on a final solution in fewer generations than if the initial population was completely

random. Each possible solution in the population is called a chromosome and each variable in the

chromosome is called a gene.

The chromosomes are then trialled on the problem to be solved and their performances are

evaluated using a fitness function. A second generation of solutions is then evolved from the

initial populations best solutions. The evolution of the next generation involves the breeding of

the best parent solutions by cross-over (gene swapping), and random mutation. Some algorithms

use an elitist factor to ensure that the best solutions from the parent generation propagate

undisturbed into the next generation. As the population size for the GA is constant, each new

generation is constructed by weeding out the poorest solutions, of the previous generation, to

make room for the new solutions made from the stronger parents. This is analogous to survival of

the fittest in nature.

19

4.2. Increasing the rate of convergence Speed of convergence of the algorithm is a critical factor when using GAs. De Garis (1990)

showed that the evolution of a controller can be affected by the starting parameters of the search

and that it is possible to reduce the speed of the search by using two fitness functions. The first

fitness function is more easily satisfied and is used to pre-evolve the solution to an appropriate

level. Once the solution reaches this level, the second fitness function is used to further improve

the solution. De Garis named the above concept Behavioural Memory.

4.3. Types of Genetic Algorithms There are numerous variations and types of genetic algorithms; however, discussing them all is

outside the scope of this thesis. The two algorithms that were used in this thesis are briefly

discussed.

4.3.1. NSGA-II

This is a multi-goal optimisation method that creates a Pareto front for the objectives. NSGA-II

was developed at the Kanpur Genetic Algorithm Laboratory and aims to minimise the fitness

function.

4.3.2. FBGAMIN

FBGAMIN is an Adaptive Mutation Breeder Algorithm (AMBA), used to optimise single goal

functions. Breeder algorithms are well suited for the optimization of real-valued function inputs,

and the adaptive mutation characteristic of the algorithm utilises a simple feedback loop within

the algorithm to ensure that the mutation rate is always at its optimum. FBGAMIN was developed

at the University of Cape Town and aims to minimise the fitness function. It uses elitism to

ensure that the best solution propagates, unchanged, to the next generation.

4.4. Applications of Genetic Algorithms to CPG networks Lewis et al (1992) used genetic algorithms to optimise the weights in a network of neurons to

control a 12 degree of freedom hexapod robot. In order to improve the convergence rate of the

GA staged evolution was used. The network consisted of 1 oscillator per leg, with two neurons

per oscillator. One neuron controlled the swing of the leg and the other the elevation of the leg.

The first stage of evolution was to evolve the parameters necessary to create oscillations in a

single two neuron network and hence a stepping motion of the leg. Once the oscillator reached

20

the desired criterion, the optimised parameters were applied to all oscillators in the network. The

second stage of evolution involved finding the connections between the oscillators, so as to

provide coordination of the legs and the generation of a successful walking pattern. The fitness

function used for the second stage of evolution rewarded both forward and backward movement

in straight line. Lewis managed to evolve both the tripod and wave gaits for the robot; however, if

further generations were computed the tripod gait always dominated the wave gait as it was the

most efficient gait for the robot.

Wiley used genetic algorithms to first evolve standing and then walking patterns for a simulated

quadruped. A reduced CPG model using square wave oscillators was used as the input to four

distinct neural networks, where each network was responsible for controlling one leg. The

frequency of the oscillator signals and the weights of the neural network were then solved using a

genetic algorithm. The fitness function was proportional to the distance moved by the quadruped

in a set direction during a specified time period. The terrain and ground friction were altered for

each experiment, and controllers capable of walking were evolved specific to each situation.

Inada and Ishii (2004) used the Matsuoka neuron model to build a CPG to control a simulated

biped with ankles, knees, hips and pitching of the waste. They used genetic algorithms to

optimise the network parameters and the targets for the algorithms were obtained by real-time

video analysis of the angles of the ankle, knee, hip and waste joints. A staged evolution approach

was used to break down the search for parameters into more manageable sub sections.

4.5. Conclusions The following conclusions were drawn from the literature survey:

Genetic algorithms are a useful tool in solving the parameters of the CPG network. Genetic algorithms can be used to evolve controllers for walking. Speed of convergence is a critical issue when using genetic algorithms. Genetic algorithms are more effective if the principles of Behavioural Memory and

Staged Evolution are applied.

21

5. THE DEVELOPMENT AND SIMULATION ENVIRONMENT

A simulation environment was created to aid the development of the walking algorithms and to

simulate the resultant walking patterns. The simulation took the form of a Graphical User

Interface (GUI) that took in a vector of joint angle trajectories. The GUI would then display a

graphical representation of legs walking using the given trajectories, as well as calculate the

accelerations, velocities and torques needed to move each of the joints.

5.1. Conventions and assumptions Figure 5 shows the naming convention for the RedBots limbs and angles. The hip angles are

defined by taking the angle from a vertical line through the hips to the thigh of the relevant leg. If

the angle is taken in the direction of movement (the right leg in Figure 5) then the angle is

positive and if the angle is taken against the direction of movement (the left leg in Figure 5) then

the angle is negative.

Figure 5 Naming conventions for RedBot simulation

Right hip angle Left hip angle

(negative)

Right knee

angle

Right foot

angle

Left knee

angle

Left foot

angle

Ground

Right thigh

Right shin

Left thigh

Left shin

Y

X

Direction of movement

z direction is out of the page

Z

22

The knee angles were calculated by taking the angle from the thigh to the shin, on the back of the

leg. Thus, if the knee angle is 0 then the shin will be bent up against the thigh and if the angle is

180 then the leg will be straight.

The above angle conventions are used through out this thesis project.

The RedBot had not arrived at the time the simulation was being coded, and thus a number of

educated guesses and assumptions were taken where necessary to build the simulation. The

following assumptions and conventions were made:

The RedBot is supported in the sagittal plane (x-y plane of Figure 5). Thus it was decided to assume there is no movement of the RedBot in frontal plane (y-z plane of Figure 5)

and the system was reduced to a two dimensional system.

The support for the RedBot is attached to the RedBots waist. It was assumed that orientation of the hips remains fixed and that they are unable to rotate about the support.

The simulation assumed 100% friction between the stance foot and the floor i.e. there is no slippage between the stance foot and the walking surface.

Motion of the hips was only allowed in the y-direction. Movement of the hips in the x-direction was constrained. Thus when walking was simulated the RedBot appeared to be

walking in place on a treadmill.

For the calculations of inertias, moments and torques, all joints were assumed to be rigid, except for the joint under consideration. Together with the assumption that there is 100%

friction between the stance foot and the ground, the section between the stance foot and

the joint under consideration could be considered to be fixed to the ground.

The lower foot was assumed to be the stance foot The feedback values for the RedBot were assumed to be the last positions the joints were

told to go to.

The mass of the plastic from which the RedBot was constructed was assumed to be negligible.

Each limb was approximated as a mass less beam with a single point mass somewhere along its length.

The mass of each thigh and shin was modelled as being the mass of the servo motor mounted in it.

The mass of the hips was modelled as being the mass of the supporting pole.

23

The distances of the point masses away from the different joints were assumed to be half the height of the servo motor used to actuate that joint.

The servo characteristics were obtained from their data sheets and are shown in detail in Table 1

of Chapter 6. The thighs and shins of the RedBot are 100mm long and the supporting hip section

80mm tall.

The above assumptions and parameters resulted in the following model of the RedBot.

Figure 6 RedBot weights and lengths used in the simulation

5.2. The simulation process and calculations In order to simulate the walking motion of the RedBot the simulation accepted a series of vectors

that described the joint angles of the RedBot at each point in time. The inputted angles were used

to calculate the x and y coordinates of each hip, knee, foot and point mass relative to the hips. The

x and y coordinates of the knees and feet were used to create a visualisation of the RedBot

walking, whilst the x and y coordinates of the point masses were used to calculate the necessary

moments of inertias and torques.

As mentioned above when calculating inertias, moments and torques all joints were assumed to

be rigid except for the joint under consideration, resulting in the section of the RedBot between

Ground

l1

l2 l1

l2

l1l1

l3 l3m1

m2 m2

m3

m3

l1=100mm

l2=17.5mm

l3=20mm

m1=100g

m2=55g

m3=31g

24

the joint under consideration and the ground being assumed to be fixed to the ground. For

example, consider the scenario in which the right leg is the stance leg and the right knee is being

examined. The right knee servo motor is then responsible for rotating the hips and the entire left

leg about the right knee. If, however, the left knee was under consideration then the left knee

servo motor would only be responsible for rotating the left shin about the left knee.

Figure 7 Torque calculation flow diagram

The following is a general method used to find the torques acting on a single joint.

The torque due to gravity on the joint was calculated as follows.

=massesall

nxngravity Mgl )()(

Where is the torque, lx is the displacement between the mass and joint in the x-direction and g is

the acceleration due to gravity.

The angular accelerations needed to move the limbs in the desired manner were calculated as

follows:

211 2

tnnn

n += +

where is the angular acceleration, is the angular position and t is the time step between

samples.

Next angles from walking algorithm Current joint angles of RedBot

Calculate net torques to move limbs

Calculate accelerations to move limbs Calculate inertia of limbs

Calculate torques from gravity on

limbs

Calculate torques required from servos

25

The rotational inertia around each joint was calculated using the following equation

=massesall

nn MrI2

Where I is the rotational inertia, M is the mass of each limb and r is the distance between the

mass and the joint.

The net torque acting on a joint was calculated as follows:

nnnnet I =)(

Once all the above calculations had been performed it was possible to calculate the torque

required by the servo motor to actuate the specific joint as follows:

)()()(

)()()(

ngravitynnetnmotor

ngravitynmotornnet

=

+=

The development GUI applied the above calculations to each joint at each sample in the vector of

joint angle trajectories. The GUI allowed the different torques and accelerations needed to

implement the joint angle trajectories to be plotted against time. This allowed for a comparison

between the torques needed and the torques available from the servo motors.

It was also desired to plot the angular velocities of each joint. The angular velocity of a single

joint was calculated as follows:

tnnnn

n += ++

1288 2111

Where is the angular velocity, t is the time step between samples and is the angular position

5.3. The final development GUI Besides being able to calculate the torques, accelerations and velocities of all the joints, the GUI

was extended to aid in the development of the different algorithms. Parameters relating to the

different algorithms could be altered through the GUI, and plots, such as neuron membrane

potentials for the CPG algorithm, could be generated. The GUI allowed different sets of

26

parameters to be saved and loaded and the simulation of the walking of the RedBot could be

converted to a movie file for presentation purposes.

Figure 8 The development and simulation GUI

The development GUI was never intended for use by anyone other than those who created it. It

was merely a tool used to ease the development of the Central Pattern Generator and If-Then rule

algorithms.

27

6. HARDWARE 6.1. Introduction to the RedBot The robot platform used in this thesis is the RedBot biped manufactured by AlegRobotics Inc.

in the United States of America. The RedBot stands 28cm tall and has 2 degrees of freedom (hip

and knee joints) per leg. For feet the RedBot has stumps and it is capable of walking between

speeds of 0.5Hz and 3Hz. The RedBot cannot run.

Figure 9 The RedBot

Motion of the RedBots limbs is controlled by servo motors. There is one servo motor per joint.

Table 1 shows the type of servo motor used in each joint as well as the servo motors weight, size

and performance characteristics.

Table 1 Joint Servo Motor Names and Characteristics

Maximum Torque (N.m) Speed (s/60) Joint Servo Motor

4.8V

Supply

6.0V

Supply

4.8V

supply

6.0V

supply

Weight

(grams)

Size

(wlh(mm))

Right Hip Futaba S9402 0.628 0.785 0.13 0.10 55 204138

Left Hip Futaba S9402 0.628 0.785 0.13 0.10 55 204138

Right knee Futaba S9602 0.256 0.265 0.11 0.09 31 351530

Left Knee Futaba S9602 0.256 0.265 0.11 0.09 31 351530

28

With a supply voltage of 6V the servo motors give better performance in terms of both speed and

torque than at 4.8V. It was thus decided to run the servos with a 6V supply.

Motion of the RedBot is constrained in the sagittal plane through an aluminium pole attached to

the RedBots hip on one end and to a pivot on the other end. This prevents the RedBot from

falling over in the frontal plane whilst still allowing it to fall over in the sagittal plane.

Joint angle feedback was achieved by tapping into the potentiometers used by the servo motors

internal circuitry for position control. Binary foot and shin switches were added to the RedBot to

detect its state for use by the rules based controller.

6.2. The microcontroller Each servo motor on the RedBot is controlled using a Pulse Width Modulated (PWM) signal. To

reduce the hardware needed to control the RedBot it was decided to use the microcontroller, and

not external circuitry, to generate the PWM signals. It was also decided that the microcontroller

would have to have provision for the addition of ankle joints to the RedBot at a later stage.

The requirements for the microcontroller are shown below

Table 2 Requirements for microcontroller used to control the RedBot

Item Quantity Reason

PWM

channels

6 One channel to control each hip and knee servo motor with an

extra two channels for the later addition of ankles

ADC

channels

8 Four channels to obtain position feedback from the hips and

knees, two channels for the future inclusion of ankles position

sensors and two channels for the possibility of foot pressure

sensors

IO Pins 4 To allow for the addition of simple binary foot switches and other

logic based IO

UART 1 To allow communications with the host PC.

The dsPIC30F4011 fulfilled all the above criteria and was chosen for use in this thesis. In

addition to this, because the dsPIC30F4011 is designed for DSP applications it is able to execute

all instructions (excluding instructions that change the program flow) in a single cycle.

29

The following table shows which pins of the dsPIC30F4011 were used for what application

Table 3 dsPIC30F4011 pin layouts

PWM signal pin layout

Servo Motor Pin Name Pin Number

Left hip OC1 23

Right hip OC2 18

Right knee OC3 22

Left knee OC4 19

Servo motor angle sensor pin layout

Servo Motor Pin Name Pin Number

Left hip AN2 4

Right hip AN3 5

Right knee AN4 6

Left knee AN5 7

Binary switch pin layout

Switch Pin Name Pin Number

Right foot - 28

Left foot - 27

Right shin - 9

Left shin - 10

6.3. RedBot to dsPICDEM2 connection board A basic connection board was built to interface the RedBot to the dsPICDEM2 development

board. The connection board was powered from a 6V DC power supply (6A max current) and

contained the necessary power connections for the RedBots servo motors. The board also

connected the PIC to the PWM or feedback connections on the RedBot. Since the supply was 6V,

it was necessary to scale all feedback signals from the 0-6V range to the 0-5V range to be

compatible with the PICs input pins. This was done by using a simple resistor divider network.

The inputs to the ADC pins were buffered using op-amps as unity gain voltage followers. The

LM324 quad op-amp was used, and thus only one chip was required to buffer all the angle sensor

feedback signals.

30

7. PRELIMINARY METHODS

Before the construction of the Central Pattern Generator could commence it was necessary to

perform a number of smaller tasks. These tasks included: programming a method to solve

differential equations, understanding the principles of tonic spiking and spike frequency

adaptation and obtaining human walking data to use as a reference for the outputs of the CPG.

7.1. Extracting human walking data from Gaitlab Since the aim of this thesis was to create a controller for biped walking it was useful to have

human hip and knee angle trajectory data available for inspection and manipulation on the

computer. The data was extracted from the file peak.dst in Gaitlab.

The following steps were taken obtain the data:

1) Gaitlab was run and the file Peak.dst was opened.

2) Peak.dst was then processed running all procedures.

3) The resulting data was then browsed and the following sets of data were copied and

imported into Microsoft Excel: JOINTANGLE LEFT HIP FLEXEXT, JOINTANGLE

RIGHT HIP FLEXEXT, JOINTANGLE LEFT KNEE FLEXEXT and JOINTANGLE

RIGHT KNEE FLEXEXT.

4) The data from the hips and knees was then scaled to correspond to the convention

described in 5.1.

5) The hip angles and scaled knee angles were then plotted.

7.2. 4th Order Runge-Kutta method The neuron models described by Izhikevich are all based on differential equations, it is thus

necessary to have a numeric method capable of solving the differential equations. The 4th Order

Runge-Kutta was chosen for this function because it is simple to implement and has relatively

high accuracy.

The aim of this experiment was to use the Runge-Kutta algorithm (given below) to solve the

differential equation, xdtdx = where x(0) = 1 using a time step of h = 0.002s,. The results of the

31

Runge-Kutta were then graphically compared to the actual solution of the equation which

is tex = .

The 4th Order Runge-Kutta algorithm is as follows:

Given an equation ),()( xtfdt

tdx = then if at any time, ti ,the current value of x ,xi, is known, it is possible to solve the next value of x, xi+1, at ti+1=ti+h as follows:

( )[ ]

),(

)21,

21(

)21,

21(

),(

26

34

23

12

1

43211

hmyhtfm

hmyhtfm

hmyhtfm

xtfm

mmmmhxx

ii

ii

ii

ii

ii

++=++=

++==

++++=+

7.3. Simulation of a single neuron Before experiments with the Central Pattern Generator as a whole could begin, it was necessary

to have an idea of how the Integrate and Fire with Spike Frequency Adaptation (IFA) neuron

behaves. The aim of this experiment was to build a flexible framework for the Central Pattern

Generator, using IFA neurons, that was easy to adapt for the inclusion of additional neurons or

coupling connections. The framework was then used to investigate the effects of the different

neuron parameters on the neurons behaviour and to find parameters that illustrate the principles

of tonic spiking and spike frequency adaptation. The parameters for the neuron were found by

trial and error.

The IFA neuron model used is based on that given by Izhikevich (2004). If there are n neurons in

a network then the ith neuron is defined as follows:

32

i

i

ithreshi

iii

n

ijjiji

gteg

cvthenvvif

vdgbvaIv

==>

++= =

)(

)(,1

Where, vi is the membrane potential (a voltage) of the ith neuron in the network, a (a current) is

the tonic input of the neuron, =

n

ijjijI

,1

is the sum of the inhibition signals (currents) from other

neurons in the network, b is the neurons membrane impedance, vthresh is the membrane potential

threshold voltage (when v exceeds vthresh the neuron is said to fire), c is the reset voltage of the

neuron, g is the potassium activation gate of the model or adaptation characteristic, which

increases by e/ every time the neuron fires via the Dirac Delta function , is the adaptation time

constant and d changes the weight of self-inhibition.

Since it is not possible to physically realise the Dirac Delta function the above model was

implemented as follows:

ii

inc

inciii

threshi

iii

n

ijjiji

gg

elabovetheintereplacesgwhere

gggandcvthenvvif

vdgbvaIv

=

+==>

++= =

mod)(

)(,1

A detailed algorithm showing the framework of the CPG as well as how the above neuron model

is applied within this framework is given below.

33

Figure 10 Algorithm for CPGFRAMEWORK1

Set n= to number of neurons in network

Create nxm array called params to store neuron variables.

m = n +8, which allows the array to store all parameters necessary for the IFA model as well as the time and the ability

for the ith neuron to receive coupling signals from the other n-i neurons in the network

Initialize all variables. All membrane potentials and potassium concentrations to be set to 0

Loop for desired simulation time period in steps of 2ms

Loop for i = 1 to n

(i.e. through all neurons in network)

Update membrane potential, current time and potassium concentration of ith

neuron in network by passing ith row of params to IFA

Is vi>vthresh?

vi=c

Set inhibition signals to other neurons to

desired values

Set inhibition signals to other

neurons to 0

Looped through all neurons?

Add current membrane potential and potassium

concentration to ith row of xv and gv respectively

Time period up?

Manipulate and/or plot xv, gv and spikev as

desired

Yes

No

Yes

Yes

No

No

Create vectors xv, gv and spikev with n rows each to store the neuron membrane potentials, potassium concentrations

and whether each neuron has fired or not (1 = yes, 0 = no) respectively for each time step in the simulation.

Add 1 to ith row of spikev

Add 0 to ith row of spikev

CPGFRAMEWORK 1

34

Figure 11 Algorithm for the IFA function

The following steps were taken to find neuron parameters that demonstrate tonic spiking and

spike frequency adaptation:

The n term in CPGFRAMEWORK1 was set to 1 as only one neuron was to be simulated. The time period for the simulation was set to 3s. The a term of the IFA model was held at zero until t = 0.5s, at which point it was set to

its desired value. At t=2.5s it was set back to zero.

Iterations of the simulation were run plotting xv, gv and a against time until satisfactory values of a, b, c d, , ginc and vthresh were found.

Initialise all variables using constants and

passed parameters (time step h = 0.002s)

=

=n

ijjijII

,1

Increment the time by h

Get next value of the membrane potential using dx=a-bx+I-g(x-d)

and the Runge-Kutta algorithm described previously

v > vthresh?

Get next value of the potassium concentration using gdg =

and the Runge-Kutta algorithm

g = g + ginc

Return updated v, g and t values to calling function

No

Yes

35

8. PRELIMANRY RESULTS 8.1. Extraction of human walking data The following graph shows the joint angle profiles extracted from Gaitlab.

-50

0

50

100

150

200

0 0.5 1 1.5 2

Time (s)

Ang

le (D

egre

es)

Left Hip Right Hip Left KneeRight Knee

Figure 12 Hip and Knee Angle Trajectories during normal human walking

By inspecting the graph it is possible to see that there are slight differences between the right and

left legs joint angle profiles. There are many possible reasons for this including:

The human body is not perfectly symmetrical and as a result the gait will not be perfectly symmetrical.

The surface on which the test subject walked may not have been perfectly level. The test subject may have had a slight injury causing an unsymmetrical gait.

The joint angle trajectories for the RedBot, however, are to be identical but 180 out of phase. It

was thus decided to use only one legs joint trajectories as a reference. The left leg was arbitrarily

chosen as the reference.

Note that the knee angle profiles contain a double bump. The flexion stage between the two

extension peaks cushions the effect of the foot on landing (Trew and Everett, 1998).

36

8.2. 4th Order Runge-Kutta results The 4th order Runge-Kutta algorithm described above was successfully implemented in

MATLAB 7. Figure 13 below compares the solution of xdtdx = as found by the Runge-Kutta

method against the actual solution of the equation.

0 0.2 0.4 0.6 0.8 11

1.5

2

2.5

3

Time (s)

Func

tion

valu

e

Actual solution (x = exp(t))Runge-Kutta solution

Figure 13 Runge-Kutta results vs. actual solution

Figure 13 shows that the Runge-Kutta solution closely approximates the ideal solution for small

values of t. This is acceptable because the IFA neuron equations are cyclic in nature and reset

themselves every time the neuron fires, and thus the equation will only be solved over small

values of t. A smaller h value would result in a more accurate approximation, however, it would

also result in an increase in the number of iterations necessary to solve for a given time period,

which increases the processing time of the algorithm. A larger h value would decrease the

accuracy of the approximation but also decrease the processing time needed to simulate a given

time period.

37

8.3. Simulation of a single neuron results CPGFRAMEWORK1 and IFA were successfully implemented in MATLAB. By experimentation

with each of the parameters within the model a broad idea of what each parameter did was

gained, however, exact relationships between the parameters and the neurons firing rate could

not be established. The basic affect of each parameter on the neurons behaviour is described

below:

c is the value to which the neurons membrane potential resets itself to after the neuron has fired.

vthresh is the threshold voltage of the neurons membrane potential. When the membrane potential reaches vthresh the neuron fires and resets its membrane potential to c.

An increase in a causes an increase in the neurons charging rate, resulting in the neuron reaching its threshold voltage quicker and increasing its firing rate.

An increase in b causes an increase in the rate of exponential decay of the neurons membrane potential, slowing the rate of charge of the neurons membrane potential and

decreasing the rate of firing of the neuron.

An increase in ginc causes a larger increase in the neurons potassium concentration every time the neuron fires, and as a result the neurons firing rate slows down more quickly,

An increase in increases the time taken for the potassium concentration of the neuron to discharge.

Making d more negative increases the self-inhibition of the neuron slowing its firing rate.

To mathematically analyse the model to obtain exact relationships for a single neuron may have

been possib