CC467Biologically inspired robotics

Introduction to Lego Mindstorms

Owen HollandComputer ScienceUniversity of Essex

In most robotics work, you are given a task, and a robot that must perform it, and you program the robot. Easy.

Why Lego is different

In the old days, there were hardly any robots.

If you wanted a task to be done by a robot, you first had to build your robot, and then you would be able to program it. Not easy…

- mechanical design and manufacture

- electronics design and manufacture

- system integration

- and then you could program the robot, often in assembly language

Why Lego is different

Now, with Lego, most of the hard bits disappear: you just stick Lego bits together to make the robot you want

- NO mechanical design and manufacture

- NO electronics design and manufacture

- NO system integration

- and you can program the robot in almost any language you want: C, Java, Lisp, Ada, Prolog, Logo, and even assembler.

- easy.

Why Lego is different

Let’s look at the history of Lego robotics.

It goes back a long way…

At MIT in the 1980s, the Mechanical Engineering Department started a course called 2.70:

“Introduction to Design”.

Students used scrap parts to build joystick-controlled machines that competed against each other at the end of the course.

Isn’t Lego just a toy?

In 1987, an Electrical Engineering and Computer Science (Course 6) student wanted a similar course in his department, and invented the 6.270 competition:

Students programmed simulated autonomous robots that competed against one another at the end of the course.

The birth of 6.270

At about the same time, a group of people at MIT Media Lab were sponsored by Lego to develop a system to enable children to build robots using Lego, and to control them using Logo, a simple programming language fashionable at the time.

One of these people, Fred Martin, began helping with 6.270, and developed some Lego-based kit to help.

LEGO/Logo

One of the outputs of Fred’s Lego sponsored project was the MIT Programmable Brick

Back to Fred Martin

Note the 6 sensor inputs and 4 motor outputs

An in-house Lego design team, obviously inspired by the MIT work, developed the Lego RCX brick.

And finally…

Note the 3 sensor inputs and the 3 motor outputs

Released in 1998 in various kits.

Little changed since.

Aimed at children.

All kits contain:

- an RCX brick

- 2-way IR communication between RCX and PC

- Lego and Lego Technic parts

- sensors and motors

Lego Mindstorms

Lego light sensor

Lego touch sensor

Lego temperature sensor

Lego rotation sensor

Lego sensors

This measures the intensity of the light coming in through the lens in the front of the sensor.

It’s quite directional.

The light coming in through the lens can come from two sources:

(1)The environment – we call it ‘ambient light’.

(2)Light coming from the red LED on the front of the sensor that has been reflected off objects in the environment. The amount reflected depends on the colour of the object.

The Lego light sensor

- Interfaces with all Lego and Technics parts

- Practically bomb-proof

- 4 types of sensors available (touch, light, rotation, temperature)

- Good battery life

- Not very expensive for whole kits

- Good support for schools

Lego Mindstorms: the Good

- Motor control is awful

- Limited sensor inputs and motor outputs

- Individual parts are very expensive

Lego Mindstorms: the Bad

- Dumbed down programming environments

Lego Mindstorms: the Ugly

Visual programming

(RCX code)

The RCX brick contains a very capable and fully featured microcontroller, the Hitachi H8/3292, running at the maximum clock speed of 16MHz.

It has 16kB of on-chip ROM (Read Only Memory) containing routines for communication with the IR channel, sensor and motor interfacing, LCD drivers, some test programs, and so on. It imposes some limitations, but nothing too serious, which is just as well because nothing can be done about it.

Is there any hope?

The RCX brick also contains 32kB of external RAM (Random Access Memory). This holds information as long as the battery is connected. It is used for two things:

(1)The ‘firmware’ that forms a kind of operating system. It is uploaded to the RAM via the IR link. The firmware supplied by Lego has some limitations (no floating point, only 32 variables).

(2)The user programs, compiled on the PC and uploaded via the IR link. Only 6kB is available for these!

Is there any hope?

Here is what happens:

- you produce a program on your PC

- the program is transformed into low-level instructions (bytecodes)

- the bytecodes are transmitted via the IR link to the RCX RAM

- the firmware interprets the bytecode and converts it into machine code using the ROM

- the processor executes the machine code

Yes, there is hope.

If we only knew what the bytecodes actually caused the microcontroller to do, we might be able write a program to generate more interesting sequences of bytecodes than we can using the Lego software.

In 1998, Kekoa Proudfoot was a student at Stanford University…

Yes, there is hope.

He reverse engineered the RCX, discovering and cataloguing all the bytecodes for the interpreter, and much else besides.

Respect to Kekoa Proudfoot

The RCX board – top view

Bottom view

Following on from Kekoa Proudfoot’s work, Dave Baum was able to write NQC – Not Quite C.

This allows you to write a program for the RCX in a language very similar to C, and to upload it to the RCX, where you can run it as normal.

NQC is text-based. Now other people have written pretty front ends so it’s just like normal programming in a typical IDE – you’ll be using John Binder’s Bricx Command Center.

Enter Dave Baum

NQC uses the original Lego firmware. If we used Kekoa Proudfoot’s discoveries, could we devise new firmware that would enable us to do even more?

YES.

In 1999 Marcus Noga began the LegOS project, designed to replace the Lego firmware with a new version allowing more or less direct control of the Hitachi processor. Smaller firmware, bigger programs (in C), and much faster execution – but for experts only.

More hope…

And now…

The newest firmware replacement is part of the LeJos system. It permits a Java Virtual machine (JVM) to run on the RCX. You can now program your robots in Java (in fact a subset of Java).

Why is this nice? Because it gives you access to lots of the neat stuff associated with Java. The LeJos package comes with classes for handling all the Lego hardware – you don’t have to build everything from scratch, which is just as well because Java is really too high level for robotics.

From the early days, enthusiasts began designing and building their own sensors. Some of these are excellent. Some are on sale. Some third-party sensors are even supported by Lego.

Hardware developments

Here’s a list of home-brewed sensors from just one site…

From the early days, enthusiasts began designing and building their own sensors. Some of these are excellent. Some are on sale. Some third-party sensors are even supported by Lego.

If you wanted to build a complex Lego robot now, you could probably find designs for the sensors you need on the Web. Some will be accompanied by the Java classes for dealing with them.

The range of hardware and software now available for Lego has transformed it from a toy into a serious educational and investigative tool.

Hardware developments

Dozens of universities in many countries, including top engineering colleges in the USA, Switzerland, Denmark, Germany, and Sweden.

What do they use it for?

Teaching programming

Teaching control

Teaching robotics

Individual projects

Student competitions etc. etc.

So who uses it?

Books:

Core Lego Mindstorms Programming by Brian Bagnold

Good intro to Mindstorms and to LeJos

Web:

Too many sites to list. Just Google any mixture of mindstorms, lego, robot, sensors, java, nqc, rcx. Lots of good stuff, and most sites have links to lots of others.

How can I find out more?

Some of the simplest robots are called Braitenberg machines.

In two weeks you’re going to build and program the one known as Vehicle 2.

A simple robot

In the space of 2 hours, working in pairs, and starting from scratch, you have to build, program, and run such a vehicle.

It’s not realistic.

So we’ll give you some strong guidance, in the form of the instructions to build a suitable vehicle…

…and in the form of a suitable program tailored to the vehicle

We have a problem

A suitable vehicle needs the following:

- a left motor (LM)

- a right motor (RM)

- a left sensor (LS)

- a right sensor (LR)

Back to basics

A suitable program needs the following:

- a way of driving each motor as a function of the reading on the sensor on the opposite side

Back to basics

And the environment needs to ensure that the change in sensory inputs produced by the movement of the vehicle leads to some interesting and obviously useful behaviour…

…with a simple performance measure so that we can have a competition

Back to basics

All vehicles will have to be run in exactly the same environment

The only suitable sensors are light sensors

Constraints

Consider the termite

Termites (and many other trail-following insects) follow trails by sensing concentrations of special chemicals (pheromones) using their antennae.

How termites work

They use a technique called osmotropotaxis.

Roughly speaking, their left legs go at a speed controlled by the strength of the sensory input to their right antenna, and their right legs go at a speed controlled by the strength of the input to their left antenna…

In other words, they operate just like little Braitenberg vehicles, and this enables them to follow trails.

How termites work

Yes…there are some trails on the floor of the arena…

Could we make visible trails on the ground, and sense them with light sensors?

…if the trail is either black or white, won’t the sensors only ever give maximum or minimum readings?

No, because the sensor takes in reflected light from an area, not a point.

If the area is wholly inside the black region, you’ll get the minimum reading (with some noise)

And if it’s wholly inside the white region, you’ll get the maximum reading (with some noise)

But surely…

…if the trail is either black or white, won’t the sensors only ever give maximum or minimum readings?

No, because the sensor takes in reflected light from an area, not a point.

If the area contains both black and white regions – if the sensor is somewhere on the borderline – then the reading will be somewhere in between the minimum and maximum (again, with some noise)

But surely…

If we space the sensors so that they are separated by just less than the width of the trail, then when both sensors read the minimum, we know we must be on the trail…

…and so moving forwards is a good idea

A basic strategy

If the sensor on the left reads the maximum, and the sensor on the right reads the minimum, we know we’re moving off the trail to the left…

…so turning right is a good idea

A basic strategy

If the sensor on the right reads the maximum, and the sensor on the left reads the minimum, we know we’re moving off the trail to the right…

…so turning left is a good idea

A basic strategy

And if both sensors read the maximum, you know you’ve completely lost it

Of course, you could devise some cunning way to get back on the trail (termites do)…

A basic strategy

What if one or both sensors give a reading somewhere in between?

The quick and dirty solution: keep on doing whatever you were doing – it won’t last

The elegant solution: you can produce one if you want, but it won’t make any difference…

But…

We can break this into two stages:

(1) How far down the trail you get

(2) If you make it to the end, how long it took

The performance measure

You can make whatever you like, but since time is so short I recommend that you use a version of the Trusty robot described in Jonathan Knudsen’s “The Unofficial Guide to Lego Mindstorms Robots”

Instruction sheets and pictures will be in the lab.

First, catch your robot

You’ll be programming your robot in NQC (Not Quite C). You’ll use the BricxCC (Bricx Control Centre) as your development environment.

When you first fire up BricxCC (by double clicking on the BricxCC icon) it looks for the RCX brick. It will find it, because you will have switched it on (using the red On-Off button) and placed it with its infra-red windows facing the IR tower and a few centimetres away from it.

And finally…the program

Click on RCX2, then OK

You’ll get this. You should also get another window

If you don’t, press F9 once or twice until you do

File, New gets you a programming window

Copy and paste the text file ‘trusty’ from your email

And save it locally as trusty (a .nqc file)

Click on Compile to compile, and on Download to put it into the RCX.

You will get helpful error messages below the program

If you can’t sort out the errors by yourself, call Renzo, the GTA

If your program won’t compile…

It’s straight from the Knudsen book – a photocopy of the relevant pages will be in the lab.

Since I can’t teach you NQC in 2.4 minutes, we’ll skip through some basics, and then go through the program.

The trusty.nqc program

NQC/RCX basics: Sensors

Sensor connections 1, 2, and 3 are known as SENSOR_1, SENSOR_2, and SENSOR_3

You have to tell each sensor connection what sort of sensor is plugged into it. For the light sensor, it’s just a matter of:

SetSensor (SENSOR_2, SENSOR_LIGHT)

NQC/RCX basics: Motors

Motor connections A, B, and C are known as OUT_A, OUT_B, and OUT_C

You set the direction of a motor by

OnFwd (OUT_A) or OnRev(OUT_B)

You set the power of a motor by

SetPower (OUT_A, x) where x runs from 0 to 7

You turn a motor off by

Off(OUT_B)

NQC/RCX basics: Tasks

NQC can run several tasks at the same time. (Unfortunately you have to be careful to ensure that tasks do not conflict, especially over resources.)

The format of a task is:

task taskname( ) {task instructions etc go here}

Every program must contain a task called main:

task ( ) main {stuff}

NQC/RCX basics: Subroutines

There are several ways of avoiding having to write out long strings of repetitive chunks of code. One is to use subroutines.

The format of a subroutine is:

sub subroutinename ( ) {code goes here}

And you call it in the main body of your code by

subroutinename ( );

(Semicolons are used as separators for expressions etc.)

NQC/RCX basics: Control structures

while (true) {stuff}

will execute stuff forever.

if (condition) do_something_simple will

do_something_simple if the condition is true

if (condition) do_something_simple

else do_something_else will do_something_else if

the condition is not true

All variables are integers. You have to declare them, otherwise the compiler will moan.

int thing;

at the beginning of your program will let you use the integer variable thing anywhere.

If you declare it within some curly brackets (as part of an expression or subroutine etc.) it won’t work if you try to use it outside the brackets (and the compiler will complain).

NQC/RCX basics: Variables

You can give constants meaningful names instead of numbers by defining them at the start of your program:

#define MONKEY_SCORE 25

means you can write MONKEY_SCORE in your code instead of just 25, and so you will be able to distinguish it from all the other constants that might be 25 – for example, SLAMMER_RECORD

NQC/RCX basics: Constants

int state;

 

#define BOTH_ON 3

#define LEFT_ON 1

#define RIGHT_ON 2

#define BOTH_OFF 0

#define INDETERMINATE 255

 

#define DARK2 35

#define LIGHT2 40

#define DARK3 40

#define LIGHT3 45

 

#define POWER 4

trusty.nqc

int state;

THIS IS THE DECLARATION OF THE VARIABLE WE WILL USE TO REPRESENT ALL THE DIFFERENT MEANINGFUL COMBINATIONS OF SENSOR STATES

trusty.nqc

#define BOTH_ON 3

#define LEFT_ON 1

#define RIGHT_ON 2

#define BOTH_OFF 0

#define INDETERMINATE 255

THESE ARE THE VALUES THE VARIABLE state CAN TAKE.

BOTH_ON MEANS BOTH SENSORS ON THE TRACK, ETC.

INDETERMINATE MEANS AT LEAST ONE SENSOR IS ON THE BORDERLINE

trusty.nqc

#define DARK2 35

#define LIGHT2 40

#define DARK3 40

#define LIGHT3 45

THESE ARE THE SENSOR INPUT VALUES FOR DECIDING WHETHER A SENSOR IS ON OR OFF THE LINE, OR ON THE BORDERLINE

trusty.nqc

Sensor tip

If the sensor port is correctly configured, you can read the value of a sensor directly from the LCD display on the RCX.

Press VIEW. You will see a little ^ pointing to input 1. The number on the LCD will be the current input of the sensor connected to input 1.

Press VIEW again. The ^ will move round one place to input 2.

Keep on pressing and it will go 3, A, B, C, and then out of VIEW mode.

#define POWER 4

THIS SETS THE POWER LEVEL THAT WILL BE USED FOR MOTORS THAT ARE ON.

trusty.nqc

task main() {

initialize();

while (true) {

if (state == BOTH_ON)

OnFwd(OUT_A + OUT_C);

else if (state == LEFT_ON) {

Off(OUT_A);

OnFwd(OUT_C);

}

else if (state == RIGHT_ON) {

Off(OUT_C);

OnFwd(OUT_A);

}

}

}

trusty.nqc

task main() {

initialize();

while (true) {

CODE HERE

}

}

THIS STARTS THE MAIN TASK, INITIALIZES EVERYTHING USING THE SUBROUTINE INITIALIZE, AND EXECUTES THE CODE IN THE MIDDLE

trusty.nqc

if (state = = BOTH_ON)

OnFwd(OUT_A + OUT_C);

else if (state = = LEFT_ON) {

Off(OUT_A);

OnFwd(OUT_C);

}

else if (state = = RIGHT_ON) {

Off(OUT_C);

OnFwd(OUT_A);

THIS GOES FORWARD FOR BOTH_ON, TURNS LEFT FOR LEFT_ON, AND TURNS RIGHT FOR RIGHT_ON

trusty.nqc

sub initialize() {

SetSensor (SENSOR_2, SENSOR_LIGHT);

SetSensor (SENSOR_3, SENSOR_LIGHT);

SetPower (OUT_A + OUT_C, POWER);

OnFwd(OUT_A + OUT_C);

start watcher;

}

THIS IS THE SUBROUTINE initialize. IT SETS THE SENSOR INPUTS CORRECTLY, STARTS THE ROBOT GOING FORWARD, AND STARTS THE TASK watcher.

trusty.nqc

task watcher() {

while (true) {

if (SENSOR_2 < DARK2) {

if (SENSOR_3 < DARK3) state = BOTH_ON;

else if (SENSOR_3 > LIGHT3) state = LEFT_ON;

else state = INDETERMINATE;

}

else if (SENSOR_2 > LIGHT2) {

if (SENSOR_3 < DARK3) state = RIGHT_ON;

else if (SENSOR_3 > LIGHT3) state = BOTH_OFF;

else state = INDETERMINATE;

}

else state = INDETERMINATE;

}

}

trusty.nqc

task watcher() {

while (true) {

ONCE STARTED, WATCHER KEEPS ON RUNNING FOR EVER

trusty.nqc

if (SENSOR_2 < DARK2) {

if (SENSOR_3 < DARK3) state = BOTH_ON;

else if (SENSOR_3 > LIGHT3) state = LEFT_ON;

else state = INDETERMINATE;

IF BOTH SENSORS ARE BELOW THEIR INDIVIDUAL DARK THRESHOLD, state IS SET TO BOTH_ON.

IF THE SENSOR ON THE LEFT IS DARK, AND THE SENSOR ON THE RIGHT IS ABOVE ITS LIGHT THRESHOLD, state IS SET TO LEFT_ON.

IF THE SENSOR ON THE LEFT IS DARK, AND THE SENSOR ON THE RIGHT IS BETWEEN ITS LIGHT AND DARK THRESHOLD, state IS SET TO INDETERMINATE

trusty.nqc

else if (SENSOR_2 > LIGHT2) {

if (SENSOR_3 < DARK3) state = RIGHT_ON;

else if (SENSOR_3 > LIGHT3) state = BOTH_OFF;

else state = INDETERMINATE;

THIS SORTS OUT RIGHT_ON AND BOTH_OFF, AND ANOTHER KIND OF INDETERMINATE

trusty.nqc

else state = INDETERMINATE;

AND THIS CATCHES THE CASE WHERE BOTH SENSORS ARE BETWEEN THRESHOLDS

THAT’S ALL YOU NEED – OFF YOU GO…

trusty.nqc