Autonomous Robotic Follower

with License Plate Recognition

Capability

Khanh Le, Luis Sosa, Bryan Diaz,

Victor Salomon

Dept. of Electrical Engineering and Computer

Science, University of Central Florida, Orlando,

Florida, 32816-2450

Abstract — A computer vision based robotic follower system

implementation is presented, and this robotic following mechanism is triggered by the use of an Automatic License Plate Recognition (ALPR) algorithm. The robot is trained at

the moment of the event to use a tracking algorithm, this system tracks a lead vehicle using a sequence of images captured on video. On this paper the focus is a low cost, single

board computer implementation, simulating the functioning as closely as possible of a regular motor vehicle.

Index Terms — ALPR, Computer Vision, Autonomous Following, Single Board Computer, License Plate Recognition.

I. INTRODUCTION

As the computing technologies advance and become

more cost effective we see more of these applied to

different areas in our daily lives, one of which is the safety

of the population in general. Because of this we see

applications of computer vision such as in the case of facial

image recognition or Automatic License Plate Recognition

(ALPR). These solutions are implemented in systems used

for applications such as automatic toll payments or access

control for residential or industrial areas.

This paper proposes a different application, using a

ALPR algorithm a robot detects a license plate number, this

extracted license plate number is then compared to a

database simulating that which a law enforcement officer

would have available, if the number extracted is matched to

a license plate number which is black listed for any given

reason, the robot will train itself to follow the said license

plate. The system design is shown in Figure 1, the main

element in the system is the single board computer, a

Raspberry Pi 2 Model B, which provides a low cost

platform to perform the processing needed for the computer

vision is processed, as well as running the other processes

involved in the testing case scenario.

II. SYSTEM COMPONENTS

A. Single Board Computer

This is called a “brain” of our whole project. The

Single Board Computer is unit where data is processing

and command is sent out to another components in

system. In this project, the most important job is how the

Robot can recognize and detect the license plate. After

that, it keep following without losing the target. We use

image-processing for implementing this function.

Raspberry Pi 2 Model B is a chosen one in pool of

microcontroller market. Camera connected to the Pi will

send image continuously to Pi, and then it will process

these data to recognize the pattern on target to detect the

license plate in black list. Also, when target is detected,

Pi will base on analyzing image of target to determine its

behavior and decide the action corresponding to this

behavior. Pi also send out command to Motor Controller

through Serial Communication for controlling the Robot

motion to follow the target. Due to the relative distance

between Robot and target, Pi will decide if motor need to

accelerate or decelerate.

Fig. 1 System Design Block Diagram

B. Motor Controller

This is a system controlling whole body of ACR. It

makes ACR run and respond to target’s behavior based

on command sent from MCU. Raspberry Pi in MCU does

image-processing for detecting a target and then make

decision of what action follower need to do. It will

command motor controller system to control servo motor

for turning or brushed DC motor for going forward or

backward and accelerating or decelerating. This

command will be made through serial communication.

This system has its own micro-controller which is

separated from MCU, which makes designing hardware

and programming to be easier. The micro-controller

communicates with servo motor directly through I/O pins

but controls the brushed DC motor by using H-Bridge as

middle guy.

C. Ultra Sonic Sensors

This type of sensors use ultra-sonic waves to

determine a given distance. One of the biggest

advantages compared to other sensors which have similar

capabilities is that this type of sensor can be used inside

or outside building because the sunlight does not affect

its operation as it does for infrared sensors, and because

of this reason, an ultrasonic sensor is relatively more

accurate.

The ultrasonic sensor has an output which emits an

ultrasonic signal and an input which receives it after this

bounces on a surface. Based on the time passed between

signal sent out and the return echo received, the distance

can be calculated, providing a high rate of accuracy, with

ranges of operation ranging from 10cm to more than

80cm.

D. Inertia Measuring Unit (IMU)

This inertial measurement unit (IMU) is a

component of navigational equipment, and it is usually

composed by a combination of accelerometers,

gyroscopes, magnetometers and sometimes other

electrical sensors inertial measurement units can measure

orientation, acceleration rates, and rotational changes.

These are often produced in different styles with

differing levels accuracy which is measured in degrees of

freedom.

The inertial measurement unit can also be used to

measure gravitational forces which are commonly called

g-forces and by measuring the various forces and keeping

track of them, the inertial measurement unit is able to

produce a linear record of these measurements, which are

then processed and the data that results is used to

calculate the inertial measurement unit’s position based

on reported velocity, direction, and time elapsed. This

data can be directly overlaid onto an electronic mapping

system that can tell the inertial measurement unit its

location and orientation with respect to a point, which

can be useful to detect a possible accident of the motor

vehicle, adding an extra safety measure to our system.

E. Power Supply

The power supply it is a very important part of the

project. It is what gives life and energy needed for the

systems to perform effectively. If the power is not

supplied efficiently, malfunctions in the subsystem may

arise. In Figure 2, a high level diagram of the power

supply system is represented.

Fig. 2 High Level Diagram of power system

1. Main DC source

The system Main DC source consists of a rechargeable

battery. This battery is responsible of supplying enough

power for the system to work efficiently. Three different

voltage regulators will be in charge of stepping down the

voltage accordingly to the need of each load

Main Dc Source consists of a rechargeable battery.

There were several requirements needed to be taken into

account in the selection of the battery. First, the battery

must be able to last at least 30 minutes at full load. Also, it

must be small and lightweight, the latter especially

important, because the performance of the ACR car might

be affected if the weight increases. After much

consideration and research, the battery chemistry selected

for the project was the Lithium Polymer Ion battery. This

battery chemistry provides a lightweight design, with

improved safety to overcharging and has a low probability

of electrolyte leakage which can negatively affect the

performance of the battery.

2. Voltage Regulators

The second part of the power supply diagram are the

voltage regulators. There were to main technologies to be

consider for our project, linear vs switching voltage

Main DC Source

Voltage Regulator

Motor

Voltage Regulator

Raspberry Pi

Camera

Voltage Regulator

MCU

Sensors

Servo Motor

(Steering)

regulators. Table #1 shows the difference between the two

technologies.

Features Linear

Voltage

Regulator

Switching

Voltage

Regulator

Function Step down

only, output

voltage must

be less than

input voltage

Steps up or Steps

Down the

voltage, can

produce multiple

outputs.

Size Small to

medium in

portable

design, may

be even larger

if heat sink is

needed

Large than linear

at low power, but

smaller in the

case where linear

requires a heat

sink

Efficiency Low to

medium

High

Noise Low Medium to high

due to ripple

effect

Output

Ripple

Very small

almost

negligible

Large

Waste

Heat

High, when

load and

voltage

difference is

high

Low, most

components will

run cool for low

power levels

Table #1.Linear Voltage Regulator Vs Switching Voltage

Regulator

The ACR car uses switching voltage regulators. Based on

the research, switching regulators are more effective and

highly efficient, in stepping down a voltage, when the

difference between the two voltages is large. Having a high

efficient regulator is necessary in our design because our

main source of power is a battery. It is necessary that the

power supplied to the system is not wasted in the form of

heat.

III. HARDWARE DETAILS

A. Image-Processing:

We are using the raspberry Pi 2 model for image

processing. There are a number of advantages of using the

Pi which is why we chose it. It has an operating system

which makes it much easier to use for the image processing

libraries. It has multiple USBs which is good because the

camera is a USB camera, we can also connect a keyboard

and a mouse to make it easier to interact with the Pi. It also

has a HDMI port so we can connect it to a monitor and see

what’s happening. Since image processing is very

performance intense we decided to buy two heat sinks and

one little fan for the main Pi processor. With the heat sinks

and the fan we are now able to overclock the Pi to

1000MHz and 500MHz for SDRAM which would make

the image processing a little faster.

B. Motor controller system

This system is composed of MSP430, Pololu motor

driver, Brushed DC motor, servo motor and voltage

level shifter between 3.3V and 5V.

1. MSP430G2553:

We pick MSP430g2553 to be this system

controller because we are familiar to this micro-

controller during Embedded System courses. Besides

that, it has same voltage level with Raspberry Pi

which is a whole project’s brain. MSP430 is able to

communicate with Raspberry Pi through Serial Port

by connecting directly TX pin on Pi to RX pin (Pin

1.1) of MSP430.

Fig. 3: MSP430 connection.

We only use MSP430 chip, not a whole launch pad,

for this project. In order to power it without the

board, we need to connect Vcc (Pin1) and Reset

(Pin16) to 3.3V. Besides that, pin20 is required to be

connected to ground. We are not going to use all the

pins of MSP430. This table #2 below shows the pins

used for connection with another components in

system.

Vcc (Pin1) 3.3V

GND (Pin20) Ground

Reset (Pin16) 3.3V

RX/P1.1 (Pin3) TX pin of Rasp Pi

P2.1 (Pin9) DIR pin of Pololu

P2.5 (Pin13) PWM pin of Pololu

P2.2 (Pin 10) Servo Motor

Table#2 Msp430 pin connection

2. Servo Motor:

The servo we use is a typical one for Remote

controlled car displayed in market place. There is no

special requirement for picking this component. In

general, Servo has 3 lines coming out for connections.

Brown is ground line, Red is Power Line and Orange

is signal line. Not same as MSP430, the servo needs

5V to be powered up. Therefore, Red Line will be

connected to 5V and then Signal Line (Orange) is

connected to P2.2 (Pin10) of Msp430. This micro-

controller will send a Pulse Code Modulation (PCM)

signal to servo in order to tell servo where the shaft

needs to go.

3. Brushed DC motor:

The model of DC brushed motor that we pick from

the marketplace is DS-540 Electric DC motor. It is

operating with full speed of 20000 rpm at 12V. We

are going to build a high speed chasing robot, so this

DS-540 are able to satisfy this speed requirement.

Also, the shaft generates a high torque to handle the

heavy load we will put on the car. The table #3 below

shows this motor specifications.

Rated Voltage 12V

Rated Current 2.6A

Full Speed 20000 rpm

Shaft Diameter 3mm

Shaft Length 10mm

Body Diameter 36mm

Total Length 68mm

Weight 161g

Table #3 DC motor specifications

4. Logic Level Shifter:

As we discuss in section 1, we use MSP430 as the

system micro-controller. It will send out signal to

Pololu motor driver to control the Brushed DC motor

(DS-540). However, logic connection of this Pololu

driver is designed to interface with 5V system, not

3.3V as MSP430 system. Therefore, we need logic

level shifter to convert signal from 3.3V level to 5V

level so that micro-controller can interface with

Pololu driver. The table #4 below show connections

for Level Shifter.

Low Level 3.3V

High Level 5V

A4 Pin9 of Msp430

B4 DIR pin of Pololu

A6 Pin13 of Msp430

B6 PWM pin of Pololu

GND Ground

Table #4 Level Shifter Connection

5. High Power Motor Driver:

This discrete MOSFET H-bridge motor driver

enables bidirectional control of one high-power DC

brushed motor. The little 1.3×0.8-inch board supports

a wide 5.5 to 30 V voltage range and is efficient

enough to deliver a continuous 15 A without a heat

sink. In this project, we supply a voltage 12V and

expected current will be around 2.5A but maybe 5A if

motor get stall. Also, this motor driver has own

protection if short circuit or overheat happen. Besides

that, the cost is only $40 and a lot cheaper than the

expensive amount for building H-bridge circuit on

PCB.

Fig. 4: motor driver board

We do not need to connect all the pins in Pololu

board. In typical motor control application, only DIR

and PWM are required for input signal. Besides that,

OUTA and OUTB will be connected to motor. 12V is

supply to V+ for powering motor and the driver. The

last one is GND connected to ground.

The operation of Pololu motor driver is really

simple to understand. DIR pin is low or high, which

determine the direction of motor (backward or

forward). Also, the PWM signal input through PWM

pin will control speed of the motor.

C. Ultra Sonic Sensor

The function of the ultra-sonic sensor is to prevent any

the robot to incur in any accidents, these sensors will be

placed around the robot in strategic locations such as the

front that the sides, in Figure 5 we see the placement chosen

for the robot, the front sensor, or sensor 1 is used to avoid

crashing with any obstacles in front of the robot, such as in

the case where the robot is autonomously following a target

Fig 5. Placement of Ultrasonic Sensors

On the sides we have sensors 2 and 3, which are used to

avoid crashing to obstacles when following a target which

is turning, if an obstacle is detected the turning will be

delayed for as long as there is no longer an obstacle which

would be an issue for the presented situation.

D. Inertia Measuring Unit (IMU)

This device will be used for two functions, first

measuring the speed of the robot, and also determining

whether there robot is in a situation which requires turning

off the different components of the system. The Razor IMU

board was selected, it contains three sensors an MEMS

triple-axis gyroscope (ITG-3200), triple-axis accelerometer

(ADXL345), and a triple-axis digital magnetometer

(HMC5883L), and this provides us with up to nine (9)

degrees of freedom. This board also has an MCU on-board,

an Atmel ATmega328 MCU, which will perform will

perform the processing related to the IMU.

Because the accelerometer in the IMU measures

acceleration and not speed a calculation to in order

determine such parameter is needed. In order to get speed,

the acceleration data has to be integrated over time, and

since this device is operating the discrete domain, an

approximation can be made and can be related by the

following equation where ‘a’ is acceleration, ‘V’ is

velocity, and ‘T’ is the sampling period.

V[n+1] = V[n] + T*a[n] (1)

Every time the data is sampled equation (1) will be

evaluated, and thus the velocity at the given time will be

evaluated and potentially displayed. Because of the nature

of this device, it important to note that the orientation in

which the IMU with respect to the vehicle, another factor is

also that the device is electromagnetic fields, so the IMU

must be placed away from certain components such as the

motors, as they might cause interference with the

magnetometer on board.

E. Power Supply

In order to design an effective power supply certain

parameters must be met for the effective design of the

system.

This table #5 provides the specific power values the

power supply system must supply, in order to provide

each subsystem the power needed to effectively work.

Items Input Voltage

(volts)

Input

current (A)

Raspberry Pi 5 2

Msp430 3.3 0.5e-3

Motor 3.6-12 Up to 6 amps

Servo ( Steering) 5 0.500

(2) Sensors 5 0.500

IMU 3.3 .215

Table #5: Voltage required for each component

1. Main DC Source:

The rechargeable battery selected to power the system

is a 14.8 Volts Lithium Polymer Ion battery. The battery

provides 5000mAh with a maximum discharge rate of 125

Amps. It only has 4 small cells, making the battery

lightweight and efficient.

2. Voltage Regulators:

Based on the parameters in Table #, three different

switching voltage regulators are necessary in order to

provide the power required for each subsystem.

The first voltage regulator, TPS54821RHLR is in

charge of stepping down the voltage from the battery (14.8

volts) to 12 volts which is delivered to the motor. This

voltage regulator can handle up to 8 amps, even though the

motor can handle up to 6 amps of current. Figure 6 shows

the schematic for the TPS54821RHLR voltage regulator

circuit. The circuit input VIN=14.8 represents the battery

input and Iout= 6 amps shows the maximum current the

motor is able to draw.

Fig 6. 12V Voltage Regulator schematic

The second voltage regulator, TPS54525PWPR is in

charge of stepping down the voltage from the battery (14.8

volts) to 5 volts. This voltage regulator supplies the power

to various components in the system. These are: Raspberry

Pi drawing 2 amps max, servomotor 0.5 Amps and two

ultrasonic sensors 0.5 amps. The voltage regulator can

handle a maximum current of 5.50 Amps but only 3 amps

are required to run all the components effectively. Figure 7

shows the schematic for the TPS54525PWPR voltage

regulator circuit. The circuit input VIN=14.8 represents the

battery input and Iout= 3.03 amps shows the maximum

current the components will draw.

Fig. 7. 5V Voltage Regulator Schematic

The third voltage regulator, TPS62177DQCR is in

charge of stepping down the voltage from the battery (14.8

volts) to just 3.3 volts. This voltage regulator provides the

IMU with 0.015 Amps and the MSP430 with just 0.2 Amps.

The voltage regulator can handle a maximum current of 0.5

amps, but the components will only draw about 0.215 amps.

Figure 8 shows the schematic for the TPS62177DQCR

voltage regulator circuit. The circuit input VIN=14.8

represents the battery input and Iout= 0.215 amps shows the

maximum current the components will draw.

Fig. 8: 3.3V Voltage Regulator Schematic

3. PCB Design:

There were several parameter taken into account when

building the PCB. Trace sizes vary depending on the

amount of current that passes through. All the components

in the PCB are surface mount. Sizes of each components

vary from 402mm to 805mm.

F. Double-side plated through hole

This board is used for professional and secure purpose.

Our project has three different board for each function of

operation, but they are required to put on same platform for

organizing. Moreover, this is high speed chasing car, so all

connections need to be secured when moving fast. We will

put all the pins needed for connection to this board and do

soldering. Therefore, there is no wire connection will be

loose when the following car is chasing the target, even if

the collision happens.

IV. SOFTWARE DETAILS

For the software we are using different libraries to

detect a license plate and be able to track it. For the tracking

library we are using OpenCV version 2.9.10 which is one

of the best open source libraries for computer vision in the

internet. For detecting the license plate letters we use a

library called OpenALPR which has as dependency

Leptonica and Tesseract-OCR libraries. The Leptonica

library is used to do image manipulations like rotation,

scaling, convolution, grayscale morphology, etc. tesseract-

ocr is the one in charge of character recognition which it

does with the help of Leptonica. OpenALPR which is the

main library that we are using, goes thru multiple steps to

finally detect the license plate. The first phase is detection

which happens only once for each image. This uses the LBP

algorithm (which is generally used for face detection) to

find the possible license plate regions (x, y, width, height).

Once the region (or regions in the case of multiple license

plates on the same image) is detected it goes to the later

pipeline phases for further processing. The detection phase

is usually the most processing-intensive phase. Next in the

pipeline is binarization which creates multiple binary

images for each plate region. Multiple images are used

because it gives the best possible chance of finding all the

characters. If we just take a single binary image then we

may miss characters if the image is too dark or too light for

example. Next we have the character analysis phase which

attempts to find character-sized regions in the pate region.

It does this by first finding all connected blobs in the license

plate region. Then it looks for blobs that are roughly the

width and height of a license plate character and have

tops/bottoms that are in a straight line with other blobs of

similar width/height. This is done multiple times in the

region. It starts by looking for small characters then

gradually looks for larger characters. The next phase is the

one that finds the edges of the license plate. The next phase

is deskew which is use to eliminate any rotation or skew in

the image. After all that comes the character segmentation

phase which tries to isolate all the characters that make up

the plate image and then the OCR analyzes each character

independently and computes all possible characters and

their confidences.

After the OpenALPR detects a license plate it will pass

the coordinates to Consensus-based Matching and Tracking

of Keypoints algorithm (written by Nebehay Georg and

Pflugfelder Roman) so that the card could track the license

plate while the cars are in motion. This algorithm works by

breaking down the object of interest into tiny parts, known

as keypoints. In each frame the algorithm tries to find those

keypoints that were already there in the initial selection of

the object of interest.

Since the Raspberry Pi will be doing the computer

vision part therefore it has to decide if it needs to speed up,

slow down, turn left, or turn right. The Raspberry Pi will be

communicating with the MSP430 via serial. The code to

determine what the car should do is very simple. When the

car is tracking the object it approximates the coordinates

that the object is with respect to the resolution of the

camera. It will also calculate the center of the object which

is what we are interested in. For example, the center could

be (65, 120) where 65 is the x-axis so is used to see how

left or right the object is and the 120 is the y-axis to see how

close or far the object is. This will only help if we determine

a reference point, which we do when we first start tracking

the object. With a reference point we can determine how far

off the target is from the reference point and act

accordingly.

Fig 9: Showing the different possible scenarios that can

occur based on object position and a reference point. “F”

is forward, “B” is back, “L” is left, and “R” is right.

Fig.10: Showing how the actual tracking algorithm looks

like in action. The white circles are some of the features

the algorithm detects on the image and then tries to point

to the center of the image. The blue circles are where most

of the white circle vectors converge to.

ACKNOWLEDGEMENT

The authors wish to acknowledge the assistance and

support of Dr. Samuel Richie and Dr Chung Yong Chan;

University of Central Florida.

REFERENCES

[1] Texas Instrument, MSP430G2553 datasheet. Webpage: http://www.ti.com/product/msp430g2553

[2] Pololu High Power Motor Driver datasheet. Webpage: https://www.pololu.com/product/755

[3] Webench Design Tool Software. Webpage: http://webench.ti.com/webench5/power/webench5.cgi?app=powerarchitect&project_id=9

[4] Texas Instrument, TPS62177 Voltage Regulator, Webpage: http://www.ti.com/product/TPS62177/technicaldocuments

[5] Texas Instrument, TPS54821Voltage Regulator datasheet Webpage: http://www.ti.com/lit/ds/symlink/tps54821.pdf

[6] Texas Instrument, TPS54525 Voltage Regulator Webpage: http://www.ti.com/lit/ds/symlink/tps54525.pdf

Khanh Le, a senior student of Electrical Engineering Department at University of Central Florida, is going to graduate in Spring 2015 semester. He will start working as Quality Engineer in Failure Analysis position with Texas Instrument right after his graduation. He is

planning in pursuing a higher degree in Electrical Engineering in the future.

Victor Hernandez Salomon is graduating with a Bachelor

of Science in Electrical

Engineering from the

University of Central Florida,

upon graduation he will join a

major company in the

Semiconductor industry. His

research interest include

biomedical applications for

analog devices. He is

currently a member Eta Kappa Nu and IEEE.

Bryan Diaz, a senior Electrical

Engineering student at the

University of Central Florida.

Currently a member of IEEE

and SHPE Organizations. After

graduation, he will pursue a

Master’s degree in Electrical

Engineering.

Luis Sosa, a senior Computer

Engineering student at the

University of Central Florida.

Currently Luis is doing an

internship with Lockheed

Martin as a software engineer.

Upon graduation Luis will be

working with Lockheed

Martin as a full time software

engineer. Luis’s areas of interest are machine learning

and NeoroEvolution of Augmenting Topologies

(NEAT).