autonomous targeting sentry(ats) group 12

Ethan King - Electrical Engineer

Daniel O’Hara – Computer Engineer

Stephen Rodriguez – Computer Engineer

James Van Gostein – Computer Engineer

Autonomous Targeting Sentry

GROUP

12

ATS

PROJECT OVERVIEW AND MOTIVATION

Prototype an autonomous turret to defend high priority areas

Detect and track targets using computer vision, determine threat and eliminate

To reduce the amount of human interaction in combat situations

OBJECTIVES

• Lightweight

• Cost efficient

• Capable of detecting, prioritizing and tracking targets autonomously

• Track targets moving at high speeds

• Manual control override

• Facial recognition

• Determine when a target is no longer a threat

• Alert target before firing

• Adjustable fire rate

SPECIFICATIONS

Description Constraint

Accuracy Rating > 70%

Number of Tracked Targets >= 3

Response Time < 1.50 seconds

Lateral Angle 140 degrees

Pitch Angle 90 degrees

Video Stream Resolution 1280 x 720 pixels

Weight < 30 lbs

Dimensions < 2’ x 2’ x 4’

Operation Time 3600 seconds

HARDWARE BLOCK DIAGRAM

Master

MCU

Yaw

Servo

Pitch

Servo

Trigger

Servo

Alarm

Cap

Touch

LEDs Laser

RS232

to I2C

Laptop

Servo

Controls

MICROCONTROLLER SELECTION

MSP430 Model G2231 G2203 F247

Operating Voltage 1.8 V – 3.6 V 1.8 V – 3.6 V 1.8 V – 3.6 V

GPIO Pins 14 16 48

Capture & Compares 2 6 10

Communication SPI, I2C SPI, I2C, UART SPI, I2C, UART

Dev Board Cost $4.30 $4.30 $75.00

Initialize

Bus

Data

MAIN MCU – MSP430G2231

Decode Data

State Flag

Transmit Pitch

Transmit Yaw

Set Trigger

Set Laser

Set Alarm

Transmit to PC

If Error

If Error

LED1 ON

LED2 ON

If Error LED3 ON

MAIN MCU – MSP430G2231

opCode (Hex) Function Method Description

0x00 to 0x8C setYawAngle(char opCode) Set yawAngle = opCode

0x90 to 0xEA setPitchAngle(char opCode) Set pitchAngle = opCode – 0x90

0xEC to 0xEF setTriggerMode(char opCode) Set trigger to off, auto, burst, single

0x8D retYawAngle(char yawAngle) Return yawAngle to PC

0xEB retPitchAngle(char PitchAngle) Return pitchAngle to PC

0xF0 to 0xF1 setAlarm(char opCode) Turn alarm off or on

0xF2 to 0xF3 setLaser(char opCode) Turn laser off or on

0xF4 to 0xF7 setLED(char opCode) Turn LEDs 1, 2, 3 off or on

0xFA to 0xFF reserved reserved

MAIN MCU – MSP430G2231

Return Type Function Description

void toBuffer(char data) Send data to the buffer

void encodeData(void) Pull data from the buffer and encode according

to specifications and push onto buffer

void decodeData(void) Pull data from the buffer and decode according

to specifications

void transmit(char address) Pull data from the buffer and transmit to another

microcontroller via I2C

void receive(void) Receive data from I2Cbus and push onto the

buffer

Initialize

CAP TOUCH MCU – MSP430G2452

Calibrate Cap

Read Cap

Set Flag

Touched Up

Touched Right

Touched Down

Touched Left

Touched Middle

pitchAngle++

yawAngle++

pitchAngle--

yawAngle--

Toggle Trigger

Encode

Buffer

Transmit

CAP TOUCH MCU – MSP430G2452

Return Type Function Description

void calibrateSensor(void) Measure capacitance of the air and store into

an array

char readTouch(void) Measures capacitance using TI’s API and return

value corresponding to up, right, etc

void toBuffer(char data) Send data to the buffer

void encodeData(void) Pull data from the buffer and encode according

to specifications and push onto buffer

void transmit(char address) Pull data from the buffer and transmit to another

microcontroller via I2C

SERVO MOTOR MCU – MSP430G2203

Initialize

Bus

Data

Determine Axis

Buffer

Apply PWM

Receive Angles

SERVO MOTOR FUNCTIONS

Return Type Function Description

Void findAxis(void) Determine the axis of received angle

void setPosition(void) Pull data from buffer and apply appropriate

PWM

void toBuffer(char data) Send data to the buffer

void receive(void) Receive data from I2Cbus and push onto the

buffer

SERVO MOTOR SPECIFICATIONS

Function Pitch Yaw Trigger

Model HS-805 HS-815 HS-81

Op. Voltage 4.80 V to 6.00 V 4.80 V to 6.00 V 4.80 V to 6.00 V

Torque 343 oz-in 343 oz-in 41.7 oz-in

Speed 0.14 S/60° 0.14 S/60° 0.09 S/60°

Current 830 mA 830 mA 280 mA

Dimensions 2.26” x 1.18” x 2.59” 2.26” x 1.18” x 2.59” 1.16” x 0.47” x 1.17”

Max Angle 90 degrees 140 degrees 90 degrees

Cost $39.99 $44.99 $17.99

RS232 I2C MCU – MSP430G2231

Initialize

Bus

Data

PC

Data

Transmit to Bus

Transmit to PC

RS232 I2C MCU – MSP430G2231

Return Type Function Description

void toBuffer(char data) Send data to the buffer

void transmit(char address) Pull data from the buffer and transmit to another

microcontroller via I2C

void receive(void) Receive data from I2Cbus and push onto the

buffer

void transmitPC(char

address)

Pull data from the buffer and transmit to another

to PC

void receivePC(void) Receive data from PC and push onto the buffer

AUDIBLE ALARM

Turbo series TMC-86-530-W

Mounting Panel Mounted

Operating mode Loud – Continuous

Operating Voltage 5-30 Vdc

Operating Frequency 2900 ± 250 Hz

Operating current 2mA at 5 Vdc

POWER SOURCE

12V/1.3Ah Sealed Lead Acid Battery

Model GH1213

Dimensions 3.80 “x 2.05” x 1.89”

Capacity 1300 mAh

Weight 1.3 lbs

Rechargeable Yes

Price $15.99

HARDWARE BLOCK DIAGRAM

Master

MCU

Alarm

3.3 Volt

Reg.

5.0 Volt

Reg

Cap

Touch

LEDs Laser

RS232

to I2C

Laptop

Power Supply

Yaw

Servo

Pitch

Servo

Trigger

Servo

Servo

Controls

PAINTBALL GUN

• AirTech E-Matrix electro pneumatic version.

• Adjustable max rate of fire through DIP switches located on PCB in the grip of gun.

• Nitrogen tank.

• Electronic Hopper.

BASE AND MOUNT

Sketch by Jon Waldmann

• 1280 x 720 pixels

• $30

• USB 2.0 port

• Automatic lighting adjustments

• Fixed position above the turret

LOGITECH

C310 HD WEBCAM

SOFTWARE FLOW DIAGRAM

Web Camera

cvtColor

goodFeaturesToTrack

cornerSubPix

calcOpticalFlowPyrLK

createTargets

setTargetThreat

trackTargets

chooseTarget

Fire

OPENCV

• Open source computer vision libraries.

• Using version 2.3.1 (newest released version)

• Maintained by Intel.

• Many useful functions for detecting and tracking objects and/or motion.

DETECTION AND TRACKING OPTIONS

• Iterative Lucas-Kanade Method with Pyramids

• Gunnar-Farneback

• Block Matching Method

• Horn-Schunck Algorithm

• Blob Tracking

• Color Tracking

• Facial Tracking

ITERATIVE LUCAS-KANADE METHOD

WITH PYRAMIDS

• Calculates optical flow between 2 images

• Given a set of points from a previous image, find those points in the new

image

• Assumes that neighbors of a pixel move in a similar fashion to that pixel itself

• Pyramids break into sub images. From there, the sub images are analyzed to

detect large motions

TARGET CREATION AND THREAT LEVEL

CALCULATION

• Keep a history of the movement of each pixel from the optical flow

calculation.

• Based on all similarly moving points in a neighboring area, create a bounding

box around each target.

• Place each target into an array of targets.

• Calculate threat level of each target based on size, velocity, and color.

• Sort target array based on threat level.

DECISION MAKING For no current target:

• Get target

• Check threat level

• Sound Warning

• Fire

For current target:

• Compare current and new target

• Choose highest priority

• If new target, warn

• Fire

MANUAL CONTROL

• Windows form application using Visual Studios 2010.

• The application will contain the video feed from the webcam and buttons to

perform the following actions by a user via on-board laptop.

• Rotate and fire the gun.

• Sound the alarm and play custom audio files.

• Rate of fire selection.

• Emergency stop button.

MANUAL CONTROL FUNCTIONS Return Type Function Name Method Description

Void rotateRight() Rotates the turret right by a set degree.

Void rotateLeft() Rotates the turret left by a set degree.

Void panUP() Tilts the turret upwards.

Void panDown() Tilts the turret downwards.

Void Fire() Fires the turret.

Void alertWarning()

Plays an audio file to warn intruders that they are

entering a dangerous zone and that they should

proceed to leave.

Void alertEngaged() Plays an audio file to warn intruders that they are about to be fired upon by ATS.

Void setFireRate(intfirerate) sets the firing rate of the turret 0 = automatic, 1 = burst, 2 = single shot.

MANUAL CONTROL – PROTOTYPE GUI

FACIAL RECOGNITION

• Security system to gain access to the manual controls of the turret.

• Scans a frame in an attempt to detect a human face using Haar-like features.

- Recognizable human facial features such as the dark areas under the eyes or cheek

color. Classifiers can also be trained to detect movement of the face such as

blinking, raising eyebrows, and flared nostrils.

• Once a face is detected the software will attempt to match the face to a face found in our

face database.

DIVISION OF LABOR

Daniel O’Hara

• Motion Tracking

• Threat Calculations

James Van Gostein

• Manual Control GUI

• Base and Mount

Ethan King

• Embedded Software

• Servo Motors

Stephen Rodriguez

• Power

• Facial Recognition

PROGRESS

10%

33%

42%

75%

75%

85%

90%

System Testing

Integration

Software

Parts Testing

Parts Aquired

Design

Research

BUDGET Part Cost Acquisition

Servo Motors $102.97 Purchased

MSP-EXP430 LaunchPad $4.30 Donated

MSP430 Capacitive Touch Booster Pack $10.00 Donated

Logitec C310 HD Webcam $29.99 Purchased

GH1213 Lead Acid Battery $15.99 To be purchased

Turbo series TMC-86-530-W Alarm $9.64 To be purchased

HK – E03358 Laser Pointer $14.99 To be purchased

PCB $33.00 To be purchased

Mounting Mats $100.00 As needed

E-Matrix, Hopper, Nitrogen tank. $150.00 Donated

Paintballs $29.99 Donated

Extra Electrical Components $30.00 As needed

Total retail cost $530.87

Total cost to group $336.58

STILL TO DO

• PCB

• Construct base and gun mounting.

• Create formula for converting pixel coordinates to angles.

• Interface laptop with embedded hardware.

• Creating targets, and threat level

• Facial recognition log-in

ISSUES AND MILESTONES

• Converting pixel coordinates to angles.

• Communication from PC to I2C bus.

• Construction of the base and gun mount.

Milestone Dates

Order PCB February 24th

Completion of Software March 9th

Completion of Base March 9th

Integration March 16th

System Testing March 23rd

Complete working prototype April 1st

QUESTIONS?