autonomous targeting sentry(ats) group 12
TRANSCRIPT
![Page 1: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/1.jpg)
Ethan King - Electrical Engineer
Daniel O’Hara – Computer Engineer
Stephen Rodriguez – Computer Engineer
James Van Gostein – Computer Engineer
Autonomous Targeting Sentry
GROUP
12
ATS
![Page 2: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/2.jpg)
PROJECT OVERVIEW AND MOTIVATION
Prototype an autonomous turret to defend high priority areas
Detect and track targets using computer vision, determine threat and eliminate
To reduce the amount of human interaction in combat situations
![Page 3: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/3.jpg)
OBJECTIVES
• Lightweight
• Cost efficient
• Capable of detecting, prioritizing and tracking targets autonomously
• Track targets moving at high speeds
• Manual control override
• Facial recognition
• Determine when a target is no longer a threat
• Alert target before firing
• Adjustable fire rate
![Page 4: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/4.jpg)
SPECIFICATIONS
Description Constraint
Accuracy Rating > 70%
Number of Tracked Targets >= 3
Response Time < 1.50 seconds
Lateral Angle 140 degrees
Pitch Angle 90 degrees
Video Stream Resolution 1280 x 720 pixels
Weight < 30 lbs
Dimensions < 2’ x 2’ x 4’
Operation Time 3600 seconds
![Page 5: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/5.jpg)
HARDWARE BLOCK DIAGRAM
Master
MCU
Yaw
Servo
Pitch
Servo
Trigger
Servo
Alarm
Cap
Touch
LEDs Laser
RS232
to I2C
Laptop
Servo
Controls
![Page 6: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/6.jpg)
MICROCONTROLLER SELECTION
MSP430 Model G2231 G2203 F247
Operating Voltage 1.8 V – 3.6 V 1.8 V – 3.6 V 1.8 V – 3.6 V
GPIO Pins 14 16 48
Capture & Compares 2 6 10
Communication SPI, I2C SPI, I2C, UART SPI, I2C, UART
Dev Board Cost $4.30 $4.30 $75.00
![Page 7: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/7.jpg)
Initialize
Bus
Data
MAIN MCU – MSP430G2231
Decode Data
State Flag
Transmit Pitch
Transmit Yaw
Set Trigger
Set Laser
Set Alarm
Transmit to PC
If Error
If Error
LED1 ON
LED2 ON
If Error LED3 ON
![Page 8: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/8.jpg)
MAIN MCU – MSP430G2231
opCode (Hex) Function Method Description
0x00 to 0x8C setYawAngle(char opCode) Set yawAngle = opCode
0x90 to 0xEA setPitchAngle(char opCode) Set pitchAngle = opCode – 0x90
0xEC to 0xEF setTriggerMode(char opCode) Set trigger to off, auto, burst, single
0x8D retYawAngle(char yawAngle) Return yawAngle to PC
0xEB retPitchAngle(char PitchAngle) Return pitchAngle to PC
0xF0 to 0xF1 setAlarm(char opCode) Turn alarm off or on
0xF2 to 0xF3 setLaser(char opCode) Turn laser off or on
0xF4 to 0xF7 setLED(char opCode) Turn LEDs 1, 2, 3 off or on
0xFA to 0xFF reserved reserved
![Page 9: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/9.jpg)
MAIN MCU – MSP430G2231
Return Type Function Description
void toBuffer(char data) Send data to the buffer
void encodeData(void) Pull data from the buffer and encode according
to specifications and push onto buffer
void decodeData(void) Pull data from the buffer and decode according
to specifications
void transmit(char address) Pull data from the buffer and transmit to another
microcontroller via I2C
void receive(void) Receive data from I2Cbus and push onto the
buffer
![Page 10: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/10.jpg)
Initialize
CAP TOUCH MCU – MSP430G2452
Calibrate Cap
Read Cap
Set Flag
Touched Up
Touched Right
Touched Down
Touched Left
Touched Middle
pitchAngle++
yawAngle++
pitchAngle--
yawAngle--
Toggle Trigger
Encode
Buffer
Transmit
![Page 11: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/11.jpg)
CAP TOUCH MCU – MSP430G2452
Return Type Function Description
void calibrateSensor(void) Measure capacitance of the air and store into
an array
char readTouch(void) Measures capacitance using TI’s API and return
value corresponding to up, right, etc
void toBuffer(char data) Send data to the buffer
void encodeData(void) Pull data from the buffer and encode according
to specifications and push onto buffer
void transmit(char address) Pull data from the buffer and transmit to another
microcontroller via I2C
![Page 12: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/12.jpg)
SERVO MOTOR MCU – MSP430G2203
Initialize
Bus
Data
Determine Axis
Buffer
Apply PWM
Receive Angles
![Page 13: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/13.jpg)
SERVO MOTOR FUNCTIONS
Return Type Function Description
Void findAxis(void) Determine the axis of received angle
void setPosition(void) Pull data from buffer and apply appropriate
PWM
void toBuffer(char data) Send data to the buffer
void receive(void) Receive data from I2Cbus and push onto the
buffer
![Page 14: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/14.jpg)
SERVO MOTOR SPECIFICATIONS
Function Pitch Yaw Trigger
Model HS-805 HS-815 HS-81
Op. Voltage 4.80 V to 6.00 V 4.80 V to 6.00 V 4.80 V to 6.00 V
Torque 343 oz-in 343 oz-in 41.7 oz-in
Speed 0.14 S/60° 0.14 S/60° 0.09 S/60°
Current 830 mA 830 mA 280 mA
Dimensions 2.26” x 1.18” x 2.59” 2.26” x 1.18” x 2.59” 1.16” x 0.47” x 1.17”
Max Angle 90 degrees 140 degrees 90 degrees
Cost $39.99 $44.99 $17.99
![Page 15: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/15.jpg)
RS232 I2C MCU – MSP430G2231
Initialize
Bus
Data
PC
Data
Transmit to Bus
Transmit to PC
![Page 16: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/16.jpg)
RS232 I2C MCU – MSP430G2231
Return Type Function Description
void toBuffer(char data) Send data to the buffer
void transmit(char address) Pull data from the buffer and transmit to another
microcontroller via I2C
void receive(void) Receive data from I2Cbus and push onto the
buffer
void transmitPC(char
address)
Pull data from the buffer and transmit to another
to PC
void receivePC(void) Receive data from PC and push onto the buffer
![Page 17: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/17.jpg)
AUDIBLE ALARM
Turbo series TMC-86-530-W
Mounting Panel Mounted
Operating mode Loud – Continuous
Operating Voltage 5-30 Vdc
Operating Frequency 2900 ± 250 Hz
Operating current 2mA at 5 Vdc
![Page 18: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/18.jpg)
POWER SOURCE
12V/1.3Ah Sealed Lead Acid Battery
Model GH1213
Dimensions 3.80 “x 2.05” x 1.89”
Capacity 1300 mAh
Weight 1.3 lbs
Rechargeable Yes
Price $15.99
![Page 19: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/19.jpg)
HARDWARE BLOCK DIAGRAM
Master
MCU
Alarm
3.3 Volt
Reg.
5.0 Volt
Reg
Cap
Touch
LEDs Laser
RS232
to I2C
Laptop
Power Supply
Yaw
Servo
Pitch
Servo
Trigger
Servo
Servo
Controls
![Page 20: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/20.jpg)
PAINTBALL GUN
• AirTech E-Matrix electro pneumatic version.
• Adjustable max rate of fire through DIP switches located on PCB in the grip of gun.
• Nitrogen tank.
• Electronic Hopper.
![Page 21: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/21.jpg)
BASE AND MOUNT
Sketch by Jon Waldmann
![Page 22: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/22.jpg)
![Page 23: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/23.jpg)
• 1280 x 720 pixels
• $30
• USB 2.0 port
• Automatic lighting adjustments
• Fixed position above the turret
LOGITECH
C310 HD WEBCAM
![Page 24: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/24.jpg)
SOFTWARE FLOW DIAGRAM
Web Camera
cvtColor
goodFeaturesToTrack
cornerSubPix
calcOpticalFlowPyrLK
createTargets
setTargetThreat
trackTargets
chooseTarget
Fire
![Page 25: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/25.jpg)
OPENCV
• Open source computer vision libraries.
• Using version 2.3.1 (newest released version)
• Maintained by Intel.
• Many useful functions for detecting and tracking objects and/or motion.
![Page 26: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/26.jpg)
DETECTION AND TRACKING OPTIONS
• Iterative Lucas-Kanade Method with Pyramids
• Gunnar-Farneback
• Block Matching Method
• Horn-Schunck Algorithm
• Blob Tracking
• Color Tracking
• Facial Tracking
![Page 27: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/27.jpg)
ITERATIVE LUCAS-KANADE METHOD
WITH PYRAMIDS
• Calculates optical flow between 2 images
• Given a set of points from a previous image, find those points in the new
image
• Assumes that neighbors of a pixel move in a similar fashion to that pixel itself
• Pyramids break into sub images. From there, the sub images are analyzed to
detect large motions
![Page 28: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/28.jpg)
TARGET CREATION AND THREAT LEVEL
CALCULATION
• Keep a history of the movement of each pixel from the optical flow
calculation.
• Based on all similarly moving points in a neighboring area, create a bounding
box around each target.
• Place each target into an array of targets.
• Calculate threat level of each target based on size, velocity, and color.
• Sort target array based on threat level.
![Page 29: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/29.jpg)
DECISION MAKING For no current target:
• Get target
• Check threat level
• Sound Warning
• Fire
For current target:
• Compare current and new target
• Choose highest priority
• If new target, warn
• Fire
![Page 30: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/30.jpg)
MANUAL CONTROL
• Windows form application using Visual Studios 2010.
• The application will contain the video feed from the webcam and buttons to
perform the following actions by a user via on-board laptop.
• Rotate and fire the gun.
• Sound the alarm and play custom audio files.
• Rate of fire selection.
• Emergency stop button.
![Page 31: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/31.jpg)
MANUAL CONTROL FUNCTIONS Return Type Function Name Method Description
Void rotateRight() Rotates the turret right by a set degree.
Void rotateLeft() Rotates the turret left by a set degree.
Void panUP() Tilts the turret upwards.
Void panDown() Tilts the turret downwards.
Void Fire() Fires the turret.
Void alertWarning()
Plays an audio file to warn intruders that they are
entering a dangerous zone and that they should
proceed to leave.
Void alertEngaged() Plays an audio file to warn intruders that they are about to be fired upon by ATS.
Void setFireRate(intfirerate) sets the firing rate of the turret 0 = automatic, 1 = burst, 2 = single shot.
![Page 32: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/32.jpg)
MANUAL CONTROL – PROTOTYPE GUI
![Page 33: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/33.jpg)
FACIAL RECOGNITION
• Security system to gain access to the manual controls of the turret.
• Scans a frame in an attempt to detect a human face using Haar-like features.
- Recognizable human facial features such as the dark areas under the eyes or cheek
color. Classifiers can also be trained to detect movement of the face such as
blinking, raising eyebrows, and flared nostrils.
• Once a face is detected the software will attempt to match the face to a face found in our
face database.
![Page 34: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/34.jpg)
DIVISION OF LABOR
Daniel O’Hara
• Motion Tracking
• Threat Calculations
James Van Gostein
• Manual Control GUI
• Base and Mount
Ethan King
• Embedded Software
• Servo Motors
Stephen Rodriguez
• Power
• Facial Recognition
![Page 35: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/35.jpg)
PROGRESS
10%
33%
42%
75%
75%
85%
90%
System Testing
Integration
Software
Parts Testing
Parts Aquired
Design
Research
![Page 36: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/36.jpg)
BUDGET Part Cost Acquisition
Servo Motors $102.97 Purchased
MSP-EXP430 LaunchPad $4.30 Donated
MSP430 Capacitive Touch Booster Pack $10.00 Donated
Logitec C310 HD Webcam $29.99 Purchased
GH1213 Lead Acid Battery $15.99 To be purchased
Turbo series TMC-86-530-W Alarm $9.64 To be purchased
HK – E03358 Laser Pointer $14.99 To be purchased
PCB $33.00 To be purchased
Mounting Mats $100.00 As needed
E-Matrix, Hopper, Nitrogen tank. $150.00 Donated
Paintballs $29.99 Donated
Extra Electrical Components $30.00 As needed
Total retail cost $530.87
Total cost to group $336.58
![Page 37: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/37.jpg)
STILL TO DO
• PCB
• Construct base and gun mounting.
• Create formula for converting pixel coordinates to angles.
• Interface laptop with embedded hardware.
• Creating targets, and threat level
• Facial recognition log-in
![Page 38: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/38.jpg)
ISSUES AND MILESTONES
• Converting pixel coordinates to angles.
• Communication from PC to I2C bus.
• Construction of the base and gun mount.
Milestone Dates
Order PCB February 24th
Completion of Software March 9th
Completion of Base March 9th
Integration March 16th
System Testing March 23rd
Complete working prototype April 1st
![Page 39: Autonomous Targeting Sentry(ATS) Group 12](https://reader030.vdocuments.mx/reader030/viewer/2022012805/61bd378d61276e740b1083d0/html5/thumbnails/39.jpg)
QUESTIONS?