![Page 1: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/1.jpg)
SKEYEBALL: REAL-TIME VISION SYSTEM FOR AN
AUTONOMOUS MODEL AIRPLANE Danko Antolovic
Pervasive Technology Laboratory, Indiana University, Bloomington
Bryce Himebaugh
Steven D. Johnson
Department of Computer Science, Indiana University, Bloomington
![Page 2: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/2.jpg)
Goals and Objectives of Skeyeball:
- Situated vision research:
- situational awareness => vehicle autonomy
- navigation
- target tracking
- collision avoidance (obstacles, terrain, other UAVs)
- Development of tools and methods for reliable embedded design:
- e.g. SPIDER avionics, under development at NASA/LRC
- Development of a platform for undergrad/graduate education
![Page 3: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/3.jpg)
Situated computer vision
Why?
-Vision is an ill-posed problem
-Research has lead to a profusion of algorithms, many ill-suited for control and guidance applications
Situated application places stringent constraints on algorithms and hardware architecture. A situated vision system must be:
-Embedded, i.e. compact and mobile
-Real-time
-Integrated into the control loop(s)
![Page 4: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/4.jpg)
The Vision Funnel
9.3 Mbytes/sec
YES/NO ?(~10 times/sec)
thresholding
edge detection
segmentation(logical analysis)
target identification
Data reduction by a factor of 10,000,000
![Page 5: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/5.jpg)
Start with a gray-scale aerial shot …
… threshold it to B/W
![Page 6: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/6.jpg)
… trace edges
… segment the edge map into connected components
![Page 7: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/7.jpg)
… calculate moments of inertia for each component
Moments are invariant under rotations and translations.
Move the camera to bring the target in the center of the vision field.
![Page 8: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/8.jpg)
15enable sigs.
4
addr
7
4
216
8
8
15
8
XC4010FPGA
back end
sampling clk.
digitized video
sync signals
LM1881
AD876
NTSCXC4010FPGA
front end
CY7C007AVDPRAM(32K)
Servos
Servo duty cycles
B/W threshold
MCF5307
Serial port
IRQ5
diagnosticsParallel port
8
duty cycles
servo motion
addr
data
data
video &threshold
servo data
PIC16F877
Servo position(analog)
4
2
controls
enable sigs.
Unified cache(DRAM only)
DRAM(8M)
Camera
To ground station (RF)
Architecture of the Vision Hardware
![Page 9: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/9.jpg)
![Page 10: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/10.jpg)
![Page 11: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/11.jpg)
![Page 12: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/12.jpg)
Measurement of the tracking speed
Maximum sweeping rate: 45 deg./sec, more than adequate for the airplane’s overflight speeds and a camera field of ca. 50 deg.
![Page 13: SK E Y E BALL: REAL-TIME VISION SYSTEM FOR AN AUTONOMOUS MODEL AIRPLANE Danko Antolovic](https://reader036.vdocuments.mx/reader036/viewer/2022081603/5681521b550346895dc05ea8/html5/thumbnails/13.jpg)
High-Priority Objectives …
- stable and unambiguous aerial tracking
- an orbiting mode, in which the aircraft autonomously circles above (or follows) an acquired target
- collision avoidance capability for stationary obstacles, to be extended in the future to avoidance of other UAVs