![Page 1: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/1.jpg)
Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats
Aaron J. CorcoranAssistant Professor
Department of Biology
UC Colorado Springs
Presentation availablewww.sonarjamming.com
@AaronJCorcoran
![Page 2: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/2.jpg)
Tyson L. Hedrick
Professor
Biology Department
University of North Carolina
Michael S. Schirmacher
Director
Wind Energy Program
Bat Conservation International
![Page 3: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/3.jpg)
Image: Nickolay Hristov
![Page 4: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/4.jpg)
Contexts for 3D tracking
Images: N. HristovA. CorcoranM. Schirmacher
![Page 5: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/5.jpg)
3D Video tracking
Radar
On-board GPS
Videography
• High accuracy
• Behavioral detail
• Site-specific
Image: N. Hristov
![Page 6: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/6.jpg)
You can track bats in 3D!!!
Use nearly any camera setup
User-friendly software
No software licenses required
![Page 7: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/7.jpg)
Overview of 3D tracking
1. Video acquisition 3. 3D reconstruction2. Camera Calibration
![Page 8: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/8.jpg)
What kind of cameras?How many cameras?Synchronization
![Page 9: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/9.jpg)
What kind of cameras?
Near infrared
• Less expensive
• Need IR lights
• Flexible camera options
Thermal/far infrared• More expensive• No lights• Greater operating range
Images: Paul Colley (Left), Nick Hristov (right)
![Page 10: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/10.jpg)
Video Acquisition
2 Camera setup 3 Camera setup
Increased accuracy
Multi-object tracking
![Page 11: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/11.jpg)
Camera Synchronization
Cabled Sync• Frame-synced images
• Specialized cameras
• May require specialized acquisition software (e.g., Streampix)
Digital Signal generator
![Page 12: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/12.jpg)
Camera Synchronization
Audio Sync• Images not frame synced
• Synced in software• 1/44100 second accuracy
• Interpolate tracked XY points
Audio signal(Bat chirp)
Jackson et al. 2016 Biology Open
![Page 13: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/13.jpg)
Camera Calibration
Intrinsics
- Focal length/sensor size
- Lens distortion
![Page 14: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/14.jpg)
Camera Calibration
Intrinsics
- Focal length/sensor size
- Lens distortion
![Page 15: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/15.jpg)
![Page 16: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/16.jpg)
Field calibration:camera positions & orientations
![Page 17: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/17.jpg)
EasyWand
Theriault et al. 2014 J. Exp. Biol.
![Page 18: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/18.jpg)
Calibration using animals themselves!
Corcoran and Hedrick , 2019 eLife
![Page 19: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/19.jpg)
![Page 20: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/20.jpg)
Corcoran and Hedrick, 2019 eLife
![Page 21: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/21.jpg)
3D track generation
Automated 2D -> 3D assignment
Flexible track generation options
• Allowable error
• Gaps in detections
Camera 1 Camera 2
X
Y
Z
Evangelista et al. 2017 Proc. Roy. B
![Page 22: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/22.jpg)
What can you do with the data?
Flight parameters
• Accelerations, ground speeds, etc…
Emergence counts/Activity patterns
Interactions with turbinesX
Y
Z
![Page 23: Aaron J. Corcoran Open-source Software for Assistant ... · Open-source Software for Large-scale, High-throughput 3D Video Tracking of Bats Aaron J. Corcoran Assistant Professor Department](https://reader034.vdocuments.mx/reader034/viewer/2022050202/5f55df4b21188119481a4fbe/html5/thumbnails/23.jpg)
Funding:
Hopeful release date spring 2020 Presentation availablewww.sonarjamming.com
@AaronJCorcoran