machine assisted plant positioning system (mapps)
TRANSCRIPT
Machine Assisted Plant Positioning System (MAPPS)
What we want:
Ecologists want to know how plants grow and survive in nature– How does the local environment effects growth
and survival?– How much does growth and survival vary from
year to year?
The problem:
• We need to:– Find the same individual plants– Measure their size precisely– Record their position precisely
• …and we need to do this year after year• …and we need to do this for THOUSANDS of
plants!
How we do this now:
• Pantograph—old fashioned drawing tool
THEN NOW
This is pretty time consuming
• https://www.youtube.com/watch?v=LNlDzhicqyk
• Requires 2 people in the field • Requires digitizing the paper maps into GIS
What our plots looks like What our data look like
Species
The Goal:
• Use image recognition software to help us collect data faster and more accurately
The Plan
A plot with plantsPlant 1
Plant 2
Plant 3
Plant 4
Plant 5
Video Camera—on tripod facing plot (or facing paper on table)
“Cursor”
Smart phone
With software for recording species and time
Marking an individual point:
1. Mark plant with cursor
Marking an individual point:
1. Mark plant with cursor
2. Scroll down menu selects species on phone with side buttons
Marking an individual point:
1. Mark plant with cursor
2. Scroll down menu selects species on phone with side buttons
3. Press button on phone to “mark” the plant
1. Mark plant with cursor
2. Scroll down menu selects species on phone with side buttons
3. Press button on phone to “mark” the plant
Phone records the time and species
Marking an individual point:
Marking a polygon:
1. Mark edge of polygon with cursor
2. Scroll down menu to species
Marking a polygon:
1. Mark edge of polygon with cursor
2. Scroll down menu to species
3. Trace plant while holding down button
Phone records species and start and stop time of button
Image processing
14:23:24
14:23:54
14:24:01
1. Video to stills 2. Cursor must be visible in each
image2. Each image has a time
Table of images
Merging image data with plant data
Object start time end time
species type
1 14:24:01 -- A. tripartita point
2 14:24:15 14:24:22 Poa secunda polygon
Table of species instances (from phone)
Frame _no. time
1 14:24:01
2 14:24:22
3 14:24:23
4 14:24:24
Table of species instances (from phone)
Simple merge by time!
Object start time end time
species type
1 14:24:01 -- A. tripartita point
2 14:24:15 14:24:22 Poa secunda polygon
Merging image data with plant data
Table of images
Frame _no. time
1 14:24:01
2 14:24:22
3 14:24:23
4 14:24:24
Table of species instances (from phone)
Simple merge by time!
Merging image data with species data
Table of images
Frame time object Species type
1 14:24:01 1 A. tripartita Point
2 14:24:22 2 Poa secunda Poly
3 14:24:23 2 Poa secunda Poly
4 14:24:24 2 Poa secunda Poly
New table with frames and species
14:24:01
1. Use openCV to find cursor color or shape in each image
2. Record x,y position3. Store in table with time of image
X
Y
Image processing
Frame time object Species X Y type
1 14:24:01 1 A. tripartita 15 3 Point
2 14:24:22 2 Poa secunda 20 14 Poly
3 14:24:23 2 Poa secunda 21 16 Poly
4 14:24:24 2 Poa secunda 22 14 Poly
Implementation• Develop smart phone app– Drop down menu to record time, object type and
species (should be easy)
• Image analysis program –openCV and Python
• Write R-script to merge tables (easy)
Potential Hurdles• Adjust for angle of cursor• Image of cursor may be blocked by plants or
person • Adjust for variation in camera position • Can it be done real-time so that maps can be
checked in the field?