vegetable seedling feature extraction using stereo color imaging ta-te lin, jeng-ming chang...
TRANSCRIPT
VEGETABLE SEEDLING FEATURE EXTRACTION VEGETABLE SEEDLING FEATURE EXTRACTION USING STEREO COLOR IMAGINGUSING STEREO COLOR IMAGING
Ta-Te Lin, Jeng-Ming ChangTa-Te Lin, Jeng-Ming Chang
Department of Agricultural Machinery Engineering,Department of Agricultural Machinery Engineering,National Taiwan University,National Taiwan University,
Taipei, Taiwan, ROCTaipei, Taiwan, ROC
INTRODUCTIONINTRODUCTION
Plant growth measurement and Plant growth measurement and modelingmodeling
Machine vision techniqueMachine vision technique Seedling characteristicsSeedling characteristics Applications in production managementApplications in production management
OBJECTIVESOBJECTIVES
Implementation of stereo machine Implementation of stereo machine vision systemvision system
Development of image segmentation Development of image segmentation algorithmalgorithm
Development of seedling feature Development of seedling feature extraction algorithm extraction algorithm
3D reconstruction of seedling structure 3D reconstruction of seedling structure and graphical representationand graphical representation
SYSTEM IMPLEMENTATIONSYSTEM IMPLEMENTATION
Rotary stageImage processing board
RS-232 interface
Rotary stage
CCD Camera
RS-232 interface
Image processing board
IMAGE PROCESSING ALGORITHMIMAGE PROCESSING ALGORITHMAcquisition of top-view and
front-view images of the seedling
Rotate stage 90º to obtainside-view image of the seedling
Geometric calibration of thethree acquire images
Training and testing of imagesegmentation using neural network
Seedling image segmentation usingneural network
Image registration to find themain stem (central point) position
Determination of leaf number andaxial direction of each leaves
Image acquisition at eachcorresponding axial direction
Calculation of leaf area, nodalcoordinates and other features
3D-reconstruction and graphicsimulation of the seedling
Basic setup procedures
Feature extraction and m
easurement procedures
BASIC SETUP PROCEDURESBASIC SETUP PROCEDURES
Acquisition of top-view andfront-view images of the seedling
Acquisition of top-view andfront-view images of the seedling
Rotate stage 90º to obtainside-view image of the seedling
Rotate stage 90º to obtainside-view image of the seedling
Geometric calibration of thethree acquire images
Geometric calibration of thethree acquire images
Training and testing of imagesegmentation using neural network
Training and testing of imagesegmentation using neural network
FEATURE EXTRACTION AND MEASUFEATURE EXTRACTION AND MEASUREMENT PROCEDURESREMENT PROCEDURES
Seedling image segmentation using neural network
Seedling image segmentation using neural network
Image registration to find themain stem (central point) position
Image registration to find themain stem (central point) position
Determination of leaf number and axial direction of each leaf
Determination of leaf number and axial direction of each leaf
Image acquisition at each corresponding axial direction
Image acquisition at each corresponding axial direction
Calculation of leaf area, nodal coordinates and other features
Calculation of leaf area, nodal coordinates and other features
3D reconstruction and graphic simulation of the seedling
3D reconstruction and graphic simulation of the seedling
IMAGE SEGMENTATIONIMAGE SEGMENTATION
IMAGE SEGMENTATIONIMAGE SEGMENTATION
RR
GG
BB
Input layerInput layer Hidden layerHidden layer Output layerOutput layer
BackgroundBackground
ForegroundForeground
R
G
B
輸入層 隱藏層 輸出層
背景
前景物件
IMAGE SEGMENTATIONIMAGE SEGMENTATION
IMAGE REGISTRATIONIMAGE REGISTRATION
A
B
C
D
0
50
100
150
0 60 120 180 240 300 360
角度
距離
點數
Angle (degree)
LEAF NUMBER AND AXIAL LEAF NUMBER AND AXIAL DIRECTION DETERMINATIONDIRECTION DETERMINATION
LEAF NUMBER AND AXIAL LEAF NUMBER AND AXIAL DIRECTION DETERMINATIONDIRECTION DETERMINATION
Seedling height heightPetiole
Stem length to petiole
Petiole angle
Leaf stalk length
Leaf width
Leaf length
Internodal length
Seedling span
Projection area
Schematic of seedling features determined with the automatic mSchematic of seedling features determined with the automatic machine vision systemachine vision system
SEEDLING CHARACTERISTICSSEEDLING CHARACTERISTICS
Stem lengthStem length HeightHeight SpanSpan Total leaf areaTotal leaf area Top fresh weightTop fresh weight Top dry weightTop dry weight Number of leavesNumber of leaves Leaf area index, LAILeaf area index, LAI Leaf lengthLeaf length Leaf widthLeaf width
GEOMETRIC CALCULATION OF TGEOMETRIC CALCULATION OF THE TOTAL LEAF AREAHE TOTAL LEAF AREA
TRACING THE LEAF EDGE TO DETRACING THE LEAF EDGE TO DETERMINE THE LEAF ANGLETERMINE THE LEAF ANGLE
y = 1.12x + 1.82
R2 = 0.921
0
20
40
60
80
100
120
0 20 40 60 80 100 120Predicted Total Leaf Area (cm
2)
Act
ual T
otal
Lea
f Are
a (c
m2 )
Comparison of predicted total leaf area to the actural total leaf arComparison of predicted total leaf area to the actural total leaf area (cabbage seedlings).ea (cabbage seedlings).
y = 0.83x - 1.22
R2 = 0.928
0
10
20
30
40
50
60
70
0 10 20 30 40 50 60 70
Predicted Total Leaf Area (cm2)
Act
ual T
otal
Lea
f A
rea
(cm
2 )
Comparison of predicted total leaf area to the actural total leaf arComparison of predicted total leaf area to the actural total leaf area (Chinese mustard seedlings).ea (Chinese mustard seedlings).
A
B
3D RECONSTRUCTION OF SEED3D RECONSTRUCTION OF SEEDLING STRUCTURELING STRUCTURE
3D RECONSTRUCTION OF SEED3D RECONSTRUCTION OF SEEDLING STRUCTURELING STRUCTURE
Serial images of Chinese mustard seedlings at various growth Serial images of Chinese mustard seedlings at various growth stages. (images are not of the same scale) stages. (images are not of the same scale)
CALIBRATION FOR FRESH WEIGHT, DCALIBRATION FOR FRESH WEIGHT, DRY WEIGHT AND LEAF AREARY WEIGHT AND LEAF AREA
•Side-view projected area•Top-view projected area•Combined projected area•Calculated total leaf area
•Top fresh weight •Top dry weight•Total leaf area
Calibration functionCalibration function
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
0 2 4 6 8 10 12 14 16
Time (day)
Fres
h W
eigh
t (g)
Top fresh weight of Chinese mustard seedlings growing under arTop fresh weight of Chinese mustard seedlings growing under artificial lighting/pot treatment.tificial lighting/pot treatment.
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
0 2 4 6 8 10 12 14 16
Time (day)
Fres
h W
eigh
t (g)
Artificial lighting/Pot
Artificial lighting/Nursery tray
Natural lighting/Pot
Natural lighting/Nursery tray
Averaged top fresh weight of Chinese mustard seedlings growing Averaged top fresh weight of Chinese mustard seedlings growing under four different treatments.under four different treatments.
CONCLUSIONSCONCLUSIONS A non-destructive machine vision system based on A non-destructive machine vision system based on
stereo color imaging was successfully developed stereo color imaging was successfully developed for the measurement of vegetable seedlings.for the measurement of vegetable seedlings.
The seedling image segmentation was based on a The seedling image segmentation was based on a back-propagation neural network that allowed for back-propagation neural network that allowed for robust segmentation of seedling from background robust segmentation of seedling from background under natural lighting conditions.under natural lighting conditions.
The registration and mapping of coordinates from The registration and mapping of coordinates from top-view and lateral images allowed for the top-view and lateral images allowed for the determination of central point and stem location of determination of central point and stem location of the seedling. Based on this information, seedling the seedling. Based on this information, seedling leaf number and axial directions can be determined.leaf number and axial directions can be determined.
Image acquisition based on the information of Image acquisition based on the information of leaf axial directions provided better accuracy in leaf axial directions provided better accuracy in extracting the seedling features.extracting the seedling features.
The leaf area of seedling can be predicted based The leaf area of seedling can be predicted based on the projection leaf area and leaf angle with on the projection leaf area and leaf angle with satisfactory accuracy.satisfactory accuracy.
The measured nodal and stem coordinates The measured nodal and stem coordinates allowed for 3D reconstruction of the vegetable allowed for 3D reconstruction of the vegetable seedling for graphic display.seedling for graphic display.
CONCLUSIONSCONCLUSIONS
THANK YOUTHANK YOU
謝 謝謝 謝