Three-dimensional shape characterization for particle aggregates using multiple projective representations

Jonathan CorriveauThesis Advisor: Dr. Shreekanth Mandayam

Committee: Dr. Beena Sukumaran and Dr. Robi Polikar

Rowan UniversityCollege of Engineering201 Mullica Hill RoadGlassboro, NJ 08028

(856) 256-5330http://engineering.rowan.edu/

Thursday, April 20, 2023

Outline

Introduction Objectives of Thesis Previous Work Approach Results Conclusions

Characterizing Shapes Shapes are described by names

Circle, Triangle, Rectangle, etc. Not possible for complicated shapes

Shapes need to be described by numbers Most shapes can be described by a set of

numbers Computers need numbers Similar shapes must have similar values Few as possible is desirable

Shapes

Rectangle Circle Triangle

Arbitrary Shape

Application

Computer Vision Face Recognition Fingerprint matching

Image 1 Image 2 Image 1 Image 2

Images Match Images Do Not Match

Phi 1: 1.058 1.2377Phi 2: 2.664 3.403Phi 3: 9.4284 7.8057Phi 4: 14.2453 13.702Phi 5: 29.8432 27.6783Phi 6: 16.0222 16.1285Phi 7: 29.4245 28.2324

Application

Character Recognition

Phi 1: 1.0292Phi 2: 2.5359Phi 3: 8.917Phi 4: 14.1381Phi 5: 29.2098Phi 6: 15.4456Phi 7: 29.1866

Descriptor DatabaseCharacter Descriptors a b

Motivation Soil Behavior

Strong relationship between stress-strain behavior of soils and the inherent characteristics of its individual particles

Inherent Particle Characteristics

Hardness, Specific Gravity Distribution

Shape and Angularity

Particle Size and Size Distribution

SEM Picture of Dry Sand

Aggregate Mixtures

Michigan Dune Sand#1 Dry Sand

Daytona Beach Sand Glass Beads

Motivation

Currently 2-D methods are not enough to characterize a soil mixture for discrete element model

Only behavior trends can be captured using 2-D models

3-D information allows a much more accurate model

3-D Shapes 3-D shapes are difficult to characterize as a set of numbers

Require sophisticated equipment Large databases of numbers to record the position of each

coordinate Aggregates of 3-D objects

A collection of 3-D particles must be characterized by a set of numbers

2-D Shapes

Computationally inexpensive Many methods already exist for characterizing

2-D shapes Can easily be implemented on a computer

with only digital images Question: How can 2-D methods help with

finding a 3-D solution?

Objectives of Thesis

Design automated algorithms that can estimate 3-D shape descriptors for particle aggregates using a statistical combination of 2-D shape descriptors from multiple 2-D projections.

Demonstrate consistency, separability and uniqueness of the 3-D shape-descriptor algorithm by exercising the method on a set of sand particle mixes.

Preliminary efforts towards the demonstration of the algorithm’s ability to accurately and repeatably construct composite 3-D shapes from multiple 2-D shape-descriptors.

Desirable Descriptor Qualities

Fundamental Qualities Uniqueness Parsimony Independent Invariance

Rotation Scale Translation

Original

Rotation Scale Translation

Additional Qualities

Reconstruction Allow for a shape to be constructed from

the descriptors Interpretation

Relate to some physical property Automatic Collection

Collection and evaluation automation Removes human error

Previous Work

Proponents Method Explanation

Sebestyn andBenson

“unrolling” a closed outline

The concept of creating a 1-D function from a 2-D boundary. Introduced by

Benson into the field of geology.

Hu2-D Invariant

Moments2-D moments that invariant to translation,

rotation, scale and reflection.

Ehrlich and Weinberg

Radius ExpansionIntroduced Fourier analysis for radius

expansion into sedimentology.

Medalia Equivalent EllipsesFits an ellipse to have similar properties

to the actual shape. Does not need outline.

Davis and Dexter Chord to PerimeterMeasures chord lengths between various

points along an outline.

Previous WorkProponents Method Explanation

Zahn and Roskies Angular BendIntroduced by Sebestyn, but made widely

known by Zahn and Roskies. Discretize an outline into a series of straight lines and angles

GranlundFourier

DescriptorsUses x+jy from the coordinates of an outline

to be analyzed by Fourier analysis.

Sadjadi and Hall3-D Invariant

Moments3-D moments that are invariant to translation,

rotation, and scale.

Garboczi, Martys, Saleh, and Livingston

Spherical Harmonics

A process similar to 3-D Fourier analysis, and requires 3-D information.

Sukumaran and Ashmawy

Shape and Angularity

Factor

Compares shapes to circles and measures their deviation. Uses a mean and standard deviation

of many particles to compare a mixes.

Radius Expansion

R1

R2R3

R4

Radius Expansion

x

y

R1()

R2()

Angular Bend

1 2

L1L2

L3

Complex Coordinates

y

x

(x1, y1)

Chord to Perimeter The covered perimeter length divided by total

perimeter determines the amount of irregularity Small ratio measures small irregularities Approaching one measures large irregularities

Chord Length

Perimeter Length

Equivalent Ellipses Two factors are calculated from ellipses

Anisometry – ratio of long to short axis of ellipse

Bulkiness – ratio of areas of figure and ellipse

Approach: Premise

2-D images of 3-D particles in an aggregate mix can be used to denote 2-D projections of a composite 3-D particle that represent the entire mixture

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2

2222

2111

,

,

,

NNND

D

D

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2-D facets of 3-D particles in mix

3-D Descriptors2-D Descriptors from Mix 2-D Descriptors from Particle

2-D facets of Composite Particle

Composite 3-D “Reconstruction”

Overview of Approach

Particles

Orientation Every particle observed offers a different

angle of a composite particle Many different facets should be represented

by the images Regularity

Similar particles should have similar shapes

Aggregate Mixtures

Michigan Dune Sand#1 Dry Sand

Daytona Beach Sand Glass Beads

Similar shapes should have similar descriptors Find a distribution for each descriptor from all

particle images Calculate both the mean and variance that

characterize the distribution Allows a set of 2-D projections to represent a

composite 3-D object using a small set of numbers

Statistics

[S1, S2, S3, S4,…… SN]

[S1, S2, S3, S4,…….SN]s3

f(s3)

m3

2

1

3-D aggregate mixes can be characterized by a set of numbers

Multiple 2-D images can be used to construct a single composite 3-D object

Very little equipment required Microscope and Camera (data collection) Computer (analysis)

From 2-D to 3-D

Shape Characterization Methods

Complex Coordinate Fourier Analysis Allows random generation of projections

from 3-D descriptors Invariant Moments

Requires less computation, less preprocessing, and is more parsimonious, but does not allow projection generation

Fourier Analysis

Object must be described as a function Function should be periodic

Fourier Transform can be applied to analyze the frequencies Low Frequencies hold general shape

information, while high frequencies carry more detail

Effective for compression since reconstruction is possible with fewer values than the original

Fourier Descriptors

0 500 1000 1500 200050

100

150

200

250

0 200 400 600 80050

100

150

200

250

Fourier Descriptors

Descriptors

Near Zero Values

Moments

Statistical moments Normalized combinations of mean,

variance, and higher order moments Moments of similar objects should share

similar moment calculations 2-D moments evaluate the images

without having to extract the boundary Parsimonious (only 7 moments)

2-D Central Moments

Equation of 2-D moment is given as:

Central moments:

dxdyyxfyxm qppq ,

dxdyyxfyyxxqp

pq ,

For a digital image the discrete equation becomes:

Normalized Central Moments are defined as:

Moments

yxfyyxxqp

yxpq ,

00

pqpq where,where, 1

2

qp

Invariant Moments

02201

211

202202 4

20321

212303 33

20321

212304

20321

21230210303217 33

20321

21230123012305 33

20321

2123003210321 33

20321

2123003213012 33

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2

2222

2111

,

,

,

NNND

D

D

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2-D facets of 3-D particles in mix

3-D Descriptors2-D Descriptors from Mix 2-D Descriptors from Particle

2-D facets of Composite Particle

Composite 3-D “Reconstruction”

Overview of Approach

Creation of Composite Particle

“Reconstruction” of 3-D Composite Particle

Three techniques were tested for constructing a 3-D composite particle using 2-D projections Extrusion Rotation into 3-D Tomographic

Extrusion Method

Rotation into 3-D Method

Tomographic Method

Implementation and Results

Experimental Setup Normalization and Results of Complex

Coordinate Fourier Analysis Invariant Moment Results Preliminary “reconstruction” results of

the different methods introduced

Experimental Setup

#1 Dry Sand

Daytona Beach Sand

Glass Bead

Optical Microscope, Digital Camera, and Computer

Data SamplesEquipment

Preprocessing of Images

Final Image Cleaned

InvertedBlack and WhiteOriginal Image

Obtaining Fourier Descriptors

120 140 160 180 200 220 240 260 280-200

-180

-160

-140

-120

-100

-80

x Coordinates

y C

oo

rdin

ate

s

0 100 200 300 400180

200

220

240

260

280

300

320

Number of Points

|x+

jy|

0 10 20 30 400

500

1000

1500

2000

Coefficient Number

Am

plit

ud

e

Edge detection of the image Plot of coordinates extracted from image

Plotted as a 1-D Function FFT of 1-D Signal

100 150 200 250 300-200

-180

-160

-140

-120

-100

-80Reconstruction using all Coefficients

x Coordinates

y C

oo

rdin

ate

sReconstruction of 2-D Projections

Reconstruction using all descriptors Reconstruction using 20 descriptors

0 10 20 30 400

500

1000

1500

2000

Coefficient Number

Am

plit

ud

e

0 10 20 30 400

500

1000

1500

2000

Coefficient Number

Am

plit

ud

e

100 150 200 250 300-200

-180

-160

-140

-120

-100

-80

x Coordinates

y C

oo

rdin

ate

s

Frequency Normalization Process

Original Image Half-Sized Image

Original Functions and FFTs

100 200 300 400

80

100

120

140

160

180

200

Number of Points

X C

oo

rdin

ate

50 100 150 200110

120

130

140

150

160

170

180

Number of Points

X C

oo

rdin

ate

0 10 20 30 400

100

200

300

400

500

600

700

Fourier Coefficient

Ma

gn

itud

e

0 10 20 30 400

100

200

300

400

500

600

700

Fourier Coefficient

Ma

gn

itud

e

Original Image Half-Sized Image

After Normalization

0 50 100 150 200 250-1

-0.5

0

0.5

1

Number of Points

No

rma

lize

d X

Co

ord

ina

te

0 50 100 150 200 250-1

-0.5

0

0.5

1

Number of Points

No

rma

lize

d X

Co

ord

ina

te

0 10 20 30 400

5

10

15

20

25

Fourier Coefficient

Ma

gn

itud

e

0 10 20 30 400

5

10

15

20

25

Fourier Coefficient

Ma

gn

itud

e

Original Image Half-Sized Image

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2

2222

2111

,

,

,

NNND

D

D

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2-D facets of 3-D particles in mix

3-D Descriptors2-D Descriptors from Mix 2-D Descriptors from Particle

2-D facets of Composite Particle

Composite 3-D “Reconstruction”

Overview of Approach

Statistics of Fourier Descriptors

-2 0 20

20

40

60

80

100

120

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

-1 -0.5 00

20

40

60

80

100

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

-1 0 10

10

20

30

40

50

60

70

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

-1 0 10

20

40

60

80

100

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

-0.5 0 0.50

10

20

30

40

50

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

-0.2 0 0.20

10

20

30

40

50

60

70

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

-0.5 0 0.50

5

10

15

20

25

30

35

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

-0.2 0 0.20

10

20

30

40

50

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

#1 Dry Sand Standard Melt Sand

Daytona Beach Sand Michigan Dune Sand

Ellipsoid Model for 3-D Shape Characterization

Radius in X – Variance of First Descriptor

Radius in Y - Variance of Second Descriptor

Radius in Z – Variance of Third Descriptor

Center of Ellipsoid – 3-D Coordinate of Descriptor Means

x

z

y

Separability of Soil Mixes using Fourier Descriptors

Glass Bead

#1 Dry

Melt

Michigan Dune

Daytona Beach

Classification Effectiveness using Fourier Descriptors

Glass Bead

#1 Dry

Melt

Michigan Dune

Daytona Beach

Invariant Moments of Similar Images

Original Rotated and Resized

Invariant Moments of Similar Images

Invariant Moments

Image 1 Image 2 Difference

1 7.1164 7.1176 0.02%

2 15.2953 15.3027 0.05%

3 12.4116 12.1704 1.94%

4 25.1942 25.2073 0.05%

5 50.3498 49.0710 2.54%

6 32.9973 33.0157 0.06%

7 50.7640 50.7842 0.04%

Invariant Moments of Dissimilar Images

Image 1 Image 2

Invariant Moments of Dissimilar Images

Invariant Moments

Image 1 Image 2 Difference

1 7.1164 7.2694 2.15%

2 15.2953 16.7749 9.67%

3 12.4116 17.4857 40.88%

4 25.1942 26.9251 6.87%

5 50.3498 52.5509 4.37%

6 32.9973 35.5863 7.85%

7 50.7640 52.5528 3.52%

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2

2222

2111

,

,

,

NNND

D

D

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2-D facets of 3-D particles in mix

3-D Descriptors2-D Descriptors from Mix 2-D Descriptors from Particle

2-D facets of Composite Particle

Composite 3-D “Reconstruction”

Overview of Approach

Statistics of Invariant Moment Descriptors

6.5 7 7.50

20

40

60

80

100

120

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

10 20 300

20

40

60

80

100

120

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

6.5 7 7.50

20

40

60

80

100

120

140

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

10 15 200

10

20

30

40

50

60

70

80

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

6.5 7 7.50

10

20

30

40

50

60

70

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

10 20 300

20

40

60

80

100

120

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

7 7.50

5

10

15

20

25

30

35

40

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

10 15 200

10

20

30

40

50

Descriptor Value

Nu

mb

er

of D

esc

ripto

rs

Daytona Beach Sand Michigan Dune Sand

#1 Dry Sand Standard Melt Sand

Separability of Soil Mixes using Invariant Moment Descriptors

#1 Dry

Melt

Michigan Dune

Daytona Beach

Classification Effectiveness using Invariant Moment Descriptors

#1 Dry

Melt

Michigan Dune

Daytona Beach

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2

2222

2111

,

,

,

NNND

D

D

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2-D facets of 3-D particles in mix

3-D Descriptors2-D Descriptors from Mix 2-D Descriptors from Particle

2-D facets of Composite Particle

Composite 3-D “Reconstruction”

Overview of Approach

Reconstruction of Projections from 3-D Descriptors

Original Image Reconstructed Image

Generation of Random Projections from 3-D Descriptors

Separability of Soil Mixes using Randomly Generated Projections

Comparison between Original and Generated Projections

1-Original

2-Generated

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2

2222

2111

,

,

,

NNND

D

D

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2-D facets of 3-D particles in mix

3-D Descriptors2-D Descriptors from Mix 2-D Descriptors from Particle

2-D facets of Composite Particle

Composite 3-D “Reconstruction”

Overview of Approach

Extrusion Method

2nd Projection of Dry Sand

1st Projection of Dry Sand

3rd Projection of Dry Sand

All Projections in 3-D Space

Implementation of Extrusion Method on Dry Sand

Projections after Extrusion

Final “Reconstruction”

Effectiveness of Extrusion “Reconstructed” Composite Particle

#1 Dry

Melt

Michigan Dune

Daytona Beach

Rotate Into 3–D Method for Dry Sand

Effectiveness of Rotation into 3-D “Reconstructed” Composite Particle

#1 Dry

Melt

Michigan Dune

Daytona Beach

Tomographic Method

Effectiveness of Tomographic “Reconstructed” Composite Particle

#1 Dry

Melt

Michigan Dune

Daytona Beach

Results of Dry Sand “Reconstruction”

Reconstruction Method

Inter-ellipsoid Distance Percentages Dry Melt Daytona Beach Michigan Dune

Extrusion 31% 40% 100% 46%

3-D Rotation 60% 73% 100% 17%

Tomographic 45% 33% 100% 85%

Distance

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2

2222

2111

,

,

,

NNND

D

D

ND

D

D

2

1

ND

D

D

2

1

ND

D

D

2

1

2-D facets of 3-D particles in mix

3-D Descriptors2-D Descriptors from Mix 2-D Descriptors from Particle

2-D facets of Composite Particle

Composite 3-D “Reconstruction”

Conclusion

Summary of Accomplishments Development of automated algorithms that can

estimate 3-D shape descriptors for particle aggregates Statistical combination of 2-D shape descriptors from multiple

2-D projections

Database containing a library of 2-D digital images for 5 aggregate mixtures

PCA and ellipsoid model to show consistency, separability and uniqueness of the algorithm

Composite 3-D shapes from multiple 2-D projections. Extrusion, Rotation and Tomographic reconstruction

Conclusions

Dissimilar soil mixes can be separated using the descriptor algorithms

Generation of random projections from the Fourier descriptors proves to be effective

Construction of a 3-D composite particle using a collection of 2-D projections appears feasible

Recommendations for Future Work

The optimal number and value of descriptors can be found, which allows the greatest separability

More work on Reconstruction Methods Extrusion – use more projections on more axes Tomographic – Rotate more images about

multiple axes and combine objects Apply composite particles created to a

discrete element model Algorithms can be applied to other

application areas (i.e. ink toner, industrial)

Acknowledgements

National Science Foundation, Division of Civil and Mechanical Systems, Geomechanics and Geotechnic Systems Program, Award #0324437

Dr. Shreekanth Mandayam, Dr. Beena Sukumaran, and Dr. Robi Polikar

Michael Kim and Scott Papson