parallel wave propagation and topological operators for fragmentation...
TRANSCRIPT
Parallel Wave Propagation and
Topological Operators for Fragmentation Simulation
Glaucio H. PaulinoGlaucio H. Paulino
Professor, Faculty CEE, MechSE, CSE
Donald Biggar Willett Professor of Engineering
6th Annual Workshop on CHARM++ and its Applications
Acknowledgments
CEE COLLABORATOR:
� Mr. Kyoungsoo Park
CS COLLABORATORS:
� Prof. Laxmikant V. Kale (UIUC)
http//cee.uiuc.edu/paulino [email protected]/15/2008 2
� Dr. Celso L. Mendes (UIUC)
� Dr. Terry L. Wilmarth (UIUC)
� Mr. Aaron Becker (UIUC)
� Mr. Isaac Dooley (UIUC)
� Prof. Waldemar Celes (PUC-Rio)
� Mr. Rodrigo Espinha (PUC-Rio)
Stress waves
Waves
http//cee.uiuc.edu/paulino [email protected]/15/2008 3
Rosakis AJ, Samudrala O, Coker D, 1999, Science 284
Outline
� Waves
� Wave Propagation
� Rayleigh wave
� Parallel computing: ParFUM
� Results: Geological simulation
Dynamic Fracture
http//cee.uiuc.edu/paulino [email protected]/15/2008 4
� Dynamic Fracture
� Cohesive zone modeling
� Node Perturbation & Edge-Swap Operation
� Results: Fracture & Compact compression tests
� Summary
� Future Work
Wave Propagation: Rayleigh Wave
� Surface Wave
� Lord Rayleigh, 1885
� Seismology, Geology, Material Science, etc
Homogeneous & Orthotropic
http//cee.uiuc.edu/paulino [email protected]/15/2008 5
� Homogeneous & Orthotropic materials (2005)
� Large-Scale 3D Analysis for Graded media
• Rayleigh L. 1885, On waves propagated along the plane surface of an elastic solid, Proc. R. Soc. Lond. A 17, 4-11
• Vinh PC, Ogden RW. 2005, On the Rayleigh wave speed in orthotropic elastic solids, Meccanica 40, 147-161.
Parallel Computing: ParFUM
� Finite Element Analysis
� Bill Gropp: Very “Easy to write code that
scales and performs poorly.”
� Time Integration
� Central difference method
Lawler OS, Chakravorty S, Wilmarth TL, Choudhury N, Dooley I, Zheng G, Kale LV, 2006, ParFUM: a parallel framework for unstructured meshes for scalable dynamic physics applications, Engineering with Computers 22, 215-235.
http//cee.uiuc.edu/paulino [email protected]/15/2008 6
� Central difference method
� Communications
� Update Rint
� Shared-node summation operation
2
1
1
2n n n nt tu u u u
+= + D + D& &&
1 int
1 1 1( )ext
n n n
-
+ + += -u M R R&&
1 1( )2
n n n n
t+ +
D= + +u u u u& & && &&
Machine Specification
� Dell Cluster [Abe]
� Peak FLOPS: 89.47 TF
� Number of Blades (nodes): 1200
� Number of CPUs (cores): 9600
� Processor: Intel 64 2.33GHz dual socket quad core
� 8 MB L2 cache (2 MB)
� Memory: 8GB (1GB) � Total: 9600 GB
http//cee.uiuc.edu/paulino [email protected]/15/2008 7
� Memory: 8GB (1GB) � Total: 9600 GB
� Dell Xeon Cluster [Tungsten]
� Peak FLOPS: 16.38 TF
� Number of nodes: 1280
� Number of processors: 2560
� Processor: Intel Xeon 3.2 GHz (32-bit)
� Memory: 1.5 GB � Total: 3840 GB
6000
8000
10000
Runtime Performance
Para
llel ru
nti
me (
sec) # of Elements: 0.4 million
� Dell Cluster [Abe]
http//cee.uiuc.edu/paulino [email protected]/15/2008 8
0
2000
4000
32 64 128 256 512 1024
Number of processors
Para
llel ru
nti
me (
sec)
91%
90%
81%62% 43%
Geology Simulation: Rayleigh Wave
Layer #1 (Graded)
4km
2km10km
z
P(t) P(t)
( ) sin(2 ) (0 2)
( ) 0 ( 2)
P t t t
P t t
p= £ £
= >
http//cee.uiuc.edu/paulino [email protected]/15/2008 9
Layer #4
Layer #3
Layer #2
Layer #1 (Graded)2km
2km
10 km
10 km
Layer #4
Homogeneous
10 km
10 km
V. Pereyra, E. Richardson, S. E. Zarantonello, Large scale calculations of 3D elastic wave propagation in a complex geology, Proceedings of the 1992 ACM/IEEE conference on Supercomputing, Minneapolis, Minnesota, 301-309.
Homogeneous Media
� Material Properties
� Dynamic Responses
CP
(km/s)CS
(km/s)ρ
(kg/m3)E (MPa) v
Media 2 1.2 2 7.02 0.219
http//cee.uiuc.edu/paulino [email protected]/15/2008 10
Tim
e (s
ec)
5
4
3
2
1
( ) 1/f r r∝
1
1.01
CR = 1.1 km/s
Graded & Layered Media
CP
(km/s)CS
(km/s)p
(kg/m3)E (MPa) v
Graded Layer #1
23.6
1.22.16
23.6
7.0240.94
0.2190.219
Layer #2 3.5 2.1 2.3 24.7 0.336
Layer #3 4.5 2.1 2.3 27.6 0.361
Layer #4 5.5 2.1 2.3 28.7 0.430
http//cee.uiuc.edu/paulino [email protected]/15/2008 11
( ) 1/f r r∝
( ) 1/f r r∝
Tim
e (s
ec)
5
4
3
2
1
1
1.25
� Waves
� Wave Propagation
� Rayleigh wave
� Parallel computing: ParFUM
� Results: Geology simulation
Dynamic Fracture
http//cee.uiuc.edu/paulino [email protected]/15/2008 12
� Dynamic Fracture
� Cohesive zone modeling
� Node Perturbation & Edge-Swap Operation
� Results: Fracture & Compact compression tests
� Summary
� Future Work
Dynamic Fracture: Cohesive Zone
� Cohesive Zone Model
http//cee.uiuc.edu/paulino [email protected]/15/2008 13
� Computational Simulation
� TRULY Extrinsic cohesive surface elements
� Several claims of extrinsic simulations in the literature are
NOT truly extrinsic (e.g. activated elements are not extrinsic)
Ce-basedTi-based
Xi XK et al., 2005,, Physical Review Letters, 94, 125510
Topology-based Data Structure
� Complete Topological Data
� Reduced Representation
� Support for Adaptive Analysis
http//cee.uiuc.edu/paulino [email protected]/15/2008 14
� Support for Adaptive Analysis
• W. Celes, G.H. Paulino, R. Espinha, 2005, Efficient handling of implicit entities in reduced mesh representations, Journal of Computing and Information Science in Engineering 5 (4), 348-359.
• W. Celes, G.H. Paulino, R. Espinha, 2005, A compact adjacency-based topological data structure for finite element mesh representation, IJNME 64(11), 1529-1556
• G. H. Paulino, W. Celes, R. Espinha, Z. Zhang, 2008, A general topology-based framework for adaptive insertion of cohesive elements in finite element meshes, EWC 24, 59-78
Entity Enumeration
Model
Entity information
Elapsed
time (s)Topological
entity
Number of
entities
Titan IV model
(linear hexahedral
mesh)
Element 1,738,240 0.097
Node 1,845,640 0.046
Facet 5,321,600 0.219
Edge 5,429,000 0.292
http//cee.uiuc.edu/paulino [email protected]/15/2008 15
Vertex 1,845,640 0.186
W. Celes, G.H. Paulino, R. Espinha, 2005, Efficient handling of implicit entities in reduced mesh representations, Journal of Computing and Information Science in Engineering 5 (4), 348-359.
4K Structured Mesh
http//cee.uiuc.edu/paulino [email protected]/15/2008 16
� Mesh Orientation Dependence
� 4 direction � Maximum error: 45º
� 8 direction � Maximum error: 22.5º
� Undesirable crack pattern
� Node Perturbation
Proposed Remediation
http//cee.uiuc.edu/paulino [email protected]/15/2008 17
� Edge Swap
0.0 0.1 0.3
Crack Length Convergence
� Simulation Outline
� Find the shortest path (e.g. )
� Node Perturbation (NP) Factor: 0.3
� Edge Swap
� Square 4K structured mesh
1tan (2.4) 67.38−
=o
= 2.6ExactL
http//cee.uiuc.edu/paulino [email protected]/15/2008 18
� Square 4K structured mesh
� Element size: 0.1
� Simulate 100 randomly perturbed meshes for each node perturbation factor
1
2.4
Results
http//cee.uiuc.edu/paulino [email protected]/15/2008 19
NP 0.3
Avg. Error = 5.5%
NP 0.3 & Edge Swap
Avg. Error = 4.5%NP0, Error = 8.2%
Crack Angle Convergence
67.38α =o
http//cee.uiuc.edu/paulino [email protected]/15/2008 20
NP Factor = 0 NP Factor (0.3) & Edge swap
Effect of Element Size & Edge-SwapA
ng
le (
º)
63.4
67.4
71.6
76.0
An
gle
(º)
55
60
65
70
75
80
with edge-swap
without edge-swap
http//cee.uiuc.edu/paulino [email protected]/15/2008 21
Element Size
Activate Edge Swap (ES)
Without Edge Swap (ES)
NP Factor = 0.3
Given Angle (α)0 10 20 30 40 50
50.2
53.1
56.3
59.7
63.4
An
gle
(
The number of appearance
Element Size = 0.1
45
50
0 0.05 0.1 0.15 0.2
Fracture Test (Verification)
0 0.0124ε =
16 mm
4.2 mm
Material PropertiesE = 3.24 GPa
v = 0.35ρ = 1190 kg/m3
GI = 352 N/m
σmax = 129.6 MPa
http//cee.uiuc.edu/paulino [email protected]/15/2008 22
16 mm
4K structured mesh (80 X 21)
Compact Compression Specimen
Incident bar Transmitter bar
E = 5.76 GPa
v = 0.42
ρ = 1182 kg/mm3
GI = GII = 4800 N/m
σmax = 105 MPa
striker
http//cee.uiuc.edu/paulino [email protected]/15/2008 23
[mm]
60
70
2035
16
Summary
� Large-Scale Parallel Wave Propagation
� 1024 processors
� Rayleigh Wave Speed in 3D Functionally Graded Media
� Crack Path Representation thru Topological representation
http//cee.uiuc.edu/paulino [email protected]/15/2008 24
� Node perturbation & Edge swap operators
� Adaptive Dynamic Fracture Simulation
� V & V
Future Work
� Wave propagation for complex geology systems
� Provide guidance to estimate Rayleigh wave speed in
smoothly graded heterogeneous media
� Incorporate data from geological surveys
� Parallel Dynamic Fracture Simulation
http//cee.uiuc.edu/paulino [email protected]/15/2008 25
� Parallel adaptive insertion of cohesive surface element
� Dynamic adaptive load balancing
� There is a lot of exciting work to do !