![Page 1: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/1.jpg)
Using GPUs for the Boundary Element Method
Christopher CooperLorena Barba
Boston University
November, 2011
1
Pan-American Advanced Studies Institute NSF Program
![Page 2: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/2.jpg)
Boundary Element Method - Formulation
‣Numerical Method for PDEs
2
∇2φ(x)
Only on boundary!
Volume mesh Surface mesh
φ(x) =�
Γψ(x,x�)dΓ�
![Page 3: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/3.jpg)
Boundary Element Method - Matrix Formulation
‣Apply for all boundary elements at
3
Γj
x = xi
x0
x1
x2
x3
x = xi
[A] {X} = [B] {Y }{X} unknown boundary values
{Y } known boundary values
![Page 4: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/4.jpg)
Boundary Element Method - Applications
4
Advanced Numerical Solutions ansol.us
Acoustics
Stokes flow
FastBEMurbana.mie.uc.edu
Electrostatics
Yokota et al. 2011
![Page 5: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/5.jpg)
Boundary Element Method - Limitations
‣Dense matrix vector multiplications scale as O(N2)!
- Assembling the RHS
- Krylov subspace linear solver (GMRES, CG, ...)
5
[A] {X} = {b}
[B] {Y } = {b}
Naive approach allows for only a few thousand elements!
![Page 6: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/6.jpg)
Fast Multipole Method (FMM)
‣ Fast interaction calculation algorithm
‣Approximates far field
6
φ(x) =N−1�
i=0
αiψ(|x− xi|)
x
xc
xi
![Page 7: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/7.jpg)
BEM - FMM
‣ BEM Matrix vector multiplications are interaction calculations
7
φ(x) =N−1�
i=0
αiG(|x− xi|)
PDE Integral N-body
![Page 8: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/8.jpg)
BEM - FMM
8
Liu . Fast Multipole Boundary Element Method
![Page 9: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/9.jpg)
BEM - FMM
‣Accelerates matrix vector computation:
- Far field is approximated -> O(N) calculations
‣No need to store the matrix
- Values computed on the fly -> O(N) storage
9
![Page 10: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/10.jpg)
BEM FMM - Previous work
10
“Petascale direct numerical simulation of blood flow on 200K cores and heterogeneous architectures”
Rahimian et al.
Fast BEM Lu et al.
![Page 11: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/11.jpg)
BEM FMM and GPUs
‣ BEM and FMM provide an important reduction in computation
11
Mesh reduction
Reduce computation
GPU
Look for further reduction from hardware!
BEM FMM
![Page 12: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/12.jpg)
BEM FMM and GPUs
‣ FMM maps well to GPUs
12
!"!# !"$ !"% !"& ! & % $ !# '& #% !&$ &(#
!#
'&
#%
!&$
&(#
(!&
!)&%
&)%$
*+,-./012.3402/,250/647831+"96/,:
;//.02.93,4831+"547<831+"5:
!"#$%&'#!"#%()
*$%&*%()
%+,-#./0"12#34+5-6789/4+5-6789:
%+,-#./0"12#37988/7988:
;/<#%%=
$-9!768
$4(>
,6!?89/45976,6"!#49+@
![Page 13: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/13.jpg)
FMM - Available resources
‣ ExaFMM: FMM library that runs on GPUs
- Shown to scale well to thousands of GPUs
- Performs in the order of Peta FLOPS
13
64 billion in 100 seconds1.0 PFlops4K GPUs on TSUBAME
Weak scaling
![Page 14: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/14.jpg)
FMM - Available resources
‣ ExaFMM: an open source FMM library that runs on GPUs
- Released in SC’11
14
![Page 15: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/15.jpg)
How far can we go?
‣ Bioelectrostatics
15Billions of elements!
![Page 16: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/16.jpg)
How far can we go?
‣Acoustics
- Helmholtz equation solved for each frequency
16
![Page 17: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/17.jpg)
FMM BEM - Implementation
17
Set up elements
Construct tree
Assemble RHS FMM
GMRES Check residual FMM
Solution
N iterations!
Only once
![Page 18: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/18.jpg)
Main challenges
‣ Porting the whole BEM to the GPU
- Avoid excessive CPU-GPU communication
18
GMRES Check residual FMM
N iterations!
![Page 19: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/19.jpg)
Main challenges
‣Constructing a BEM in the ExaFMM framework
- Needed for large number of elements
- Use ExaFMM’s domain decomposition for parallelization
19
![Page 20: Using GPUs for the Boundary Element Method - Nvidia · PDF fileUsing GPUs for the Boundary Element Method Christopher Cooper Lorena Barba Boston University November, 2011 1 Pan-American](https://reader031.vdocuments.mx/reader031/viewer/2022022002/5a7b84207f8b9a72118bf997/html5/thumbnails/20.jpg)
Conclusions and future work
‣ BEM FMM technology is a very good candidate to be accelerated with GPUs
‣ Intelligent use of tools such as ExaFMM will allow us to solve cool engineering problems!
20