intermediate mass black hole and numerical relativity

intermediate mass black hole and numerical relativity

Zhoujian Cao Institute of Applied Mathematics, AMSS 2013-7-1

Category of Black Holes

• Super massive black hole:

M: 10^5—10^9 Msun

• Stellar massive black hole:

M: 1-10s Msun

• Intermediate massive black hole:

M: 10s—10^5 Msun (mainly in globular cluster)

Category of Black Holes Binary

ALIA

Xuefei Gong, et al, CQG 28, 094012 (2011)

1:1000

1:1

Advanced LIGO

Abadie, et al, PRD 85, 102004 (2012)

Wave form template for GW detection

• PN templates: for early stage of inspiralling• EOBNR (effective one body model together

with numerical relativity): for full inspiral + merger + ring down stage; works well for mass ratio less than 1:8 and extreme mass ratio BBH

• But no reliable template for mass ratio 1:10 to 1:1000

From a given separation of the two BHs, when mass ratio increases the number of orbit increases quickly. This requires that the numerical simulation with full GR increases much consequently. In contrast to 1:1, 1:100 needs 10 times more computation cost.

PN estimation

Computational cost

1:1, 9 days 1:100, 20 days

LSSC cluster II, 128 CPUs, for last 2 orbits computational cost 1 to 20!!

Challenge of large mass BBH to NR

• Compared to 1:1, the computational cost of 1:100 BBH increase roughly 200 times!!

• For typical simulation of 1:1 BBH, 14 days are needed. So by straight forward method to 1:100, roughly 1year is needed!!

Possible ways out

• 1. Physical level: approximation method, such as self force frame work (but still first order yet), ……

• 2. Numerical Algorithm level: implicit scheme [R. Lau et al, PRD 84, 084023 (2011)], combine Cauchy evolution to null evolution, ……

• 3. Computer level: improve scalability to use more CPUs, use GPU, ……

Possible ways out

• 1. Physical level: approximation method, such as self force frame work (but still first order yet), ……

• 2. Numerical Algorithm level: implicit scheme [R. Lau et al, PRD 84, 084023 (2011)], combine Cauchy evolution to null evolution, ……

• 3. Computer level: improve scalability to use more CPUs, use GPU, ……

Mesh refinement scheme

F. Loeffler et al, CQG 29, 115001 (2012)

Level #2 Level #1

Parallel mesh level algorithmCompared to traditional mesh refinement method, we can improve the scalability of the code and get 2x speed up

Cowork with Zhihui Du, S. Brandt and F. Loeffler, 2013

GPU acceleration

For system biology, Yamazaki, Igarashi, Neural Networks, 2013

For GW data analysis, Zhihui Du, et al, CQG 29, 235018 (2012)

Einstein solver with GPU

Cowork with Quan Yang, Zhihui Du, 2012

Structure of AMSS-NCKU GPU code

Two groups MPI processes, one for cpu and one for gpu

MPI + OpenMP + CUDA

Primary test of AMSS-NCKU GPU code

Titan: top 1 super computer around the world (now Tianhe 2)

1024x16 cores + 1024 GPUs

Summary

• Challenge from GW detection:

AdvLIGO—1:150

ALIA ---1:1000

• Parallel mesh level calculation method—2x speed up

• GPU implementation to NR---have got roughly 5x speed up; 30x speed up? in progress