millennium data dissemination 2012-11-26mpa institute seminar1 possible extensions how ambitious...

MPA institute seminar 1

Millennium Data Dissemination

2012-11-26

possible extensionshow ambitious can/should we

be?

Raw data:Particles

FOF groups and Subhalos

Density fields

Subhalo merger trees

Synthetic galaxies (SAM)

Mock catalogues

Mock images

2012-12-17 Millennium Workshop 2

SUBFIND Tree Builder

Halo Merger Trees

SAMSSP

FOF GroupFinder

(Sub-)HalosFOF GroupsSimulation Particles

Gadget

MoMaF’

SAM Galaxies

Light-cone Galaxies

SkyMaker’SExtractor

Synthetic Images

SExtractor Catalogues

PSF, Noise,Background

Scaling

IGM

MRObs



Raw data

• Serve particles from Millennium simulations– 10 billion/snapshot, ~25TB

• from database?– with spatial queries– links to halos/FOF groups

• through JobRunner services– cutout– gridding + FFTs etc

2012-11-26


Other simulations?

• Hydro Example: – http://gavo.mpa-garching.mpg.de/HydroClusters– online visualisation:

http://galformod.mpa-garching.mpg.de/phoxybrowser

• Others– Millennium+gas?– Eagle– …

• How?– In DB ok for objects– Visualisation support for “fluffy things”

2012-11-26

http://gavo.mpa-garching.mpg.de/HydroClusters






Halo Merger Trees

SAMSSP

FOF GroupFinder


Gadget

MoMaF’

SAM Galaxies

Light-cone Galaxies


Synthetic Images



Scaling

IGM

MRObs



Group catalogs• Have

– FOF– SUBFIND subhalos– Mhalo merger trees– Dhalo merger trees

• More properties– shapes, density profiles (ala AIP, or ala MR-XXL): good for lensing– others?

• More?– BDM, NEXUS, ORIGAMI, …– All of Knebe etal

• LSS objects?– Voids, filaments, walls in the DB– “more precise” environmental queries possible– per grid cell or as object (“stacking voids”)

2012-11-26


Halo Merger Trees

SAMSSP

FOF GroupFinder


Gadget

MoMaF’

SAM Galaxies

Light-cone Galaxies


Synthetic Images



Scaling

IGM

MRObs



SAM catalogues

• More models– Durham, MPA

• Complete MR-XXL?–~200x Millennium, 80TB

• Extensions

2012-11-26


SFH-s

• Allows SEDs from post-processing• how to store? – rows vs arrays (in SQLServer)– functions (easy in SQLServer):

• create spectra– “exact” observer frame bands

• PCA amplitudes in the database– find nearby galaxies for given observed

spectrum2012-11-26


SAM online

• GALFORM online !!!– http://galformod.mpa-garching.mpg.de/dev/jobr

unner/

– on millimil

• Do we want others?– Galacticus?

• Need constraints, validated models.– How afraid should we be of “bad science”?

• SAM online on Millennium?– input to MRObs pipeline

2012-11-26

http://galformod.mpa-garching.mpg.de/dev/jobrunner/




Halo Merger Trees

SAMSSP

FOF GroupFinder


Gadget

MoMaF’

SAM Galaxies

Light-cone Galaxies


Synthetic Images



Scaling

IGM

MRObs



Light cones

• In Database: not just from MPA!– common model– linked to SAMs

• More– deep: new code for Millennium-II– all sky: MR-XXL– with scaling!

• What properties needed?– who wants them? – what for (lensing?)?

• Light cone building online– Will soon make our MoMaF available on JobRunner– new code in preparation (for MR-II etc)

2012-11-26


Halo Merger Trees

SAMSSP

FOF GroupFinder


Gadget

MoMaF’

SAM Galaxies

Light-cone Galaxies


Synthetic Images



Scaling

IGM

MRObs



MRObs

• Store of images will keep growing– Overzier!

• Create images on demand– telescope models custom?

• Post processing as well?– Sextractor– other?

• Improvement galaxy images– realistic structure (weak lensing systematics, strong

lensing structure)– from simulations ?– form observations?

2012-11-26


Halo Merger Trees

SAMSSP

FOF GroupFinder


Gadget

MoMaF’

SAM Galaxies

Light-cone Galaxies

SkyMaker’

SExtractor

Synthetic Images



Scaling

IGM

MRObs


Lensing


Lensing

• Under construction – with Ben Metcalf

• Online?–Why not, JobRunner will handle it.

• How much?– what is required for weak lensing?

2012-11-26


Even more

• L-Galaxies online++– updating physics modules?–MCMC mode?

• Dark matter light cones?– for lensing– what do we need? particles, fields?

• Interface– dashboard (M.Egger)– improve SQL interface (Roys talk)

2012-11-26


Galformod infrastructure

• Currently– 12+1 12-core nodes– 180 TB raw storage– 20TB DB storage

• What do others have?– AIP ~500TB– JHU ~5PB (?)

• What do we want?

2012-11-26

100 billion …


What do we want need?

• Depends on answers to all previous questions• What can we afford?– Where can we get the money?

• Where do we put “it”?– RZG?

• Pool our resources together?– beyond mirror– distributed over Virgo nodes?

• Larger scale collaborations (EUDat)?– see Hermann’s talk

2012-11-26

millennium data dissemination 2012-11-26mpa institute seminar1 possible extensions how ambitious...

Documents