integrating cern software with docker
TRANSCRIPT
Integrating CERN software with DockerRicardo [email protected]
● Jakob Blomer● Nikola Hardi● Domenico Giordano● Spyridon Trigazis
Credits
● European Organization for Nuclear Research
● Founded in 1954
● 22 Member States○ With many others contributing to experiments
● CERN’s Mission is Fundamental Research
CERN and the LHC
CERN and the LHC
CERN Computing● 8500 physical nodes● 280000 cores● 34000 VMs● 2800 users● >200 PB
LHC Grid
CernVM-FS● Main mean for software distribution● POSIX read-only● FUSE● Standard HTTP● Caching● On demand file transfer● Data Integrity
https://github.com/cvmfs/cvmfs
Containers at CERN● Reusable, reproducible, easy to share units
○ Reusable Analysis○ Interactive Analysis
● Continuous Integration● Infra Services● ...
Step 1: Access to software● Software already at every site via CVMFS● Integrated via a Docker volume plugin (v1)● Shared mounts on the host, bind mounts
https://gitlab.cern.ch/cloud-infrastructure/docker-volume-cvmfs
Step 2: Access to images● Well behaved use cases
○ Infra, CIs usually have smallish, layered images● Others less
○ Users like the single, easy to share unit○ Conditions data, calibration, etc○ Large (~1GB) to Very Large (18GB?)
● Issue with large clusters (100s nodes)
https://github.com/cvmfs/docker-graphdriver
Step 2: Access to images● Docker graph driver (v2 plugin)● AUFS and Overlay2
https://github.com/cvmfs/docker-graphdriver
{ "version": "0.1", "layers": [ { "digest": "ae79f251470513c2a0ec750117a81f2d58...", "repo": "test.cern.ch" }, ... { "digest": "6f01dc62e444044e3ce494269837ef0aed...", "repo": "test.cern.ch" } ]}
CVMFS Graph Driver - Tests● Fresh Swarm Cluster of ~100 nodes
○ VMs on our production cloud○ openstack coe cluster create --label docker_storage_driver=cvmfs ...
● Performance Benchmark Suite○ docker service create --mode global ...○ 100 replicas pulling from the same registry at ~same time○ Medium sized image of ~1GB and very large image of ~10GB
CVMFS Graph Driver - Tests● Regular image results (no CVMFS)
~1GB
~10GB
● CVMFS aware thin image
CVMFS Graph Driver - Tests
~1GB ~10GB
Summary● Integrated CernVM-FS via Docker plugins● Benefiting from existing cache infrastructure● Gains from File vs Layer granularity
○ Startup time, network usage● Further tests
○ Multi-cloud deployments○ Warm vs Cold cache