Timothy Carr@shadowphax

● Provide a dynamic and elastic compute / storage environment for researchers.

● Get rid of “under the desk“ syndrome.

● Help build platforms and capabilities to deliver them in a sustainable manner for research consumption.

Solving the lack of Research

Compute

South African Data-Intensive Reseach Cloud (SADIRC)

● Expanding the ARC by onboarding SKA South Africa and SANSA.

● The main driver behind SADIRC is to provide collaborative access to all South African researchers including those at under-resourced institutions.

● ARC was a prototype and will now form part of SADIRC.

● Openstack

○ IAAS + Research Hardware (GPUs)

○ Research environments can be tailored accordingly.

○ Openstack is just a set of APIs which provides a level of abstraction to infrastructure.

○ Many development projects now build for some type of cloud of which Openstack generally features.

Solution

● Configuration deployment:○ MAAS ( Metal as a Service)○ OS: Ubuntu 14.04 LTS○ Ubuntu Juju (5)○ Hypervisor: QEMU ○ OpenVSwitch ○ Ceph (RBD)

Deployment● Juju Charms

○ Lots of limitations with updating charms.○ Looking to move away - Deploy with

Ansible. ○ 1500 MTU virtual switch issue, L&M Rel○ Federated Authentication.

● ARC Core Services:○ x3 Dell R620○ x3 LXC containers of the following:

■ RabbitMQ, Percona MySQL■ Nova, Keystone, Glance ■ Keystone, Dashboard, etc etc..

○ Juju resolves failed services with HA.

Projects

● Written in GOlang.● Build and maintain machine images.● Ships as a single binary Packer is not

a configuration management replacement.

● Multitude of builders - Microsoft Azure, GCP, AWS, Openstack

● IDIA images are baked and then deployed using Terraform rolling updates.

Terraform.io ● Architect your entire infrastructure stack in code.

● Written in GOlang.● Single binary, no dependencies.● Multiple Providers -Microsoft

Azure, GCP, AWS, Openstack.● Scale your environment.

singularity.lbl.gov

● Simple installation.● Container images can only be

built as root. ● Read-only for non-root users. ● Host storage and networking

stack used by Singularity, no passthrough required.

● UID/GID mapping to the container● Storage taken care of for you. ● Native support for GPUs / IB and

X server applications.

singularity.lbl.gov

● Maintenance of singularity containers through the use of makefiles.

IMAGE=jupyter-casa.imgDEF=jupyter-casa.def

.PHONY: bootstrap

all: bootstrap

${IMAGE}:sudo singularity create -s 6000 ${IMAGE}

bootstrap: ${IMAGE}sudo singularity bootstrap ${IMAGE} ${DEF}

upgrade:sudo singularity exec --writable ${IMAGE} apt-get updatesudo singularity exec --writable ${IMAGE} apt-get upgradesudo singularity exec --writable ${IMAGE} conda update --all

clean: sudo rm ${IMAGE}

● Jupyter Kernels are hosted inside Singularity containers.

● A more complex suite of astronomy applications are compiled and built into these singularity containers.

● Containers are stored on the BeeGFS volume and shared across the entire suite of IDIA worker machines.

{ "argv": ["/data/exp_soft/containers/jupytercasa.img" "-m", "casapy", "-f", "{connection_file}"], "mimetype": "text/plain", "display_name": "Juypter-Casa", "language": "casa"}

jupyter/kernels/JupyterCasa/kernel.json

● Shared BeeGFS Posix Storage. ● User Authentication SSH public key stored

in LDAP, ldappasswd for web-based services.

● Astronomy applications packaged in containers, stored on shared file system.

● Jupyter Hub research execution portal.● Terraform python client used to scale

resources. ● CLI to execute container apps.

IDIA Research Environment

Thank You !