20141103 cern open_stack_paris_v3

Tim Bell

@noggin143

tim.bell@cern.ch

04/11/2014 2Tim Bell - OpenStack Paris

Answering fundamental questions…

How to explain particles have a mass?

Brout-Englert-Higgs

Boson

04/11/2014 Tim Bell - OpenStack Paris 3

Answering Fundamental Questions…

Where has all the anti-matter gone ?

04/11/2014 Tim Bell - OpenStack Paris 4

Answering Fundamental Questions…

What is the mass of the Universe made of?

We can only

see 5% of its

estimated mass

~25% Dark matter?

~70% Dark energy?

04/11/2014 Tim Bell - OpenStack Paris 5

Answering Fundamental Questions…

Why is Gravity so weak ?

Extra dimensions ?

Gravitons ?

04/11/2014 Tim Bell - OpenStack Paris 6

04/11/2014 7Tim Bell - OpenStack Paris

04/11/2014 8Tim Bell - OpenStack Paris

04/11/2014 9Tim Bell - OpenStack Paris

Collisions

04/11/2014 10Tim Bell - OpenStack Paris

A Big Data Challenge

04/11/2014 11

In 2014,

• ~ 100PB archive with additional 27PB/year

• ~ 11,000 servers

• ~ 75,000 disk drives

• ~ 45,000 tapes

• Data should be kept for at least 20 years

In 2015, we start the accelerator again

• Upgrade to double the energy of the beams

• Expect a significant increase in data rate

Tim Bell - OpenStack Paris

LHC data growth • Estimating

400PB/year by

2023

• Compute needs

expected to be

around 50x current

levels if budget

available

04/11/2014 Tim Bell - OpenStack Paris 12

0.0

50.0

100.0

150.0

200.0

250.0

300.0

350.0

400.0

450.0

Run1 Run2 Run3 Run4

CMS

ATLAS

ALICE

LHCb

2010 2015 2018 2023

PB

per

year

The CERN Meyrin Data Centre

04/11/2014 13Tim Bell - OpenStack Paris

http://goo.gl/maps/K5SoG

04/11/2014 Tim Bell - OpenStack Paris 14

04/11/2014 15Tim Bell - OpenStack Paris

Good News, Bad News

04/11/2014 Tim Bell - OpenStack Paris 16

• Additional data centre in Budapest now online

• Increasing use of facilities as data rates increase

But…

• Staff numbers are fixed, no more people

• Materials budget decreasing, no more money

• Legacy tools are high maintenance and brittle

• User expectations are for fast self-service

We are not Special!• Challenge the must-have lists at project start

• Are those requirements really justified ?

• Accumulating technical debt stifles agility

• There is no Moore’s Law for people• Automation needs APIs, not documented procedures

• Find open source communities and contribute• Understand ethos and architecture

• Stay mainstream, stay up to date

04/11/2014 Tim Bell - OpenStack Paris 17

CERN Tool Chain

04/11/2014 Tim Bell - OpenStack Paris 18

Status

• Started project in 2011 with Cactus

• In production since July 2013 with Grizzly• 2 upgrades without major incidents or VM downtime

• 4 OpenStack Icehouse clouds at CERN• Largest is ~70,000 cores on ~3,000 servers

• 3 other instances with 45,000 cores total

• Expected to pass 150,000 cores in total by Q1 2015

• All non-CERN specific code is upstream

04/11/2014 19Tim Bell - OpenStack Paris

compute-nodescontrollers

compute-nodes

Nova Cells Scaling Architecture

20

Child Cell

Geneva, Switzerland

Child Cell

Budapest, HungaryTop Cell - controllers

Geneva, Switzerland

Load Balancer

Geneva, Switzerland

controllers

04/11/2014 Tim Bell - OpenStack Paris

IN2P3

Lyon

Onwards the Federated Clouds

Public Cloud such

as Rackspace

CERN Private Cloud

72K cores

ATLAS Trigger

28K cores

CMS Trigger

12K cores

Brookhaven

National Labs

NecTAR

Australia

Many Others on

Their Way

04/11/2014 Tim Bell - OpenStack Paris 21

ALICE Trigger

12K cores

Hooke’s Law for Cultural Change• Under load, an

organisation can

extend proportional to

external force

• Too much stretching

leads to permanent

deformation

04/11/2014 Tim Bell - OpenStack Paris 22

The Agile Experience

04/11/2014 Tim Bell - OpenStack Paris 23

Cultural Barriers

04/11/2014 Tim Bell - OpenStack Paris 24

Standing on the Shoulders of Giants

04/11/2014 25Tim Bell - OpenStack Paris

Thanks to all Community Members!

04/11/2014 26

• Details at http://openstack-in-

production.blogspot.fr

• CERN code is

upstream or at http://github.com/cernops

• CERN & Industry

Collaboration at http://cern.ch/openlab

Tim Bell - OpenStack Paris