mochi: composable lightweight data services for …...– provide a reusable toolkit of components...

6TH JLESC WORKSHOP

MOCHI: COMPOSABLELIGHTWEIGHT DATA SERVICES FOR HPC

erhtjhtyhy

PHIL CARNSMathematics andComputer Science DivisionArgonne National Laboratory

November 30, 2016

Kobe, Japan

MOTIVATING TRENDS IN HPC DATA AND STORAGE

HARDWARE TRENDS

More layers are being added to the storage hierarchy

Tradeoffs in price, maturity, capacity, longevity, performance, and density

MemoryMemory

Burst BufferBurst Buffer

Parallel File SystemParallel File System

Campaign StorageCampaign Storage

ArchiveArchive

MemoryMemory

Parallel File SystemParallel File System

ArchiveArchive

HPC Before 2016

HPC After 20161-2 PB/sec Residence – hours Overwritten – continuous

4-6 TB/sec Residence – hours Overwritten – hours

1-2 TB/sec Residence – days/weeks Flushed – weeks

100-300 GB/sec Residence – months-year Flushed – months-year

10s GB/sec (parallel tape Residence – forever

HPSS

Parallel

Tape

Lustre

Parallel File

System

DRAM

Slide adapted from

Gary Grider (LANL)

SOFTWARETRENDS

4

Application Data

Application

CheckpointsExecutables

and Libraries

Intermediate

Data

Products

SPINDLE SCR

FTIDataWarp

MDHIMKelpie

� Alternative to one-size-fits-all “parallel file system for data management” model

� Data services are provisioned on demand

� Co-designed with applications in some cases

Data services are

specializing to provide

application and domain-

specific functionality

WHAT’S NEXT?Enabling the evolution of data-intensive scientific services

5

� Support emerging hardware (e.g., NVM), storage hierarchies, and deployment

scenarios

� Lower the barrier to entry for new specialized services

– Avoid building new services from the ground up

– Provide a reusable toolkit of components and “microservices” for critical

functionality

– Compose to as needed for the task at hand

� Example microservices: key/value storage, group membership, replication

� Example composed scientific service: “Genomics Query Service”

ENABLING DATA SERVICES

ROB ROSS, PHILIP CARNS, KEVIN HARMS,

JOHN JENKINS, AND SHANE SNYDER

GARTH GIBSON, CHUCK CRANOR,

QING ZHENG, AND GEORGE AMVROSIADIS

JEROME SOUMAGNE AND JOE LEE

GALEN SHIPMAN AND BRAD SETTLEMYER

Argonne National Laboratory

Carnegie Mellon University

The HDF Group

Los Alamos National Laboratory

EXAMPLE DATA SERVICEEarly proof of concept: lightweight object storage service

7

� RPC framework: Mercury

� Concurrency: Argobots

� Network layer: CCI

� Storage access:

NVML/libpmem

ACCESS LATENCY

8

� Can we retain the performance

characteristics of NVM across

remote/distributed services?

� Must also be resource-friendly (no busy-

polling on network, limited core usage)

Evaluation on InfiniBandnetwork with RAM storage

ACCESS LATENCY

9






Protocol modes:

� Eager mode, data is packed into

RPC msg


ACCESS LATENCY

10






Protocol modes:


RPC msg

� Data is copied to/from pre-registered

RDMA buffers


ACCESS LATENCY

11






Protocol modes:


RPC msg

� Data is copied to/from pre-registered

RDMA buffers

� RDMA “in place” by registering

memory on demand


12

� Objective: Provide computation cache as a

service to minimize fine-scale model executions

� Approach:

– Start with a key/value store

– Distributed approximate nearest-neighbor

query for interpolation candidates

� Status:

– Mercury-based in-memory DB service

– Investigating distributed, incremental

nearest-neighbor indexing

– Applying technique to multiple applicationsImport/export

DB instances

Distributed DB

Application domain

Query 6D space for

nearest neighbors Co-designing a fine-scale model database

EXAMPLE DATA SERVICE

DATA RESEARCH AT ANL

� Mochi:

– Specialized data services

– http://www.mcs.anl.gov/research/

projects/mochi/

� Darshan:

– Large scale I/O characterization


projects/darshan/

� CODES:

– Discrete event simulation of HPC

storage and networks


projects/codes/ 13

Observe:

Darshan

Implement:

Mochi

Model:

CODES

Mochi is just one component

www.anl.gov

THANK YOU!

THIS WORK WAS SUPPORTED BY THE U.S. DEPARTMENT OF ENERGY, OFFICE OF

SCIENCE, ADVANCED SCIENTIFIC COMPUTING RESEARCH, UNDER CONTRACT

DE-AC02-06CH11357.

WE ARE HIRING! THE MATHEMATICS AND COMPUTER SCIENCE DIVISION OF

ARGONNE NATIONAL LABORATORY IS SEEKING OUTSTANDING PEOPLE AT

MULTIPLE CAREER LEVELS.

HTTP://BIT.LY/2G8MOPV

(COMPUTER SCIENCE RESEARCH)

HTTP://BIT.LY/2FSPJW5

(SOFTWARE DEVELOPMENT)

mochi: composable lightweight data services for …...– provide a reusable toolkit of components...

Documents