elixir cz · multi-layer user network – physical (optical fibers) – optical, 400 gbps core (now...
TRANSCRIPT
e-Infrastructures &
ELIXIR CZ
Tomáš Košňar, CESNET
Role of e-infrastructures
● consensual IT environment for any scientific area
● comprehensive, widely acceptable solutions and tools– data transport ~ network
– data processing
– data storage
– identity management, AAI tools
– monitoring
– security
● robustness, performance, stability of resources and services
National e-infrastructure landscape
● CESNET
– legal body (a.l.e.) 1996 – public, state universities, CAS
– primary national network infrastructure; coordinating national grid, cloud and storage activities; collaborative environment, AAI, security, …
● CERIT-SC
– @ Masaryk University in Brno
– flexible clouds, development, innovations● IT4Innovations
– @ Technical University Ostrava
– National Supercomputing Centre● till 2019 formally independent (each on the national roadmap)● now → single national e-infrastructure e-INFRA CZ
– consortium (formal agreement); coordinated by CESNET
● long collaboration (since 1998 formally)
● MetaCentre – distributed grid & cloud environment
● founding members of ELIXIR CZ
● symbolic architecture
e-INFRA CZ
Role of e-infrastructures in ELIXIR CZ
● responsibility for IT resources and capacity building
– IT capacity purchased and operated by e-infrastructures
● separate funding stream (EU development funds for capital investment)● current project - CERIT-SC/MU coordinating, CERIT-SC and CESNET
delivering
– investment strategy → computing capacity, storage, some commercial SW
● building distributed e-infrastructure for ELIXIR CZ
– needs of life science community
– integrate into the national e-infrastructure
– controlled by ELIXIR CZ
– side effect → automated/by default access to/utilization of back-end services that are “a must” for any infrastructure without needs to provide them separately.. (monitoring, security, ticketing systems, etc..)
user network● multi-layer– physical (optical fibers)– optical, 400 Gbps core (now 100),
FDWDM– IP/MPLS, 400 GE core (now 100),
up to 100 Gb/s access● over-provisioned● special care
– topology, capacity distribution– e-infrastructure components– significant user data sources– high speed ports at network edge
● standard access for ELIXIR CZ members
● special solutions for ELIXIR CZ significant/specific resources when needed - have to discuss in advance !!!
e-INFRA CZ – network architecture concept 2020+
● MetaCentre component● https://www.metacentrum.cz/● distributed – located at universities, CAS● contributors – CESNET, CERIT-SC,
universities● federated approach● unified management, monitoring, user
support● currently 18000 CPU cores (x86_64)
● HTC, SMP servers● nodes with NVIDIA GP-GPU
● 6 PB storage (scratch, home)● NFS, GPFS, HDFS
(Hadoop/Spark)● object (Ceph, Swift, S3)
e-INFRA CZ - computing infrastructure
CERIT-SC
MetaCentreCESNET coordination
multiple resources owners
IT4innovations
● MetaCentre component
● computing environments
– grid – batch systems, PBSPro
– cloud – OpenNebula, OpenStack
– map-reduce – Hadoop, Spark
– large application/SW portfolio
● international communities support - LHC, Auger, CTA, Belle, ELIXIR (RI ELIXIR, EXCELERATE, EOSC Life), ESA (Sentinel data), in EOSC projects (ELIXIR, ICOS, ELI)
● 2000+ active Czech users
e-INFRA CZ - computing infrastructure
CERIT-SC
MetaCentreCESNET coordination
multiple resources owners
IT4innovations
● “standard“ e-infrastructure resources/services
● ELIXIR CZ capacity building
– own resources● exclusive access
– priority queues
– SW packages
e-INFRA CZ and ELIXIR - computing
CERIT-SC
MetaCentreCESNET coordination
multiple resources owners
IT4innovations
● ELIXIR CZ capacity building
– compute cluster● 42 servers, 1500 CPUs @ ICS MU and IOCB Prague
– another 45 servers MetaCentre capacity in Brno
– specific MetaCentre queue dedicated to ELIXIR CZ
– SW
● centrally provided resources, virtualization for extending availability– e.g. PEAKS SW, Win based, in virtual appliance and thus shared by
more groups (with the help of national high speed network capacity)
● discussions about the feasibility of commercial SW - black boxes, not prepared for shared environment (setup, licensing model), rather expensive
– coordinated with e-infrastructures investments
e-INFRA CZ and ELIXIR - computing
● originally distributed storage infrastructure of HSM monolithic storages with filesystem access
● towards object storage technology– different levels of network utilization & response
● S3, Ceph, RBD● power requirements ~1.8kW/Mkč + cooling
– community based storage infrastructure● users can contribute and benefit from
geographical distribution (replications) ~ MetaCentre like philosophy
– expected investment ~ 27MKč/year ?
● total physical capacity – 34 PB (27+7) now, 62 PB (+28 os) in 2020; available capacity lower (ways of storing ~ number of copies, etc..)
e-INFRA CZ - storages
● storage services - https://du.cesnet.cz/– file transfer/temporary storage (FileSender) - https://filesender.cesnet.cz– sync‘n‘share (ownCloud) - https://owncloud.cesnet.cz– file system access– for personal use - https://du.cesnet.cz → register to VO Storage– for group use → contact support– object storage access → contact support– archival storage (inc. regular check-sums), repositories → contact support– CESNET storage support: [email protected]
● ELIXIR CZ capacity building– focusing working space– currently extended from 0.5 PB to 2 PB (GPFS)– object storage 1 PB for cloud use– data of ELIXIR CZ services, used databases copies, copies of data in
home directories
e-INFRA CZ - storages
● IMG Praha
– building IT capacities together across several RI/institutions
● IMG, ELIXIR CZ, CZBI, OPENSCREEN, CCP, e-INFRA CZ
● coordinated by ELIXIR CZ
– ”data lake” model support → data from several sources/infrastructures for more effective processing, cross correlations etc..
– appropriate network architecture/capacity → e-INFRA CZ (CESNET)
– model of cooperation to follow at other localities/institutions
Community based capacity building
● operation of stable services in secured environment
– incl. virtualization services and containerization support
– cpPredictor, CCMI, Chipster, web ELIXIR CZ
● resources for services (incl. workflow engines)
– Chipster, Galaxxy, RepeatExplorer, FireProt
● complex environment for development (services/tools) including GUI for bioinformatics (Chipster and Galaxy)
● compute resources for ELIXIR users
● application SW
– commercial ~ CLC Workbench, PEAKS Studio, Mascot
– open source systems available through the whole MetaCentre
Services delivery overview
● storage for working copies of archives
– CESNET/e-INFRA CZ data services
● “back-office” services
– AAI scheme & tools
– support for training and education, ...
● ways to adapt e-infrastructure services to meet specific life science needs
– discussions on SLA, responsibility of partners
– monitoring, acceptable use policies and rules
Services delivery overview
● co-leadership of the ELIXIR Compute platform, task co-leadership in AAI and Clouds● leadership of the EOSC-Life project wp on access control and mgmt. + active
involvement in cloud wp; leadership of the CINECA project AAI wp● ELIXIR Authentication and Authorization Infrastructure – ELIXIR AAI
– major internationally visible contribution
International scope
ELIXIR AAI
External authentication(e-infrastructures)
Relying services
eduGAIN IdPs Common IdPs
ELIXIR Proxy IdP ELIXIR Directory Bona fide management
Dataset authorisation management (REMS)
Group/role mgmt (PERUN)
Credential translation
EGA eLearning
Cloud Beacon
wiki
Data archive
… …
Attribute self-management
Step-up AuthN
Why together ?
● Close interaction and cooperation makes sense
– collaborative model (vs. user-provider)
● synergistic effects of collaborative development gives better value to both sides and thus to users community
● more effective and better solutions, easier cooperation with others (international)
● faster & better understanding each other and user community needs
● …
● ELIXIR CZ: e-infrastructures = partners
e-infrastructures
life science area
ELIXIR CZ
● providing and operating IT resources
– https://www.elixir-czech.cz/
– https://wiki.metacentrum.cz/wiki/Elixir
● developing specific IT solutions
– resources configuration...SW development
– fully integrated with national and international “pure” e-infrastructure activities (e.g. EOSC)
● helping collaboration with other research infrastructures
e-infrastructures for ELIXIR CZ summary
Thank you !