big data - unibs.it · big data analytics long term archiving tape library high performance data...
TRANSCRIPT
![Page 2: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/2.jpg)
Cineca’s used storage
![Page 3: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/3.jpg)
Cineca’s scenario
European projects
Italian
projects
Projects
FAIR
principles
Services &
resourcesCloud
HPC
Big Data
analytics
Long term
archiving
Tape
library
High
performance
data transfer
![Page 4: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/4.jpg)
![Page 5: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/5.jpg)
Cloud
HPC
plain fs
Long
term
archive
![Page 6: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/6.jpg)
Big Data and Analytics
Giorgio Pedrazzi
![Page 7: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/7.jpg)
Technologies
• Cloud computing: Openstack, Docker, Singularity
• Hadoop ecosystem: Hive, Pig, Mahout, Spark
• Open source applications: R, H2O.ai, Stanford NLP, Knime5
• Commercial software: Stata, SAS, Matlab, 5
![Page 8: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/8.jpg)
Data repository
![Page 9: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/9.jpg)
EUHIT Portal
![Page 10: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/10.jpg)
EuHIT
EuHIT is a consortium that aims at integrating cutting-edge
European facilities for turbulence research across national
boundaries.
![Page 11: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/11.jpg)
EUDAT
A truly pan-European Infrastructure
EUDAT offers common data services,
supporting multiple research communities as
well as individuals, through a geographically
distributed, resilient network of 35 European
organisations
Our vision is to enable European
researchers and practitioners
from any research discipline to
preserve, find, access, and
process data in a trusted
environment, as part of a
Collaborative Data
Infrastructure
![Page 12: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/12.jpg)
EUDAT Service Suite
http://www.eudat.eu/services
![Page 13: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/13.jpg)
EUDAT data management
![Page 14: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/14.jpg)
Persistent Identifiers (PID)
• EUDAT relies on the B2HANDLE service to associate persistent identifier to
digital objects
• Its focus is the registration of data in an early state of the scientific process,
where lots of data is generated and has to become referable to collaborate with
other scientific groups or communities.
14 © CINECA
Handle resolution
![Page 15: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/15.jpg)
Why High Performance Computers in HBP?
Brain simulation
Data analytics
Image
processing
Visualisation
The human brain
is COMPLEX!
Illustration: Brown Bird Design
![Page 16: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/16.jpg)
High Performance Analytics & Computing Platform
Our mission
Build and operate a
supercomputing, data and
visualization infrastructure
enabling scientists to:• Run large-scale, data intensive,
interactive brain simulations up to the
size of a full human brain
• Manage the large amounts of data
used and produced in the HBP
• Manage complex workflows, data
analysis and visualization workloads
![Page 17: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/17.jpg)
High Performance Analytics & Computing Platform
Our role in the Human Brain Project
• Providing the base infrastructure for the HBP:supercomputers, storage, network, other resources
• Development of software and technology to– Facilitate usage of the infrastructure for researchers
– Make more efficient use of the infrastructure, e.g.• Simulation technology capable to exploit modern and
future supercomputers
• Visualization tools: working with large-scale imaging or simulation data
• Enabling the data federation and data-intensive computing
• Interactive computing technology
• Supporting developers and users
![Page 18: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/18.jpg)
Federated base infrastructure for the HBP
![Page 19: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/19.jpg)
Unified access to federated resources
Middleware: unified access to resources
![Page 20: Big data - unibs.it · Big Data analytics Long term archiving Tape library High performance data transfer . Cloud HPC plainfs Long term archive. Big Data and Analytics ... • Opensourceapplications:R,H2O.ai,StanfordNLP,Knime5](https://reader034.vdocuments.mx/reader034/viewer/2022042207/5ea9a1e21936e55254108742/html5/thumbnails/20.jpg)
QUESTIONS