“big data” at the natural resource ecology laboratory randall b. boone
DESCRIPTION
“Big Data” at the Natural Resource Ecology Laboratory Randall B. Boone Research Scientist, Natural Resource Ecology Laboratory and Associate Professor, Department of Ecosystem Science and Sustainability ISTeC Big Data Forum Colorado State University April 18, 2013. Department of Ecosystem - PowerPoint PPT PresentationTRANSCRIPT
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 1
“Big Data” at theNatural Resource Ecology Laboratory
Randall B. Boone
Research Scientist, Natural Resource Ecology Laboratory andAssociate Professor, Department of Ecosystem Science and Sustainability
ISTeC Big Data ForumColorado State University
April 18, 2013
Department of EcosystemScience and Sustainability
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 2
Integrating across scales through
Top-down
and
Bottom-up
approaches
D. Ojima
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 3
Agent-based Modeling and Big Data
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 4
G-Range, A Global Rangeland Model
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 5
T. Hilinski
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 6
T. Hilinski
Potential Vegetation: Mean Annual NPP (gC/m2) 1961-2006 Mean
DayCent and Century Simulations
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 7
Managing Big Data at NREL
Our data are big, but well described
Unlike some industrial applications, the system must be exceedingly flexible
Must be responsive to a variety of users (e.g., more diverse uses than the ISTeC Cray)
Calculation-intensive uses
Users are unlikely to have the ability to parallelize tools
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 8
Rubel Cluster – The Backbone of NREL’s “Mid-Performance Computing”
Distributed memory computer cluster
256 processors
500 Gflops
463 GB memory
2 TB immediate storage
50-60 TB extended storage, plus 120 TB coming online
Private 1 Gbps Ethernet interconnect, with 10 Gbps access to storage
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 9
Rubel Cluster
Provides a testing platform for the CRAY for some
Others use alternatives, such as R, which can make use of multiple processors
Good, but not high performance computing
A mix of desktop and cluster analyses yield “mid-performance computing”
But it is effective!
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 10
Tom Hilinski
Apr 18, 2013 ISTeC Big Data Forum, Colorado State University 11
Rubel – Past, Present, and Future