teragrid science gateways nancy wilkins-diehr area director for science gateways [email protected]...

TeraGrid Science Gateways

Nancy Wilkins-DiehrArea Director for Science Gateways

[email protected]

TeraGrid 09 Education Program, June 22, 2009


You in this audience are the future of computational science

You can tell me more about the value of gateways than I can tell you

First let me give you some background so we’re on the same page, then let’s begin a discussion

Today I hope to demonstrate

•What are gateways?–Why are gateways worth the effort?–What do they allow scientists to do that they couldn't do

without gateways? –What are some specific examples of this? –Why are these examples important?


GatewaysA natural result of the impact of the internet on worldwide

communication and information retrieval

•Implications on the conduct of science are still evolving– 1980’s, Early gateways, National Center for Biotechnology Information

BLAST server, search results sent by email, still a working portal today– 1992 Mosaic web browser developed– 1995 “International Protein Data Bank Enhanced by Computer Browser”– 2004 TeraGrid project director Rick Stevens recognized growth in scientific

portal development and proposed the Science Gateway Program– Today, Web 3.0 and programmatic exchange of data between web pages

•Simultaneous explosion of digital information– Growing analysis needs in many, many scientific areas– Sensors, telescopes, satellites, digital images and video– #1 machine on Top500 today is 1000x more powerful than all combined

entries on the first list in 1993

•Scientists developing their own tools to take advantage of the internet and help address the digital data problem


Only 17 years since the release of Mosaic!

NSF understands the impact technology can have on science

•“Virtual environments have the potential to enhance collaboration, education, and experimentation in ways that we are just beginning to explore.”

•“In every discipline, we need new techniques that can help scientists and engineers uncover fresh knowledge from vast amounts of data generated by sensors, telescopes, satellites, or even the media and the Internet.”


Gateways are a terrific example of interfaces that can support

transformative science

Today, there are approximately 35 gateways using the TeraGrid


Why are gateways worth the effort?

•Increasing range of expertise needed to tackle the most challenging scientific problems–How many details do you

want each individual scientist to need to know?•PBS, RSL, Condor•Coupling multi-scale codes•Assembling data from multiple sources

•Collaboration frameworks


#! /bin/sh#PBS -q dque#PBS -l nodes=1:ppn=2 #PBS -l walltime=00:02:00#PBS -o pbs.out#PBS -e pbs.err#PBS -Vcd /users/wilkinsn/tutorial/exercise_3../bin/mcell nmj_recon.main.mdl

+( &(resourceManagerContact="tg-login1.sdsc.teragrid.org/jobmanager-pbs") (executable="/users/birnbaum/tutorial/bin/mcell") (arguments=nmj_recon.main.mdl) (count=128) (hostCount=10) (maxtime=2) (directory="/users/birnbaum/tutorial/exercise_3") (stdout="/users/birnbaum/tutorial/exercise_3/globus.out") (stderr="/users/birnbaum/tutorial/exercise_3/globus.err"))

=======# Full path to executableexecutable=/users/wilkinsn/tutorial/bin/mcell

# Working directory, where Condor-G will write # its output and error files on the local machine.initialdir=/users/wilkinsn/tutorial/exercise_3

# To set the working directory of the remote job, we# specify it in this globus RSL, which will be appended# to the RSL that Condor-G generatesglobusrsl=(directory='/users/wilkinsn/tutorial/exercise_3')

# Arguments to pass to executable.arguments=nmj_recon.main.mdl

# Condor-G can stage the executabletransfer_executable=false

# Specify the globus resource to execute the jobglobusscheduler=tg-login1.sdsc.teragrid.org/jobmanager-pbs

# Condor has multiple universes, but Condor-G always uses globusuniverse=globus

# Files to receive sdout and stderr.output=condor.outerror=condor.err

# Specify the number of copies of the job to submit to the condor queue.queue 1

Not just ease of useWhat can scientists do that they

couldn’t do previously?•Linked Environments for Atmospheric Discovery (LEAD) - access to radar data•National Virtual Observatory (NVO) – access to sky surveys•Ocean Observing Initiative (OOI) – access to sensor data•PolarGrid – access to polar ice sheet data•SIDGrid – expensive datasets, analysis tools•GridChem –coupling multiscale codes

•How would this have been done before gateways?TeraGrid 09 Education Program, June 22,

2009

Gateways democratize access to high end resources

•Almost anyone can investigate scientific questions using high end resources–Not just those in the research groups who log in to

supercomputers–Gateways allow anyone with a web browser to explore

•Opportunities can be uncovered via google–My 11-year-old son discovered nanoHUB.org himself while his class was studying Bucky Balls

•Foster new ideas, cross-disciplinary approaches–Encourage students to experiment

•But used in production too–Significant number of papers resulting from gateways including

GridChem, nanoHUB–Scientists can focus on challenging science problems rather

than challenging infrastructure problemsTeraGrid 09 Education Program, June 22,

2009

Tremendous Opportunities Using the Largest Shared Resources -

Challenges too!•What’s different when the resource doesn’t belong just to me?–Resource discovery–Accounting–Security–Proposal-based requests for resources (peer-reviewed access)

•Code scaling and performance numbers•Justification of resources•Gateway citations

•Tremendous benefits at the high end, but even more work for the developers•Potential impact on science is huge

–Small number of developers can impact thousands of scientists–But need a way to train and fund those developers and provide them with appropriate tools


Gateways in the marketplaceKids control telescopes and share images

•“In seconds my computer screen was transformed into a live telescopic view”–“Slooh's users include

newbies and professional astronomers in 70 countries”

•Observatories in the Canary Islands and Chile, Australia coming soon •5000 images/month since 2003•Increases public support for investment in these facilities


Learn about the jet stream using LEADCan be done with live weather data

•portal.leadproject.org•Education link

– LEAD to learn modules

•Polar jet stream


•North American Mesoscale (NAM) model– Two jet streams in North America –

polar and subtropical, polar is more prominent

– Polar generally moves west to east, but sometimes forms large loops to the north and south

•First image is a view of wind speed at 45, 55, 65, and 75 m/s.

•Also can see polar jet stream’s effect on surface temperatures


•Move ahead to 6z (1 AM EST) on Jan 30th

•Uncheck the box next to the 55 m/s wind speed and the 45 m/s wind speed•Looking at the 300 mb pressure and surface temperature, notice the upper-level ridge in the western part of the country and the trough in the eastern part of the country•Western states are experiencing warmer temperatures than the eastern states•Polar jet stream appears to be in the area of the largest surface temperature gradients and where the 300 mb geopotential height contours are tightest, making it more important to forecasters than the sub-tropical jet stream•During the winter months, the surface temperature gradients are strongest and farthest south, causing the polar jet stream to be stronger in the winter than the summer.


•Move ahead to 18z (1 PM EST) on January 30th

•Uncheck the box next to the 65 m/s wind speed and the Wind Barbs, check the box next to Upward Motion•Regions of upward motion at the entrance and exit of the high speed “jet streak” support the development of storm systems •Test your knowledge

–http://www.atmos.millersville.edu/~lead/JS_Questions.html


Social Informatics Data GridCollaborative access to large, complex datasets

•SIDGrid is unique among social science data archive projects– Streaming data which change over

time•Voice, video, images (e.g. fMRI), text, numerical (e.g. heart rate, eye movement)

•https://sidgrid.ci.uchicago.edu– Login and click portal– Search for "elan example"

•Public project with video,audio and annotation data

•Preview the data with Flash•Search•Export the data

–Can visualize and annotate with SIDGrid Elan software

» https://sidgrid.ci.uchicago.edu/about/getting_started

•Run the analysis scripts. TeraGrid 09 Education Program, June 22, 2009

http://www.ci.uchicago.edu/research/files/sidgrid.mov

Gateways and Gadgets•Web technologies change fast

– Must be able to adapt quickly

•Gateways and gadgets– Gateway components

incorporated into any social networking page

– 75% of 18 to 24 year-olds have social networking websites

•iPhone apps?•Web 3.0

– Beyond social networking and sharing content

– Standards and querying interfaces to programmatically share data across sites

•Resource Description Framework (RDF), SPARQL


Open Life Sciences Gateway

•Add a BLAST query to your igoogle page and run on the TeraGrid•Google.com, add stuff•Gadget authorization

– Click the link of “Personalize the gadget”– Use account/password scidac/scidac to login in

•Here is a short test sequence:•>MRLFKTRKSTDTYSTLAAQQQQQQQQQQQHQAEGSNISHSSNSSSNKSHTPATCSNRLNKSIVSSTSISSSLPDLHDKSPVMILSCTTLASNGATATAAVTATATGTAATSGGSLQQQQQQHLQHQQQQQPLRTATPTCLLSGRQTPSAI• Check the blast type to "blastp" and leave the job submission site as "any site". Click OK, you will find a new blast job is created on Job History Portlet (Gadget).


Tremendous Potential for Gateways

•In only 17 years, the Web has fundamentally changed human communication•Science Gateways can leverage this amazingly powerful tool to:–Transform the way scientists collaborate–Streamline conduct of science– Influence the public’s perception of science

•Reliability, trust, continuity are fundamental to truly change the conduct of science through the use of gateways–High end resources can have a profound impact

•We need you to help shape this very exciting future!



Now that you know more about gateways, what ideas come to mind for gateways in your area of interest?


Have fun at TG09

Contact me any timeNancy Wilkins-Diehr, [email protected]

teragrid science gateways nancy wilkins-diehr area director for science gateways [email protected]...

Documents