july, 2007 gcb sage workshop sage – architecture and api
TRANSCRIPT
![Page 1: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/1.jpg)
July, 2007 GCB SAGE workshop
SAGE – Architecture and API
![Page 2: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/2.jpg)
July, 2007 GCB SAGE workshop
Early Tile-Display Software: JuxtaView
• Visualization of 2D datasets with predictive pre-fetching.
• Slow user interaction and inefficient data access
NCMIR – microscopy (2800x4000 24 layers)
Scripps – Bathymetry and digital elevation
![Page 3: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/3.jpg)
July, 2007 GCB SAGE workshop
Early Tile-Display Software: MagicCarpet
• Smart mip-mapped 2D data access
• Very interactive
• Needs datasets to be pre-processed
![Page 4: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/4.jpg)
July, 2007 GCB SAGE workshop
Early Tile-Display Software: Vol-a-Tile
• Visualization of 3D datasets• Transfer function and UI
![Page 5: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/5.jpg)
July, 2007 GCB SAGE workshop
The Sage Pixel Streaming Architecture• Stream uncompressed pixels generated by apps to display walls• Designers tried to achieve a desktop-like environment for tile displays
![Page 6: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/6.jpg)
July, 2007 GCB SAGE workshop
Adaptive Rendering
• Adapt to the display client– Laptop, high-resolution monitor, tiled display
• Rendering capabilities– Pixel drawing, polygon rendering– Network capacity
• Adapt to output resolution– “Resize” and “move” events– Pixel up-scaling or down-sampling– Increase rendering resolution
![Page 7: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/7.jpg)
July, 2007 GCB SAGE workshop
Pixel Sources
• Visualization applications– Software rendering– Hardware rendering
• Legacy applications– RDP, ARD, VNC– TeraVision (hardware capture)
• Video streaming
![Page 8: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/8.jpg)
July, 2007 GCB SAGE workshop
Working in Display-Rich Environments
Remote laptop
High-resolution maps
Live video feeds
3D surface rendering
Volume Rendering
Remote sensing
![Page 9: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/9.jpg)
July, 2007 GCB SAGE workshop
SAGE Components
• Capture the pixels
• Partitioning of the images
• Routing the pixels
• Layout on the display
• User interaction
![Page 10: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/10.jpg)
July, 2007 GCB SAGE workshop
Basic Pipeline
• With SAGE we separate the rendering and display.
•Rendering machine is connected to display driven by thin client by fast network
![Page 11: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/11.jpg)
July, 2007 GCB SAGE workshop
Pipeline Gets Complicated…•More complicated scenarios
•Rendering machine sends pixels to multiple displays.
•There are multiple rendering machines (cluster)
•Assume that the rendering is done remotely across big fat networks
•Provide source (rendering software) with user interaction feedback
![Page 12: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/12.jpg)
July, 2007 GCB SAGE workshop
Ideal Scenario
![Page 13: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/13.jpg)
July, 2007 GCB SAGE workshop
Why Raw Pixel Streaming
• Pixels are transmitted raw (without compression) because networks are getting faster/cheaper.
• We don’t spend time compressing and decompressing on the CPU
• New SAGE supports DXT compression
![Page 14: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/14.jpg)
July, 2007 GCB SAGE workshop
Related Work
• WireGL / Chromium– Streams OpenGL primitives– Slow. User interactive apps suffer.– Not designed for fat long distance networks– Complex configuration
• XDMX– Highly researched. Very flexible.– Streams X primitives– Slow– Not designed for fat long distance networks– No support for mullions
![Page 15: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/15.jpg)
July, 2007 GCB SAGE workshop
Related Work … contd.
• IBM’s SGE– Hardware switched solution– Streams raw pixels– Not scalable– Expensive and legacy.
• TeraVision– Hardware and open source software solution– Streams raw pixels from any video source– Scalable– Expensive– No support for dynamic routing.
![Page 16: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/16.jpg)
July, 2007 GCB SAGE workshop
Architecture …• Free Space
manager provides central control between apps, UI and system
• SAGE applications send their streams directly to the display nodes
![Page 17: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/17.jpg)
July, 2007 GCB SAGE workshop
Free Space Manager
• Central control unit of SAGE.
• Setups the entire system based on configuration files
• Communicates with UI clients
• Window movement and resize require messages to be passed between the Free Space Manager and apps
• Sends SAGE status messages to apps.
![Page 18: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/18.jpg)
July, 2007 GCB SAGE workshop
SAGE Application Interface Library (SAIL)
• Library for writing SAGE applications
• Interfaces with the Free Space Manager to receive and send system messages
• When a SAGE application gives SAIL a new video frame, it sends pixels directly to the appropriate portions of the display tiles.
![Page 19: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/19.jpg)
July, 2007 GCB SAGE workshop
More Details• Frame syncing
– Every frame displayed within SAGE is synced with a TCP message.
– So 60 fps = 60 messages per second
• Windowing system– FrenchWindows
• Collaborative features– UI can be launched and used by multiple users at the
same time– New SAGE (v2.x) provides multiple mouse pointers.
![Page 20: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/20.jpg)
July, 2007 GCB SAGE workshop
Basic Code to ‘SAGEify’ an Existing AppsailConfig scfg; // Setup SAGE structure
scfg.cfgFile = "sage.conf";scfg.appName = “myapp";scfg.rank = 0;
sageRect renderImageMap;renderImageMap.left = 0.0;renderImageMap.right = 1.0;renderImageMap.bottom = 0.0;renderImageMap.top = 1.0;scfg.imageMap = renderImageMap;scfg.colorDepth = 24;scfg.pixFmt = TVPIXFMT_888;scfg.rowOrd = BOTTOM_TO_TOP;
sageInf.init(scfg); // Init SAGEwhile (1) {
.. <render video frame> ..
sageInf.swapBuffer( rgbBuffer ); // send rendered video frame to SAGE }
![Page 21: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/21.jpg)
July, 2007 GCB SAGE workshop
SAGE UI
• Connects to SAGE over the network.
• Main window is divided into two sections: – top which represents
the display the UI is connected to
– bottom which holds information about the applications currently running.
![Page 22: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/22.jpg)
July, 2007 GCB SAGE workshop
SAGE UI
• Resize or move application windows like on a desktop
• Change ‘depth’ order of windows
![Page 23: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/23.jpg)
July, 2007 GCB SAGE workshop
SAGE UI
• Maximize the windows
• Preserve (or not) the aspect ratio of rendering app.
![Page 24: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/24.jpg)
July, 2007 GCB SAGE workshop
SAGE UI
• Performance monitoring built in
• Bandwidth (Network and Display)
• FPS (Network and Display)
• Number of nodes used by an app
![Page 25: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/25.jpg)
July, 2007 GCB SAGE workshop
SAGE UI
• Performance graphs
• Built in logging – Global option
accessible via ‘Performance’ menu
![Page 26: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/26.jpg)
July, 2007 GCB SAGE workshop
SAGE UI
• Session recording and playback
![Page 27: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/27.jpg)
July, 2007 GCB SAGE workshop
SAGE UI
• Can connect to multiple displays at same time
![Page 28: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/28.jpg)
July, 2007 GCB SAGE workshop
SAGE UI
• Chat features as a back channel
![Page 29: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/29.jpg)
July, 2007 GCB SAGE workshop
SAGE Application Model
• Application– Rendering component– User interface component
• Layout on the screen controlled by the FreeSpace Manager
• Pixels captured by SAIL– SAGE Application Interface Library
• Streaming network protocol
![Page 30: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/30.jpg)
July, 2007 GCB SAGE workshop
SAIL – The SAGE Application Interface Library
• The SAGE programming API.
• Simple interface allows you to specify a SAGE displayand connect to it
• glSwapBuffer like call
![Page 31: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/31.jpg)
July, 2007 GCB SAGE workshop
OpenGL App SAGEified// headers for SAGE#include "sail.h"#include "misc.h" void reshape(int width, int height){ // blah blah}
//display functionvoid redraw(void){ // draw code glReadPixels(0, 0, winWidth, winHeight,
GL_RGB, GL_UNSIGNED_BYTE, rgbBuffer); sageInf.swapBuffer((void *)rgbBuffer); glutSwapBuffers(); • }•
int main(int argc, char *argv[]){ // application code glPixelStorei( GL_UNPACK_ALIGNMENT, 1 ); sailConfig scfg; scfg.cfgFile = "sage.conf"; scfg.appName = "render"; scfg.rank = 0; scfg.ip = argv[2]; scfg.resX = 400; scfg.resY = 400; sageRect renderImageMap; renderImageMap.left = 0.0; renderImageMap.right = 1.0; renderImageMap.bottom = 0.0; renderImageMap.top = 1.0; scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = BOTTOM_TO_TOP; sageInf.init(scfg); cout << "sail initialized " << endl; glutMainLoop();}
![Page 32: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/32.jpg)
July, 2007 GCB SAGE workshop
Write a Native SAGE App• // for SAGE• #include <sail.h>• #include <misc.h>• • // SAGE Stuff• int winWidth, winHeight;• sail sageInf; // sail object• • // initialize SAGE• sailConfig scfg;• scfg.cfgFile = "sage.conf";• scfg.appName = “myApp";
• scfg.rank = rank;• scfg.ip = NULL;• • scfg.resX = (extent.w / extent.zoom);• scfg.resY = (extent.h / extent.zoom);• • sageRect renderImageMap;• renderImageMap.left = 0.0;• renderImageMap.right = 1.0;• renderImageMap.bottom = 0.0;• renderImageMap.top = 1.0;•
•
scfg.imageMap = renderImageMap;scfg.colorDepth = 24;scfg.pixFmt = TVPIXFMT_888;scfg.rowOrd = TOP_TO_BOTTOM;
sageInf.init(scfg); // create zoom adjusted image bufferbuffer = new unsigned char[(extent.w / extent.zoom) * (extent.h / extent.zoom) * (3)];// clear the zoom adjusted image buffermemset(buffer,0,(extent.w / extent.zoom)*(extent.h / extent.zoom)* 3); // Main while loopwhile (!quit){ // Generate pixels and copy into ‘buffer’ // swap buffer sageInf.swapBuffer(buffer);}
![Page 33: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/33.jpg)
July, 2007 GCB SAGE workshop
Building and Installing SAGEFrom Source
– Dependencies– Compilation– Configuration
![Page 34: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/34.jpg)
July, 2007 GCB SAGE workshop
SAGE Dependencies
• QUANTA 0.4 (www.evl.uic.edu/cavern/quanta)• Readline (runtime and development packages):
from GNU project, http://cnswww.cns.cwru.edu/~chet/readline/rltop.html
• SDL libraries for the display side, http://www.libsdl.org
• Some test programs need GLUT to compile (atlantis, atlantis-mpi, ...), http://freeglut.sourceforge.net
![Page 35: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/35.jpg)
July, 2007 GCB SAGE workshop
Compiling SAGE
• Modifying makefiles• Open the appropriate Makefile in the "sage/src"
directory and edit the following lines:– Set QUANTA_DIR to the directory where you built
QUANTA – Set the appropriate flag for your shell preference:– If you use bash, set MYFLAGS –DSAGE_BASH– If you use csh, set MYFLAGS –DSAGE_CSH
![Page 36: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/36.jpg)
July, 2007 GCB SAGE workshop
Compiling SAGE .. Contd.
• Execute make install in the "sage/src" directory.
• Execute make install in the "sage/src/nwProtocol" directory.
• Execute make install in sage/app and sage/app/atlantis.
![Page 37: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/37.jpg)
July, 2007 GCB SAGE workshop
Edit Configuration Files
• Go to sage/bin
• Edit fsManager.conf
• Edit sage.conf
• Edit stdtile.conf (or whatever tile config file specified in sage.conf)
![Page 38: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/38.jpg)
July, 2007 GCB SAGE workshop
Sample sage.confdisplayBinDir SAGE_DIR/binappBinDir SAGE_DIR/bin
appList
render {configName localnodeNum 1 Init 100 100 1000 1000exec 127.0.0.1 render 0 127.0.0.1nwProtocol tvTcpModule.so}
atlantis {configName localnodeNum 1Init 100 100 1000 1000exec 127.0.0.1 atlantis 0 127.0.0.1nwProtocol tvTcpModule.soconfigName UDPnodeNum 1Init 100 100 1000 1000exec 127.0.0.1 atlantis 0 127.0.0.1nwProtocol tvUdpModule.so}
endList
![Page 39: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/39.jpg)
July, 2007 GCB SAGE workshop
Sample sage.conf … contd.tileConfiguration stdtile.conf
receiverBaseSyncPort 12000receiverBufNum 20receiverStreamPort 21000fullScreen 1
sailBaseSyncPort 11000
![Page 40: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/40.jpg)
July, 2007 GCB SAGE workshop
Sample stdtile.conf
• For 1 node only :
TileDisplayDimensions 1 1 Mullions 0.625 0.625 0.625 0.625Resolution 1280 1024PPI 90Machines 1
DisplayNode Name localhostIP 127.0.0.1Monitors 1 (0,0)
![Page 41: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/41.jpg)
July, 2007 GCB SAGE workshop
Stdtile.conf For Multiple NodesTileDisplay
Dimensions 5 3 Mullions 0.625 0.625 0.625 0.625Resolution 1280 1024PPI 90Machines 15
DisplayNode Name yorda1-10IP 10.0.8.121Monitors 1 (0,2)
DisplayNode Name yorda2-10IP 10.0.8.122Monitors 1 (0,1)
DisplayNode Name yorda3-10IP 10.0.8.123Monitors 1 (0,0)
DisplayNode Name yorda4-10IP 10.0.8.124Monitors 1 (1,2)
•
DisplayNode Name yorda5-10IP 10.0.8.125Monitors 1 (1,1)
DisplayNode Name yorda6-10IP 10.0.8.126Monitors 1 (1,0)
DisplayNode Name yorda7-10IP 10.0.8.127Monitors 1 (2,2)
DisplayNode Name yorda8-10IP 10.0.8.128Monitors 1 (2,1)
DisplayNode Name yorda9-10IP 10.0.8.129Monitors 1 (2,0)
![Page 42: July, 2007 GCB SAGE workshop SAGE – Architecture and API](https://reader035.vdocuments.mx/reader035/viewer/2022062315/56649db35503460f94aa3232/html5/thumbnails/42.jpg)
July, 2007 GCB SAGE workshop