visualization in health grid environments: a novel service and business approach

Visualization in Health Grid Environments: a Novel Service and Business Approach

Frank Dickmann1, Mathias Kaspar1, Benjamin Löhnhardt1, Nick Kepper2,3, Fred Viezens4, Frank Hertel4, Michael Lesnussa2, Yassene Mohammed5,

Andreas Thiel6, Thomas Steinke7, Johannes Bernarding4, Dagmar Krefting8, Tobias A. Knoch2,3, Ulrich Sax9

1Department of Medical Informatics, University of Göttingen, Robert-Koch-Straße 40, 37075 Göttingen, Germany.

2Biophysical Genomics, Dept. Cell Biology & Genetics, Erasmus MC, Dr. Molewaterplein 50, 3015 GE Rotterdam, The Netherlands.

3Biophysical Genomics, Genome Organization & Function, BioQuant Center/ German Cancer Research Center, Im Neuenheimer Feld 267, 69120 Heidelberg, Germany.

4Otto-von-Guericke University, Institute for Biometrics and Medical Computer Science, Leipziger Str. 44, 39120 Magdeburg, Germany.

5RRZN – Regional Compute Centre for Lower Saxony, Leibniz Universität Hannover, Schloßwender Straße 5, 30159 Hannover, Germany.

6OFFIS / R&D Division Health, Escherweg 2, 26121 Oldenburg, Germany. 7Zuse Institute Berlin (ZIB), Takustrasse 7, 14195 Berlin-Dahlem, Germany.

8Department of Medical Informatics, Charite - University Medicine, Campus Benjamin Franklin (CBF), Hindenburgdamm 30, 12200 Berlin, Germany.

9Department of Information Technology, University Medicine Göttingen, Robert-Koch-Straße 40, 37075 Göttingen, Germany.

{fdickmann, mathias.kaspar, benjamin.loehnhardt}@med.uni-goettingen.de, [email protected], {fred.viezens, frank.hertel,

johannes.bernarding}@med.ovgu.de, [email protected], [email protected], [email protected], [email protected],

[email protected], [email protected], [email protected].

Abstract. Advanced visualization technologies are gaining major importance to allow presentation and manipulation of high dimensional data. Since new health technologies are constantly increasing in complexity, adequate information processing is required for diagnostics and treatment. Therefore, the German D-Grid initiative started to build visualization centers in 2008, which have recently been embedded into the existing compute and storage infrastructure. This paper describes an analysis of this infrastructure and the interplay with life science applications for 3D and 4D visualization and manipulation. Furthermore, the performance and business aspects regarding accounting, pricing and billing are investigated. The results show the viability and the opportunities for further optimization of this novel service approach and the possibilities for a sustainable business scenario.

Key words: MediGRID, distributed visualization, accounting and billing, telemedicine, service business model.

2 Visualization in Health Grid Environments: a Novel Service and Business Approach

1 Introduction

The MediGRID community represents biomedical life science users as a part of the D-Grid1 initiative in different projects since 2005. A grid computing infrastructure has been built according to healthcare requirements as well as biomedical research applications ranging from medical imaging to genome analysis [1]. Beside the initial MediGRID project, Services@MediGRID, MedInfoGRID, PneumoGRID, Gap-SLC and WissGrid projects and the Medical Grid Forum are working on further grid aspects within life sciences2. While grid infrastructures are capable of processing biomedical data of increasing complexity, new adequate approaches of presenting the related data are necessary because existing grid services do not tackle this issue. The presentation approach concerns visualization of high dimensional data. Here, multiple dimensions are presented in 2D or 3D including temporal changes in order to reduce the level of complexity [2]. This contributes to the fact that the human visual perception can handle complex structures, coherencies and dependencies more easily [3]. With the new generations of high performance graphics chips, data presentation for professional visualization purposes have become affordable, e.g. in Cave Automatic Virtual Environments (CAVETM), but not yet widely available. For professional multi-dimensional output in a workplace environment, 3D autostereoscopic screens offer a screen with perceivable depth by using parallax barrier technology [4]. Additionally present medical IT products do not support interfaces to the grid and expensive hardware is required, e.g., Picture Archive and Communication Systems (PACS) displaying medical imaging data.

While grid computing can offer the required storage and compute power, high quality graphics output is still rare. Nevertheless, visualization nodes with high end graphic capabilities integrated within the grid computing infrastructure can provide high quality graphics output to a broader range of scientists. This approach follows the original idea of offering visualization devices through the grid by Foster and Kesselmann as in the case of radio telescopes [5]. Thus, the goal of the presented solution in this paper is to supply scientists and researchers with an on-demand high performance graphics and compute platform.

The German life science grid community is tackling this issue by establishing a grid based infrastructure for visualization resources. These resources are high performance grid nodes equipped with special high-end graphics output devices. Since the beginning of 2009 the community projects – MediGRID, Services@MediGRID and MedInfoGRID – have set up grid clusters enhanced by high-end graphics output devices in Berlin, Göttingen, and Heidelberg in Germany. The sites of Berlin and Göttingen are also equipped with 3D autostereoscopic screens.

Here, we describe and analyze the current infrastructure in respect to the interaction with life science applications for visualization and manipulation, in terms of the technical feasibility and performance of this concept. Sustainability of such a complex service depends on the translation into adequate business models regarding

1 D-Grid initiative: http://www.d-grid.de. 2 MediGRID: http://www.medigrid.de, Services@MediGRID: http://services.medigrid.de,

MedInfoGRID: http://www.medinfogrid.de, Grid forum: http://www.tmf-ev.de. The other projects have been initiated recently and the websites are not available yet.

Visualization in Health Grid Environments: a Novel Service and Business Approach 3

accounting, pricing and billing. The results show further opportunities for optimization and that the complexity can be handled and used for a sustainable business scenario in life sciences with its complex needs.

2 Visualization in the Grid

In the following we describe (a) the technical specifications of the present installations and (b) a performance analysis that shows the advantages of this infrastructure by using life science applications.

2.1 Hardware, OS and Software Specifications

The visualization resources in Berlin, Göttingen and Heidelberg are based on HP Scalable Visualization Array (SVA) / XC cluster technology [6]. Each node of the cluster is configured with two NVIDIA Quadro FX5600 cards. For different visualization workloads each cluster is built from two types of nodes: a) 8x HP xw8600 workstations for default and MPI parallelized visualization applications (e.g. based on MPI and VTK), and b) one or two HP Proliant DL785 32 core server equipped with 128 or 256 GB RAM.

3D autostereoscopic screens are installed in research labs at the Charité and ZIB (Berlin) and at the department of Medical Informatics (Göttingen). The screens with diagonals of 42 inch (Full-HD screen resolution of 1920x1080 pixels) and 27 inch (WUXGA screen resolution of 1920x1200 pixels.) support up to five viewers simultaneously. The 3D autostereoscopic screens are connected to the cluster resources via KVM switches (EVS-4 by Thinklogical) via optical fiber and are therefore stationary.

The Remote Graphics Software (RGS) within the HP SVA implementations consists of a sender component and a receiver component. Its functionality is similar to well-known remote desktop software client/server products like Virtual Network Computing (VNC). It offers the possibility to transmit full HD resolution even through a low bandwidth network with a minimum loss of performance and quality. RGS achieves this by advanced image compression methods combined with an additional user option to adjust the picture quality level. In theory, an RGS session can be viewed by an unlimited number of guests. In order to use 3D autostereoscopic output of OpenGL based applications TechViz XL was chosen. TechViz transforms the OpenGL stream into an autostereoscopic output and directly sends it to the graphics card driver.

2.2 Integration into the D-Grid Environment

To integrate the visualization resources into the D-Grid computing environment, the required MediGRID grid middleware stack had to be installed. This includes e.g.,


installation of the Globus Toolkit on the head node of the visualization clusters. The XC-SVA preconfigured batch system for visualization jobs is based on SLURM/LSF and was made interoperable with Globus. Application workflows of the MediGRID Virtual Organization (VO) are handled by the Grid Workflow Execution Service (GWES). Visualization workflows impose additional constraints regarding usage time (usually daily working hours). To handle the constraints GWES still needs further optimization.

3 Performance Analysis

To test the feasibility of this new grid approach, a test bed is defined by using life science applications: These are: 3D-Slicer [7], GLOBE 3D Genome Browser [8], and the Visual Molecular Dynamics (VMD) [9]. The analysis includes local and distant collaboration tests, and a performance test.

3.1 Test Bed Applications

3D-Slicer is a free open source visualization and image processing tool designed for medical imaging [10]. Its BSD license allows unrestricted commercial use. 3D-Slicer offers representation of multi-dimensional medical data, time series of volume data, multi-modal images or simulated data. It also allows export of visualization scenes in standard formats, and is further extensible by external stand-alone applications..

The GLOBE 3D Genome Browser is a grid based virtual “paper tool” developed for the analysis, manipulation and understanding of multi-dimensional genomic data in a 3D environment. The Genome Browser is designed for research on genomic information in a holistic manner.

VMD [11] is a tool designed to visualize, model and analyze biological systems such as proteins, nucleic acids and lipid bilayer assemblies. VMD can be used to animate and analyze trajectories of molecular dynamics simulations (MD). It can also act as a graphical front end for an external MD program by displaying and animating a molecule undergoing simulation on a remote computer.

3.2 Test Bed Parameters for Collaboration and Performance Tests

The test bed focuses on the rendering speed and usability of the applications in terms of latency. Therefore, the frames per second were measured for different tests. The influence of the 3D autostereoscopic rendering by TechViz and of the RGS quality setup were considered.

The RGS performance was compared to a Lenovo ThinkPad T60 laptop (1,83 GHz CPU; ATI Mobility Radeon X1300 with 64 MB dedicated; 1 GB RAM) as a very common computer model. For the 3D tests the autostereoscopic monitors were used in full resolution; the RGS and the T60 were tested with a SXGA resolution of 1280x1024 pixel. TechViz was used for the 3D output. Due to the lack of an FPS


indicator 3D-Slicer was tested by judging the level of latency. 3D-Slicer is also not compatible with TechViz. All three applications used only one CPU core.

3.3 Collaboration and Performance Tests

The perception of depth within the 3D displays has been successfully tested for up to five viewers with VMD and the Genome Browser. The models of the Genome Browser appear to be more realistic. Distant collaboration of the RGS has been successfully tested with networks of 1 GBit/s, 54 MBit/s wireless LAN and asynchronous private internet access network. Within the wireless and high-bandwidth cable environment the transmission at 100% quality is possible. Nonetheless, it depends on the network load and the screen resolution. Sharing the graphics output with two or more guests does not affect the RGS performance significantly.

The performance of 3D-Slicer was tested with the model SPL Abdominal Atlas [12] on three different test runs: i) conventional layout, ii) 3D only layout, iii) volume visualization. The cluster offered an overall good performance without latency except iii) displayed with RGS. The laptop showed latency but was still usable for ii) and showed no latency for i). For iii) the latency was too high on the laptop. The use of RGS reduces the perceived performance.

For the test of the Genome Browser three test runs were defined: i) 1 chromosome, ii) 3 chromosomes, and iii) 20 chromosomes. The usage of RGS reduces the performance as well as for 3D-Slicer. The measurement of FPS indicated a reduction of 41 percent on average for SXGA. While there was no significant performance difference between the two test resolutions, a reduction of 95 percent on average was measured by using WUXGA and TechViz. On average the cluster was 3442 percent faster than the laptop at SXGA by using RGS.

The tests of the VMD were performed using the X-ray diffraction structure of xylose (glucose) isomerase from actinoplanes missouriensis (PDB-ID 9XIM). With this structure three test runs were applied: i) no zoom, ii) continuous rotation, and iii) 5x zoom with rotation. The measured FPS indicated a performance decrease between WUXGA and SXGA by 33 percent on average. Furthermore, the FPS decreased by 54 percent on average with RGS enabled at SXGA. Applying TechViz in WUXGA reduced the FPS by 87 percent on average. The cluster was faster by 91 percent on average than the laptop at SXGA by using RGS.

4 Accounting, Pricing and Billing Approaches

According to our performance study, visualization services can be offered via resources which are integrated into the grid. Accounting determines pricing and billing of the services and is to be defined by possible use cases. The infrastructure costs are reflected in the service prices. Thus, the service supplier structure needs to be analyzed. For billing prices have to be combined with the appropriate metrics to measure the actual usage of each service.


Regarding the distributed complex character of the grid, accounting, pricing, and billing of visualization services need to be integrated into the technical and organizational structure of D-Grid.

4.1 Implementation of Accounting in D-Grid

Accounting in D-Grid is based on the Distributed Grid Accounting System (DGAS) [13] and encompasses participating D-Grid resource providers, infrastructure resources and services. Resource usage is collected locally and transmitted to the central DGAS service at the RRZN. Accounting information are accessible per user, per VO, per site and per D-Grid supervisor (ROC-Manager) level via the HLRmon website and the command line tools of DGAS. Each level aggregates the information and can be filtered on a per job view. HLRmon and the DGAS command line tools are web based and use certificate based authentication and authorization. Currently, the implementation focuses on the CPU related accounting metrics. Metrics for storage accounting are planned to be established. [14]

To establish legally liable relationships amongst, resource providers, general service providers in D-Grid, and professional service providers, who will be represented by a Virtual Organization, service level agreements (SLA) are required [15]. SLAs will offer reliability for the professional service providers as well as the users. Because a VO is not legally liable a responsible institution is required to act as the partner of SLA contracts on behalf of the VO. Professional service providers can then distribute their grid based IT services and fulfill the requirements of their customers – the users. SLAs between the VO MediGRID and the resource providers as well as the general service provider RRZN for accounting are in the process.

4.2 Use Cases for Visualization Services

According to the technical specifications and the designated workflow there are eight visualization use cases in the grid: local and distant remote usage, each with and without including additional compute grid tasks and each with and without sharing the graphics output for collaboration purpose.

Compute& Storage

Local / Remote Visualization

Compute& Storage

Fig. 1. Use cases as a value chain of visualization in the grid.

The use case of local usage of visualization resources includes the stationary 3D autostereoscopic displaying of applications and the local usage via RGS. For remote visualization just RGS is applicable (3.2). Further computational grid tasks can be performed before and after the visualization itself in order to prepare data sets for visualization, and to analyze interactivly modified data from a visualization session. These use cases are sequential and can be described using a value chain [16] (Fig. 1).


4.3 Metrics Definition for Accounting of Visualization Services

There are usually two parameters used for the accounting of the consumed computing power: the CPU time and the wall clock time [17]. The wall clock time measures the runtime of a job on a node regardless whether the CPU is fully occupied at all times or not. The real-time character of visualization jobs require the accounting of the complete time of a node occupation since other visualization jobs cannot be processed on the same node since the HP SVA allows only one session per node. Thus, the complete session time of visualization jobs needs to be measured and represents the accounting parameter for interactive visualization jobs processed by grid resources. Additionally, the pre-login time after a successful reservation is to be assigned to the requesting grid user. This is necessary because the reservation blocks resources and therefore incurs opportunity costs. Thus, the overall visualization time is similar to the wall clock time (Fig. 2). Due to the lack of parallelized visualization applications using distributed graphics rendering power, here the accounting measures the use of exactly one node. For a future use of distributed graphics rendering power, the accumulated session times will be required to be accounted.

Due to the defined use cases (4.2) the accounting of compute and storage resources within the grid is also required. The batch processed compute jobs, the occupied storage space and the real-time session usage for visualization should be combined in a holistic accounting approach. In other words, the more diverse the use cases are the more complex accounting of visualization jobs becomes due to the increasingly different individual metrics involved.

t

at = Timea = Accounted session timeRt = ReservationLt = LoginEt = Session end

EtLtRt

pre-login session Fig. 2. Accounted visualization session time.

4.4 Cost Factors for Pricing

Visualization services factors consist of hardware, software, service and support as a whole IT service [18]. Hardware will be offered by resource providers as well as basic software, e.g., operating systems. General services, mostly the middleware, are offered by the D-Grid Integration Project such as accounting and monitoring services. The discipline-related tools and applications are offered by professional service providers. Each provider also offers support to users and other providers. The end-user is required to pay for all of these services. [19]


The costs of resource providers comprise hardware investments, software licenses and subsequent annual expenditures regarding energy, maintenance and administration. The prices of resource providers are usually defined on a total cost of ownership calculation. Since the local visualization scenarios require 3D screens, the facility costs should also be included.

Since customers like to receive full service as well as the invoice from a single point of contact, professional service providers should aggregate all relevant costs, including its own total cost of ownership.

Considering visualization as a grid service, the present grid economy definition [20] needs to be extended to addressing real-time services. Since the graphics performance of the contemporarily available resources is very similar, the graphics performance does not yet affect the choice of resources. Further development will increase diversity due to different choices of visualization implementations. Thus, visualization resources will become more diverse and this will have direct impact on the choice of appropriate resources.

Due to its real-time character and the limited number of resources the prices for visualization services should also be adjusted to varying demands during the day: prices during peak hours should obviously be higher than during non-peak hours. This will generate a price competition between visualization resource providers in different time zones. Additionally, as on-demand services for collaborative use cases visualization services have restrictions due to the fact that resources have to be available at an appointed time. Thus, low prices can be charged for advance reservations because they support resource providers in order to optimize the utilization of the visualization resources. Nonetheless, visualization resources can also act as common compute nodes and their utilization can be optimized by compute jobs.

The price for a visualization session can be charged on the usage basis or on a flat rate. A flat rate will reduce complexity for the end-user and the professional service provider but there is also a risk of excessive usage by the end-users. Thus, constant price monitoring will be vital for visualization grid business models.

4.5 Distribution and Billing

The defined accounting and pricing concept for visualization services in the grid are relevant for distribution in a grid service market. Furthermore, the additional value of the visualization services has to be emphasized and communicated to the customers. According to the use cases, the integration into an on-demand high performance compute infrastructure and the availability of worldwide collaboration are the predominant customer values. Beyond, the customers need to have an easy access to visualization services. Thus, brokers are necessary in order to bring together customers and professional service providers [16]. For the visualization services, the broker also needs to support resource and/or room reservation, estimated session duration, and ideally estimates the costs based on the reservation parameters.

Customers will be billed based on the accounting and pricing information according to their contracts with professional service providers. The resulting accounting information is to be aggregated on a per service base for each customer and provides the invoice data.


5 Conclusion and Outlook

Obviously advanced visualization technologies for the presentation and manipulation of multi-dimensional data sets would largely profit from the usage of grid visualization infrastructures. The analysis of this infrastructure by a feasibility test showed that the envisioned goal: visualization resources can be accessible to multiple viewers. In an institutionally and internationally distributed research environment as in the life sciences with a growing demand for visualization applications the grid will be able to support collaboration and therefore scientific progress. Nevertheless, adequate visualization tools for a distributed environment are still missing. The performance of the tested life science applications can be significantly increased by using more than one CPU and/or GPU core. Therefore, the development of visualization applications for life sciences has to take parallelization into account especially in respect to future market sustainability. This also addresses parallelization of the graphics rendering process. This would result in an enormous increase in speed of 3D applications, which is the best competitive advantage marketable.

Based on this technological advancement accounting, pricing and distribution aspects were analyzed. Accounting requires additional metrics as well as integration between the resources, general services and professional services within D-Grid. The integration of an accounting mechanism will be crucial for further commercial viability. Since total costs of the infrastructure determine the visualization service price, every involved grid resource provider and service provider need to have an audit regarding their own costs. Distribution will require a customer relationship management according to the requirements of life science customers to achieve commercial success. Transparent billing integration into enterprise resource planning systems can increase acceptance by customers and professional service providers.

Consequently, visualization in (health) grid environments could be a big opportunity for novel service and business approaches with increased complexity due to the integration of various resources into a virtual “high-performance” desktop environment and thus offers great possibilities for sustainable valorization.

Acknowledgements

This publication was supported by the alliance projects Services@MediGRID (FKZ 01IG07015A-G), MedInfoGrid (FKZ 01G07016A) and the D-Grid Integration Project (FZK 01IG07014) funded by the German Federal Ministry of Education and Research (BMBF).

References

1. Krefting, D., Bart, J., Beronov, K., Dzhimova, O., Falkner, J., Hartung, M., Hoheisel, A., Knoch, T.A., Lingner, T., et al.: MediGRID: Towards a user friendly secured grid infrastructure. Future Generation Computer Systems 25(3), 326-336 (2009)


2. Holden, B.J., Pinney, J.W., Lovell, S.C., Amoutzias, G.D., Robertson, D.L.: An exploration of alternative visualisations of the basic helix-loop-helix protein interaction network. BMC Bioinformatics 8, 289 (2007)

3. Noirhomme-Fraiture, M., Nahimana, A.: Visualization. In: Symbolic Data Analysis and the SODAS Software (Diday, E., Noirhomme-Fraiture, M., eds.), pp. 109-120. Wiley-Interscience, New York, NY, USA (2008)

4. Dodgson, N.A., Moore, J.R., Lang, S.R., Multi-View Autostereoscopic 3D Display. In: The International Broadcasting Convention (IBC'99), pp. 497-502, Amsterdam, http://www.cl.cam.ac.uk/~nad10/pubs/IBC99-Dodgson.pdf, (1999)

5. Foster, I., Kesselman, C.: The Grid 2: Blueprint for a New Computing Infrastructure. Morgan Kaufmann (2003)

6. HP Scalable Visualization Array, http://h20311.www2.hp.com/HPC/cache/281455-0-0-0-121.html

7. 3D Slicer, http://www.slicer.org 8. Mangan, M.E., Williams, J.M., Lathe, S.M., Karolchik, D., Lathe, W.C., 3rd: UCSC

genome browser: deep support for molecular biomedical research. Biotechnol Annu Rev 14, 63-108 (2008)

9. VMD - Visual Molecular Dynamics, www.ks.uiuc.edu/Research/vmd 10. Pieper, S., Halle, M., Kikinis, R.: 3D SLICER. Proceedings of the 1st IEEE International

Symposium on Biomedical Imaging: From Nano to Macro 2004 1, 632-635 (2004) 11. Kepper, N., Foethke, D., Stehr, R., Wedemann, G., Rippe, K.: Nucleosome geometry and

internucleosomal interactions control the chromatin fiber conformation. Biophysical Journal 95(8), 3692-3705 (2008)

12. Talos, I., Jakab, M., Kikinis, R., SPL Abdominal Atlas. Surgical Planning Laboratory, Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA., http://www.na-mic.org/publications/item/view/1266 (2008)

13. The Distributed Grid Accounting System (DGAS), http://www.to.infn.it/grid/accounting/main.html

14. Wiebelitz, J., Brenner, M., Konzept für das Accounting im D-Grid 1.0. Regionales Rechenzentrum für Niedersachsen (RRZN), Hannover, Accessed: 2008.11.02, http://www.rrzn.uni-hannover.de/fileadmin/ful/mitarbeiter/wiebel/Accounting/DGI-2_FG-5-2_Acct-Konzept_08.pdf (2008)

15. McKee, P., Taylor, S., Surridge, M., Lowe, R., Ragusa, C.: Strategies for the Service Market Place. In: Grid Economics and Business Models, pp. 58-70 (2007)

16. Stanoevska-Slabeva, K., Talamanca, C., Thanos, G., Zsigri, C.: Development of a Generic Value Chain for the Grid Industry. In: Grid Economics and Business Models, pp. 44-57 (2007)

17. Thigpen, W., Hacker, T.J., Athey, B.D., McGinnis, L.F., Distributed Accounting on the Grid. In: Proceedings of the 6th Joint Conference on Information Science (Caulfield, H.J., Chen, S.-H., Cheng, H.-D., Duro, R.J., Honavar, V., Kerre, E.E., Lu, M., Romay, M.G., Shih, T.K., et al., eds.), pp. 1147-1150. JCIS / Association for Intelligent Machinery, Inc., Research Triangle Park, North Carolina, USA, (2002)

18. Zarnekow, R., Brenner, W., A product-based information management approach. In: Proceedings of the 11th European Conference on Information Systems, ECIS 2003, Naples, Italy, http://is2.lse.ac.uk/asp/aspecis/20030183.pdf, (2003)

19. Altmann, J., Ion, M., Bany Mohammed, A.: Taxonomy of Grid Business Models. In: Grid Economics and Business Models, pp. 29-43 (2007)

20. Buyya, R., Abramson, D., Venugopal, S.: The Grid Economy. Proceedings of the IEEE 93(3), 698-714 (2005)

visualization in health grid environments: a novel service and business approach

Documents