Open Source Grid Computing in the Finance Industry Alex Efimov STFC Kite Club

Download Open Source Grid Computing  in the Finance Industry Alex Efimov STFC Kite Club

Post on 24-Jan-2016




0 download


Open Source Grid Computing in the Finance Industry Alex Efimov STFC Kite Club Knowledge Exchange Advisor UK CERN Technology Transfer Officer +44 (0)1223 422405. STFC Introduction. - PowerPoint PPT Presentation


Open Source Grid Computing in the Finance IndustryAlex EfimovSTFC Kite ClubKnowledge Exchange AdvisorUK CERN Technology Transfer (0)1223 422405STFC IntroductionThe STFC was formed on 1 April 2007 by the merger of the Particle Physics and Astronomy Research Council (PPARC) and the Council for the Central Laboratory of the Research Councils (CCLRC)It is one of seven national research councils in the UK. Annual budget is around 600m: funds the best research within in particle physics, astronomy, nuclear physics and space science; enables the research community to have access to the best facilities in the world;increases the UK technology capability, engagement with industry and knowledge transfer.High costs of establishing a partnershipAsymmetric information flowsDifferences between scientific and business culturesInsufficient level of market level readinessLack of skills and experienceEquity gap STFC helps to overcome the following problemsgLite Commercialisation ProjectConstellation Technologies LtdConstellation Technologies Ltd, registered in October 2007 in the UKA spin-off company from STFCBased at Rutherford Appleton Laboratory (RAL)Develops and distributes the computer virtualization solutions based on gLite Grid middlewareEmploys IT experts from CERN and UK universitiesContact: Nick Trigg,, CLIK/RAL Constellation Technologies, Ltd. will provide following commercial services:Commercial technical support and trainingDevelop and distribute commercial enterprise and global solutions based on the gLite middlewareTechnical consultancy to software vendors and end-usersBuild commercial e-Infrastructure based on gLite and offer it on pay-as-you-use basis (Utility Computing Model)Grid in Finance: Current Status Vast majority of large financial institutions use Grid The largest share of the market (around 75%) DataSynapse and Platform Open source presence (Condor) is negligibleProprietary vendors know better what customers in the financial industry needGrid in FinanceInfrastructure LevelReducing overheadsImproving manageabilityPlanning and flexibilityApplication LevelRisk ManagementPortfolio ManagementDerivatives PricingTransactional activitiesFacing New ChallengesComplexity of modern IT infrastructures: physical servers, virtual machines, clusters, Grids, geographical distributionCost of electricityCredit crunchFurther pressures to reduce costsOpenness to the acceptable security conceptCase StudiesBank 1 Around 30k computers in more than 10 data centresDecentralised IT: resources belong to departmentsUse DataSynapse: more than 200 Grids Want to try virtualisationBank 2Around 20k computers in 9 data centresCentralised IT: all users are on hard SLAsUse Platform and CondorLittle or no interest in virtualisation Finding SolutionsImproving utilisation rates through market based algorithms for resource allocationAccessing external infrastructures on-demandUsing a single management platform for all computing resourcesOpportunity for gLitegLite for FinanceGrant Reference: ST/G000263/1Principal Investigator: Dr Neil Geddes, STFC (RAL)Project Title: Industrial research to integrate gLite (Grid software) into a platform for the commercial trading of computer resourcesFunds Awarded: 198,791.11Improving UtilizationHardware Servers: (-) low utilisation rates, scalability problemsUtilisationVirtual Servers: (+) improved utilisation rates, better scalability, easy disaster recovery(-) increased number of servers to manage, incompatible virtualization platforms Enterprise/Departmental Grid:(+) improves utilisation rates of physical servers, enables collaboration(-) limited scalability, lack of interoperability between vendors, limited efficiency of policy based mechanisms100%0%GridCloud Computing:(+) no need to own hardware, shared access, improved utilisation through pay-as-you-use(-) incompatible platforms, fair price is dubious to usersgLite Architecture for Finance File CatalogueLogging &Book-keepingResourceBrokerComputingResource Information ServiceJob StatusAuthor.&Authen.InputOutputUser/Grid interfaceDemandOfferVRMCompetitionHardwareOperating SystemVirtualisationEnterprise GridEnterprise CloudCloudConstellation TechnologiesIncompatible StandardsCreates pools of resources Higher utilisation ratesIncompatible StandardsInterfacesInterfaces and Market MechanismsKey differentiators: Open source no vendor lock-in ScalabilityPIPSS Features Maximum STFC contribution: 125k per yearMaximum project duration: 3 yearsCalls: quarterlyMaximum STFC contribution : 110k per yearMaximum project duration: 12 monthsCalls: applications may be made at any timeMini-PIPSS Features STFC Funding for PartnershipsTechnology TransferHow to get involved:Apply for Mini-PIPSS and PIPSS grantConsultancy projects with industryConstellation Technology Ltd is looking for people with gLite knowledge and programming skillsAll technologies mentioned above (virtualization, Grid, Cloud Computing) are intended to improve utilisation rates.Virtualization improves utilization but also introduces a problem. The growing number of virtual machines represents a new challenge to enterprises. Virtualization vendors offer solutions to manage pools of virtual machines but they are incompatible and they cant create mixed pools of VMs and physical servers. They also cant dial with geographically distributed infrastructures.Together virtualization and Grids the granularity of enterprise IT infrastructures. Instead of a uniform set of physical servers and storage enterprises now deal with a mixture of physical and virtual servers and departmental Grids. Though on average utilization rates improved it is still far from optimum. Further progress is constrained because this complexity of existing enterprise infrastructures makes it difficult to manage using the traditional policy based approach.Cloud Computing is gaining popularity because using IT infrastructures on demand and pay-as-you-use basis is more efficient than owning hardware. However, there is still no market for this service because various providers use incompatible platforms and users dont know the fair price. At the moment Resource Broker is policy based. Constellation Technologies is developing the Virtual Resource Marketplace software to replace the existing generic brokering process with the one based on market algorithms.*