parallelizing large excel-based calculations on windows hpc server & azure
DESCRIPTION
Parallelizing Large Excel-Based Calculations on Windows HPC Server & Azure. Please silence cell phones. Computational Finance. Computational Finance emphasizes practical numerical methods rather than mathematical proofs and focuses on techniques that apply directly to economic analyses. - PowerPoint PPT PresentationTRANSCRIPT
April 10-12, Chicago, IL
Parallelizing Large Excel-Based
Calculations on Windows HPC Server &
Azure
April 10-12, Chicago, IL
Please silence cell phones
3
Computational Finance
Computational Finance emphasizes practical numerical methods rather than mathematical proofs and focuses on techniques that apply directly to economic analyses.
4
Quantitative AnalyticsMany problems in mathematical finance entail the computationof a particular integral. In many cases these integrals can be valuedanalytically, and in still more cases they can be valued using numerical integration, or computed using a PDE. However when the number of dimensions (or degrees of freedom) in the problem is large, PDEs and numerical integrals become intractable, and in these cases Monte Carlo Methods often give better results.
Black-Scholes Formula Equity Options
5
Monte Carlo Methods
Any method which solves a problem by generating suitable random numbers and observing that fraction of the numbers obeying some property or properties. The method is useful for obtaining numerical solutions to problems which are too complicated to solve analytically.
• Generate a set of (pseudo) random numbers in a known domain• Check whether each point is within the Area• Estimate the Area as a subset of the known domain
The higher the number of samples the more precise the calculation
6
Add-ins XLLs
Data Sources
Data Feeds RTDs
Microsoft Excel Computational Environment
VBA
Application.ScreenUpdating = False Calculate Range(Cells(36 + 5 * (k - 1), 22), Cells(40 + 5 * (k - 1), 60)).Select Selection.Copy Range("V26").Select Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, SkipBlanks
Application.ScreenUpdating = True
7
On-premise
HPC – Deployment Choices
Spreadsheet
8
On-Premise
Traditional HPC environmentDedicated compute nodes
Workstation nodes (CoW) & Server Scavenging
Supports specialized hardware (Infiniband, GPU, etc.)
2008 R2
On-premise head node and broker nodesAdministration- deployment, monitoring, diagnostics, & reporting
Scheduler – FCFS, Balanced, Pools, Preemption
Runtimes - Parametric sweep, MPI, SOA
9
Hybrid HPC – Burst to Azure
2008 R2SP1/SP2
Combine power of on-premise compute resources with scale-out capability of Azure.Cluster burst to Azure –
Add cloud resources as necessary
Worker Roles
VMs
On-premise head node & broker nodesFull Admin and Scheduler capabilities
Work (jobs/tasks) pushed to Azure nodes when available
Workloads: parametric sweep, MPI, SOA
10
HPC Hosted in Azure (HPC Scheduler on Azure)
Application hosted entirely in Azure No on-premise Head Node
Application is accessed fromRich client on premise
Portal
Web Application
Multiple business modelsPackaged application is sold to a customer
Application is available as a service in the cloud
Scheduler and Runtimes Supported in AzureWorkloads: Parametric Sweep, SOA, MPI
No Head Node (Scheduler Service, no Admin Services)
11
Compute Nodes
Spreadsheet
Head Node
Brokers
Request S
ession
Broker
Assignment
Task Submission
Task Assignment
Results
Results
11
Offloading Excel Calculations on a Cluster
12
WCF BrokerNode(s
)
Applications
Excel 2010
HPC/Excel Client
HPC/Excel Add-in
Macros
VBA Driver
Excel Driver
Excel Driver
HPC/Excel Service
Excel 2010
.NET / COM
CLI
Network ShareExcel Workbook
Desktop Client Windows HPC Cluster
Network Resources
Excel Components
13
Overview
Save temporary copy of workbook
Start Excel and Load Workbook
HPC_Initialize
HPC_Partition (Loop) HPC_Execute
HPC_Merge
Desktop HPC Cluster
HPC_ExecuteHPC_Execute
14
OverviewPorting a workbook for the cluster (1)Identify Iterative CalculationUsually a loop or similar construct
Identify Business LogicBusiness logic should be unchangedRe-use code for desktop, cluster calculation
Identify Startup CodeUsually some pre-run or startup requirements
15
OverviewPorting a workbook for the cluster (2)Add Client "Control" CodeAdd Skeleton MacrosConvert Desktop Code to Cluster Functions
Add Local Test FunctionsConsolidate Desktop and Cluster Code
16
Original VBA CodeIterative loop calculation
Function RunModel()
' set up variables NumIterations = Range("C8").Value
' run calculation in a loop For n = 1 To NumIterations rslt = CalculateSingleIteration(n) ConsolidateResults rslt Next n
' complete call UpdateCharts
End Function
17
Modified VBA CodeBusiness logic is moved, but
unchangedFunction HPC_Execute( data As Variant ) As Variant
' run single iteration HPC_Execute = CalculateSingleIteration(data)
End Function
Function HPC_Merge( data As Variant )
' insert results into workbook ConsolidateResults data
End Function
April 10-12, Chicago, IL
DemoDiamond Sponsor Platinum Sponsor
19
Win a Microsoft Surface Pro!
Complete an online SESSION EVALUATION to be entered into the draw.
Draw closes April 12, 11:59pm CTWinners will be announced on the PASS BA Conference website and on Twitter.
Go to passbaconference.com/evals or follow the QR code link displayed on session signage throughout the conference venue.
Your feedback is important and valuable. All feedback will be used to improve and select sessions for future events.
April 10-12, Chicago, IL
Thank you!Diamond Sponsor Platinum Sponsor