benchmark testing results: telvent arcfm solution running on microsoft sql server · 2017-01-30 ·...
TRANSCRIPT
BenchmarkTestingResults:TelventArcFM
SolutionRunningonMicrosoftSQLServer Benchmark testing confirms the superior performance and scalability of Telvent ArcFM Solution
10.0.1 SP1 running on SQL Server 2008 R2 Enterprise and Windows Server 2008 R2 Enterprise
Technical White Paper Published: October 2011 Applies to: Microsoft SQL Server 2008 R2 Enterprise
Abstract
In May and June 2011, Telvent and Microsoft worked together to conduct a series of performance and
scalability tests for Telvent ArcFM Solution 10.0.1 service pack 1 (SP1) running on Microsoft SQL Server
2008 R2 Enterprise and Windows Server 2008 R2 Enterprise.
The benchmark results validate that ArcFM Solution is a highly scalable system on SQL Server under real-
world conditions simulating a large utility with 10 million customers. The ArcFM geographic information
systems (GIS) functionality of ArcFM Solution running on SQL Server demonstrated excellent performance
and scalability under a heavy load. The Responder functionality also performed very well under a heavy
call load, handling 70,000 incoming calls per hour for 15 hours.
The benchmark results confirm that ArcFM Solution on SQL Server can scale and perform to meet the
needs of the world’s largest utilities. Utilities can be confident that ArcFM Solution 10.0.1 SP1, deployed
on SQL Server 2008 R2 Enterprise and Windows Server 2008 R2 Enterprise, provides a reliable foundation
for a smart grid solution with a leading price-to-performance ratio.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
2
©2011 Microsoft Corporation. All rights reserved. This document is provided “as-is.” Information and
views expressed in this document, including URL and other Internet Web site references, may change
without notice. You bear the risk of using it.
This document does not provide you with any legal rights to any intellectual property in any Microsoft
product. You may copy and use this document for your internal, reference purposes.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
3
Table of Contents
Introduction ............................................................................................................................................. 4
Brief Overview of Telvent ArcFM Solution .................................................................................................. 5
ArcFM..................................................................................................................................................... 6
Responder .............................................................................................................................................. 6
Benefits of SQL Server for ArcFM Solution ................................................................................................. 7
Benchmark Testing Confirms Performance and Scalability of ArcFM Solution on SQL Server ................... 9
Benchmark Testing Goals ......................................................................................................................... 9
Benchmarking Test Scenarios................................................................................................................. 10
Test Database .......................................................................................................................................... 10
ArcFM Testing Process ............................................................................................................................. 11
Responder Testing Process ....................................................................................................................... 12
Benchmark Testing Setup ...................................................................................................................... 13
Database and Application Servers ........................................................................................................... 13
Initial Setup and Tuning ........................................................................................................................... 14
Benchmark Testing Results .................................................................................................................... 14
Summary of Results for ArcFM ................................................................................................................. 15
Summary of Results for Responder .......................................................................................................... 16
Conclusion ............................................................................................................................................. 16
Additional Information .......................................................................................................................... 17
About Telvent ........................................................................................................................................... 17
About the Microsoft Worldwide Utilities Group ....................................................................................... 17
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
4
Introduction Today’s utilities face the challenges of tighter IT budgets, reduced in-house
development staff, and the need to quickly deliver business benefits through
applications. Utilities also need to adapt to the rapid changes in smart grid technology,
the electricity distribution system of the future.
Telvent and Microsoft work together to provide integrated solutions that can help
utilities meet these challenges and better use their distribution assets for maintenance
and growth. By working together to integrate key aspects of their solutions—including
Telvent ArcFM Solution, Microsoft SQL Server, and Windows Server—Telvent and
Microsoft address the most pressing needs of the smart grid age, including performance
metrics, response times, and security.
ArcFM Solution running on SQL Server and Windows Server provides a complete, out-of-
the-box solution for facilities management designed to meet the needs of utility
businesses. The solution offers a map-centric, intuitive way to model, design, maintain,
and manage facility and land–based information.
In May and June 2011, Telvent, a Microsoft Gold Certified Partner, joined with Microsoft
to conduct a series of benchmarking tests to verify the performance and scalability of
ArcFM Solution 10.0.1 SP1 running on SQL Server 2008 R2 Enterprise and Windows
Server 2008 R2 Enterprise.
The results of the benchmark testing were impressive. Highlights of the results include:
• ArcFM Solution on SQL Server was able to support the requirements of a typical
10 million–customer utility for all processes tested.
• The Responder functionality successfully processed 70,000 calls per hour for 15
consecutive hours (with no user load simulated during the testing).
• The ArcFM Solution user experience in the benchmark test was comparable to
the user experience in tests run locally on significantly smaller datasets and with
a smaller load.
• ArcFM Solution running on SQL Server performed equally well on a large system
(32 cores, 512 GB RAM) and a smaller system (8 cores, 24 GB RAM).
This white paper presents an overview of ArcFM Solution and of SQL Server, and details
the results of the benchmark testing. It also includes links for further information.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
5
Brief Overview of Telvent ArcFM Solution Telvent ArcFM Solution is an extension of the ESRI ArcGIS platform, a market-leading
geographic information systems (GIS) software platform. ArcFM Solution consists of a
family of models and a set of sophisticated tools that provide a comprehensive
application for electric, gas, water, and wastewater utilities, and for telecommunication
providers. ArcFM Solution enables maintenance and data viewing using company-
specific business rules, and includes applications to configure the data models and tools
to work together.
Telvent applications gain leverage for enterprise GIS through the use of open
technologies. Key components of this strategy include an industry-standard, open-
development environment, open databases, and Component Object Model (COM)
architecture. All Telvent application software takes advantage of this object-oriented
architecture and provides for fully networked solutions in a client-server environment.
The ArcFM Solution suite was developed using Visual Basic 6 (VB6), C++, and C#; these
languages can also be used for customization and extension of the applications.
Figure 1 shows the ArcFM Solution architecture for utilities.
Figure 1. ArcFM Solution architecture for utilities
For more information, see ArcFM Solution Overview.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
6
Microsoft and Telvent conducted benchmark testing on two core elements of the ArcFM
Solution—ArcFM and Responder—which together provide a solid foundation for the
smart grid.
ArcFM
ArcFM is a complete GIS utility solution for modeling, editing, maintaining, and
managing facility asset data both in the field and on the desktop. ArcFM includes tracing
tasks to automate utility operations, and an extensive set of editing tools such as
automated snapping and attribute inheritance. ArcFM also includes tools to quickly
define and recall map sheet collections for map production. Users can create and edit a
session in the field, then send it back to the enterprise to be used for editing the
database, or geodatabase, using the Session Manager (see Figure 2).
Figure 2. ArcFM Session Manager
For more information about ArcFM, see Telvent Smart Grid Solutions Suite: ArcFM
Enterprise GIS.
Responder
Responder is a GIS-hosted outage management system (OMS) that takes advantage of
Microsoft .NET and ASP technologies to enable trouble call and outage incident
management in a web-based, scalable, and configurable desktop environment. The
Responder Trouble Call Analysis (TCA) Engine uses a sophisticated and iterative
prediction algorithm to determine which interruptible network device caused an
outage. High-performance GIS display capabilities provide utility personnel with a spatial
view of trouble call locations, enabling faster analysis of outages and immediate
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
7
dispatch of crews (see Figure 3). The application also supports historical archiving and
performance indices reporting.
Figure 3. Responder
For more information, see Smart Grid Solution Suite: Responder.
Benefits of SQL Server for ArcFM Solution SQL Server 2008 R2 Enterprise provides a reliable, cost-effective, low-maintenance
database framework for ArcFM Solution. SQL Server is a comprehensive, integrated, and
enterprise-ready data management software that can support the largest and most
process-intensive deployments.
Using SQL Server, Telvent customers can take advantage of reduced licensing, hardware,
administration, and support fees, in addition to enterprise-class performance and
scalability. Hundreds of organizations are currently running 10 terabyte (TB) and larger
transactional databases on SQL Server.1
1 http://www.microsoft.com/presspass/itanalyst/docs/06-30-09EnterpriseDatabaseManagementSystems.pdf
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
8
Running ArcFM Solution on SQL Server provides customers with many benefits. These
benefits include:
• Scalability and performance.
SQL Server includes many new features that help Telvent customers scale up
and scale out as their businesses grow.2
• Lower hardware costs.
SQL Server runs on standard commodity server hardware, providing
dramatically lower total cost of ownership (TCO) for utilities compared to
mainframe computers.
• Lower software costs.
With SQL Server, customers can typically enjoy a 3:1 reduction over the largest
competitor in licensing costs.3
• Simpler systems management and lower staffing costs.4
SQL Server database administrators (DBAs) can typically manage three times as
many physical databases as a competitor’s DBAs.5
• Lower maintenance costs.6
SQL Server first-year maintenance costs are up to 77 percent less than those of
the largest competitor solutions.
• Fewer security vulnerabilities.
SQL Server has consistently had fewer security vulnerabilities than the largest
competitor solutions.7
With SQL Server, Telvent customers can save with reduced licensing, hardware,
administration, and support fees, which translate into substantially lower costs over the
life of the system.
For more information, see the SQL Server home page.
2 http://www.microsoft.com/sqlserver/2008/en/us/performance-scale.aspx
3 http://www.microsoft.com/sqlserver/2008/en/us/value-calc.aspx
4 http://www.microsoft.com/sqlserver/2008/en/us/compare-oracle.aspx
5 http://www.alinean.com/PDFs/Microsoft_SQL_Server_and_Oracle-Alinean_TCA_Study_2010.pdf
6 http://www.microsoft.com/sqlserver/2008/en/us/compare-oracle-calc.aspx
7 NIST National Vulnerability Database, http://nvd.nist.gov/
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
9
Benchmark Testing Confirms Performance and
Scalability of ArcFM Solution on SQL Server As part of their ongoing commitment to provide utilities with truly scalable GIS systems,
Telvent and Microsoft conducted scalability and performance testing of the ArcFM and
Responder functionalities of ArcFM Solution running on SQL Server 2008 R2 Enterprise
and Windows Sever 2008 R2 Enterprise. The benchmark testing confirmed that
Telvent customers can choose SQL Server as a database platform for ArcFM Solution
with confidence for several reasons:
• The lower cost of using SQL Server does not translate to a compromise in
performance.
Some of the largest and smallest utilities in the world use ArcFM Solution. The
benchmark testing results demonstrate that SQL Server can scale easily across
the spectrum of usage patterns without any significant performance
degradation.
• Geometric network operations are not affected by system load or server
capacity.
In the benchmark testing, a geodatabase powered by ArcFM Solution and
managed by SQL Server showed no significant changes in performance as loads
were increased from very small to very large. Performance remained consistent
from a single user to 400 concurrent users.
• ArcFM functionality scales well as system usage increases over time.
The benchmark testing looked at the effects of a rapidly growing dataset when
multiple users added features to the geometric model over time. It confirmed
that with SQL Server as a database, users will not experience a significant
decrease in performance in creating, editing, and saving ArcFM sessions as their
dataset grows.
Benchmark Testing Goals The goal of the benchmark testing was to confirm that ArcFM and Responder running on the
Microsoft platform can meet the requirements of the largest Telvent utility customers.
To test the ArcFM functionality, Microsoft and Telvent simulated the most common GIS
activities typically conducted at Telvent customer sites. Table 1 lists the goals for ArcFM.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
10
Table 1. Goals for ArcFM benchmark testing
Panning and zooming
within a map
Verify that performance for these processes does not degrade
as the database size and load increase
Tracing the geometric
network
Creating and completing
edit sessions
To test Responder, Microsoft and Telvent used the custom load–simulation tool
TroubleMaker to simulate trouble or outage calls. Table 2 lists the goals for the Responder
benchmark testing.
Table 2 Goals for Responder benchmark testing
Prediction
performance testing
(call processing)
Process at least 50,000 trouble calls/trouble events per
hour
User experience
testing
Verify that as the database size and load increase,
performance does not degrade and remains comparable to
that of SQL Server competitors
Note that Telvent and Microsoft also set the goal of identifying and taking advantage of
any SQL Server 2008 R2 Enterprise performance improvement opportunities.
Benchmarking Test Scenarios To ensure that the testing was as realistic as possible, Telvent and Microsoft used test
scenarios that simulated a very large utility with approximately 10 million customers.
Test Database One of the goals of the benchmark testing was to verify that performance remained
constant as the database size and load increased. Two full database configurations were
therefore used in the testing:
• 32 cores and 512 GB of RAM
• 8 cores and 24 GB of RAM
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
11
The geodatabase was designed to emulate the characteristics of a large city:
• 10 million customers
• 3.0 million service points
• 0.9 million transformers
• 1.9 million primary conductors, overhead (OH)
• 1.3 million primary conductors, underground (UG)
• 2.7 million secondary conductors, OH
• 2.6 million secondary conductors, UG
ArcFM Testing Process To test ArcFM, Telvent used Citrix EdgeSight to simulate common client activities
running concurrently for 30 minutes:
• Panning and zooming within the map (400 simultaneous users for 30 minutes)
• Tracing (400 simultaneous users for 30 minutes)
• Editing (200 simultaneous users for 30 minutes)
Note: The number of simultaneous users used in the benchmark testing is twice the
known number of concurrent users at any Telvent customer site.
Panning and Zooming
The Pan and Zoom test simulated 400 simultaneous users panning and zooming within a
map and using the following procedure:
1. Set minimum and maximum XY coordinates to establish a working area in the
map.
2. Select a random XY coordinate within the designated area and pan to it.
3. Select a scale between 1:1,000 and 1:100,000.
Tracing
The Trace test simulated 400 simultaneous users performing several tracing operations
on Dynamic Protective Device (DPD) features using the following procedure:
1. Open the Locator tool and perform an attribute search for all DPD features.
2. Enter a number in the ObjectID field to locate a random DPD feature.
3. Zoom to the located DPD feature and center it in the map.
4. Select the Downstream Trace tool and click on the DPD feature to initiate a
trace.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
12
5. Clear the results and perform a Downstream Protective Device trace on the
same feature.
6. Clear the results and return to the Locator tool to search for another random
DPD feature.
Editing
The Edit test simulated 200 simultaneous users completing an editing session using the
following procedure:
1. Open Process Framework and create a new session. Name it based on the
current time.
2. Start the edit session in ArcMap, and zoom to a specific location on the map.
3. Sketch a number of poles (approximately 50).
4. Sketch a primary overhead conductor along the placed poles.
5. Save and close the session.
Responder Testing Process To test Responder, Microsoft and Telvent used Citrix EdgeSight and TroubleMaker to
simulate 70,000 trouble or outage calls per hour. These calls were submitted and
processed through the Responder Prediction Engine for 15 consecutive hours. This load
is more than double the load Telvent has observed at their largest existing customer
sites.
Two types of Responder tests were conducted:
• Prediction performance
• User experience
Prediction Performance Testing
The benchmark test used the TroubleMaker event simulation agent to submit calls at
various rates (between 25,000 calls/hour and 100,000 calls/hour). Telvent reviewed the
accuracy of the outage-prediction metrics (based on calls). Telvent also confirmed that the
number of outage groups created from the predictions (referred to as “incidents” in
Responder) was the expected number given the prediction thresholds for the areas.
User Experience Testing
The benchmark test calculated the time to perform various client activities such as cancel,
create, confirm, and restore an incident/outage. Activities were performed with various
system loads, and calls were submitted concurrently.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
13
Benchmark Testing Setup The benchmarking test was conducted in the Microsoft Enterprise Engineering Center (EEC)
labs in Redmond, Washington, using standard, commodity servers and storage to confirm a
realistic cost of ownership for the solution.
Database and Application Servers For the ArcFM testing, the environment consisted of one database server and 12 application
servers in a Citrix farm (using Citrix XenApp 5.0).
For the Responder testing, the environment consisted of three business logic servers and
one Responder message-router server running the Responder business logic. Testing also
included a Responder web server and the Citrix EdgeSight server farm to provide the
simulated user load.
Figure 4 shows the hardware configuration used.
Figure 4. Benchmark testing environment
Table 3 shows details of the hardware used in the benchmark testing.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
14
Table 3. Details of hardware used
Backend tier Database server
32 cores, 512 GB RAM;
also reduced to
8 cores, 24 GB RAM
Middle tier
Responder business logic servers (3) 8 cores, 64 GB RAM
Responder web servers (1) 8 cores, 64 GB RAM
Responder message router server (1) 8 cores, 64 GB RAM
License server (1) 2 cores, 4 GB RAM
Domain controller (1) 2 cores, 4 GB RAM
Frontend tier
Citrix farm servers (12) 8 cores, 48 GB RAM
Citrix EdgeSight simulation (5) 8 cores, 32 GB RAM
The database server was connected to a XioTech Magnitude 3D 3000 storage area
network (SAN) with one hundred and ten 146 GB-15K-FC drives.
Initial Setup and Tuning The databases and application servers were set up according to the ArcFM and
Responder configuration guides. Telvent installed Citrix XenApp 5.0 (an application
management system that virtualizes, centralizes, and manages applications) along with
Citrix EdgeSight, and configured both to support up to 500 concurrent users.
Note that the testing included a large number of domain users to ensure real-world
scenarios. Each user profile was set to “roaming” to ensure the environments were
consistent across the Citrix server farm.
Benchmark Testing Results The benchmark testing results confirm that Telvent ArcFM and Responder running on
SQL Server and Windows Server are highly scalable systems that provide the
performance required by large Telvent customers.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
15
Summary of Results for ArcFM The benchmark testing showed that ArcFM is capable of supporting more than 400
users simultaneously panning different portions of the map without degradation of the
redraw performance.
Table 4 summarizes the results for the various ArcFM processes tested:
Table 4. Summary of results for ArcFM
Pan
and
zoom
• Pan and zoom scaled successfully up to 400 simultaneous users
(there was no degradation in system performance between a single
user and 400 users).
• In addition, even with 400 users, there was no performance
degradation between the large, scaled-up database configuration
(32 cores, 512 GB RAM) and the scaled-down configuration (8 cores,
24 GB RAM). This confirms that smaller hardware configurations can
provide high performance and scalability.
• The time of operation for pan and zoom in all scenarios remained at
0.5 seconds.
Tracing
• Tracing scaled successfully up to 400 simultaneous users (there was
minimal degradation in system performance between a single user
and 400 users).
• In addition, even with 400 users, there was no performance
degradation between the large, scaled-up database configuration
(32 cores, 512 GB RAM) and the scaled-down configuration (8 cores,
24 GB RAM). This confirms that smaller hardware configurations can
provide high performance and scalability.
• Time of operation for tracing functions varied by only milliseconds.
Edit
session
• The time of operation for opening and closing edit sessions
increased by less than 5 seconds when scaling from one user to 200
concurrent users.
• The time of operation for saving and closing a session grew by less
than 5 seconds when scaling from one user to 200 concurrent users.
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
16
Summary of Results for Responder The benchmark testing showed that the ArcFM Solution Responder functionality is able
to support high call volumes, with a maximum call processing of 70,000 calls per hour
over the course of 15 hours.
Table 5 summarizes the results for the Responder processes tested:
Table 5. Summary of results for Responder
Prediction
performance
testing (call
processing)
• Responder processed 70,000 calls per hour for an extended
period (approximately 15 hours, no user load).
User
experience
testing
• User experience metrics were comparable with tests run
locally on significantly smaller datasets with less of a load and
a smaller geodatabase size.
• User experience time measures for create, confirm, restore,
complete, edit, and close an incident/outage tasks were
comparable with tests run locally on significantly smaller
datasets with less of a load and a smaller geodatabase size.
• Note that the user experience measures for cancel,
consolidate, and move upstream an incident/outage tasks
were slow and/or timed out. The performance issues for these
particular operations under significant load and scale were
identified as issues with the Responder solution code, and will
be addressed in future versions.
Conclusion The benchmark testing results confirm that ArcFM and Responder are highly scalable
systems capable of supporting a geodatabase containing 10 million customers on SQL
Server 2008 R2 Enterprise and Windows Server 2008 R2 Enterprise.
The benchmark testing showed that the cost effectiveness of the Microsoft platform
does not mean a compromise in performance. ArcFM Solution running on SQL Server
showed no significant performance degradation as loads were increased from a single
user to 400 concurrent users. For ArcFM functionality, the testing showed that
performance remained consistent between a larger hardware configuration and a
Benchmark Testing Results: Telvent ArcFM Solution and Microsoft SQL Server
17
smaller, less powerful system. For Responder, the user experience for most tasks was
comparable to when run locally on significantly smaller datasets with a smaller load.
Highlights of the benchmark testing include the following results:
• Both ArcFM and Responder were able to support the real-world processes for a
typical 10 million–customer utility within target time windows for all processes
tested.
• Responder successfully processed 70,000 calls per hour through the Responder
Prediction Engine for 15 consecutive hours. This demonstrates that Responder can
successfully handle an unusually high call volume for an extended period of time.
• ArcFM functionality showed zero degradation when scaling to a significantly larger
database with more load and on a less powerful server. This proves the ability of
ArcFM running on SQL Server to scale while maintaining performance.
Telvent customers can be confident that ArcFM Solution running on the Microsoft
platform can meet their performance and scalability requirements.
Additional Information The following references provide more information about Telvent and Microsoft.
About Telvent
Telvent (NASDAQ: TLVT) is a global IT solutions and business
information services provider dedicated to helping improve efficiency, safety, and security
for the world’s leading companies. Telvent serves markets that are critical to the
sustainability of the planet, including the energy, transportation, agricultural, and
environmental sectors. For more information, visit www.telvent.com.
About the Microsoft Worldwide
Utilities Group The Microsoft Worldwide Utilities Group offers platform and partner solutions that
empower utilities to thrive in today’s market-driven environment through optimized
business operations in customer care, generation, tracing and risk management,
transmission and distribution, regulatory compliance, and enterprise services. For more
information, contact your local Microsoft sales representative, or visit
www.microsoft.com/utilities.