highly available biztalk concepts around the implementation of biztalk server 2006 in a high...
TRANSCRIPT
Highly Available BizTalk
Concepts Around the Implementation of BizTalk Server 2006 in a High Availability Environment
Daniel Toomey & Patrick Hoodpresenters
Goal of This Presentation
Not highly technical (i.e. no code) Not demo-heavy Discussion of the main concepts and strategy
of HA in an integration solution Understand Microsoft recommended
practices
Definition of High Availability (HA)
“…is a system design protocol and associated implementation that ensures a certain absolute degree of operational continuity during a given measurement period.” (www.wikipedia.org/wiki/High_Availability)
Definition of High Availability (HA)
Redundancy of each functional component Seeks to eliminate “single point of failure” Single component failure triggers recovery
mechanisms that are transparent to users of the system
High Availability vs High Performance
High Availability is about Failover Does not necessarily involve load balancing Active/Passive configuration Scaling Out
High Performance is about… Performance!! Typically about load balancing and managing
high throughput Active/Active configuration Scaling Out or Scaling Up Not necessarily Highly Available
High Availability and Integration
?
High Availability in BizTalk
BizTalk Components Databases (SQL Server) Services (Host Instances) Adapters (Send / Receive) Enterprise Single Sign-On (SSO)
Process for ensuring High Availability is different for each of these components
BizTalk Groups
Out-of-the-box functionality for BizTalk allows for the easy (and default) establishment of “BizTalk Groups”
A BizTalk Group is a collection of servers that host BizTalk services (hosts) which operate upon the same Message Box(es)
All hosts within a BizTalk Server Group are based upon the same set of configuration and message storage databases
Automatic Load Balancing
HA for BizTalk Databases
SQL Server supports BizTalk through data persistence: Stores all configuration, business rules, message state
and tracking info Stores the messages themselves Separates data from hosts that process the data
The most critical component in a BizTalk architecture Can be implemented as a SQL Server Cluster
(active / passive mode)
HA for BizTalk Databases
HA for BizTalk Databases
1. Create global domain accounts
2. Configure the SQL Server cluster before BizTalk installation
3. Install BizTalk
4. Run the BizTalk Configuration Wizard in custom configuration mode
5. Specify the SQL Server cluster for the BizTalk databases
HA for BizTalk Databases
Failover Behaviour in BizTalk: BizTalk databases are temporarily unavailable during
failover In-process host instances are recycled until connection
to the SQL Server is automatically restored Isolated host instances are paused, an error is
generated in the BizTalk Server 2006 Application log and receive locations are disabled
Once connection to the SQL databases is restored, document processing resumes normally and receive locations are enabled
HA for BizTalk Databases
SQL Server Database Mirroring Not currently a supported solution for ensuring
high availability of the Microsoft BizTalk Server 2006 databases
Potential problems maintaining transactional consistency in the BizTalk databases
Log Shipping is the recommended practice for Disaster Recovery
HA for BizTalk Hosts
Hosts provide logical containers for functionality: Receiving Sending Processing
Recommended practice is to create hosts for each separate functionality Creates security boundaries Easier management & scalability
HA for BizTalk Hosts
In-process Hosts Run inside of BizTalk runtime process Contain all non-Web-based artefacts:
Orchestrations Adapter send handlers Adaptor receive handlers
(except for HTTP & SOAP)
Isolated Hosts Do not run inside of BizTalk runtime process HTTP and SOAP receive handlers
HA for BizTalk Hosts
BizTalk Server 2006 lets you separate hosts and run multiple host instances to provide high availability
No additional clustering or load-balancing mechanism required because BizTalk Server 2006 automatically distributes workload across multiple computers through host instances
However, hosts running the receive handler for the following adapters require a load-balancing mechanism such as Network Load Balancing (NLB) to provide high availability: HTTP SOAP BizTalk Message Queuing (MSMQT)
HA for BizTalk Hosts (Receiving)
Scaled Out Receiving Hosts
HA for BizTalk Hosts (Receiving)
Scaled Out Receiving Hosts (multiple clients)
HA for BizTalk Hosts (Receiving)
Using host instances on multiple computers: FILE Adapter (point host instances to same UNC path) SQL Adapter (point host instances to same database table)
Using host instances on multiple computers with NLB: HTTP Adapter (subscribe to a shared clustered URL) Web Services Adapter (NLB distributes incoming messages) SharePoint Adapter (subscribe to a shared URL) MSMQT Adapter (NLB distributes incoming messages)
Using a clustered BizTalk host (req. Enterprise Edition): FTP Adapter POP3 Adapter (multiple concurrent connections) MSMQ Adapter
HA for BizTalk Hosts (Receiving)
Adapter Type Default Config NLB Cluster Clustered Host
FILE
HTTP
SOAP
SQL
WSS
FTP
POP3
MSMQ
HA for BizTalk Hosts (Processing)
Scaled Out Processing Hosts
HA for BizTalk Hosts (Processing)
Scaled Out Processing Hosts Orchestration state is maintained centrally
in SQL Server, not locally on each BizTalk Server computer
BizTalk load balances automatically One instance can complete a process
started by another instance Proof – of – Concept
BizTalk Host Load Balancing
Used “CallOrchestration” sample from SDK Inserted Delay shapes and trace messages to
log the step and the processing server Deployed to two servers in a BizTalk Group Submitted 1000 files Analysed the resulting logs
For more than 25% of the files, processing steps were divided across more than one individual server (i.e. host instance)
HA for BizTalk Hosts (Sending)
Scaled Out Sending Hosts
HA for BizTalk Hosts (Sending)
Scaling Out Sending Hosts Similar to Processing Hosts – Host & Data Independence Special Considerations:
FTP Send Adapter Run in a clustered BizTalk Host Supports only one host instance running at a time
MSMQ Adapter Cluster the MSMQ Service Cluster the BizTalk Host in the same group Configure MSMQ Send Handler within clustered host
BizTalk Host Clustering
Only necessary for certain adapters Requires BizTalk 2006 Enterprise Edition Requires BizTalk Servers to be configured as a Windows
Server Cluster first Considerations:
Non-clustered host should not be run on a Windows Server cluster where Enterprise SSO is clustered
More info: http://msdn2.microsoft.com/en-us/library/aa560059.aspx
Network Load Balancing (NLB)
As previously mentioned, the following adapters require a load-balancing mechanism such as Network Load Balancing (NLB) to provide high availability: HTTP SOAP BizTalk Message Queuing (MSMQT)
Can load-balance the BAM portal & BAS website Provides High Availability at the Network level, rather
than the Resource level
Network Load Balancing (NLB)
NLB farm of servers appears as one server to clients
Distributes load between the servers in the farm
Each server in the NLB farm is aware of each other and automatically handle server unavailability
Each server is fully self-contained BizTalk grouping provides balancing on
hydration of long-running processes
Network Load Balancing (NLB)
Network Load Balancing (NLB)
Easier and more flexible management Rolling OS update & software deployment
Uninterrupted availability and fault tolerance Server failure & hardware update/replacement
Better scalability True horizontal scalability Up to 32 servers in an NLB farm Multiple farms via DNS round-robin
Network Load Balancing (NLB)
Option of Hardware-based or Software-based NLB solution
Hardware-based solution consists of a specialised network appliance e.g.
F5 Networks Radware Cisco Foundary Alteon
Windows NLB
Full software NLB implementation Supported on all versions of Windows
2003 Server Supported on Windows 2000 Advanced
Server and Datacenter Server Editions Generally a 5-10% overhead per server MSCS and Windows Network Load
Balancing (NLB) are NOT supported on the same set of nodes
Windows NLB
Consider NICs & Unicast vs. Multicast
Mode & Number of NICs Use
Single network adapter in unicast mode A cluster in which ordinary network communication among cluster hosts is not required and in which there is limited dedicated traffic from outside the cluster subnet to specific cluster hosts.
Multiple network adapters in unicast mode A cluster in which ordinary network communication among cluster hosts is necessary or desirable. It is also appropriate when you want to separate the traffic used to manage the cluster from the traffic occurring between the cluster and client computers.
Single network adapter in multicast mode A cluster in which ordinary network communication among cluster hosts is necessary or desirable but in which there is limited dedicated traffic from outside the cluster subnet to specific cluster hosts.
Multiple network adapters in multicast mode A cluster in which ordinary network communication among cluster hosts is necessary and in which there is heavy dedicated traffic from outside the cluster subnet to specific cluster hosts.
Windows NLB
Port-rules – multiple-host or single-host Affinity - can be set to:
None Single-client (or sticky-IP) Class C
Host Priorities For BizTalk NLB, recommend multiple
host, no affinity, even priority
Enterprise Single Sign-On (SSO)
Critical part of the BizTalk infrastructure Helps to secure information for the receive
locations Master Secret Server
Stores the encryption key used to secure data in the credentials database
Must configure the first computer where SSO is installed as the Master Secret Server
Enterprise Single Sign-On (SSO)
If Master Secret Server fails, currently running operations continue* but cannot encrypt new credentials
BizTalk Server dependency on Master Secret Server:
*Following link suggests that BizTalk runtime will fail after a period of time if Master Secret Server fails (courtesy of Tim Goodsell):
http://www.microsoft.com/technet/prodtechnol/biztalk/2006/library/bts06clustering/c5fe4626-ff82-45cd-a62f-848fdc61e6ff.mspx?mfr=true
High Availability for Ent SSO
Master Secret Server CANNOT exist on an NLB cluster
Master Secret Server can be moved from BizTalk NLB servers (often to SQL Server infrastructure)
Master Secret Server can be clustered
Summary
In an multi-system environment, High Availability means securing not only the individual systems themselves but also the integration architecture
BizTalk Server 2006 can be implemented to support High Availability using a variety of techniques and configurations for the various components: OTB functionality via BizTalk Server Groups SQL Server Failover Cluster Windows NLB Cluster Clustered Hosts
References
BizTalk & High Availability:
Planning for High Availabilityhttp://msdn2.microsoft.com/en-us/library/aa558765.aspx
Planning Your Platform for Fault Tolerancehttp://msdn2.microsoft.com/en-us/library/aa560135.aspx
Creating a Highly Available BizTalk Server Environmenthttp://msdn2.microsoft.com/en-us/library/aa560847.aspx
Sample BizTalk Server High-Availability Scenarioshttp://msdn2.microsoft.com/en-us/library/aa578057.aspx
Providing High Availability for BizTalk Server Databaseshttp://msdn2.microsoft.com/en-us/library/aa559920.aspx
High Availability for the BizTalk Base EDI Adapterhttp://msdn2.microsoft.com/en-us/library/aa561569.aspx
High Availability and the Microsoft Operations Frameworkhttp://msdn2.microsoft.com/en-us/library/aa560207.aspx
References
Enterprise Single Sign-On:
High Availability for Enterprise Single Sign-Onhttp://msdn2.microsoft.com/en-us/library/aa560674.aspx
Backup, Cluster, Move & Restore Enterprise SSOhttp://msdn2.microsoft.com/en-us/library/aa559192.aspxhttp://msdn2.microsoft.com/en-us/library/aa560589.aspxhttp://msdn2.microsoft.com/en-us/library/aa559364.aspxhttp://msdn2.microsoft.com/en-us/library/aa561823.aspx
Window Server Cluster:
Using Windows Server Cluster to Provide High Availability for BizTalk Hostshttp://msdn2.microsoft.com/en-us/library/aa560059.aspx
Improving Fault Tolerance in BizTalk Server 2006 by Using a Windows Server Cluster (includes BizTalk MSCS and clustering the SSO) http://www.microsoft.com/technet/prodtechnol/biztalk/2006/library/bts06clustering/6e6fee7b-1317-4e70-ab21-a145f2aa8594.mspx?mfr=true
References
Windows Server 2003 Network Load Balancing:
http://articles.techrepublic.com.com/5100-6345_11-5208171.html http://www.west-wind.com/presentations/loadbalancing/NetworkLoadBalancingWindows2003.asp http://technet2.microsoft.com/WindowsServer/en/library/65319bac-2efe-4764-8752-
d091447dddbe1033.mspx?mfr=true http://technet2.microsoft.com/WindowsServer/en/library/884c727d-6083-4265-ac1d-
b5e66b68281a1033.mspx?mfr=true
NLB Multicast vs. Unicast:
http://searchwincomputing.techtarget.com/tip/0,289483,sid68_gci1230735,00.html http://searchwincomputing.techtarget.com/tip/0,289483,sid68_gci1229240,00.html
Questions?