Enabling Database as a Service With the Microsoft Private Cloud Stack-V11

Download Enabling Database as a Service With the Microsoft Private Cloud Stack-V11

Post on 09-Jul-2016




4 download


Enabling Database as a Service With the Microsoft Private Cloud Stack-V11


Enabling Database as a Service with the Microsoft Private Cloud stackA Building Clouds Blog eBook | Enterprise Cloud Group CAT TeamAuthor: Bruno Saille (Microsoft)Published: May 2015, Version 1.1AbstractThis paper is for IT Professionals and Database Administrators who would like to enable Database as a Service (DBaaS) for their end users. Providing DBaaS to applications owners or developers is a topic that comes up quite often these days in our interactions with customers. The Microsoft Private Cloud stack offers several ways to provide DBaaS. This document summarizes the options, and which requirements may lead you to prefer one option compared to the others. While this will mainly be in the context of Microsoft SQLServer, we will also cover options to achieve DBaaS for other types of database software.This paper applies to WindowsServer2012R2, Microsoft SystemCenter2012R2, and Microsoft WindowsAzurePack for WindowsServer. 2015 Microsoft Corporation. All rights reserved. This document is provided "as-is." Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it. Some examples are for illustration only and are fictitious. No real association is intended or inferred. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. You may modify this document for your internal, reference purposes.Some information relates to pre-released product which may be substantially modified before its commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here.Table of ContentsIntroduction: First, lets define Database as a Service4Delivering DBaaS scenarios through System Center and the Windows Azure Pack5Preparing the VM Clouds fabric7Templates for virtual machines with SQL Server9Preparing the SQL Servers fabric10Enabling DBaaS for tenants, through plans and subscriptions12Looking at the tenant experience18Plan subscription18Creating a virtual machine with SQL Server installed20Creating a database24Some requirements that make Windows Azure Pack a good fit for DBaaS31Frequently Asked Questions about Database as a Service in the context of Windows Azure Pack32Do I need to use both the VM Clouds and the SQL Server resource providers?32What if I like the shared approach, but want to provide more dedicated servers to my tenants?33How is the location of new databases determined in the shared model?34How can I can provide additional value-added services for my tenants?34Can I pre-provision resources for my tenants when they subscribe to my plans?35Using Windows Azure Pack, can I also enable Database as a Service for other types of database software?35My processes are largely based on ITIL today. How can I combine ITIL with Database as a Service?36Can I use the APIs to interact with the databases, as an administrator or a tenant?36How can I ensure that my tenants get their fair share of performance when using the shared model with the SQL Server resource provider?36When using IaaS to deploy SQLServer virtual machines, could my tenants also deploy highly-available virtual machines?37Can I get data for potential chargeback with my tenants?38Other options with System Center38Looking beyond Database as a Service: Why leverage Microsoft technologies to virtualize and manage SQL Server40References42Introduction: First, lets define Database as a ServiceEveryone has a different definition based on their requirements, but ultimately Database as a Service (DBaaS) means giving end users application owners, developers, and others the ability to request and provision database engine components in a self-service manner. For some enterprises and service providers, responding to requests for database services is an activity that takes a lot of time from DBAs on a daily basis, and automating it frees up valuable time for other value-added tasks, reduces human error, and increases consistency and confidence in operations.Depending on the requirements, the scope of a DBaaS implementation may include coverage for all or part of the following considerations: What components should be deployed? Deployment of a database, an instance, a machine with MicrosoftSQLServer installer, or a set of machines including SQLServer? Should or will deployments occur on a shared fabric only, or will we want to enable dedicated scenarios where database components are dedicated to a specific tenant? What about the service level and high availability aspects? Are we talking about deploying commodity databases without any high availability capabilities, or surfacing highly available database components? How much of the lifecycle of the database component should be covered? Is it just about the deployment and delegation of the database components, or should it cover the full lifecycle (resize, monitoring, database user management, decommissioning, etc.)?There is no good or bad answer to these questions, it just depends on what IT and business issues you are trying to solve with your DBaaS implementation.Whatever the end solution chosen, it will likely consist of multiple building blocks or layers like these: The fabric is the foundation layer providing compute, network, and storage resources for hosting the SQLServer database components. This fabric can benefit from the performance and scale capabilities from WindowsServer and SQLServer. For example, increased Hyper-V guest VM scale and performance (virtual processors, memory, disk IOPS), storage flexibility and choices for both virtual machines and databases (SMB support, Storage Spaces, etc.). Resource consolidation and pooling are often the first phase of a SQLServer private cloud project leading to DBaaS, meaning that optimizing the footprint and performance of the fabric is generally an important topic. The self-service interface (portal and APIs) sits as the top layer to receive requests and let users know about the status and progress of those requests, and possibly interact with their delegated resources A management and automation layer to integrate the self-service interface with the fabric. The management layer may also provide added benefits like in-depth application-level monitoring, inventory, patching, and backup/restore capabilities.This document will focus on how the Microsoft stack addresses these different layers in the context of DBaaS, leveraging Microsoft WindowsAzurePack (WAP) as the main portal, and it will also mention other options as well.Finally, while DBaaS is a key benefit of the Microsoft Private Cloud stack in SQLServer environments, there are other reasons why WindowsServer and MicrosoftSystemCenter are a great platform to virtualize and manage SQLServer. Some of these reasons are detailed in the last section of this document. Delivering DBaaS scenarios through System Center and the Windows Azure PackIn conjunction with WindowsServer and SystemCenter, Windows Azure Pack[footnoteRef:1] (WAP) enables enterprises and service providers to deliver a subset of MicrosoftAzure services in their datacenters. WAP provides consistency with Azure through the user experience in the portal, and also aims at providing consistency through the APIs. Both the portal and the APIs ship as part of WAP. [1: http://bit.ly/1JGfr94] WAP provides several services out of the box, including web sites, VM Clouds, and SQLServer, among others. These services are surfaced by WAP through the corresponding resource providers.WAP admin portalWAP tenant portalMore specifically in the context of this discussion:1. The VM Clouds resource provider enables tenants to deploy and manage virtual machines, which could host SQL Server, on top of a Hyper-V fabric managed by the Virtual Machine Manager (VMM) component of System Center. This is typically what most people would call Infrastructure as a Service (IaaS), and it maps to the dedicated scenarios.2. The SQL Server resource provider (described here[footnoteRef:2] and in this session from Ignite 2015[footnoteRef:3]) enables tenants to deploy and manage databases on top of a SQLServer fabric. This is sometimes referred to as a Platform as a Service (PaaS) approach, and it maps to the shared scenarios. [2: http://bit.ly/1GMIlk1] [3: http://channel9.msdn.com/Events/Ignite/2015/BRK3501] With these two resources providers and in the context of WAP, the three layers mentioned in the introduction look like the figure below. The figure is simplified, because VM Clouds are also managed by System Center.Microsoft Private Cloud stackWhether the VMs or databases are highly available or not depends on how the underlying fabric was designed. For example, the SQLServer resource provider can work with SQLServer hosts using AlwaysOn configurations.In the next section of this document, we will look more closely at each provider to understand their capabilities out of the box, and what additional content is available to help you realize the DBaaS potential.Note: Both resource providers can be used in plans surfaced to tenants. Whether you would use one or both of them depends on your requirements, and the resources at your disposal. It is also possible to mix and match, that is, to have some plans offering virtual machines and databases, and other plans only offering one of these. For more details on this topic, see the section titled Do I need to use both the VM Clouds and the SQL Server resource providers? in the FAQ section of this document.Preparing the VM Clouds fabricThe VM Cloud fabric consists of clouds leveraging compute, network and storage on Hyper-V hosts, on which tenants will be given the ability to deploy virtual machines hosting SQLServer.The fabric for virtual machines is made up of clouds, as defined and managed in System Center Virtual Machine Manager (VMM). Once the clouds are defined in VMM, and VMM is connected to WAP through Service Provider Foundation (SPF), the clouds show up in the WAP admin portal:Notice how the actual clouds are not detailed on the previous screenshot; they are defined and configured in VMM, and just reused by WAP. The actual architecture used under the hood (clustering or not, using SAN arrays, Storage Spaces, etc.) is up to the administrator when defining each cloud. In an actual implementation, we could expect the Gold cloud to have more features, higher availability or performance. The way that you craft your VM Clouds is really up to you and your requirements.The clouds will then be used when creating plans, as explained later on in this document.Templates for virtual machines with SQL ServerAn important aspect of the VM Clouds resource provider is the Gallery tab, where you can view and import currently installed VM role gallery items:These are templates which include deployment of the operating system and some additional applications.You can find sample VM Role gallery items using Web Platform Installer (WebPI) as explained in Shawn Gibbs post here[footnoteRef:4], and a more general reference for resources related to VM Role Gallery Items has been posted here[footnoteRef:5] by Charles Joy. [4: http://bit.ly/1GMIlk1] [5: http://bit.ly/1KBb8tj] We have released VM Role Gallery Items for SQL Server, as detailed in these blogs posts here[footnoteRef:6], and here[footnoteRef:7]. [6: http://bit.ly/1hZURCS] [7: http://bit.ly/1CA33Ao] More specifically: A SQL Server 2012 SP1 advanced template (leveraging an unattended installation) A SQL Server 2014 advanced template (leveraging an unattended installation) A SQL Server 2012 SP1 sysprepped template (this is an update to the VM Role gallery item initially published by the VMM team on Web Platform Installer)You can use these as foundation, to help you get started. You can also leverage the VM Role Example kit[footnoteRef:8] to start your own. [8: http://bit.ly/1KBbgc2] Preparing the SQL Servers fabricThe SQLServer fabric consists of groups of physical or virtual SQL Servers, on which tenants will be given the ability to deploy databases. Those servers can be physical or virtual, and require[footnoteRef:9] SQLServer2008SP3, SQLServer2008R2SP2, SQLServer2012SP1, or SQLServer2014. [9: http://bit.ly/1c4dLZj] Here is an example with a small environment with five SQLServer servers, and you can see how they were assigned to groups:The groups can be organized as you like, for example per site, by usage (production, development), or even by business units this can be used to ensure that tenants only use specific hosts when creating databases, since the plan they subscribe to will be attached to specific SQLServer groups (more on that later).Just like with VM Clouds, the way SQLServer is configured on these servers and groups is up to the administrator. This means some of the servers and groups may be using AlwaysOn[footnoteRef:10] capabilities or different storage architectures (and costs). This also highlights the capability to tier your SQLServer groups, just like you can do it for VM Clouds. You can view this in a similar manner to what Microsoft Azure provides for SQL Azure Database Basic, Standard and Premium offerings[footnoteRef:11]. More details on performance-related questions can also be found in the FAQ section, including support for Resource Governor on SQLServer2014 introduced in Update Rollup 5. For more information, see When using the shared model with the SQL Server Resource Provider, how can I ensure my tenants get their fair share of performance? and Do I need to use both the VM Clouds and the SQL Server resource providers? in the FAQ section of this document. [10: http://bit.ly/1JftT4n] [11: http://bit.ly/1hs3XTv] The SQL Servers view in the WAP administrator portal with Update Rollup 5 installed. Notice the new tab for Resource Pool Templates and the new Resource Pools column, both related to SQLServer Resource Governor support.In the WAP administrator portal, adding a SQLServer instance to the SQLServer fabric can be achieved like this:Note that the SQLServer has to be preprovisioned (although you could automate that too, as we will be discussing later). Another prerequisite is that the SQLServer instance must have SQLServer authentication enabled, because it is required to register the server with the SQLServer resource provider. While this requirement may sound surprising for some of our enterprise readers, remember that WAP is also often used in service providers scenarios, where the SQLServer fabric servers are likely to be in a hoster domain, fully decoupled from the tenants domains.Update: Update Rollup (UR) 6[footnoteRef:12] also adds the capability in the user interface to edit the properties of a hosting server. Only the capacity can be edited once databases have been created. [12: http://bit.ly/1OVb6Cr] This capability was already available via API in Update Rolup (UR) 5.Enabling DBaaS for tenants, through plans and subscriptionsAfter we have the fabric set up, we need to create plans that tenants can subscribe to. At this stage, we only need to provide a name for the plan and choose the services it will contain:By default, the plan is created as private so that nobody can subscribe to it until it is made public, and its services are not configured yet:We can then configure how each service is going to be used in our DBaaS plan:1. VM Clouds: We need to pick which cloud is going to be leveraged by this planIt is possible to set optional quotasWe define to which networks the VM can be connected these can be directly mapped to the actual underlying network or be virtualized (fully isolated or routed through a gateway).An important step is to define the gallery items that can be used by this plan. In this scenario, the gallery items chosen are related to SQLServer.Finally, this is where you can define which actions will be available to tenants for their virtual machines.2. SQLServersWe need to add SQLServer groups to our plan, also with limits about what can be used in these groups. Note: We will get back to the Max Additional Size Per Database If Add-Ons Acquired (MB) option shortly.Also note that this screen is a bit different after installing Update Rollup (UR) 3 and 5, as UR3 adds a Database Windows Authentication option, and UR5 adds the Resource Pool Template option if the select group supports SQLServer Resource Governor. Both these additions are discussed later in this document.In this example, only one group was added to this plan, but adding multiple groups is possibleFinally, we can optionally configure add-ons that tenants can subscribe to, to augment the services they have in a plan. Add-on subscriptions are tracked and can be charged for.In this scenario, I created an add-on called SQL Server Quota Upgrade and configured it for the Production group, with these settings:This add-on can be linked to the plan, using the Link Add-On option from the plans dashboard page:Once all of this is done, we just have to make our plan public, so that tenants can see it and subscribe:Looking at the tenant experiencePlan subscriptionLets take the example of a tenant who has no subscription yet:This tenant requests a new subscription, and selects the Database as a Service plan:Once this is done, the page will refresh and the user now has options on the left pane to create virtual machines and virtual networks (because the plan included the VM Clouds resource provider) and databases (because the plan included the SQLServer resource provider):Creating a virtual machine with SQL Server installedNote These screenshots will be related to deploying the SQL Server 2014 Advanced VM role, which deploys an unattended installation of SQLServer2014, with additional optional settings. Different VM roles including the ones you may want to create or customize may have different configuration settings.The wizard to create a new VM RolePicking the VM Role gallery item to deployFirst, we need to enter the VM nameThen we enter settings like the VM size, local administrator password, network and domain to join.Entering SQL Server specific dataNow we can see the virtual machine being provisioned.Creating a databaseThe wizard to create a new databaseSpecifying the database name:Specifying a user name Note that Update Rollup (UR) 3 had changed two behaviors regarding this screen: the ability to use Windows Authentication for the database credentials, and the rights assigned to the chosen user this is detailed later in this document.When the database has been provisioned, the tenant has access to several actions.One of the things the tenant can view is the connection string information. This will be used when working with the database.Actual network routing from the tenant location (possibly external in a service provider scenario) to the SQLServer shared fabric has to be implemented by the fabric administrator.Other possible actions for the tenant are to reset the user password for that database, and resize the database if current quota allows for it.Update: In July 2014, Update Rollup (UR) 3[footnoteRef:13] has changed two behaviors related to database creation by tenants: [13: http://bit.ly/1KBdND6 ] Database Credentials user: Prior to UR3, you had to specific a SQL Server username and password, that would be created as a new login and assigned rights (see next item). Starting with UR3, tenants can also leverage Windows Authentication, if the plan has been configured to allow for it see the Database Windows Authentication option at the bottom of the screenshot belowThe option to enable Windows Authentication when adding a SQL Server group to a planWhen Windows Authentication is enabled, a tenant can then choose between both authentication methods when adding the administrator for the new database. Entering a SQL user works in a similar manner.In the same Create Database wizard, entering a Windows user or group requires us to specify the corresponding name. Rights assigned to tenants through the Database Credential user, and type of database created: Prior to UR3, the SQL Server login specified was assigned db_owner (dbo) rights to the new database (and to that database only). Starting with UR3, with SQL Server 2012 or later, and with Contained Database Authentication enabled, WAP will not give tenants dbo rights on databases created after the update is applied, and the database will be created as contained. This change was done for security reasons, to restrict the ability to perform and restore DB backups and change database size, or remove the ability for tenants to delete their databases outside of WAP. Migration scripts [footnoteRef:14]for existing tenant databases are also provided. [14: http://bit.ly/1JGYRWn ] On a side note, starting with UR3, WAP also creates its management databases as contained, which enables support for SQL Server AlwaysOn for Disaster Recovery of these databases.Note If you require isolation at the machine level, you can have a look at the sample approach described in the FAQ What if I like the shared approach, but want to provide more dedicated servers to my tenants?Lets spend some time on database resize and quota:By default, the database is created as dynamically expanding, with the size specified in the plan configuration. Location was automatically determined based on current usage in this specific SQL Server group, and can be seen in the connection string (see the screenshot before the previous one).A tenant cannot resize the database if quota has been reached already:Now, in our scenario, if the tenant actually goes and subscribe to the SQL Server Quota Upgrade add-on, he/she would now get the option to resize the database:The menu shows how to request a new add-onIn this example, there is only one add-on availableThanks to the additional quota provided by the add-on, the user now has the ability to increase the database size.The change is reflected in the databases view, in the tenant portalSome requirements that make Windows Azure Pack a good fit for DBaaSTwo requirements that would make WAP a good approach for Database as a Service include: Consistency with Microsoft Azure (consistency of user interface, APIs, approaches). Built-in multitenancy; subscriptions enforce the multi-tenancy aspects, which can also be seen with the different resources (potential use of network virtualization with tenants, database delegation, etc.).These two items enable WAP as a great approach for Database as a Service, because an important part of the features can be achieved out of the box, with the ability to focus on customizing value-added scenarios using the built-in APIs on top of that platform. For example, actions executed in the context of the admin and the tenant portals can be automated by calling the associated REST APIs or PowerShell (see the FAQ below). It is also possible to create your own admin or tenant extensions, or a custom resource provider. We also see some service providers leverage the APIs and build their own portal.3Frequently Asked Questions about Database as a Service in the context of Windows Azure PackDo I need to use both the VM Clouds and the SQL Server resource providers?Well, here is another way to ask the question: Do you need both the shared and dedicated scenarios?Out of the box, the SQLServer resource provider covers the shared scenarios. Given that the resource provider was designed from the start as a way to manage SQLServer databases, it has that Database Friendly Platform as a Service approach. Its often used in conjunction with the Web Sites Resource Provider, which it can interact with when a Web Site also requires a database.So what about dedicated scenarios?You might be tempted to think you need dedicated scenarios for performance. For example, weve heard many times the question about customizing VM roles to separate logs and databases on separate disks, per SQLServer best practices. Interestingly enough, while this is doable (VM roles can attach multiple disks), the shared scenarios may actually be even more compatible with delivery of high performance and highly available databases, since you can design your SQLServer fabric with that performance goal in mind, and slice it in such a way that tenants can subscribe to different levels of services and performance. The plans would tie to SQLServer groups with different characteristics. WAP does not require a specific way to design the shared SQLServer infrastructure (except by requiring SQLServer authentication to be enabled), and you can rely on the existing SQL Server best practices. With that in mind, it might eventually be easier to provide dedicated performance through the shared scenarios, rather than over-complicating the dedicated scenarios.You might also be tempted to associate dedicated scenarios with security. In reality, when a database is provisioned by the SQLServer Resource Provider, only the user specified for that database is added as a database owner, so there is a level of isolation from a security and delegation standpoint, even if its not at the operating system level as it would be when deploying a full virtual machine with SQLServer. [Plus, should you or your tenants be interested in using the shared approach AND in dedicating SQL Servers to specific tenants, the next question in this FAQ explains a way to do that.]This is where its important to determine what you need most in the dedicated scenarios. For example, here are some reasons to leverage VM Clouds for Database as a Service, which may or may not apply to your specific requirements: The VM Clouds Resource Provider has the benefit of allowing more flexibility in what you provide to users. Through VM Roles, you could also provide Reporting Services, other SQLServer-related components that are not supported today by the SQLServer Resource Provider, or other types of database software. If you are mainly delivering Infrastructure as a Service in your private cloud, leveraging the VM Clouds Resource Provider for DBaaS might also provide consistency across the board for management, disaster recovery, etc.Note Being more of an Infrastructure as a Service offering, the VM Clouds Resource Provider will by default require your users to enter more non-database-related data, like the domain to join, etc. Depending how you set up multitenancy, you could lock down some of these settings in the VM role gallery items templates, hide them in provisioning scripts provided to tenants, or even customize the end user experience through custom tenant and admin extensions for the WAP portal.What if I like the shared approach, but want to provide more dedicated servers to my tenants?Out of the box when working with the SQL Server Resource Provider, and in the spirit of a Platform as a Service approach, tenants do not control which server the database is created on. They are just guaranteed that only the user they choose is created and has access to this database, providing the associated security and isolation. They can determine the target server by looking at the connection string that is provided after the database is created, but WAP handles the placement and quota allocation as needed, based on the plans the tenant has signed up for.Some enterprises and service providers have been enquiring about the ability to dedicate a SQL Server from the shared fabric to a specific tenant (because - especially with service providers - this is a service some customers may be willing to pay for).A first answer is to leverage the Infrastructure as a Service (IaaS) features also supported by WAP, and this especially makes sense if these organizations are already planning to leverage IaaS. A tenant can be provided the ability to deploy gallery items (virtual machines possibly deploying SQL Server at the same time, through the VM Clouds resource provider).Another answer is also provided in this blog post[footnoteRef:15], for those enterprises and service providers who are looking to achieve this within the realm of the SQL Server resource provider. This explains how, through automation, a tenant can request their own dedicated and scalable capacity within the SQL Server Resource Provider shared fabric. The associated sample PowerShell script is provided. [15: http://bit.ly/1F2qkkc ] How is the location of new databases determined in the shared model?This works differently depending on whether the database will eventually be using a resource governor-enabled server or not. Non-resource governor enabled databases will fill a server before allocating databases on another server, and databases may be spread across servers in the order the create requests are received regardless of subscriptions. Resource-governor enabled databases spread evenly on available servers, all databases belonging to a subscription will be assigned to the same server, and the server with most space will be chosen when the first database for the subscription is created.How can I can provide additional value-added services for my tenants?An example of a value-added service is the ability for users to backup and restore their database-enabled resources, whether they are virtual machines or databases. This is where automation can play a role, as it is possible to provide a script running in the administrator context and providing backup and restore at the Hyper-V fabric level, or at the SQLServer fabric level.There would then be multiple choices to surface the corresponding action to the tenants. For example, when using Service Management Automation (SMA) or another automation engine, you could surface automation triggers through the user interface of your choice. Currently, SMA does not provide an integration in the tenant portal out of the box, but WAP offers extension capabilities and you could create your own custom resource provider to surface these different value added actions.Torsten Kuhn has blogged[footnoteRef:16] about how to take the Hello World sample resource provider from the Windows Azure Pack SDK and extend it to call a SMA Runbook. [16: http://bit.ly/1kaxdlR ] Can I pre-provision resources for my tenants when they subscribe to my plans?Consider this scenario: when your tenants subscribe to a DBaaS plan, you may want to pre-provision some databases for them, or pre-provision a domain controller in their own virtualized and isolated network to be used with their own application later.Charles Joy has published some great samples on how to provision resources on a behalf of a tenant using the WAP API and automation.You can see the introductory post to the full series here[footnoteRef:17]. In particular, part 4[footnoteRef:18] includes samples to deploy virtual machines, and part 5[footnoteRef:19] includes a sample to deploy a database. These methods can be used in the context of Database as a Service, or with WAP in general. [17: http://bit.ly/1dIGZxh ] [18: http://bit.ly/1OUXJC8] [19: http://bit.ly/1F2rwUY] Using Windows Azure Pack, can I also enable Database as a Service for other types of database software?For shared deployments, WAP also includes a MySQL Resource Provider out of the box, and it can be added in the same plan as the SQLServer and the VM Clouds resource providers. It is also be possible to create your own resource provider, as Torsten explained in this blog post[footnoteRef:20]. This approach can be used to create a resource provider for another type of database software. [20: http://bit.ly/1GVrtI4 ] For dedicated deployments, there are sample templates available for virtual machines running MySQL and Oracle DB 12c.A VM Role Gallery Items for Oracle DB 12c sample is available here[footnoteRef:21], which includes the following capabilities : [21: http://bit.ly/1hZURCS ] Deploying Database Software, and specifying the Oracle Home User Deploying a database. Creating a schema, with a specific name and the associated userMy processes are largely based on ITIL today. How can I combine ITIL with Database as a Service?This is something we get asked a lot, especially in the enterprise space where there is often an IT Service Management (ITSM) service catalog already used to manage requests and approvals. It is definitely possible to approve requests for new databases or virtual machines in the ITSM solution, and trigger WAP API calls to deploy the resources on behalf of the user. Now, as organizations try to embrace the IT as a Service mindset, a balance has to be found so that traceability and approvals required by ITIL processes do not offset the agility and economics of the cloud.To help frame this discussion, the last part of this blog post[footnoteRef:22], looks at a fictitious example where approvals would happen at the plan level, leveraging the WAP APIs. A similar approach could be used for other usages of WAP, not just Database as a Service. [22: http://bit.ly/1F2rwUY] Can I use the APIs to interact with the databases, as an administrator or a tenant?Yes, this blog post[footnoteRef:23] includes some sample scripts, and the reference SDK can be found here[footnoteRef:24]. [23: http://bit.ly/1F2rwUY] [24: http://bit.ly/1KHNEm7 ] Update: Update Rollup (UR) 6[footnoteRef:25] also adds new API calls and a PowerShell cmdlet (Get-MgmtSvcSqlHostingServerConsistency) to detect potential inconsistencies between what the SQL Server Resource Provider expects, and what is currently hosted on the SQL Server fabric (databases, resource pools, workloads, groups). [25: http://bit.ly/1OVb6Cr] Sample usage : $report = Get-MgmtSvcSqlHostingServerConsistency -HostingServerId "wdp90u" -IncludeInSyncResources -AdminUri "https://wapserver:30004" -Token $Token DisableCertificateValidation(Getting the token is explained in the previous blog post with API samples).How can I ensure that my tenants get their fair share of performance when using the shared model with the SQL Server resource provider?Shared performance is related to how you prepare the shared SQLServer infrastructure. Here are some examples: Within an instance, you can use SQLServer resource governor to allocate resource equally. When an additional server needs to be added to the SQLServer group, you could run Transact-SQL queries through SQLPS PowerShell module to configure the resource governor in a consistent manner with other servers in the group, or if you need to update its settings across a group of serversUpdate: With the release of Update Rollup (UR) 5[footnoteRef:26] in February 2015, support for resource governor in SQL Server 2014 has been added natively to SQLServer Resource Groups in WAP. Resource Governor support can be added to Resource Groups, and Resource Pools Templates can be created and assigned to plans offering database as a service. More details can be found in Rupi Sureshkumars blog post here[footnoteRef:27]. [26: http://bit.ly/1ESCXfW] [27: http://bit.ly/1FLwtjJ ] With multiple instances on the same server, your options for fair share with a resource governor may require using resource limits such as cap_cpu, and leveraging the default pool of each instance so that all workloads in the instance have to honor them. This is because Resource Governor is mostly designed to manage the resources inside a single SQLServer instance.In this context, using single-instance SQLServers may be the most straightforward way to leverage the out of the box support for Resource Governor post-UR5.The SQLServer fabric servers can be physical or virtual. When virtualized, it is also possible to use Hyper-V Quality of Service (QoS) features to ensure fair share of the different virtual machines in the SQLServer fabric. Note that Hyper-V supports guest clustering, and even shared virtual hard disks, so going the virtual route does not prevent you from using clusters.More generally, since we are working at the SQLServer fabric infrastructure level, this is where monitoring might also be key. By using a monitoring solutions like Operations Manager as part of the System Center suite, you can monitor how your SQLServer fabric is doing and, when getting closer to potentially breaching SLAs, you could add servers to the shared infrastructure to distribute the load. This could even be automated.Note If you want to go as far as dedicating a group of servers to specific tenants, you could also look at the solution described at the question What if I like the shared approach, but want to provide more dedicated servers to my tenants?When using IaaS to deploy SQLServer virtual machines, could my tenants also deploy highly-available virtual machines?While VM roles do not support shared virtual hard disks today, it is possible to deploy a SQLServer AlwaysOn configuration supported by a file share witness. We have published a sample set of templates and runbooks[footnoteRef:28] handling this scenario in the context of the Cloud Platform System[footnoteRef:29] (CPS), but these samples could also be used outside of CPS. [28: http://bit.ly/1dIQFaU] [29: http://bit.ly/1umvRtw] Can I get data for potential chargeback with my tenants?There is a billing interface to access usage data collected by a resource provider. Both the VM Clouds and SQLServer resource providers include built-in mechanisms to query and report on usage data for showback and chargeback.Out of the box, the Service Reporting[footnoteRef:30] feature consumes the data from the VM Clouds resource provider, and you can then use standard analysis tools like Excel or Performance Point. And you could have your own query mechanisms to access Database related data. [30: http://bit.ly/1DPQoba] Alternatively, you can also leverage solutions like Cloud Cruiser[footnoteRef:31], should you require more reports out of the box, or working with other types of clouds. Microsoft now includes a customized Express version of Cloud Cruisers cloud financial management solution with WindowsServer2012R2 via WindowsAzurePack (WAP). This version also consumes the usage data collected by the SQLServer resource provider. [31: http://bit.ly/1ceN1Fv] Other options with System CenterIt is also possible to create your own Database as a Service architecture by leveraging System Center. While this is more of a Do It Yourself (DIY) approach, the stack includes all the features needed to provide self-service, automation, and provisioning of database components. If this is an approach that makes sense in your environment, the SQL Server Self Service Kit[footnoteRef:32] includes sample solutions to help you get started. [32: http://bit.ly/1QgoJJG] In this sample solution, the Service Manager integration is optional. ITSM requests are monitored by the sample Orchestrator runbooks, and those runbooks could monitor another ITSM solution.This sample solution has also been ported to use Service Management Automation (SMA) as the automation engine instead of Orchestrator, depending on your requirements: SQL Server Self-Service Kit SMA Edition[footnoteRef:33] (*). [33: http://bit.ly/1ESE2En ] And, should you want to enable DBaaS for other types of database software like Oracle DB 12c, we also provide a sample Oracle Self-Service Kit[footnoteRef:34] [34: http://bit.ly/1IbIKzN ] Compared to WAP, this alternative approach using System Center components and automation may be a good fit if your requirements include: Leveraging existing components or skills in your environment. For example, the Orchestrator instance used here can certainly be used to automate other actions, and is not limited to Database as a Service scenarios Interoperating with complex business processes, especially from an ITIL standpoint. We saw previously that there are ways to balance cloud concepts with ITIL in the context of WAP, but that usually requires at least a few changes in the approach, to avoid offsetting the benefits of the cloud. If you have very limited options to change existing ITIL processes, this second approach may make more sense until the service provider mindset gets more adopted by your organizationIts also important to note the potential limitations of this approach, compared to WAP: Your runbooks will have to manage the full lifecycle of the resources. For example, the SQL Server Self Service Kit sample solution does not include resizing or decommissioning of databases, so you will need to expand it for that task. On the flipside, the resulting solution is really crafted exactly to your needs. Multitenancy is not built-into the sample solutions: The sample runbooks need to touch the provisioned virtual machines, for example to provision instances and databases. This means they apply to environment without multitenancy, or with a single tenant, or where tenant networks are routed. This may or may not be a limiting issue, depending on what you are trying to achieve.(*) A note about the SQL Server Self-Service Kit SMA EditionThe main benefit of the SMA Edition version was to showcase the same scenarios as the original SQL Server Self-Service Kit, this time using a full PowerShell-based automation engine (SMA). This was mainly done to provide a learning opportunity for users comparing the runbooks in both automation engines, and to see how they were ported to SMA. Its important to understand that the same limitations mentioned before still apply, since the sample runbooks directly update the target VMs and do not leverage the WAP admin APIs. The SMA Edition also does not include runbooks monitoring Service Manager ITSM requests.Looking beyond Database as a Service: Why leverage Microsoft technologies to virtualize and manage SQL ServerThis document has focused on the ability to enable Database as a Service (DBaaS). While DBaaS is certainly a key benefit of the Microsoft Private Cloud stack, there are other reasons why Windows Server and System Center are a great platform to virtualize and manage SQLServer, including (but not limited to) the following:1. Scale and performance of Hyper-V, from an architecture and guest VM standpoint: 64 virtual processors and 1 TB RAM guest virtual machines, 1 million IOPS from a single VM[footnoteRef:35], 64-node Hyper-V clusters, NUMA support, and low overhead vs a physical SQL Server computer. You can read more about it here[footnoteRef:36]. [35: http://bit.ly/1zIPkuR ] [36: http://bit.ly/1KahAGI] 2. High availability at the host and application level: Using Virtual HBA and/or the Shared Virtual Hard Disks (VHDX) feature, it is possible to combine in-guest clustering with host clustering. Plus features like Cluster Aware Updating also provide easier patching operations.3. Flexible and cheaper architecture and storage options: Hyper-V provides support for existing investments in storage arrays, such as Offloaded Data Transfer (ODX) support, thin/trim provisioning, and other flexible and cost effective options relying on file shares, while still maintaining performance and transparent failover when used with clustering. Storage Spaces also includes tiering now, and Hyper-V supports online resize of VHDs4. VM mobility, backup and disaster recovery options: Technologies at play here include Hyper-V Replica, Shared Nothing Live Migration, and Network Virtualization. As part of System Center, Data Protection Manager also includes SQLServer-specific workload support.5. Application level monitoring and compliance: The SQLServer Management Pack for Operations Manager includes in-depth monitoring scenarios for SQLServer components, whether they are hosted on physical or virtual machines. Performance, availability and even configuration data including policy based management are surfaced in monitoring views, dashboards, and reports.6. Automation and advanced patching: Automation enables scenarios like SQLServer maintenance tasks, advanced sequenced patching, etc.7. Hybrid scenarios with Microsoft Azure, either at the networking layer (for example with SQLServer databases on premise and web tiers in Microsoft Azure, leveraging Virtual Networks) or the application layer (including options added in recent versions of SQLServer). And you can still get a single pane of glass for monitoring, leveraging on-premises, SQLAzure and Microsoft Azure management packs8. Finally, licensing costs could also come into play as a benefit too, when virtualizing with Hyper-V and managing with System CenterReferencesThis section provides links for additional information about topics discussed in this paper. Building Clouds Blog (http://aka.ms/bcb) Windows Azure Pack overview (http://bit.ly/1JGfr94) Overview of the SQL Server Resource Provider for Windows Azure Pack (http://bit.ly/1GMIlk1) Ignite 2015 session : Provisioning SQL Database-as-a-Service in the Azure Pack (http://channel9.msdn.com/Events/Ignite/2015/BRK3501) Service Models feed and the Web Platform Installer (WebPI) (http://bit.ly/1DIDqMy) References for Creation, Configuration, and Automation of VM Role Gallery Items (http://bit.ly/1KBb8tj) Sample VM Role Gallery Items for SQL Server (http://bit.ly/1hZURCS) Update to the Sample VM Role Gallery Items for SQL Server (http://bit.ly/1CA33Ao) VM Role Example kit (http://bit.ly/1KBbgc2) SQL Server Requirements for Windows Azure Pack (http://bit.ly/1c4dLZj) Configure SQL AlwaysOn Availability Groups in Windows Azure Pack (http://bit.ly/1JftT4n) SQL Azure Database Basic, Standard and Premium offerings (http://bit.ly/1JftT4n) Update Rollup (UR) 6 for Windows Azure Pack (http://bit.ly/1OVb6Cr) Update Rollup (UR) 3 for Windows Azure Pack (http://bit.ly/1KBdND6) Migration scripts for UR3 (http://bit.ly/1JGYRWn) Dedicating a part of the SQL Server fabric to a specific tenant (http://bit.ly/1F2qkkc) Custom Resource Providers in Windows Azure Pack - Extending the Hello World Sample calling a SMA Runbook (http://bit.ly/1kaxdlR) AutomationThe New World of Tenant Provisioning with Windows Azure Pack (Blog post series) (http://bit.ly/1dIGZxh) Part 4 (http://bit.ly/1OUXJC8) shows how to deploy Virtual Machines on behalf of a tenant Part 5 (http://bit.ly/1F2rwUY) shows how to deploy SQL Server databases on behalf of a tenant, and discusses how to interface with ITIL processes Custom Resource Providers in Windows Azure Pack Moving from Hello World to your own Resource Provider (http://bit.ly/1GVrtI4) VM Role Gallery Items for Oracle DB 12c (http://bit.ly/1hZURCS) Combining ITIL with Database as a Service (http://bit.ly/1F2rwUY) Reference API for the SQL Server Resource Provider (http://bit.ly/1KHNEm7) Update Rollup (UR) 5 for Windows Azure Pack (http://bit.ly/1ESCXfW) Manage Tenant Database Workloads with Resource Governor in Azure Pack (http://bit.ly/1FLwtjJ) Sample set of templates and runbooks for the Cloud Platform System (CPS) (http://bit.ly/1dIQFaU) Cloud Platform System (CPS) Overview (http://bit.ly/1umvRtw) Chargeback: Service Reporting feature overview (http://bit.ly/1DPQoba) Chargeback: Partner Cloud Cruiser (http://bit.ly/1ceN1Fv) SQL Server Self Service Kit (http://bit.ly/1QgoJJG) SQL Server Self-Service Kit SMA Edition (http://bit.ly/1ESE2En) Oracle Self-Service Kit (http://bit.ly/1IbIKzN) 1 million IOPS from a single VM (http://bit.ly/1zIPkuR) at TechEd Europe 2012 Windows Server 2012 Hyper-V Delivers On Scalability and Performance for Virtualized Enterprise Applications (http://bit.ly/1KahAGI) Document Revision HistoryDate PublishedVersionCommentsMay 20151.1 Updated the document to cover updates in Update Rollups 3, 5 and 6 for Windows Azure Pack Added a more explicit FAQ entry for sample scripts for the SQL Server Resource Provider Added a FAQ entry to the CPS sample templates for SQL Server AlwaysOn Updated formattingMay 20141.0 First published version


View more >