Enabling Database as a Service with the Microsoft Private Cloud stackA Building Clouds Blog eBook | Enterprise Cloud Group CAT Team

Author: Bruno Saille (Microsoft)

Published: May 2015, Version 1.1

Abstract

This paper is for IT Professionals and Database Administrators who would like to enable “Database as a Service” (DBaaS) for their end users. Providing DBaaS to applications owners or developers is a topic that comes up quite often these days in our interactions with customers. The Microsoft Private Cloud stack offers several ways to provide DBaaS. This document summarizes the options, and which requirements may lead you to prefer one option compared to the others. While this will mainly be in the context of Microsoft SQL Server, we will also cover options to achieve DBaaS for other types of database software.

This paper applies to Windows Server 2012 R2, Microsoft System Center 2012 R2, and Microsoft Windows Azure Pack for Windows Server.

© 2015 Microsoft Corporation. All rights reserved. This document is provided "as-is." Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it.

Some examples are for illustration only and are fictitious. No real association is intended or inferred.

This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. You may modify this document for your internal, reference purposes.

Some information relates to pre-released product which may be substantially modified before it’s commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

Table of ContentsIntroduction: First, let’s define “Database as a Service”..............................................................................4

Delivering DBaaS scenarios through System Center and the Windows Azure Pack....................................5

Preparing the VM Clouds fabric...............................................................................................................7

Templates for virtual machines with SQL Server.................................................................................9

Preparing the SQL Servers fabric...........................................................................................................10

Enabling DBaaS for tenants, through plans and subscriptions...............................................................12

Looking at the tenant experience..........................................................................................................18

Plan subscription...............................................................................................................................18

Creating a virtual machine with SQL Server installed........................................................................20

Creating a database...........................................................................................................................24

Some requirements that make Windows Azure Pack a good fit for DBaaS...............................................31

Frequently Asked Questions about Database as a Service in the context of Windows Azure Pack...........32

Do I need to use both the VM Clouds and the SQL Server resource providers?....................................32

What if I like the shared approach, but want to provide more dedicated servers to my tenants?.......33

How is the location of new databases determined in the shared model?.............................................34

How can I can provide additional value-added services for my tenants?..............................................34

Can I pre-provision resources for my tenants when they subscribe to my plans?.................................35

Using Windows Azure Pack, can I also enable Database as a Service for other types of database software?...............................................................................................................................................35

My processes are largely based on ITIL today. How can I combine ITIL with Database as a Service?....36

Can I use the APIs to interact with the databases, as an administrator or a tenant?............................36

How can I ensure that my tenants get their fair share of performance when using the shared model with the SQL Server resource provider?................................................................................................36

When using IaaS to deploy SQL Server virtual machines, could my tenants also deploy highly-available virtual machines?..................................................................................................................................37

Can I get data for potential chargeback with my tenants?....................................................................38

Other options with System Center............................................................................................................38

Looking beyond Database as a Service: Why leverage Microsoft technologies to virtualize and manage SQL Server.................................................................................................................................................40

References.................................................................................................................................................42

3

Introduction: First, let’s define “Database as a Service”Everyone has a different definition based on their requirements, but ultimately “Database as a Service” (DBaaS) means giving end users — application owners, developers, and others — the ability to request and provision database engine components in a self-service manner. For some enterprises and service providers, responding to requests for database services is an activity that takes a lot of time from DBAs on a daily basis, and automating it frees up valuable time for other value-added tasks, reduces human error, and increases consistency and confidence in operations.

Depending on the requirements, the scope of a DBaaS implementation may include coverage for all or part of the following considerations:

- What components should be deployed? Deployment of a database, an instance, a machine with Microsoft SQL Server installer, or a set of machines including SQL Server?

- Should or will deployments occur on a shared fabric only, or will we want to enable dedicated scenarios where database components are dedicated to a specific tenant?

- What about the service level and high availability aspects? Are we talking about deploying commodity databases without any high availability capabilities, or surfacing highly available database components?

- How much of the lifecycle of the database component should be covered? Is it just about the deployment and delegation of the database components, or should it cover the full lifecycle (resize, monitoring, database user management, decommissioning, etc.)?

There is no good or bad answer to these questions, it just depends on what IT and business issues you are trying to solve with your DBaaS implementation.

Whatever the end solution chosen, it will likely consist of multiple building blocks or layers like these:

- The fabric is the foundation layer providing compute, network, and storage resources for hosting the SQL Server database components. This fabric can benefit from the performance and scale capabilities from Windows Server and SQL Server. For example, increased Hyper-V guest VM scale and performance (virtual processors, memory, disk IOPS), storage flexibility and choices for both virtual machines and databases (SMB support, Storage Spaces, etc.). Resource consolidation and pooling are often the first phase of a SQL Server private cloud project leading to DBaaS, meaning that optimizing the footprint and performance of the fabric is generally an important topic.

- The self-service interface (portal and APIs) sits as the top layer to receive requests and let users know about the status and progress of those requests, and possibly interact with their delegated resources

- A management and automation layer to integrate the self-service interface with the fabric. The management layer may also provide added benefits like in-depth application-level monitoring, inventory, patching, and backup/restore capabilities.

This document will focus on how the Microsoft stack addresses these different layers in the context of DBaaS, leveraging Microsoft Windows Azure Pack (WAP) as the main portal, and it will also mention other options as well.

4

Finally, while DBaaS is a key benefit of the Microsoft Private Cloud stack in SQL Server environments, there are other reasons why Windows Server and Microsoft System Center are a great platform to virtualize and manage SQL Server. Some of these reasons are detailed in the last section of this document.

Delivering DBaaS scenarios through System Center and the Windows Azure PackIn conjunction with Windows Server and System Center, Windows Azure Pack1 (WAP) enables enterprises and service providers to deliver a subset of Microsoft Azure services in their datacenters. WAP provides consistency with Azure through the user experience in the portal, and also aims at providing consistency through the APIs. Both the portal and the APIs ship as part of WAP.

WAP provides several services out of the box, including web sites, VM Clouds, and SQL Server, among others. These services are surfaced by WAP through the corresponding resource providers.

WAP admin portal

1 http://bit.ly/1JGfr94

5

WAP tenant portal

More specifically in the context of this discussion:

1. The VM Clouds resource provider enables tenants to deploy and manage virtual machines, which could host SQL Server, on top of a Hyper-V fabric managed by the Virtual Machine Manager (VMM) component of System Center. This is typically what most people would call Infrastructure as a Service (IaaS), and it maps to the “dedicated” scenarios.

2. The SQL Server resource provider (described here2 and in this session from Ignite 20153) enables tenants to deploy and manage databases on top of a SQL Server fabric. This is sometimes referred to as a Platform as a Service (PaaS) approach, and it maps to the “shared” scenarios.

With these two resources providers and in the context of WAP, the three layers mentioned in the introduction look like the figure below. The figure is simplified, because VM Clouds are also managed by System Center.

2 http://bit.ly/1GMIlk13 http://channel9.msdn.com/Events/Ignite/2015/BRK3501

6

Microsoft Private Cloud stack

Whether the VMs or databases are highly available or not depends on how the underlying fabric was designed. For example, the SQL Server resource provider can work with SQL Server hosts using AlwaysOn configurations.

In the next section of this document, we will look more closely at each provider to understand their capabilities out of the box, and what additional content is available to help you realize the DBaaS potential.

Note: Both resource providers can be used in plans surfaced to tenants. Whether you would use one or both of them depends on your requirements, and the resources at your disposal. It is also possible to mix and match, that is, to have some plans offering virtual machines and databases, and other plans only offering one of these. For more details on this topic, see the section titled “Do I need to use both the VM Clouds and the SQL Server resource providers?” in the FAQ section of this document.

Preparing the VM Clouds fabricThe VM Cloud fabric consists of clouds leveraging compute, network and storage on Hyper-V hosts, on which tenants will be given the ability to deploy virtual machines hosting SQL Server.

7

The fabric for virtual machines is made up of clouds, as defined and managed in System Center Virtual Machine Manager (VMM). Once the clouds are defined in VMM, and VMM is connected to WAP through Service Provider Foundation (SPF), the clouds show up in the WAP admin portal:

Notice how the actual clouds are not detailed on the previous screenshot; they are defined and configured in VMM, and just reused by WAP. The actual architecture used under the hood (clustering or not, using SAN arrays, Storage Spaces, etc.) is up to the administrator when defining each cloud. In an actual implementation, we could expect the “Gold” cloud to have more features, higher availability or performance. The way that you craft your VM Clouds is really up to you and your requirements.

The clouds will then be used when creating plans, as explained later on in this document.

8

Templates for virtual machines with SQL ServerAn important aspect of the VM Clouds resource provider is the Gallery tab, where you can view and import currently installed VM role gallery items:

These are templates which include deployment of the operating system and some additional applications.

You can find sample VM Role gallery items using Web Platform Installer (WebPI) as explained in Shawn Gibbs’ post here4, and a more general reference for resources related to VM Role Gallery Items has been posted here5 by Charles Joy.

We have released VM Role Gallery Items for SQL Server, as detailed in these blogs posts here6, and here7.More specifically:- A SQL Server 2012 SP1 “advanced” template (leveraging an unattended

installation)- A SQL Server 2014 “advanced” template (leveraging an unattended

installation)- A SQL Server 2012 SP1 “sysprepped” template (this is an update to the

VM Role gallery item initially published by the VMM team on Web Platform Installer)

You can use these as foundation, to help you get started. You can also leverage the VM Role Example kit8 to start your own.

4 http://bit.ly/1GMIlk15 http://bit.ly/1KBb8tj6 http://bit.ly/1hZURCS7 http://bit.ly/1CA33Ao8 http://bit.ly/1KBbgc2

9

Preparing the SQL Servers fabricThe SQL Server fabric consists of “groups” of physical or virtual SQL Servers, on which tenants will be given the ability to deploy databases. Those servers can be physical or virtual, and require9 SQL Server 2008 SP3, SQL Server 2008 R2 SP2, SQL Server 2012 SP1, or SQL Server 2014.

Here is an example with a small environment with five SQL Server servers, and you can see how they were assigned to groups:

The groups can be organized as you like, for example per site, by usage (“production”, “development”), or even by business units – this can be used to ensure that tenants only use specific hosts when creating databases, since the plan they subscribe to will be attached to specific SQL Server groups (more on that later).

Just like with VM Clouds, the way SQL Server is configured on these servers and groups is up to the administrator. This means some of the servers and groups may be using AlwaysOn10 capabilities or different storage architectures (and costs). This also highlights the capability to “tier” your SQL Server groups, just like you can do it for VM Clouds. You can view this in a similar manner to what Microsoft Azure provides for SQL Azure Database Basic, Standard and Premium offerings11. More details on performance-related questions can also be found in the FAQ section, including support for Resource Governor on SQL Server 2014 introduced in Update Rollup 5. For more information, see “When using the shared model with the SQL Server Resource Provider, how can I ensure my tenants get their fair share of performance?” and “Do I need to use both the VM Clouds and the SQL Server resource providers?” in the FAQ section of this document.

9 http://bit.ly/1c4dLZj10 http://bit.ly/1JftT4n11http://bit.ly/1hs3XTv

10

The “SQL Servers” view in the WAP administrator portal with Update Rollup 5 installed. Notice the new tab for Resource Pool Templates and the new Resource Pools column, both related to SQL Server

Resource Governor support.

In the WAP administrator portal, adding a SQL Server instance to the SQL Server fabric can be achieved like this:

Note that the SQL Server has to be preprovisioned (although you could automate that too, as we will be discussing later). Another prerequisite is that the SQL Server instance must have SQL Server authentication enabled, because it is required to register the server with the SQL Server resource provider. While this requirement may sound surprising for some of our enterprise readers, remember that WAP is also often used in service providers’ scenarios, where the SQL Server fabric servers are likely to be in a hoster domain, fully decoupled from the tenants’ domains.

Update: Update Rollup (UR) 612 also adds the capability in the user interface to edit the properties of a hosting server. Only the capacity can be edited once databases have been created.

12 http://bit.ly/1OVb6Cr

11

This capability was already available via API in Update Rolup (UR) 5.

Enabling DBaaS for tenants, through plans and subscriptionsAfter we have the fabric set up, we need to create plans that tenants can subscribe to. At this stage, we only need to provide a name for the plan and choose the services it will contain:

12

By default, the plan is created as private so that nobody can subscribe to it until it is made public, and its services are not configured yet:

13

We can then configure how each service is going to be used in our DBaaS plan:

1. VM Clouds:

We need to pick which cloud is going to be leveraged by this plan

14

It is possible to set optional quotas

We define to which networks the VM can be connected – these can be directly mapped to the actual underlying network or be virtualized (fully isolated or routed through a gateway).

15

An important step is to define the gallery items that can be used by this plan. In this scenario, the gallery items chosen are related to SQL Server.

Finally, this is where you can define which actions will be available to tenants for their virtual machines.

2. SQL Servers

We need to add SQL Server groups to our plan, also with limits about what can be used in these groups. Note: We will get back to the Max Additional Size Per Database If Add-Ons Acquired (MB) option shortly.

Also note that this screen is a bit different after installing Update Rollup (UR) 3 and 5, as UR3 adds a Database Windows Authentication option, and UR5 adds the Resource Pool Template option if the select group supports SQL Server Resource Governor. Both these additions are discussed later in this document.

16

In this example, only one group was added to this plan, but adding multiple groups is possible

Finally, we can optionally configure add-ons that tenants can subscribe to, to augment the services they have in a plan. Add-on subscriptions are tracked and can be charged for.

In this scenario, I created an add-on called “SQL Server Quota Upgrade” and configured it for the “Production” group, with these settings:

17

This add-on can be linked to the plan, using the Link Add-On option from the plan’s dashboard page:

Once all of this is done, we just have to make our plan public, so that tenants can see it and subscribe:

Looking at the tenant experiencePlan subscriptionLet’s take the example of a tenant who has no subscription yet:

18

This tenant requests a new subscription, and selects the “Database as a Service” plan:

19

Once this is done, the page will refresh and the user now has options on the left pane to create virtual machines and virtual networks (because the plan included the VM Clouds resource provider) and databases (because the plan included the SQL Server resource provider):

Creating a virtual machine with SQL Server installedNote These screenshots will be related to deploying the “SQL Server 2014 Advanced” VM role, which deploys an unattended installation of SQL Server 2014, with additional optional settings. Different VM roles – including the ones you may want to create or customize – may have different configuration settings.

20

The wizard to create a new VM Role

Picking the VM Role gallery item to deploy

21

First, we need to enter the VM name

22

Then we enter settings like the VM size, local administrator password, network and domain to join.

23

Entering SQL Server specific data

Now we can see the virtual machine being provisioned.

Creating a database

The wizard to create a new database

24

Specifying the database name:

Specifying a user name – Note that Update Rollup (UR) 3 had changed two behaviors regarding this screen: the ability to use Windows Authentication for the database credentials, and the rights assigned

to the chosen user – this is detailed later in this document.

25

When the database has been provisioned, the tenant has access to several actions.

26

One of the things the tenant can view is the connection string information. This will be used when working with the database.

Actual network routing from the tenant location (possibly external in a service provider scenario) to the SQL Server shared fabric has to be implemented by the fabric administrator.

Other possible actions for the tenant are to reset the user password for that database, and resize the database if current quota allows for it.

Update: In July 2014, Update Rollup (UR) 313 has changed two behaviors related to database creation by tenants:

“Database Credentials” user: Prior to UR3, you had to specific a SQL Server username and password, that would be created as a new login and assigned rights (see next item). Starting with UR3, tenants can also leverage Windows Authentication, if the plan has been configured to allow for it – see the “Database Windows Authentication” option at the bottom of the screenshot below

The option to enable Windows Authentication when adding a SQL Server group to a plan

13 http://bit.ly/1KBdND6

27

When “Windows Authentication” is enabled, a tenant can then choose between both authentication methods when adding the administrator for the new database. Entering a SQL

user works in a similar manner.

In the same “Create Database” wizard, entering a Windows user or group requires us to specify the corresponding name.

Rights assigned to tenants through the “Database Credential” user, and type of database created: Prior to UR3, the SQL Server login specified was assigned db_owner (dbo) rights to

28

the new database (and to that database only). Starting with UR3, with SQL Server 2012 or later, and with Contained Database Authentication enabled, WAP will not give tenants dbo rights on databases created after the update is applied, and the database will be created as contained. This change was done for security reasons, to restrict the ability to perform and restore DB backups and change database size, or remove the ability for tenants to delete their databases outside of WAP. Migration scripts 14for existing tenant databases are also provided.

o On a side note, starting with UR3, WAP also creates its management databases as contained, which enables support for SQL Server AlwaysOn for Disaster Recovery of these databases.

Note If you require isolation at the machine level, you can have a look at the sample approach described in the FAQ “What if I like the shared approach, but want to provide more dedicated servers to my tenants?”

Let’s spend some time on database resize and quota:

By default, the database is created as dynamically expanding, with the size specified in the plan configuration. Location was automatically determined based on current usage in this specific SQL Server group, and can be seen in the connection string (see the screenshot before the previous one).

A tenant cannot resize the database if quota has been reached already:

14 http://bit.ly/1JGYRWn

29

Now, in our scenario, if the tenant actually goes and subscribe to the “SQL Server Quota Upgrade” add-on, he/she would now get the option to resize the database:

The menu shows how to request a new add-on

In this example, there is only one add-on available

Thanks to the additional quota provided by the add-on, the user now has the ability to increase the database size.

30

The change is reflected in the databases view, in the tenant portal

Some requirements that make Windows Azure Pack a good fit for DBaaSTwo requirements that would make WAP a good approach for Database as a Service include:

- Consistency with Microsoft Azure (consistency of user interface, APIs, approaches).- Built-in multitenancy; subscriptions enforce the multi-tenancy aspects, which can also be seen with

the different resources (potential use of network virtualization with tenants, database delegation, etc.).

These two items enable WAP as a great approach for Database as a Service, because an important part of the features can be achieved out of the box, with the ability to focus on customizing value-added scenarios using the built-in APIs on top of that platform. For example, actions executed in the context of the admin and the tenant portals can be automated by calling the associated REST APIs or PowerShell (see the FAQ below). It is also possible to create your own admin or tenant extensions, or a custom resource provider. We also see some service providers leverage the APIs and build their own portal.

31

Frequently Asked Questions about Database as a Service in the context of Windows Azure PackDo I need to use both the VM Clouds and the SQL Server resource providers?Well, here is another way to ask the question: Do you need both the shared and dedicated scenarios?

Out of the box, the SQL Server resource provider covers the shared scenarios. Given that the resource provider was designed from the start as a way to manage SQL Server databases, it has that “Database Friendly” Platform as a Service approach. It’s often used in conjunction with the Web Sites Resource Provider, which it can interact with when a Web Site also requires a database.

So what about dedicated scenarios?

You might be tempted to think you need dedicated scenarios for performance. For example, we’ve heard many times the question about customizing VM roles to separate logs and databases on separate disks, per SQL Server best practices. Interestingly enough, while this is doable (VM roles can attach multiple disks), the shared scenarios may actually be even more compatible with delivery of high performance and highly available databases, since you can design your SQL Server fabric with that performance goal in mind, and “slice” it in such a way that tenants can subscribe to different levels of services and performance. The plans would tie to SQL Server groups with different characteristics. WAP does not require a specific way to design the shared SQL Server infrastructure (except by requiring SQL Server authentication to be enabled), and you can rely on the existing SQL Server best practices. With that in mind, it might eventually be easier to provide dedicated performance through the shared scenarios, rather than over-complicating the dedicated scenarios.

You might also be tempted to associate dedicated scenarios with security. In reality, when a database is provisioned by the SQL Server Resource Provider, only the user specified for that database is added as a database owner, so there is a level of isolation from a security and delegation standpoint, even if it’s not at the operating system level as it would be when deploying a full virtual machine with SQL Server. [Plus, should you or your tenants be interested in using the shared approach AND in dedicating SQL Servers to specific tenants, the next question in this FAQ explains a way to do that.]

This is where it’s important to determine what you need most in the dedicated scenarios. For example, here are some reasons to leverage VM Clouds for Database as a Service, which may or may not apply to your specific requirements:

- The VM Clouds Resource Provider has the benefit of allowing more flexibility in what you provide to users. Through VM Roles, you could also provide Reporting Services, other SQL Server-related components that are not supported today by the SQL Server Resource Provider, or other types of database software.

- If you are mainly delivering Infrastructure as a Service in your private cloud, leveraging the VM Clouds Resource Provider for DBaaS might also provide consistency across the board for management, disaster recovery, etc.

32

Note Being more of an Infrastructure as a Service offering, the VM Clouds Resource Provider will by default require your users to enter more non-database-related data, like the domain to join, etc. Depending how you set up multitenancy, you could lock down some of these settings in the VM role gallery items templates, hide them in provisioning scripts provided to tenants, or even customize the end user experience through custom tenant and admin extensions for the WAP portal.

What if I like the shared approach, but want to provide more dedicated servers to my tenants?Out of the box when working with the SQL Server Resource Provider, and in the spirit of a Platform as a Service approach, tenants do not control which server the database is created on. They are just guaranteed that only the user they choose is created and has access to this database, providing the associated security and isolation. They can determine the target server by looking at the connection string that is provided after the database is created, but WAP handles the placement and quota allocation as needed, based on the plans the tenant has signed up for.

Some enterprises and service providers have been enquiring about the ability to dedicate a SQL Server from the shared fabric to a specific tenant (because - especially with service providers - this is a service some customers may be willing to pay for).

A first answer is to leverage the Infrastructure as a Service (IaaS) features also supported by WAP, and this especially makes sense if these organizations are already planning to leverage IaaS. A tenant can be provided the ability to deploy gallery items (virtual machines possibly deploying SQL Server at the same time, through the VM Clouds resource provider).

Another answer is also provided in this blog post15, for those enterprises and service providers who are looking to achieve this within the realm of the SQL Server resource provider. This explains how, through automation, a tenant can request their own dedicated and scalable capacity within the SQL Server Resource Provider shared fabric. The associated sample PowerShell script is provided.

15 http://bit.ly/1F2qkkc

33

How is the location of new databases determined in the shared model?This works differently depending on whether the database will eventually be using a resource governor-enabled server or not. Non-resource governor enabled databases will fill a server before allocating databases on another server, and databases may be spread across servers in the order the create requests are received regardless of subscriptions. Resource-governor enabled databases spread evenly on available servers, all databases belonging to a subscription will be assigned to the same server, and the server with most space will be chosen when the first database for the subscription is created.

How can I can provide additional value-added services for my tenants?An example of a value-added service is the ability for users to backup and restore their database-enabled resources, whether they are virtual machines or databases. This is where automation can play a role, as it is possible to provide a script running in the administrator context and providing backup and restore at the Hyper-V fabric level, or at the SQL Server fabric level.

There would then be multiple choices to surface the corresponding action to the tenants. For example, when using Service Management Automation (SMA) or another automation engine, you could surface automation triggers through the user interface of your choice. Currently, SMA does not provide an integration in the tenant portal out of the box, but WAP offers extension capabilities and you could create your own custom resource provider to surface these different value added actions.

Torsten Kuhn has blogged16 about how to take the “Hello World” sample resource provider from the Windows Azure Pack SDK and extend it to call a SMA Runbook.

16 http://bit.ly/1kaxdlR

34

Can I pre-provision resources for my tenants when they subscribe to my plans?Consider this scenario: when your tenants subscribe to a DBaaS plan, you may want to pre-provision some databases for them, or pre-provision a domain controller in their own virtualized and isolated network to be used with their own application later.

Charles Joy has published some great samples on how to provision resources on a behalf of a tenant using the WAP API and automation.

You can see the introductory post to the full series here17. In particular, part 418 includes samples to deploy virtual machines, and part 519 includes a sample to deploy a database. These methods can be used in the context of Database as a Service, or with WAP in general.

Using Windows Azure Pack, can I also enable Database as a Service for other types of database software?For shared deployments, WAP also includes a MySQL Resource Provider out of the box, and it can be added in the same plan as the SQL Server and the VM Clouds resource providers. It is also be possible to create your own resource provider, as Torsten explained in this blog post20. This approach can be used to create a resource provider for another type of database software.

For dedicated deployments, there are sample templates available for virtual machines running MySQL and Oracle DB 12c.

A VM Role Gallery Items for Oracle DB 12c sample is available here21, which includes the following capabilities :- Deploying Database Software, and specifying the Oracle Home User- Deploying a database. - Creating a schema, with a specific name and the associated user

My processes are largely based on ITIL today. How can I combine ITIL with Database as a Service?This is something we get asked a lot, especially in the enterprise space where there is often an IT Service Management (ITSM) “service catalog” already used to manage requests and approvals. It is definitely possible to approve requests for new databases or virtual machines in the ITSM solution, and trigger

17 http://bit.ly/1dIGZxh 18 http://bit.ly/1OUXJC819 http://bit.ly/1F2rwUY20 http://bit.ly/1GVrtI4 21 http://bit.ly/1hZURCS

35

WAP API calls to deploy the resources on behalf of the user. Now, as organizations try to embrace the “IT as a Service” mindset, a balance has to be found so that traceability and approvals required by ITIL processes do not offset the agility and economics of the cloud.

To help frame this discussion, the last part of this blog post22, looks at a fictitious example where approvals would happen at the plan level, leveraging the WAP APIs. A similar approach could be used for other usages of WAP, not just Database as a Service.

Can I use the APIs to interact with the databases, as an administrator or a tenant?Yes, this blog post23 includes some sample scripts, and the reference SDK can be found here24.

Update: Update Rollup (UR) 625 also adds new API calls and a PowerShell cmdlet (Get-MgmtSvcSqlHostingServerConsistency) to detect potential inconsistencies between what the SQL Server Resource Provider expects, and what is currently hosted on the SQL Server fabric (databases, resource pools, workloads, groups).Sample usage : $report = Get-MgmtSvcSqlHostingServerConsistency -HostingServerId "wdp90u" -IncludeInSyncResources -AdminUri "https://wapserver:30004" -Token $Token –DisableCertificateValidation

(Getting the token is explained in the previous blog post with API samples).

How can I ensure that my tenants get their fair share of performance when using the shared model with the SQL Server resource provider?Shared performance is related to how you prepare the shared SQL Server infrastructure. Here are some examples:

Within an instance, you can use SQL Server resource governor to allocate resource equally. When an additional server needs to be added to the SQL Server group, you could run Transact-SQL queries through SQLPS PowerShell module to configure the resource governor in a consistent manner with other servers in the group, or if you need to update its settings across a group of servers

Update: With the release of Update Rollup (UR) 526 in February 2015, support for resource

22 http://bit.ly/1F2rwUY23 http://bit.ly/1F2rwUY24 http://bit.ly/1KHNEm7 25 http://bit.ly/1OVb6Cr26 http://bit.ly/1ESCXfW

36

governor in SQL Server 2014 has been added natively to SQL Server Resource Groups in WAP. Resource Governor support can be added to Resource Groups, and Resource Pools Templates can be created and assigned to plans offering database as a service. More details can be found in Rupi Sureshkumar’s blog post here27.

With multiple instances on the same server, your options for fair share with a resource governor may require using resource limits such as cap_cpu, and leveraging the “default” pool of each instance so that all workloads in the instance have to honor them. This is because Resource Governor is mostly designed to manage the resources inside a single SQL Server instance.

In this context, using single-instance SQL Servers may be the most straightforward way to leverage the out of the box support for Resource Governor post-UR5.

The SQL Server fabric servers can be physical or virtual. When virtualized, it is also possible to use Hyper-V Quality of Service (QoS) features to ensure fair share of the different virtual machines in the SQL Server fabric. Note that Hyper-V supports guest clustering, and even shared virtual hard disks, so going the virtual route does not prevent you from using clusters.

More generally, since we are working at the SQL Server fabric infrastructure level, this is where monitoring might also be key. By using a monitoring solutions like Operations Manager as part of the System Center suite, you can monitor how your SQL Server fabric is doing and, when getting closer to potentially breaching SLAs, you could add servers to the shared infrastructure to distribute the load. This could even be automated.

Note If you want to go as far as dedicating a group of servers to specific tenants, you could also look at the solution described at the question “What if I like the shared approach, but want to provide more dedicated servers to my tenants?”

When using IaaS to deploy SQL Server virtual machines, could my tenants also deploy highly-available virtual machines?While VM roles do not support shared virtual hard disks today, it is possible to deploy a SQL Server AlwaysOn configuration supported by a file share witness. We have published a sample set of templates and runbooks28 handling this scenario in the context of the Cloud Platform System29 (CPS), but these samples could also be used outside of CPS.

Can I get data for potential chargeback with my tenants?There is a billing interface to access usage data collected by a resource provider. Both the VM Clouds and SQL Server resource providers include built-in mechanisms to query and report on usage data for showback and chargeback.

27 http://bit.ly/1FLwtjJ 28 http://bit.ly/1dIQFaU29 http://bit.ly/1umvRtw

37

Out of the box, the Service Reporting30 feature consumes the data from the VM Clouds resource provider, and you can then use standard analysis tools like Excel or Performance Point. And you could have your own query mechanisms to access Database related data.

Alternatively, you can also leverage solutions like Cloud Cruiser31, should you require more reports out of the box, or working with other types of clouds. Microsoft now includes a customized Express version of Cloud Cruiser’s cloud financial management solution with Windows Server 2012 R2 via Windows Azure Pack (WAP). This version also consumes the usage data collected by the SQL Server resource provider.

Other options with System CenterIt is also possible to create your own Database as a Service architecture by leveraging System Center. While this is more of a Do It Yourself (DIY) approach, the stack includes all the features needed to provide self-service, automation, and provisioning of database components. If this is an approach that makes sense in your environment, the SQL Server Self Service Kit32 includes sample solutions to help you get started.

In this sample solution, the Service Manager integration is optional. ITSM requests are “monitored” by the sample Orchestrator runbooks, and those runbooks could monitor another ITSM solution.

30 http://bit.ly/1DPQoba31 http://bit.ly/1ceN1Fv32 http://bit.ly/1QgoJJG

38

This sample solution has also been ported to use Service Management Automation (SMA) as the automation engine instead of Orchestrator, depending on your requirements: SQL Server Self-Service Kit “SMA Edition”33 (*).

And, should you want to enable DBaaS for other types of database software like Oracle DB 12c, we also provide a sample Oracle Self-Service Kit34

Compared to WAP, this alternative approach using System Center components and automation may be a good fit if your requirements include:

- Leveraging existing components or skills in your environment. For example, the Orchestrator instance used here can certainly be used to automate other actions, and is not limited to Database as a Service scenarios

- Interoperating with complex business processes, especially from an ITIL standpoint. We saw previously that there are ways to balance cloud concepts with ITIL in the context of WAP, but that usually requires at least a few changes in the approach, to avoid offsetting the benefits of the cloud. If you have very limited options to change existing ITIL processes, this second approach may make more sense until the service provider mindset gets more adopted by your organization

It’s also important to note the potential limitations of this approach, compared to WAP:

- Your runbooks will have to manage the full lifecycle of the resources. For example, the SQL Server Self Service Kit sample solution does not include resizing or decommissioning of databases, so you will need to expand it for that task. On the flipside, the resulting solution is really crafted exactly to your needs.

- Multitenancy is not built-into the sample solutions: The sample runbooks need to touch the provisioned virtual machines, for example to provision instances and databases. This means they apply to environment without multitenancy, or with a single tenant, or where tenant networks are routed. This may or may not be a limiting issue, depending on what you are trying to achieve.

(*) A note about the SQL Server Self-Service Kit “SMA Edition”

The main benefit of the SMA Edition version was to showcase the same scenarios as the original SQL Server Self-Service Kit, this time using a full PowerShell-based automation engine (SMA). This was mainly done to provide a learning opportunity for users comparing the runbooks in both automation engines, and to see how they were ported to SMA. It’s important to understand that the same limitations mentioned before still apply, since the sample runbooks directly update the target VMs and do not leverage the WAP admin APIs. The SMA Edition also does not include runbooks monitoring Service Manager ITSM requests.

33 http://bit.ly/1ESE2En 34 http://bit.ly/1IbIKzN

39

Looking beyond Database as a Service: Why leverage Microsoft technologies to virtualize and manage SQL ServerThis document has focused on the ability to enable “Database as a Service” (DBaaS). While DBaaS is certainly a key benefit of the Microsoft Private Cloud stack, there are other reasons why Windows Server and System Center are a great platform to virtualize and manage SQL Server, including (but not limited to) the following:

1. Scale and performance of Hyper-V, from an architecture and guest VM standpoint: 64 virtual processors and 1 TB RAM guest virtual machines, 1 million IOPS from a single VM35, 64-node Hyper-V clusters, NUMA support, and low overhead vs a physical SQL Server computer. You can read more about it here36.

2. High availability at the host and application level: Using Virtual HBA and/or the Shared Virtual Hard Disks (VHDX) feature, it is possible to combine in-guest clustering with host clustering. Plus features like Cluster Aware Updating also provide easier patching operations.

3. Flexible and cheaper architecture and storage options: Hyper-V provides support for existing investments in storage arrays, such as Offloaded Data Transfer (ODX) support, thin/trim provisioning, and other flexible and cost effective options relying on file shares, while still maintaining performance and transparent failover when used with clustering. Storage Spaces also includes tiering now, and Hyper-V supports online resize of VHDs

4. VM mobility, backup and disaster recovery options: Technologies at play here include Hyper-V Replica, Shared Nothing Live Migration, and Network Virtualization. As part of System Center, Data Protection Manager also includes SQL Server-specific workload support.

5. Application level monitoring and compliance: The SQL Server Management Pack for Operations Manager includes in-depth monitoring scenarios for SQL Server components, whether they are hosted on physical or virtual machines. Performance, availability and even configuration data – including policy based management – are surfaced in monitoring views, dashboards, and reports.

6. Automation and advanced patching: Automation enables scenarios like SQL Server maintenance tasks, advanced sequenced patching, etc.

7. Hybrid scenarios with Microsoft Azure, either at the networking layer (for example with SQL Server databases on premise and web tiers in Microsoft Azure, leveraging Virtual Networks) or the application layer (including options added in recent versions of SQL Server). And you can still get a single pane of glass for monitoring, leveraging on-premises, SQL Azure and Microsoft Azure management packs

8. Finally, licensing costs could also come into play as a benefit too, when virtualizing with Hyper-V and managing with System Center

35 http://bit.ly/1zIPkuR 36 http://bit.ly/1KahAGI

40

41

ReferencesThis section provides links for additional information about topics discussed in this paper.

Building Clouds Blog (http://aka.ms/bcb) Windows Azure Pack overview (http://bit.ly/1JGfr94) Overview of the SQL Server Resource Provider for Windows Azure Pack (http://bit.ly/1GMIlk1) Ignite 2015 session : Provisioning SQL Database-as-a-Service in the Azure Pack

(http://channel9.msdn.com/Events/Ignite/2015/BRK3501) Service Models feed and the Web Platform Installer (WebPI) (http://bit.ly/1DIDqMy) References for Creation, Configuration, and Automation of VM Role Gallery Items

(http://bit.ly/1KBb8tj) Sample VM Role Gallery Items for SQL Server (http://bit.ly/1hZURCS)

o Update to the Sample VM Role Gallery Items for SQL Server (http://bit.ly/1CA33Ao) VM Role Example kit (http://bit.ly/1KBbgc2) SQL Server Requirements for Windows Azure Pack (http://bit.ly/1c4dLZj) Configure SQL AlwaysOn Availability Groups in Windows Azure Pack (http://bit.ly/1JftT4n) SQL Azure Database Basic, Standard and Premium offerings (http://bit.ly/1JftT4n) Update Rollup (UR) 6 for Windows Azure Pack (http://bit.ly/1OVb6Cr) Update Rollup (UR) 3 for Windows Azure Pack (http://bit.ly/1KBdND6) Migration scripts for UR3 (http://bit.ly/1JGYRWn) Dedicating a part of the SQL Server fabric to a specific tenant (http://bit.ly/1F2qkkc) Custom Resource Providers in Windows Azure Pack - Extending the Hello World Sample calling a

SMA Runbook (http://bit.ly/1kaxdlR) Automation–The New World of Tenant Provisioning with Windows Azure Pack (Blog post series)

(http://bit.ly/1dIGZxh) o Part 4 (http://bit.ly/1OUXJC8) shows how to deploy Virtual Machines on behalf of a

tenanto Part 5 (http://bit.ly/1F2rwUY) shows how to deploy SQL Server databases on behalf of a

tenant, and discusses how to interface with ITIL processes Custom Resource Providers in Windows Azure Pack – Moving from Hello World to your own

Resource Provider (http://bit.ly/1GVrtI4) VM Role Gallery Items for Oracle DB 12c (http://bit.ly/1hZURCS) Combining ITIL with Database as a Service (http://bit.ly/1F2rwUY) Reference API for the SQL Server Resource Provider (http://bit.ly/1KHNEm7) Update Rollup (UR) 5 for Windows Azure Pack (http://bit.ly/1ESCXfW) Manage Tenant Database Workloads with Resource Governor in Azure Pack

(http://bit.ly/1FLwtjJ) Sample set of templates and runbooks for the Cloud Platform System (CPS)

(http://bit.ly/1dIQFaU) Cloud Platform System (CPS) Overview (http://bit.ly/1umvRtw) Chargeback: Service Reporting feature overview (http://bit.ly/1DPQoba) Chargeback: Partner Cloud Cruiser (http://bit.ly/1ceN1Fv) SQL Server Self Service Kit (http://bit.ly/1QgoJJG)

42

SQL Server Self-Service Kit “SMA Edition” (http://bit.ly/1ESE2En) Oracle Self-Service Kit (http://bit.ly/1IbIKzN) 1 million IOPS from a single VM (http://bit.ly/1zIPkuR) at TechEd Europe 2012 Windows Server 2012 Hyper-V Delivers On Scalability and Performance for Virtualized

Enterprise Applications (http://bit.ly/1KahAGI)

Document Revision HistoryDate Published Version CommentsMay 2015 1.1 Updated the document to cover updates in

Update Rollups 3, 5 and 6 for Windows Azure Pack

Added a more explicit FAQ entry for sample scripts for the SQL Server Resource Provider

Added a FAQ entry to the CPS sample templates for SQL Server AlwaysOn

Updated formattingMay 2014 1.0 First published version

43