oss for managed edgecloud with zenoss - netappa partner solution, zenoss service dynamics, that...

In partnership with

White Paper

Software-Defined Operations for Managed EdgeCloud with Zenoss Devon Snyder, NetApp; Kent Erickson, Zenoss

October 2017 | WP-7260

2 Software-Defined Operations for Managed EdgeCloud with Zenoss

© 2017 NetApp, Inc. All Rights Reserved. © 2016 NetApp, Inc. All rights reserved.

TABLE OF CONTENTS

1 Introduction ........................................................................................................................................... 4

2 Managed EdgeCloud Solution Overview............................................................................................ 4

2.1 Value Proposition ............................................................................................................................................4

3 Key Assurance Use Cases and Value ................................................................................................ 5

3.1 Unified Fault Identification, Usage Tracking, and Capacity Data Collection ....................................................5

3.2 Rapid Fault Identification and Scope Analysis ................................................................................................5

3.3 Proactive Identification of Tenant Capacity Needs ..........................................................................................6

3.4 Low-Effort Solution Administration and Scale-Up and Scale-Down Capability ...............................................6

3.5 Customer Visibility into Monitoring Results .....................................................................................................7

3.6 Extension Capability to Support Differentiated Service Offerings ...................................................................7

4 Zenoss Overview .................................................................................................................................. 7

4.1 Differentiation from Public Cloud .....................................................................................................................9

5 Zenoss Assurance Architecture Overview ........................................................................................ 9

5.1 Resource Manager ....................................................................................................................................... 10

5.2 Collectors ...................................................................................................................................................... 10

5.3 Model Database Management System ......................................................................................................... 11

5.4 Event Database Management System .......................................................................................................... 11

5.5 Performance Management Database Management System ........................................................................ 11

5.6 Service Impact .............................................................................................................................................. 11

5.7 Zenoss Service Dynamics Analytics Data Warehouse ................................................................................. 12

5.8 Control Center ............................................................................................................................................... 13

5.9 Requirements ................................................................................................................................................ 13

5.10 Hardware Requirements ............................................................................................................................... 13

6 Assurance Deployment Overview ..................................................................................................... 14

6.1 Installation ..................................................................................................................................................... 14

6.2 Configuration ................................................................................................................................................. 17

6.3 Organizing the Managed EdgeCloud Installation .......................................................................................... 18

6.4 Adding Users ................................................................................................................................................ 18

6.5 Problem Notification ...................................................................................................................................... 19

6.6 Product Walkthrough .................................................................................................................................... 21

6.7 Service Reporting ......................................................................................................................................... 26

6.8 Tenant Access for Real-Time Information ..................................................................................................... 27

7 Assurance Commercial Overview ..................................................................................................... 30



7.1 Zenoss Service Dynamics: On Premises ...................................................................................................... 31

7.2 Zenoss as a Service: Cloud-Based Hosting .................................................................................................. 31

8 Conclusion .......................................................................................................................................... 31

LIST OF TABLES

Table 1) Required use cases met by Zenoss software. ..................................................................................................8

LIST OF FIGURES

Figure 1) Managed EdgeCloud overview. ......................................................................................................................4

Figure 2) Relationships among Zenoss architectural components. .............................................................................. 10

Figure 3) Service Impact model. ................................................................................................................................... 12



1 Introduction

This white paper was written for service-provider technical staff who are assessing the NetApp® Managed

EdgeCloud solution or who desire to improve their service for Managed EdgeCloud tenants. It is also

intended for those who seek to develop additional services for Managed EdgeCloud tenants. It introduces

a partner solution, Zenoss Service Dynamics, that allows service providers to manage all of their

Managed EdgeCloud deployments in one system, with complete visibility of storage, compute,

networking, and hypervisor resources.

After reading this document, you will understand the value that Zenoss software brings to your Managed

EdgeCloud offering. You will also have a working knowledge of the software, its daily use, and key

features, and you will see how it can help you to expand your Managed EdgeCloud business.

2 Managed EdgeCloud Solution Overview

The NetApp Managed EdgeCloud solution is a turnkey, cloud-in-a-box, converged infrastructure solution

that is preintegrated with Cisco, NetApp, and Microsoft products. Managed EdgeCloud extends cloud

infrastructure onto a customer’s premises and enables managed private-cloud services to be delivered

through a complete solution. The cloud in a box consists of Windows Server and Hyper-V running on

Cisco UCS Mini, NetApp FAS and All Flash FAS (AFF) storage, and Cisco Integrated Services Router

(ISR). This configuration provides secure backhaul to a service provider’s infrastructure. The components

arrive preassembled in a 24U cabinet with power and battery backup included.

2.1 Value Proposition

Managed EdgeCloud enables you, as the provider of cloud services, to quickly deliver a managed private

cloud offering to your customers. Each EdgeCloud unit is managed and supported remotely and delivered

seamlessly as a subscription. Because all units are managed by you from a secure single pane of glass,

you can scale your customer base efficiently. This consistently delivered, preintegrated infrastructure

reduces risk and provides a clear integration process. A centralized, cloud-based web portal powered by

the Windows Azure Pack delivers a seamless, role-based experience to your customers.

Managed EdgeCloud with Cisco and NetApp places computing at the source of demand. Customers get

the dedicated local resources that their workloads require with the simplicity and cost efficiency of the

public cloud.

Figure 1) Managed EdgeCloud overview.



3 Key Assurance Use Cases and Value

Service providers typically use the NetApp Managed EdgeCloud solution to provide private cloud

functions to many dozens of tenants. A tenant can choose to deploy private cloud functions in one

location or in many different locations. In each location, private cloud services are delivered with a

converged infrastructure solution consisting of compute, network, storage, and security.

Service providers who deploy Managed EdgeCloud solutions are typically responsible for delivering

reliable service to multiple tenants with hundreds of individual devices across many dozens of locations.

Without a centralized assurance function, any of these devices could physically or logically fail, run short

of resources, or otherwise malfunction. This would disrupt the services that the tenant is paying for and

lead to dissatisfied customers, missed SLAs, and even financial penalties.

Reliable service delivery is provided by the service assurance function. Here are the key use cases:

• Unified fault detection, usage tracking, and capacity data collection across all Managed EdgeCloud device types

• Rapid fault identification and scope analysis

• Proactive identification of tenant capacity needs

• Low-effort assurance solution administration, including the ability to grow or shrink capacity

• Extension capability to support differentiated service offerings

The following sections describe these use cases in detail.

3.1 Unified Fault Identification, Usage Tracking, and Capacity Data Collection

A unified assurance solution tracks the proper operation of all the components of the Managed

EdgeCloud solution.

The typical approach that service providers have used to monitor device health across multiple locations

is to employ separate monitoring tools: one for Microsoft Windows, one for Cisco UCS, one for the Cisco

router, one for NetApp storage, and one for the UPS. However, this approach has the following issues:

• Administrative complexity. Separately installing, configuring, and maintaining five different software products at the service provider scale can consume substantial amounts of time and resources.

• Scale complexity. Each element manager takes a different approach to operating at the service provider scale, resulting in uneven costs as scale needs change.

• Licensing complexity. Individual element managers have different requirements for growing and shrinking to meet changing needs.

A unified solution addresses all of these issues:

• Administrative ease. A unified solution is installed once and configured centrally. Only one system is maintained, which reduces the amount of team skills and training required for ongoing operations.

• Scale ease. A unified solution offers a simple way to add or remove resources to meet changing needs and presents an identical installation process at each tenant location.

• Licensing simplicity. A unified solution provides predictable costs and can support adding or removing elements without incurring permanent license costs.

3.2 Rapid Fault Identification and Scope Analysis

Fault identification is an essential use case. Knowing that a piece of equipment is failing and requires

service is vital to meeting SLAs. Also, it is crucial to be able to instantly identify the specific tenant and

site that are affected by an outage.

Here are the use cases in fault identification:



• Gather faults generated by device self-monitoring. For example, NetApp FAS automatically detects power supply failures and generates a fault event.

• Test devices for proper behavior. For example, a Cisco integrated service router (ISR) might work properly, but the WAN or LAN connection might be interrupted.

• Check that metrics are within range. For example, a UCS Mini contains temperature sensors. At high temperatures, a service provider might need to shut down equipment to prevent damage.

• Link faults to specific tenants. Individual tenants might have varying SLAs, and automatically linking a device failure to a specific tenant can ensure that SLAs are met and penalties avoided.

• Link faults to specific tenant sites. With potentially hundreds of pieces of geographically distributed equipment, it is important to know where to send repair teams and equipment.

• Identify related faults and provide root-cause analysis. In a private cloud environment, a failure in one component can generate many related failure events. Root-cause analysis sifts through a sudden flood of errors and identifies the specific fault at the core of the flood. For example, a blade server that is inadvertently removed from a chassis causes faults in the blade server, the Hyper-V hypervisor, and each virtual machine (VM) running on the hypervisor.

• Automatic fault clearing. When a system is in failure mode, a fault should be persistently shown. When recovery action is completed, the fault should be cleared. This reduces administrative work and enables precise calculation of the impact of a fault on the SLA.

• Multiple notification methods. Different teams must be notified when different types of faults occur. Some teams might work best with a service desk system such as ServiceNow. External providers might require notification by email. Collaborative teams might rely on Slack. Supporting multiple methods in one system enables rapid response by the proper responder.

3.3 Proactive Identification of Tenant Capacity Needs

Service providers using Managed EdgeCloud might find that some of their tenants rapidly grow their use

of the private cloud service. As they grow, the resources of the Managed EdgeCloud pod eventually

become exhausted. Growing tenants represent an opportunity to increase revenue by delivering an

additional pod. On the other hand, some tenants might have difficulty using private cloud services. If

these difficulties are not addressed, the tenant might cancel the private cloud service.

Knowing how tenants are using the resources in each pod is critical to growing private cloud revenues.

Resource usage tracking. In a private cloud environment, tracking the use of compute processor and

memory, storage utilization and operations rates, and local and wide-area NIC usage are the key

performance indicators. You should retain these indicators for longer than a year to help you identify

seasonal usage patterns that could provide revenue opportunities.

Resource exhaustion identification. As tenants add VMs, they use available resources. With

unchecked growth, new and existing VMs might run more slowly or even stop entirely. You want to know

when resources are running short so that you can take corrective action with the customer, for example

by ordering more capacity or reducing usage.

Proactive notification. It can take some time to add capacity, so it is important to identify potential

resource exhaustion well in advance. Days or weeks of advance notice might be required to deliver

additional hardware to remote locations.

3.4 Low-Effort Solution Administration and Scale-Up and Scale-Down Capability

When you are operating a service assurance solution, you want to keep ongoing costs low and have the

ability to scale up or down. Delivering efficient operations for a monitoring implementation should not be

challenging. The effort needed to add or remove assurance at each new tenant site should be small, and

adjusting centralized capacity to reflect changes in the installed base should be easy and predictable.

• Add a new tenant site. How much work is it to set up the assurance for a new tenant location?



• Understand the necessary resources.

• Maintain reliable, secure communications.

• Deploy without agents.

• Add capacity to and remove it from central operations.

• Provide centralized backup and restore functions.

• Automatically discover tenant site resources and build or maintain impact models.

3.5 Customer Visibility into Monitoring Results

Your tenants want to know that they're getting the service they're paying for. Regular service performance

reports are standard, but customers also expect web access to real-time data, which helps them locate

the source of an application issue quickly and act appropriately.

The following capabilities are key:

• Tracking and reporting availability across each Managed EdgeCloud instance as a whole and by individual device.

• Tracking and reporting devices that are overextended and are being asked to provide more resources than are available. This is often a cause of application performance issues.

• Secure tenant access to real-time monitoring data over the internet.

• Individually customized, automatically delivered reports.

3.6 Extension Capability to Support Differentiated Service Offerings

As you engage with your tenant customers, you might identify new revenue opportunities. Therefore you

should expect the assurance solution to be open to extensions. There should be standard extensions

from the vendor or extensions that you can build yourself; or you might choose to contract with an outside

provider to create extensions for you.

When reviewing extension options, consider the following issues:

• Review the library of available extensions. Are they published only by the vendor, or is there an active community of contributors?

• Is there a published API?

• Is there a published software development kit (SDK) for extension development?

• Are extension development training and services available?

4 Zenoss Overview

Zenoss Service Dynamics redefines monitoring for service providers supporting Managed EdgeCloud

deployments. Monitoring is not just about interrogating devices; it fills the larger business need of tracking

the health of tenant services delivered across multiple locations. Zenoss Service Dynamics helps service

providers deliver high-quality managed-cloud services by avoiding service disruptions and providing

actionable intelligence for Managed EdgeCloud infrastructure, applications, and services. In addition, it

can also integrate with orchestration, asset tracking, and incident management systems to tie together

multiple service management processes.

We have summarized the required use cases for the Managed EdgeCloud solution provider and those

met by the Zenoss software. If you want to consider alternative solutions, Table 1 can help you to

compare key functions.



Table 1) Required use cases met by Zenoss software.

Details Zenoss Solution

Alternative Solution

Use case: Unified fault identification, usage tracking, and capacity data collection

NetApp storage support ✓

Cisco server support ✓

Cisco networking support ✓

Microsoft Hyper-V support ✓

Use case: Rapid fault location and scope analysis

Gather device self-monitoring faults ✓

Test devices for proper behavior ✓

Check that KPI metrics are within range ✓

Link faults to specific tenant sites ✓

Identify related faults with root-cause analysis ✓

Automatic fault clearing ✓

Multiple notification methods ✓

Use case: Tenant monitoring access and reporting

Standard availability reporting ✓

Standard performance reporting ✓

Customizable report development ✓

Secure tenant access to monitoring data through application ✓

Extend your own tenant portal with monitoring data ✓

Use case: Tenant capacity identification

Tracks resource usage ✓

Identifies resource exhaustion ✓

Proactive capacity notifications ✓

Use case: Low-effort solution administration and scale up/down capability

Configured and operated centrally ✓

Secure, reliable communications ✓

Centralized backup and restore functions ✓

Easy to scale up or down ✓

Simple, predictable licensing ✓

Automatically discover tenant site resources ✓



Details Zenoss Solution

Alternative Solution

Automatically build dependency models ✓

Use case: Support differentiated offerings

Broad library of extensions ✓

Published API and SDK ✓

Developer training ✓

Developer services ✓

4.1 Differentiation from Public Cloud

One of the biggest challenges in selling Managed EdgeCloud solutions is convincing fans of public

clouds. Zenoss software can help with that.

Two of the key difference makers ship automatically with the standard product:

• Giving your customers direct access to your monitoring environment with role-based access control helps them understand that private clouds lower many business risks.

• Customers can quickly eliminate the Managed EdgeCloud infrastructure as a source of application problems, reducing downtime.

• You have the opportunity to introduce custom service offerings that can lead to greater revenue opportunities.

• Customers can feed critical events directly into the service desk of their choice by adding the Zenoss incident management extension.

• Customers can publish critical events from their Managed EdgeCloud to their favorite monitoring tool.

• They can manage the Windows and Linux guest operating systems that are running in their VMs, including services like Microsoft SQL Server, MySQL, IIS, Apache, MSMQ, RabbitMQ, and more.

• They can add servers, network devices, etc. from outside their Managed EdgeCloud installation by using the Zenoss collector they already have.

• They can build management reports customized for their needs and have them delivered automatically.

All of these capabilities are standard product features.

5 Zenoss Assurance Architecture Overview

The Zenoss Service Dynamics architecture is designed to accommodate the unique requirements of

monitoring Managed EdgeCloud service infrastructures. The architecture helps to reduce complexity by

providing a unified, end-to-end view of the interconnected resources and dependencies supporting

service delivery across physical, virtual, or cloud-based environments.

Zenoss Service Dynamics is built with a service-oriented architecture that makes monitoring just as

dynamic as your Managed EdgeCloud customer deployments. You can abstract monitoring services so

that they can share compute and storage resources. Abstraction also allows you to drive more rapid

implementation of monitoring, streamline updates and upgrades, build out high availability, and scale up

or out more easily.

Zenoss Service Dynamics is designed and built with the following architectural components:

• Resource Manager



• Service Impact

• Analytics

• Control Center

• Collectors

Resource Manager, Service Impact, Analytics, and Control Center are deployed in one central location,

and collector VMs installed in each Managed EdgeCloud site collect and forward monitoring data.

Figure 2) Relationships among Zenoss architectural components.

5.1 Resource Manager

Resource Manager is the foundation of Zenoss Service Dynamics, providing unified monitoring, inventory,

modeling, and event management. It monitors all the components of multiple Managed EdgeCloud tenant

sites through a single, unified, agentless platform. Highly flexible and scalable, Resource Manager

combines enterprise-grade event management with performance and availability monitoring so that you

can see everything in one place. Resource Manager receives its data from an agentless collector VM

deployed in each Managed EdgeCloud location.

5.2 Collectors

Collectors are an agentless way of gathering performance and availability data. Data collection is

accomplished using a variety of protocols, such as NetApp ONTAP® data management software, WinRM,

SNMP, and HTTP, and can be extended and customized to leverage data from additional APIs. The

ability to simultaneously collect data by using these protocols makes collectors extremely efficient,

spanning physical and logical elements of the Managed EdgeCloud <pods>.



You can support very large deployments of Managed EdgeCloud with total monitoring loads of more than

a million data points per second. In a Managed EdgeCloud deployment, a single collector appliance is

deployed in each Managed EdgeCloud instance, making it simple and efficient to add new tenant sites. A

single Zenoss Service Dynamics system can support 100 Managed EdgeCloud sites. For larger

deployments, a central Zenoss system provides a consolidated view across tens of thousands of VMs.

5.3 Model Database Management System

The model database management system (DBMS) uses MariaDB to store modeling states for the devices

loaded into the system. The model state includes every device and its monitored components, the

configuration of the device and components, its management information base, and the changes in state

as they occur.

5.4 Event Database Management System

The event DBMS uses MariaDB to store collected event records, updates in state, and the count of

duplicate events. Every detail associated with an event is stored in the event DBMS. Apache Lucene, a

high-performance, fully featured text search engine, is used to increase the search and index

performance of all events. Duplicate events are stored as a count on the original event, along with a last-

seen field that is updated with the date and time of the event. This prevents unneeded growth in the event

system and storms of duplicate events from flooding the event console. The event engine is capable of

processing an extremely high volume of data, up to 100 million events daily.

5.5 Performance Management Database Management System

Zenoss Service Dynamics delivers big-data performance, availability, and scale by implementing Apache

HBase as a centralized data store. With this powerful big-data back end, it is possible to store a vast

amount of performance and availability data at the collected resolution with no rollup required. This can

give clear, granular insight into upcoming tenant capacity needs, as well as provide long-term storage to

help identify seasonal trends.

On the front end, Zenoss Service Dynamics uses OpenTSDB for reading and writing at scale to the big-

data back end. Unlike traditional relational databases, OpenTSDB can easily handle the high transaction

volume of time series data required in a real-time monitoring environment. In addition to handling high

volumes of complex data, OpenTSDB provides a performance processing pipeline that allows you to

scale the solution effectively.

5.6 Service Impact

Given the dynamic nature of a Managed EdgeCloud deployment, in which relationships between tenant

VMs and Managed EdgeCloud infrastructure are constantly changing, pinpointing the root cause of an

issue is never easy. When customers control their own virtualization, the overall system cannot be static,

and performing root-cause analysis can be a time-consuming and inefficient process.

Service Impact contains the dynamic service model in which relationships and dependencies are mapped

between Managed EdgeCloud compute, storage, and network infrastructure and the VMs that each

supports. Figure 3 depicts a Service Impact model.



Figure 3) Service Impact model.

Services identify the infrastructure groupings that support tenants and tenant sites. For example, a tenant

might have one or several Managed EdgeCloud sites covered by a single SLA. You can create services

for each tenant from their list of VMs. You can also provide application-centric infrastructure monitoring by

creating groups of VMs for one or more applications.

After the tenant services are defined, Service Impact uses advanced modeling and domain-awareness

logic to pull in all relevant Managed EdgeCloud infrastructure elements to complete the Service Impact

model. These resources—such as Windows Hyper-V operating systems, Cisco UCS blades or chassis,

NetApp FAS storage, and Cisco switches—are discovered and mapped into the relevant service

dependency graphs, as shown in Figure 3.

Service Impact uses this information to process events associated with these resources against root-

cause analysis and confidence-ranking engines to generate service events that pinpoint the most likely

cause of an incident. These events alert Managed EdgeCloud administrators to tenant outages, help

decrease triage time, and expedite root-cause analysis.

5.7 Zenoss Service Dynamics Analytics Data Warehouse

The Zenoss Service Dynamics Analytics (Analytics) product provides data warehouse analysis, report

generation, report scheduling, and custom dashboards. Analytics directly leverages the resource and

service models to report on service levels and risk assessment. The open business intelligence engine

that delivers this functionality creates rich, scalable analytics for both historical reporting and trending

information. Analytics can be deployed as part of the primary Zenoss Service Dynamics pool or in its own

pool. This functionality allows organizations to use resource-intensive reporting and analytics features in

the way that creates the least impact on monitoring operations.

The Analytics architecture combines extract, transform, and load processes with data warehouse and

reporting tools to extract and report data. Analytics aggregates data over time and is meant for long-term

trending. It purges data from the data warehouse at regular intervals, rolling up hourly or daily aggregates

on whatever schedule you define.

This aggregate information is critical to understanding health and utilization trends, allowing you to

forecast capacity needs and to anticipate availability and performance problems. You can draw

conclusions about upcoming operational issues and infrastructure restraints that could impact service



delivery by examining service availability, service performance, or CPU usage over time and using trend

extrapolation.

5.8 Control Center

Control Center is an out-of-band management and application orchestration system for Docker

containers. ZSD 5 daemons execute in Docker containers to simplify scalability and deployment of ZSD 5.

Control Center load balances and manages the services and collectors among a pool of server

resources. It centralizes log access for the Zenoss containers running in every site by using Kibana, and it

provides a method to back up and restore ZSD 5. Finally, Control Center monitors the health of the

services it manages.

5.9 Requirements

Control Center and Resource Manager require real or virtual hosts that implement the 64-bit version of

the x86 instruction set and support one of the required operating systems. Hardware requirements vary

by role (master or resource pool host) and by the services assigned to a host's pool.

5.10 Hardware Requirements

Master Virtual Machines

The default resource pool contains the Control Center master host and several hosts for Resource

Manager services. Hosts in the default resource pool should meet the following minimum requirements:

• 8 CPU cores (64-bit only; real or virtual)

• 32GB RAM

• 1 network interface controller (must support TCP/IP)

• The network latency among all hosts in a resource pool should be less than 5 milliseconds

• Local storage is recommended and SAN storage is supported

Master host storage requirements vary by collection rate, backup frequency, and the type of data stored

in specific locations. For example, a Resource Manager collection rate of 25,000 metrics per second

typically requires about 1TB of storage in 60 days, depending on the OpenTSDB compaction settings.

Resource Pool Virtual Machine

The hosts in resource pools (other than the default) need enough RAM, CPU, and storage resources to

support the services assigned to the pool. The network latency among all hosts in a resource pool should

be less than 5 milliseconds. The hardware specifications for all hosts in a resource pool should be

identical.

The storage requirements for hosts (other than the master) exclude space for application data, which is

stored on the master host and mounted everywhere else. Storage is needed only for Linux, Swap, and

Docker.

Collector Virtual Machine

The VMs supporting collectors at tenant sites typically need 16GB of memory, 4 CPU cores, and 50GB of

local storage.

Storage requirements for hosts (other than the master) exclude space for application data, which is stored

on the master host and mounted everywhere else. Storage is needed only for Linux, Swap, and Docker.



Operating System Requirements

Control Center, Resource Manager, and collector VMs require the 64-bit version of the following Linux

distributions:

• Red Hat Enterprise Linux (RHEL) 7.0 or 7.1

• CentOS 7.0 or 7.1

All versions of the Linux kernel included in these releases and all subsequent updates are supported.

However, Zenoss encourages you to keep the kernel up to date. Control Center and Resource Manager

are tested on operating system platforms that are installed and configured with standard options.

The RHEL and CentOS 7.x distributions offer a variety of server configurations. Docker and Control

Center are tested and supported with the minimal installation configuration, when the NFS and NTP

packages are installed as well.

Control Center relies on the system clock to synchronize its actions. The installation procedures include

steps to add the Network Time Protocol (NTP) daemon to all hosts. By default, the NTP daemon

synchronizes the system clock by communicating with standard time servers available on the internet.

Alternatively, you can configure the daemon to use a time server in your environment instead.

6 Assurance Deployment Overview

We implemented Zenoss Service Dynamics in a NetApp development lab to validate correct operations

with a Managed EdgeCloud installation. This section summarizes the installation process and

configuration steps. It also presents an overview of product features and looks at service-level reports of

particular interest to organizations that provide private cloud services.

6.1 Installation

The Zenoss product supports single and multiple host configurations with centralized and distributed

options. We chose to install it using a single Hyper-V VM instance with 8 cores and 64GB of memory.

This configuration has enough capacity to support hundreds of VMs running in 10 Managed EdgeCloud

installations.

Managed EdgeCloud service providers generally choose to add a local collection point to each customer-

situated Managed EdgeCloud instance. This arrangement meets a number of critical needs, including a

smaller network attack footprint by restricting the number of open ports needed. It also offers better

resilience against changing WAN conditions by taking advantage of data queueing and standard timeout

features as well as easier configuration of customer-specific credential and overlapping address spaces.

The administrative overhead of maintaining collector software at each location is quite low, because no

data is stored locally and all configuration and version maintenance is performed centrally.

Here are the installation steps:

1. A VM is provisioned and the Centos operating system is loaded.

2. Zenoss Control Center is loaded to provide Docker management, storage, and networking services.

3. The Docker application Zenoss Resource Manager is loaded onto Control Center.

4. Zenoss Service Impact is added to Resource Manager.

Note: We chose not to install the custom report development environment, Analytics, which runs in a separate VM to make sure that reports do not affect production operations.

Zenoss Control Center is generally accessed through a browser. A command line interface and API allow

scripting options. The Zenoss Resource Manager application runs in several dozen Docker containers,

each running small services. Control Center configures Docker networking for the application, which



makes sure that all services are running. Networking configuration also centralizes container logs,

provides central application storage, and runs backup and restore services.

For our single-system deployment, we had one Control Center resource pool. A resource pool runs a set

of containers. For customer-site collection services, we would create a pool for each site that runs a copy

of the data collection containers.

When you are selling a service like Managed EdgeCloud, you must allow for growth. Control Center

makes adding capacity very simple. You can support a much larger set of customers by adding additional

hosts (VMs) to the default pool. Control Center distributes the containers across all the hosts in the pool,

which makes adding capacity very easy.

Our 8-core, 64GB VM was oversized for the needs of the pilot environment. Zenoss works with you to

determine the correct size for your needs.



Control Center also tracks resource usage across the hosts so that you can see if more capacity is

necessary. Self-monitoring is a useful feature, because you do not need to develop monitoring on your

own.

The Docker containers for the Zenoss application are called services instances in Control Center. Control

Center automatically starts, monitors, and restarts them if necessary.



6.2 Configuration

After we installed Control Center and loaded the Resource Manager application into it, the next step was

to configure our environment. After we created an application administrator, we needed to add the

devices from our Managed EdgeCloud test bed.

Resource Manager provides a wizard for adding all of your devices at once. Of course, you can add more

at any time. There is also a bulk-loading command line option and a scripting API that integrates with

provisioning tools like SaltStack.

It is straightforward to add devices because there are no agents to deploy. Devices are added to the

system, modeled, and monitored using network-friendly protocols like ONTAP, WinRM, and SNMP. The

wizard asks for the device type, IP address, and credentials and then adds all the devices in a

background process.

When adding a device failed, we found it useful to examine the details logs. The logs clearly showed that

we had not configured the necessary read-only credentials properly, and we were able to fix the device

configuration and successfully start monitoring.



6.3 Organizing the Managed EdgeCloud Installation

When you are managing a dozen or more tenant sites, you must be able to keep track of the equipment

at each site, quickly spot sites with problems, and drill down to the device fault. We used the system

organizer feature in Resource Manager to do this. Organizers are simple groups of devices that are

created through the UI or through batch and scripting interfaces.

We used a three-tier model in our pilot installation. Our top tier was the pods level, which we chose to

represent all tenants. Each tenant had a separate organizer under the pods level. Below this level, we

built technology organizers for the Cisco UCS, NetApp FAS, the Hyper-V servers, ISR, and System

Center. Our single site is named Managed EdgeCloud1.

You can see how this looks in the following screenshot. We have focused on the list of Hyper-V servers.

We also wanted a dashboard map of our locations, so we used the locations organizer to put our

Managed EdgeCloud devices in one location and our central Zenoss server in another. Location

organizers have an address field.

6.4 Adding Users

Out of the box, the Zenoss system uses application security, although it can be integrated with Active

Directory (or a Lightweight Directory Access Protocol server) as well.

The system uses role-based access control to assign view-only, monitoring user, or administrative

application control to each account. You must also add an email address for each account.



You can also safely provide users with their own accounts so that they can see what you see. To do so,

complete the following steps:

1. Create a user group for each tenant. Then create accounts for their users and remove all their default privileges.

2. Give the group read-only privileges for just the system organizer that represents their Managed EdgeCloud installation. If the group has several Managed EdgeCloud installations in use, give them the same privileges for those too.

3. Add the tenant accounts to the user group, and give them the URL to your Zenoss installation. They will see their Managed EdgeCloud installation just like you do, but nothing else.

Offering your tenants this level of visibility into the infrastructure supporting their applications is a strong

differentiator from public cloud platforms.

6.5 Problem Notification

The Zenoss system includes a flexible notification system based on the event management system. The

first step is choosing which events require automatic action. That is done by setting up a trigger, or a rule

that matches a set of event fields.

We decided to set up a very basic trigger; for devices that are in production, the notification system alerts

the user about all critical events. Triggers can be much more specific, however. For example, you might

want to create separate triggers for customer Managed EdgeCloud installations and central Zenoss

resources. Indeed, different people could be made responsible for different triggers.



Triggers link to notification actions. There is an expandable set of notification actions that includes email,

traps, scripts, Slack, and more. Zenoss offers a two-way integration to service desk products too,

although we did not implement that feature. We matched our trigger to an email notification, which worked

well for our testing needs.



6.6 Product Walkthrough

When you log in for the first time following configuration, you see a dashboard. Dashboards are

individually customizable and can also be shared.

Our dashboard shows a map of status by location, a list of devices with current issues, and the status of

each Managed EdgeCloud pod. You can add or remove and rearrange dashboard portlets to fit your

individual needs, and you can create other dashboards for specific purposes as well.



The event console lists fault events from devices, availability issues, and detected performance

thresholds. If you do not have a service desk, you can manage events by acknowledging them, closing

them, and attaching comments to them.

Several features for managing event floods work out of the box. In the following screenshot, the far-right

column lists event counts. The system automatically detects duplicate events. All you see is the first and

last time the event is triggered and the total count. The system is also Layer-3 network-topology aware,

so you are not alerted for every device in a customer's Managed EdgeCloud installation if the WAN

connection to it is interrupted. You see only the connection alert.

Each Managed EdgeCloud device is extensively monitored with automatic configuration detection, an

event console that is specific to the device, and a set of graphs.

The following screenshot depicts a set of graphs for a NetApp ONTAP storage cluster, showing the types

of performance data tracked for the hard disks. Support for NetApp systems is excellent because NetApp



is a long-term Zenoss customer, and Zenoss developed their support system in close collaboration with

the NetApp IT team.

Cisco UCS support is best in industry as well, and Zenoss and Cisco have a close working relationship.

Zenoss was the first monitoring company to release UCS support, and development of this support

relationship is ongoing.



Automatic dependency analysis is another useful feature. In the following screenshot, we are focused on

a single UCS blade server, and we can see the Hyper-V server running on it. We can also see the local

disks, virtual host bus adaptors, and other components that rely on that server. This system is useful for

diagnostics, to spot related issues, and to quickly scan performance graphs.

Turning to Microsoft Hyper-V, we can see the critical resources that support the customer’s VMs. The

monitoring support goes beyond basic resource utilization to specific Microsoft metrics like pressure.



Memory pressure tells you quickly whether a customer has added too many VMs to a host. If memory

pressure is under 100, then the system is OK. If it is over 100, then the customer might need to acquire

another Managed EdgeCloud pod or stop one or more unimportant VMs.

Network device support is straightforward. The following screenshot show throughput on an interface with

the Range menu pulled down. The Zenoss software keeps metrics data for long-term trending as long as

you like, although the default is 90 days. You can add capacity forecasting to any metric as well, to let you

know ahead of time if the customer is running out of resources.



6.7 Service Reporting

To assess service-level reporting for tenants, we evaluated the built-in reports. We found the Organizer

Availability report to be very useful. Using the System Organizer structure that we built, this report lets us

select a time period and provides overall and granular availability by technology type.

We could also look at device availability for every device in a customer's site and export the results to

CSV if necessary.

The first Organizer Availability report we reviewed was for simple network availability. The following report

focuses on performance issues, specifically on threshold violations within the designated time period. You

can report separately on availability and performance with one report.

You can provide your customers with more than traffic-light visibility, another great differentiator for

Managed EdgeCloud over a public cloud platform.



Zenoss also includes a custom report development utility called Analytics. We did not review this feature,

but the following screenshot shows a sample from the standard report.

6.8 Tenant Access for Real-Time Information

There are two ways to give a tenant immediate access to live data about their Managed EdgeCloud

deployment. They can use the same Zenoss Resource Manager user interface that we did, with view-only

access to only their own devices. This can be done with standard product features, using the organizers

we created previously.

The following screenshot shows a dashboard that we built for our tenant customers. When they log on,

they see the overall status of each of their installations and a separate list of specific devices that have

had problems.



Tenants can select any of their devices and all of the information that Zenoss Service Dynamics has

collected for the device, including any events and historical performance information.



Finally, tenants can run custom reports against their devices and export the data to Excel.

We can also integrate monitoring data into our customer portal by using the product’s API set. A set of

APIs was specifically built for portal integration, which requires code writing. The following screenshot

shows a sample of what can be built from the portal integration APIs.



7 Assurance Commercial Overview

Zenoss software is available as a hosted service or as an on-premises installation at a service provider.

Pricing for the two methods is identical and is provided as a yearly subscription. The price of the

subscription varies depending on the number of managed resources and is adjusted up or down as the

number of managed resources changes.

With the hosted service, Zenoss supplies the computing resources, installs and maintains the software,

and is responsible for backup and recovery actions. A 99.9% availability SLA is included in the offer. This

is an attractive option for service providers who want to focus on the business services around Managed

EdgeCloud.

With on-premises installation, the service provider is responsible for providing computing resources and

ongoing software management. This is an attractive option for service providers who want to maintain

control over their customers’ monitoring data records and/or to extensively customize the software

operations.

The software calculates the number of managed resources in use at any time and provides that

information in a license report. For a Managed EdgeCloud unit, the number of managed resources is the

sum of the following items:

• VM count

• Windows OS/hypervisor count

• UCS fabric interconnect

• UCS servers

• NetApp FAS

• Cisco ISR

For example, a Managed EdgeCloud Standard unit has up to 237 managed resources, as the total of the

following components:

• Up to 225 VMs

• 4 Windows server/hypervisors

• 2 UCS fabric interconnects



• 4 UCS servers

• 1 NetApp FAS 2552

• 1 Cisco ISR 4431

A unit on which the customer is not running the full complement of VMs has fewer managed resources.

The subscription price for a service provider is based on the total number of managed resources in use

across all customer sites and includes a volume discount. The service provider’s subscription price is set

on a quarterly basis according to the number of managed resources in use at the beginning of the

quarter. This is particularly attractive for a service provider that is experiencing rapid growth.

Zenoss Service Dynamics is available on premises or as a cloud service (Zenoss as a service). Typically,

Zenoss experts assist in the initial deployment, particularly for the on-premises solution.

7.1 Zenoss Service Dynamics: On Premises

For hundreds of organizations, Zenoss Service Dynamics (installed on premises) is the right solution for

their needs. The advanced, service-centric unified monitoring capabilities that Zenoss Service Dynamics

provides, coupled with reliability, ease of management, and low cost, allows customers to successfully

monitor and manage their IT services.

7.2 Zenoss as a Service: Cloud-Based Hosting

If you prefer a hosted solution, Zenoss as a service (ZaaS) enables you to take advantage of Zenoss

Service Dynamics capabilities within minutes.

As a hosted service, ZaaS allows your team to focus on delivering value instead of just keeping the lights

on. Because ZaaS is a hosted service, you do not have to worry about deploying Zenoss Service

Dynamics, managing or upgrading your Zenoss installation, or architecting your IT monitoring

infrastructure for high availability. The Zenoss team takes care of all that for you.

With ZaaS, you’ll always have the latest version of the Zenoss Service Dynamics platform monitoring your

environment, along with an SLA that ensures 99.9% availability. Deploying ZaaS for Managed EdgeCloud

is particularly easy. Simply deploy a lightweight Zenoss virtual appliance for each tenant site in your

environment and start monitoring.

8 Conclusion

Zenoss Service Dynamics is an effective monitoring system for service providers selling Managed

EdgeCloud solutions.

First, it offers a single source for monitoring data for all the hardware and software elements of a

Managed EdgeCloud. Storage, compute, networking, and virtualization are all in one place. In addition to

monitoring, it shows the logical connections among all elements. This makes your staff efficient in tracking

how your service is performing and makes investigating and resolving issues much faster.

Second, you can focus monitoring on your customers and their SLAs, not just on the equipment in their

facility, an important distinction when you’re selling a service and not just shipping boxes.

Third, it’s easy to grow the system to additional customers, and just as easy to shrink it. One VM is all it

takes to monitor a new customer site. The ability to change scale lowers business risk. You do not need

to overinvest at the beginning, and you will not be stuck with fixed costs that are too high if you lose any

customers.

And finally, if you want true simplicity, you do not have to operate the Zenoss system yourself. With their

hosted offering, Zenoss can do that you. This includes everything from providing the necessary compute

resources to running backups and providing disaster recovery. Your staff can focus entirely on customers.



Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exact product and feature versions described in this document are supported for your specific environment. The NetApp IMT defines the product components and versions that can be used to construct configurations that are supported by NetApp. Specific results depend on each customer’s installation in accordance with published specifications.

Copyright Information

Copyright © 2017 NetApp, Inc. All rights reserved. Printed in the U.S. No part of this document covered by copyright may be reproduced in any form or by any means—graphic, electronic, or mechanical, including photocopying, recording, taping, or storage in an electronic retrieval system—without prior written permission of the copyright owner.

Software derived from copyrighted NetApp material is subject to the following license and disclaimer:

THIS SOFTWARE IS PROVIDED BY NETAPP “AS IS” AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL NETAPP BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

NetApp reserves the right to change any products described herein at any time, and without notice. NetApp assumes no responsibility or liability arising from the use of products described herein, except as expressly agreed to in writing by NetApp. The use or purchase of this product does not convey a license under any patent rights, trademark rights, or any other intellectual property rights of NetApp.

The product described in this manual may be protected by one or more U.S. patents, foreign patents, or pending applications.

RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.277-7103 (October 1988) and FAR 52-227-19 (June 1987).

Trademark Information

NETAPP, the NETAPP logo, and the marks listed at http://www.netapp.com/TM are trademarks of NetApp, Inc. Other company and product names may be trademarks of their respective owners.

http://mysupport.netapp.com/matrix

http://www.netapp.com/TM

oss for managed edgecloud with zenoss - netappa partner solution, zenoss service dynamics, that...

Documents