ucs manager technical deep dived2zmdbbm9feqrf.cloudfront.net/2013/eur/pdf/brkcom-2001.pdfucs manager...

© 2013 Cisco and/or its affiliates. All rights reserved. BRKCOM-2001 Cisco Public

UCS Manager Technical Deep Dive

BRKCOM-2001


Agenda

• Introduction

• UCS Architecture, Innovations, Topology

• Physical Building Blocks

• Logical Building Blocks

• Typical Use Cases (Live UCS Demo)

• Focus on new 2.1 operational and networking features

3


Introduction

• This session will not cover UCS hardware components

‒ Please check out BRKCOM-1005 UCS System Architecture Overview for that

• This session will not cover Storage best practices

‒ Please check out BRKCOM-2002 UCS Storage Best Practices for that

• This session will not cover VM FEX in detail

‒ Please check out BRKCOM-2005 UCS Fabric and VMs for that

• On the other hand, this session will cover UCSM quite a bit!

What to expect

4


UCS Architectural Goals

5

• Decouple admin overhead from

number of managed endpoints

Deepest possible abstraction of

hardware and connectivity

Reduce server provisioning time

Fully programmable platform


Unified Computing System (UCS)

Single Point of Management

Unified Fabric

Stateless Servers with Virtualized Adapters


UCS Manager Embedded– manages entire system

UCS Fabric Interconnect 48 or 96 Port 10Gb FCoE with Unified Ports

UCS Fabric Extender Remote line card

4 x 10GE or 8 x 10GE

UCS Blade Server Chassis Flexible bay configurations

UCS Server Industry-standard architecture Blade and rack-mount, 2 and 4 socket

UCS Virtual Adapters Choice of multiple adapters

UCS Physical Building Blocks

7


Server provisioning pain points

• Provisioning new servers to deploy or scale an application is time

consuming

• Customers standardize on equipment for a given app

‒ amount of memory, number of NICs, HBA vendor, boot characteristics, etc.

• Deploying a new application typically involves placing an internal request

to the server team which needs to locate the appropriate server, rack

mount it, cable everything, install an OS, ensure connectivity and pass the

ball back to the app team

‒ typically this takes days or weeks

• How can we simplify this?


Decouple hardware config from hardware

Server Name UUID Boot Order

Boot Parameters WWN/WWPN MAC Address

SAN Attributes LAN Attributes


Introducing Service Profiles

• UCS Servers are somewhat similar to VMs

• Configuration files store server characteristics:

‒ Boot parameters

‒ NIC and HBA configuration (MAC, WWN, etc.)

‒ UUID

‒ Various policies (what happens when a link fails,

etc.)

• All those items can be grouped in pools

• Those service profiles are then mapped to

physical servers to very quickly provision one

or more appropriate servers

PHYSICAL SERVERS

Server Profiles

Server Name

UUID

MAC

WWN

Boot info

LAN Config

SAN Config


Stateless Computing

• Server identity no longer has to be tied

to physical server hardware

Profiles provide identity

Seamless server mobility

Stateless blades

• Boot over network (LAN or SAN)

Boot order and boot devices are part of the

pre-defined logical server profile

On-board disks can be used for temp,

swap, etc.

Server Name: LS-A

UUID: 56 4d cd 3f 59 5b 61…

MAC : 08:00:69:02:01:FC

WWN: 5080020000075740

Boot Order: SAN, LAN

SAN LAN

Chassis-1/Blade-1

Chassis-9/Blade-5


Pools

• UUID

• MAC

• WWNN / WWPN

Policies • Boot Devices/Order

• Host F/W

• QoS

• Local Disk

• …

Templates

• VNIC

• VHBA

Templates

• Service Profile

Service Profile

“Web-1”

Service Profile

“Web-2”

Service Profile

“Web-N”

. . .

Network

• VLANs

• VSANs

• Pin Groups

UCS Logical Building Blocks

12

• Connectivity


How to create Service Profiles

Templated

Manual

Identity

(MAC Address, WWN, Etc.)

Behavior

(Firmware, QoS, Etc.)

Connectivity

(vNICs, vHBAs,)

Configuration Methods

Service Profile Physical Blade

Cloned


Concept of resource pools

blade pool

policies

qualifications blade pools blades

P1

P3

C1S1

C1S1

C2S3

C3S4

C1S1

C4S8

adapters

memory

storage

adapters

CPU memory

P2


More modularity: service profile policies

Ethernet Adapter Policy

Tx- queues

Rx – queues

CoS

BIOS Policy

Quiet boot

Hyper-threading

VT-technologies

Characteristics:

• Consumed at instantiation of

Service Profile (SP)

• Linked to specific

components of SP

• Are usable through multiple

Organizations

• Allow update of policy to

apply to those SPs that

consume it


Templates, policies and Service Profiles

Service Profile Template:

Web Server

• UUID: WebUUPool

• Boot Order: WebBootPolicy

• Host Firmware: WebFWPolicy

• SOL: SOLPolicy

• WWNN: WebWWNNPool

Adapter Policy: Eth0 VLAN: Web_FrontEnd Fabric: A MAC: WebMacPool-A

Adapter Policy: fc0 VSAN: Web-A Fabric: A WWPN: WebWWPNPool-A

Service Profile: Prod-Web-S1

Service Profile:Prod-Web-S12

• UUID: 10:02:03:04:05

• Boot Order: VM / SAN

Target 50:00:00:34:FA

• Host Firmware: 9.0.03.146

• WWNN: 20:00:00:00:10

• WWPN: 20:00:00:00:20

• MAC: 20:03:CA:00:05

• UUID: 10:02:03:04:F3

• Boot Order: VM / SAN

Target 50:00:00:34:FA

• Host Firmware: 9.0.03.146

• WWNN: 20:00:00:00:50

• WWPN: 20:00:00:00:40

• MAC: 20:03:CA:00:FA


Management Interface: UCSM

17

• Java-based GUI • Java Webstart client

download • Single pane of glass

for all UCS-related tasks

© 2013 Cisco and/or its affiliates. All rights reserved. BRKCOM-2001 Cisco Public 18

Today: release 2.1 (“Del Mar”)


UCS Manager: behind the scenes

UCSM Management Information Model

Summary:

• Data Management Engine (DME)

contains full representative object

model stored in the Management

Information Tree (MIT)

• Interaction with DME occurs via

Northbound External Interface Layer

• Northbound interfaces all resolve

down to XML API

• Application Gateways couple UCSM

with end points

UC

SM

UCS CLI

Data Management Engine MIT

Management Controller Application

Gateway

Nic AG

Host Aggr

Port AG

Dcos AG

NXOS(A) UCS 6100

NXOS(B) UCS 6100

IOM (CMC) UCS 2100

B200/B250 (BMC)

MEZZ_CARD M*KR

UCS UOS

telnet, ssh, netconf

Smash CLP Component

Native XML API

Custom Application GUI


Northbound Connections

UC

SM

Data Management Engine

Transaction Services

Management Information Tree

Tools e.g. Curl,

Poster plugin

Visore UCSM GUI External Management Application

Validation

XM

L

HTT

P

H

TTP

S

Behavior and Orchestration Services

Object Services

XM

L

HTT

P

H

TTP

S

XM

L

HTT

P

H

TTP

S

XM

L

HTT

P

H

TTP

S


UCS Management Information Tree (MIT)

All physical & logical components that

comprise UCS are represented in a

hierarchical management information

model (MIT)

Each node in the tree represents one

or more managed objects that

contains its configuration and

operational state.


UCS MIT - Naming

• Data in the MIT is referenced hierarchically

• Relative Name (RN) is typically a combination of object type prefix + naming property value for that object.

• E.g. for blades object type prefix=blade, the naming property is ID, so if the ID of this blade is 1, the RN=blade-1

• chassis-1

• blade-8

• fan-module-8

• Adaptor-1

• Distinguished Names (DN) are built from a combination of RNs.

• sys/chassis-1/blade-6/adaptor-1


DME: Applying A Change in State

All Modifications Are: • Validated Centrally

• Deployed

Asynchronously

• Success or Failure

Communicated by

asynchronous

Notifications to the

Event Stream.


UCS: a model-driven framework

• UCSM information model: logical abstraction of hw/sw components in UCS

• The DME modifies objects in the MIT first

• Physical endpoints are modified after by AGs

• This approach separates business logic from platform implementation

• Makes it easy to add components or write code without worrying about hardware idiosyncrasies for example


Overview of HA in UCS

L1 to L1

L2 to L2

Mgmt Plane: active / standby

Data Plane: active/active


It’s demo time!


Live UCS Demonstration (No PPT!)

• UCSM overview

• Templates, pools, policies and SPs

‒ what’s new in 2.1 (incremental allocation, SP renaming, LAN conn policies)

‒ fault management: clearing up any confusion

• Networking refresher

‒ what’s new in 2.1 (VLAN groups, IGMP, etc.)

• Quick overview of XML API

‒ Powershell toolkit, Developer Connection, Platform Emulator

27


Annex: new 2.1 features


Release 2.1: over 15 new features

29


Operational Enhancements

• Service Profile Renaming

• FSM Tab Enhancements

• Fault Suppression

• ID Pool Sequential Allocation

• UCSM Upgrade Validation

• UCSM Auto Install

• Scheduled backups


• Users will have the ability to rename a Service Profile in its current place in the org tree

• Old IDs (e.g., UUID, MAC, WWN) are retained

• New audit logs, events, and faults reference the new name

• Existing audit logs, events, and faults from the original Service Profile remain available under the original name

• No server reboot is necessary

• Only Service Profiles can be renamed. Nothing else.

Customers can simplify operations by aligning

Service Profile names with their naming conventions

Customer benefits

Feature details

Service Profile renaming

Operational

Enhancements


FSM Tab Enhancement: GUI

• GUI now shows all FSM Stage details in additional to overall FSM status details.


Sequential ID Pool allocation

Much better control over SP to ID pools (MAC, WWN, etc.) allocation!


UCSM Upgrade Validation

• Starting with Del Mar (2.1) it is possible to do a dry run of system upgrades

• Automatic rollback in case of problem


Firmware auto-install

• 5 different features actually:

• Host Firmware Package creation using simple mode

• Deprecation of Management Firmware Package

• Removal of ignore compatibility check

• Firmware Monitoring

• Firmware Auto Install

• Install Infrastructure Firmware

• Install Server Firmware


• Administrators can utilize a simple wizard-like interface to specify which version of firmware they would like to upgrade either their infrastructure or servers to

• Sequencing of firmware updates is handled automatically to ensure the least downtime

• Intermediate user acknowledgement during fabric upgrade allows users to verify that elements such as storage are in an appropriate state before continuing the upgrade

Users can utilize a tool to automatically sequence

and apply upgrades to each system endpoint, simplifying the process

Customer benefits

Feature details

Firmware Auto Install

Operational

Enhancements


• Fault suppression offers the ability to lower severity of designated faults for a maintenance window, preventing Call Home and SNMP traps during that period

• Predefined policies allow a user to easily place a server into a maintenance mode to suppress faults during maintenance operations

Customers can align UCSM fault alerting with their

operational activities

Customer benefits

Feature details

Fault suppression

Operational

Enhancements

User can start or schedule fault suppression on a per SP basis or Org-level!


Fault Suppression Overview

1. Users can now “Start/Stop Fault Suppression” in order to suppress

transient faults and Call-home/SNMP notifications

2. Supported on both physical (Chassis, Server, IOM, FEX) and logical

entities (Org, Service Profile)

3. Users can specify a time window during which fault suppression will

take into effect

4. A fault suppression status indicator to show different states (Active,

Expired, Pending)

5. Fault Suppression policies that contain a list of faults raised during

maintenance operations will be provided. Users cannot create or

customize policies.


Fault Suppression behavior

1. When fault suppression is started

Fault is marked with different severity (conditioned) &

lifecycle (suppressed)

Suppressed faults will not be shown to user by default

Call-home/SNMP notifications suppressed

2. When fault suppression is stopped

Any outstanding faults will be..

re-evaluated

Call-home/SNMP notifications sent


• Org-aware VLANs

• VLAN compression

• VLAN groups

• IGMP Querier

• Flexible networking resources consumption

‒ Exclude network (LAN/SAN) aspects from server admin privilege

‒ LAN & SAN connectivity policies

Networking Features


• VLANs today are not confined to specific Orgs

• They are available as a free-for-all resource

‒ Any SP vnic can use any VLAN

• 2.1 changes this with Org-aware VLANs

‒ Opt-in feature: by default, an Org has access to all VLANs

‒ If enabled, Org is limited to specific VLANs

Org-aware VLANs


VLAN Permissions in a Org Hierarchy

org::Org name=“root”

org::Org name=“eng”

org::Org name=“hr”

org::Org name=“dev”

org::Org name=“personnel”

org::Org name=“benefits”

fabric::VlanReq name=“one”

fabric::VlanReq name=“three”

fabric::VlanReq name=“three”

fabric::VlanReq name=“two”

{} * All VLANs

{} * All VLANs

{} * All VLANs

{one,two}

{one,two,three}

{three}


Org-aware VLANs


Org-aware VLANs

• We created a VLAN called ORGWARE with Org membership = DEMO

• SPs that are not in Org DEMO do not see the ORGAWARE VLAN


• Optimizes (P,V) to increase supported limit • Increase Scale from 32K to up to 64K* on the

6248/6296

• *Caveat: -Actual Limit dependent on environment/configuration

• Enable higher scale in logical ports (also called P,V ports) • Important for customers with:

• ESX DVS or Nexus 1kV • Multi-tenant – Service Provider/Enterprise • High # of Servers with high # of VLAN –

GET/Enterprise/Commercial/SLED

Customer benefits

Feature details

Logical Port Count Optimization Networking

User will be able to enable or disable (default) feature in global policies


VLAN Compression – Use case

• Cloud provider, all virtualized clusters

• vSwitch based deployments

• Larger set of VLANs in each group eg : 1-500 in cluster 1 and 501-1000 in cluster 2

• Dedicated NAS array/VLANs per cluster or customer. No VLAN overlap between clusters.

• Example

‒ 5 clusters of 32 ESX hosts (160 hosts total)

‒ Each ESX host has 4 static vNICs (2 per fabric)

‒ Each cluster has 150 VLANs that do not overlap across clusters

‒ Each vNIC carries 150 VLANs

‒ Number of VIFs: 160 x 4 = 640

‒ 8 Ethernet border ports carrying all the VLANs

‒ Number of VLANS: 5 x 150 = 750

‒ PV count = 160 x 2 x 150 + 8 * 750 = 54,000


Memory – CPU Trade Offs

• VLAN compression automatically groups VLANs that have the same L2

forwarding rules

• VLAN Compression helps to reduce the CPU load on switching

processes during operational state transitions such as port flap

• Because the CPU load is lower, (P,V) limit increased to 64,000

• Because of memory and CPU requirements, can only be done on the

6200 platform


VLAN Compression

VLAN Compression

Disabled

VLAN Compression

Enabled

Springfield - 6100 14,000 NA

Mammoth - 6200 32,000 64,000

Support for higher P,V Count Limit on 6200 Platform


VLAN Groups

• Simplify disjoint VLANs

provisioning

• Support org-based VLAN

ownership

• A VLAN Group has a name

• Defined under LAN Cloud

• Created by the network admin

• Contains zero or more VLAN

name references

VLAN Group G1

Lan Cloud

VLAN Group G2

VLAN a

VLAN b

VLAN c

VLAN d

VLAN a

VLAN b

VLAN c

VLAN d


VLAN Groups

• VLAN Groups can be applied to Ethernet uplink ports

• VLAN membership can be defined once and the group can then be

applied to multiple Ethernet uplink ports.

• VLAN overlaps between disjoint VLANs can be detected easily

• When VLAN membership changes in a VLAN group, VLANs are uniformly

reconfigured across all applicable uplink ports

• Optionally, VLAN Group has a name reference to a native VLAN


Disjoint VLANs

• Network admin creates VLAN

Group

• Add VLANs (name references) to

group

• Assign uplink port to Group

• Raise faults if there are VLAN

overlaps

• Existing Capitola 2.0 Disjoint

features are retained for backward

compatibility

VLAN Group G1

Lan Cloud

VLAN Group G2

VLAN a

VLAN b

VLAN c

VLAN d

VLAN a

VLAN b

VLAN c

Eth1/2

Eth1/1

Eth1/4

Eth1/3


Uplink Port Assignment to VLAN Groups

• The diagram below shows an example with disjoint VLANs

• Instead of assigning each uplink port to each individual VLAN, the user

can assign uplink ports to VLAN groups

VLAN Group 1

VLAN 100-200

VLAN Group 2

VLAN 300-600

BIF 1 BIF 2 BIF 3 BIF 1 BIF 2 BIF 3

UCS A UCS B

SP1


IGMP Querier

• IGMP sent by hosts to express interest in mcast group

• IGMP snooping lets L2 switches learn location of hosts and send mcast

to those interest (avoid flooding)

• IGMP querier lets switch/router periodically probe hosts for group

membership (avoids potential issues where IGMP snooping entry

disappears because of quiet/silent host)

• Before 2.1, IGMP snooping always on and not user configurable

• Before 2.1, no IGMP querier function natively available on UCS


IGMP Querier

• With 2.1, several new options (end-host mode and switch mode)

‒ IGMP snooping on, Querier on

Queries generated by FI sent downstream only (not on uplink ports) unless FI in switch mode

‒ IGMP snooping on, Querier off

Standard default behavior. Requires external mcast router or querier.

‒ IGMP snooping off, Querier off

FIs flood mcast traffic

‒ IGMP snooping off, Querier on

Invalid and blocked by UCSM


IGMP Querier


New server-admin role

• Goal: limit server-admin role to purely server related tasks. Don’t expose

networking (LAN/SAN)

• Works in tandem with new LAN and SAN Connectivity Policies


LAN/SAN Connectivity Policies

• Determine how LAN or SAN are configured

• Server-admin can reference LAN/SAN conn. policies when creating SPs

• Policies always in updating mode: changes are immediately reflected


Miscellaneous


UCS Manager GUI: front-end to the XML API

• All GUI interactions result

in XML messages

• Useful for crafting XML

messages for integration

to 3rd party tools

Windows XP Pro

C:\Documents and Settings\Username\Application Data\Sun\Java\Deployment\log\.ucsm

Windows 7

C:\Documents & Settings\Username\AppData\LocalLow\Sun\Java\Deployment\log\.ucsm

Mac

\home_directory\Library\Caches\Java\log\.ucsm


UCS Manager GUI

“Copy XML” to capture XML output for individual elements


Built-In XML Object Browser

http://<ucsm>/visore.html


developer.cisco.com

• UCS Powershell Library

• UCS Python SDK

• UCSPE: UCS Platform Emulator that fully exports the XML API

• SNMP/MIB

• SCOM

62


Call to Action

• Visit the Cisco Campus at the World of Solutions to experience Cisco innovations in action

• Get hands-on experience attending one of the Walk-in Labs

• Schedule face to face meeting with one of Cisco’s engineers

at the Meet the Engineer center

• Discuss your project’s challenges at the Technical Solutions Clinics

63