sun logical domain quick start guide

Part No 821-0316-11Revision 1.1 1/13/10

SUN™ LogicaL DomaiNSQuick Start GuidePeter A. Wilson, Systems Technical MarketingGary Combs, Systems Technical Marketing

Sun Microsystems, Inc.

Table of contents

Sun™ Logical Domains Quick Start guide ...........................................................1

Straightforward use case ..................................................................................... 2

How this article is organized ................................................................................ 3

initial Server Setup .........................................................................................5

Before you begin ................................................................................................. 5

Gather documentation .................................................................................... 5

Assign IP addresses ............................................................................................. 5

Set up the service processor ................................................................................ 6

Obtaining the DHCP-assigned address .............................................................. 6

Assigning a static IP address ............................................................................ 7

Check firmware revision ...................................................................................... 8

Update system firmware ..................................................................................... 8

Configure the Solaris OS .................................................................................... 10

Command-line setup ..................................................................................... 10

Browser user interface setup ......................................................................... 10

Apply operating system patches ......................................................................... 11

Download the Logical Domains software ............................................................ 11

Install Logical Domains Software ....................................................................... 12

control and i/o Domain Setup ....................................................................... 13

Initial server state ............................................................................................. 13

Create a virtual disk server ................................................................................ 14

Create a virtual switch ...................................................................................... 14

Create a virtual console device .......................................................................... 15

Free CPU, MAU, and memory resources .............................................................. 15

Save the configuration in the service processor ............................................... 16

Virtual Disk Backend Setup ............................................................................ 17

Virtual disk options ........................................................................................... 17

Locate or create a disk partition ......................................................................... 17

guest Domain creation ................................................................................. 20

Allocate resources to a guest domain ................................................................. 20

Create the guest and allocate resources ............................................................. 21

Boot from the guest domain’s console ................................................................ 22

Use Solaris ZFS snapshots and clones ................................................................. 23

Housekeeping .............................................................................................. 25

Save configurations to the service processor ....................................................... 25

Use proper shutdown procedures ....................................................................... 25

Removing guest domains .................................................................................. 25

Removing Logical Domains software .................................................................. 26


Summary ..................................................................................................... 27

About the authors ............................................................................................. 28

Acknowledgments ............................................................................................ 28

Ordering Sun documents ................................................................................... 28

Accessing Sun documentation online ................................................................. 29

Sun Microsystems, Inc.1 Sun Logical Domains Quick Start Guide

.

Chapter 1

Sun™ Logical Domains Quick Start guide

Sun™ Logical Domains, or Sun LDoms, is Sun’s built-in, no-cost hardware

virtualization technology for servers and blade systems powered by UltraSPARC®

T1, T2, and T2 Plus processors. These systems are equipped for massive scalability

using chip multithreading (CMT) technology in which each of up to eight processor

cores support up to eight concurrent execution threads each for a total of up to

64 threads per processor and up to 256 threads per server. The systems based on

these processors deliver excellent performance for highly threaded applications,

along with the lowest power consumption per thread of the current generation of

processors.

LDoms allows server resources to be partitioned and allocated to virtual machines.

Resources that can be partitioned include individual CPU threads (virtual CPUs),

the processor’s cryptographic acceleration modular arithmetic units (MAUs),

memory, PCI root nexus nodes (or “PCI buses”), and network interface units (NIUs).

Architecturally, LDoms is a Type 1 hypervisor because the technology runs on a bare

machine and provides each domain with the illusion that it is running on its own

dedicated hardware — in this case hardware having the SPARC® sun4v architecture.

With LDoms, the control domain manages the hypervisor. I/O is performed through

virtual devices to one or more I/O domains that actually perform physical I/O on

behalf of a domain (these are sometimes referred to as service domains). Figure

1 illustrates a server with a control domain that manages the hypervisor, two

I/O domains that perform physical I/O, and three guest domains. In a typical

installation, a combined control and I/O domain performs both functions .

Hypervisor

Sun CMT ServerPCI A PCI B

LogicalDomain 1

Guest OSImage

ControlDomain

LogicalDomain

Manager

LogicalDomain 2

Guest OSImage

LogicalDomain 3

Guest OSImage

I/ODomain 1

Solaris™10 OS

I/ODomain 2

Solaris10 OS

Figure 1. Logical Domains supports multiple guest domains each having their own secure partition of server resources. Physical I/O is performed by I/O domains that ‘own’ one or more PCI root nexus nodes, or buses.

When this document was prepared, Sun™

Logical Domains 1.2 was the current version

and the document applies to versions 1.2

and greater. Take care to use the latest

recommended versions of LDoms, the

Solaris™ Operating System, and firmware.

Recommended sets of software can be

found in the Logical Domains Release Notes

for the current version.


Where Logical Domains differs significantly from other Type 1 hypervisors on the

market today is that it allows physical server resources (such as CPUs, memory,

MAUs, and even PCI root nexus nodes) to be partitioned, not time shared, between

domains. This provides each domain with dedicated resources and increases isolation

between domains. Partitioning helps to avoid issues such as one domain depriving

other domains of needed resources. Partitioning potentially increases cache

performance by fixing the domain’s relationship to a CPU in contrast to timesharing.

While separating CPU and memory resources is prudent from a resource allocation

standpoint, LDoms also allows I/O resources to be shared in flexible ways to allow

both sharing and separation of resources to improve performance, reliability, and

availability.

Straightforward use caseThe purpose of this Sun BluePrints™ article is to provide a simple set of steps for

setting up and running a server with Logical Domains software. This guide is not

intended to be a comprehensive beginner’s guide to LDoms. Instead, it intends to

cover the use case of using Logical Domains in a laboratory environment for the first

time. Thus, we assume a minimal amount of supporting infrastructure, specifically a

desktop or laptop computer connected via network to the server. We do not require,

for example, the use of a network install server.

This article was originally written to describe Logical Domains 1.2 software. The

procedures shown in this document apply to version 1.2 and later, and they may

show references to Logical Domains 1.2 software. We recommend using the latest

version of Logical Domains software for your platform together with the latest

recommended versions of Solaris, patches, and firmware as described in the

applicable Logical Domains Release Notes.

Documentation for all Logical Domains versions is available from http://docs.

sun.com/app/docs/prod/ldoms.

We assume that your environment contains the following:

Server• . A Sun SPARC Enterprise® server based on UltraSPARC T2 or T2 Plus

processors and at least one internal disk drive.

Operating system• . The server runs the version of the Solaris™ Operating System

recommended in the Logical Domains Release Notes for the current LDoms

version.

Network environment• . Both the server’s lights-out management port and its

primary (NET 0) interfaces are connected to a network. The server and its service

processor ports are assigned static IP address.

Serial port• . The server’s serial port should be accessible from the workstation or

laptop either directly or through a terminal concentrator.


Workstation or laptop• . A workstation or laptop is needed to access the server. It

should have a Web browser with a suitable Java™ software plugin.

These assumptions do not cover UltraSPARC T1 processor-powered servers such as the Note – Sun SPARC Enterprise T1000 and T2000 servers. This guide focuses on administering servers through Sun’s Integrated Lights-Out Management (ILOM) software using both its command-line interface and browser user interface (BUI). For UltraSPARC T1 server platforms that use Sun’s older lights-out management software, ALOM commands will need to be used instead.

How this article is organizedAs this article will demonstrate, it takes only seven commands to set up a guest

domain using Sun™ Logical Domains software using a given virtual disk image:

# ldm create guest1# ldm set-vcpu 4 guest1# ldm set-mem 4g guest1# ldm add-vnet vnet0 primary-vsw0 guest1# ldm add-vdsdev /ldompool/domain1/diskimagefile vol1@primary-vds0# ldm add-vdisk vdisk0 vol1@primary-vds0 guest1# ldm bind guest1

There is, however, some preparation needed to set up a new server, update firmware,

install Logical Domains software, and set up virtual devices and disks. That is the

bulk of this article that takes you through the steps from providing a server’s initial

network identity to booting your first logical domain. Although many of these steps

can be condensed by using the Logical Domains Configuration Assistant mentioned

in the sidebar, the purpose of this document is to convey an understanding of the

process and mechanics of setting up a system to run Logical Domains software and

creating guest domains on the system.

Chapter 2, “Initial Server Setup,” walks you through the tasks that you’ll need •

to perform just once: giving your server a network identity, updating the system

firmware, initializing the Solaris OS, and downloading and installing the Logical

Domains Manager software.

Chapter 3, “Control and I/O Domain Setup,” describes how to turn the OS •

running on the server into a control and I/O domain in the context of Logical

Domains software. The chapter shows how to set up the virtual disk and network

infrastructure used by guest domains.

Chapter 4, “Virtual Disk Backend Setup,” describes the benefits of using Solaris •

ZFS™ software to support virtual disks, and it walks you through the steps involved

in carving out a spare disk partition to use as a Solaris ZFS file system.

Chapter 5, “Guest Domain Creation,” shows how to create a logical domain and •

connect it to the virtual infrastructure created in Chapters 3 and 4.

Even these seven steps can be shortend by

using the Logical Domains Configuration

Assistant described in Appendix D of the

Logical Domains Administration Guide.

In a single command, the ldmconfig

command can set up a virtual disk backend

file, create, and start one or more guest

domains.


Chapter 6, “Housekeeping,” describes how to store your current configuration in •

the service processor and to follow proper shutdown procedures. It explains how

to remove the guest domain you’ve created and restore your server to its initial

state.

Chapter 7, “Summary,” summarizes the seven commands that can be used to •

create new logical domains and suggests directions for additional exploration.


Chapter 2

initial Server Setup

This chapter covers the steps that you need to take in order to get your server

connected to the network with current firmware installed.

Before you beginBefore you begin, make sure that your server is set up so that it meets the basic

assumptions under “Straightforward use case” in the previous chapter. This guide

assumes that you have a new, out-of-the-box server with current firmware and the

version of the Solaris OS required by the Logical Domains Release Notes is installed.

gather documentationEven though this guide is intended to give you a complete, self-contained set of

instructions for setting up a first domain using Logical Domains software, it is still

useful to have the basic LDoms documentation on hand to answer questions and

provide more background. Visit http://docs.sun.com/app/docs/prod/

ldoms to download appropriate versions of the following documents:

Logical Domains Release Notes• . This document will give you any last-minute news

on the release and any unresolved issues. You should not run into any issues

following the steps in this guide.

Logical Domains Administration Guide• . This document explains how to administer

Logical Domains software, illustrating a large number of scenarios that are beyond

the scope of this document.

Logical Domains Reference Manual• . This document provides all of the options for

all Logical Domains-related commands in a concise format.

assign iP addressesYou will need three IP addresses as you work through this guide, one for each of the

following uses:

Service processor Network Port. This is the port labeled NET MGT.•

Control domain. This is the address of the primary domain on the server, the one •

that boots with no Logical Domains software installed. This is typically the port

labeled NET 0.

Guest domain. This is the address that the guest domain will use when it connects •

to the network, and for the purpose of this example is the same NET 0 port as the

control domain.

If you need to upgrade your operating

system image, perform these initial setup

steps before performing the installation.


Obtain these addresses, along with the appropriate netmasks, and router IP

addresses, from your network administrator. Putting the control domain and guest

domain IP addresses on the same subnet makes it easy to set up communication

between the control domain and the guest domain.

Set up the service processorThe first step is to connect to the service processor serial port and assign an IP

address to the service processor’s network port. This procedure shows you how to

accomplish this using ILOM commands. With power applied to the server, connect to

the serial port and press Return to obtain a login prompt.

obtaining the DHcP-assigned addressBy default, the service processor attempts to obtain an address from a DHCP

server, if one is available. If your network is supported with DHCP, you will need to

determine the address that was assigned so that you can connect via the browser

user interface later on.

Log in with the user root and the initial password changeme. Then use the show

/SP/network command to view the assigned address. In this example, the

address is 10.6.162.25.

For help connecting to the server’s serial

port, please refer to the administration

guide for your server available at

http://docs.sun.com.


SUNSP00144F4641BD login: rootPassword: changemeWaiting for daemons to initialize...

Daemons ready

Sun(TM) Integrated Lights Out Manager

Version 3.0.2.50

Copyright 2008 Sun Microsystems, Inc. All rights reserved.Use is subject to license terms.

Warning: password is set to factory default.

-> show /SP/network

/SP/network Targets:

Properties: commitpending = (Cannot show property) dhcp_server_ip = none ipaddress = 10.6.162.25 ipdiscovery = static ipgateway = 10.6.163.254 ipnetmask = 255.255.252.0 macaddress = 00:14:4F:46:EE:A1 pendingipaddress = 10.6.162.25 pendingipdiscovery = static pendingipgateway = 10.6.163.254 pendingipnetmask = 255.255.252.0 state = enabled

Commands: cd set show

assigning a static iP addressIf you wish to assign a static IP address to the service processor, or override the

address allocated by DHCP, follow these steps.

Log in with the user root and the initial password changeme. Then type the

following commands to set the service console’s IP address and other parameters:

.


.

SUNSP00144F4641BD login: rootPassword: changemeWaiting for daemons to initialize...

Daemons ready

Sun(TM) Integrated Lights Out Manager

Version 3.0.2.50

Copyright 2008 Sun Microsystems, Inc. All rights reserved.Use is subject to license terms.

Warning: password is set to factory default.

-> set /SP/network state=enabled-> set /SP/network pendingipdiscovery=static-> set /SP/network pendingipaddress=your-ip-address-> set /SP/network pendingipnetmask=your-netmask-> set /SP/network pendingipgateway=your-router-address-> set /SP/network commitpending=true

Now that the service processor has been set up to have a network identity, you can

choose to make further configuration steps through the browser user interface or

through the serial port.

check firmware revisionAt the service processor’s command prompt, type the following command line to

obtain the system software version:

-> show /HOST sysfw_version

You can determine your firmware revision from the command output. Firmware

revision numbers are sequential for each server type, so consult http://www.

sun.com/bigadmin/patches/firmware to determine the most current revision.

Locate your server model and see if the listed system firmware revision is greater

than the one installed on your server. If it is, download the latest firmware onto your

workstation and uncompress it. The result should be a package file whose name

ends in .pkg. If your have the latest firmware installed, skip the next section.

Update system firmwareYou can both check firmware revisions and update firmware through the service

console’s browser user interface (BUI) by directing your browser to http://

service-processor-address. Click the System Information tab, then the

Versions tab, and you’ll see a display similar to that shown below.


.

The server power must be turned off before you update the firmware. To turn off

the power, click the Remote Control tab, then the Remote Power Control tab. Select

Graceful Shutdown and Power Off from the pop-up menu and click Save.

To update the service processor firmware, click the Maintenance tab, then click

Firmware Upgrade. Click the Enter Upgrade Mode button.

Now browse to where you have stored the firmware package on your desktop or

laptop, and then click the Upload button.


.

The upgrade takes several minutes. Do not interrupt the process.

configure the Solaris oSIf you haven’t already initialized the Solaris OS that is installed on the server, now

is the time. You will need to go through a sequence of steps to give the operating

system its identity, including its name, time zone, and network settings. A new

Solaris OS instance will ask you for this information the first time that it boots.

You need to first power on the server, and then execute the initialization steps

through a command-line interface (via the service console) or through the BUI.

command-line setupTo work from the command line, you can use an ssh connection and log in to the

service processor’s network port. Use the ILOM start /SYS command to turn

on the system’s power, and then connect to the server console through the service

processor.

-> start /SYSAre you sure you want to start /SYS (y/n)? yStarting /SYS-> start /SP/consoleAre you sure you want to start /SP/console (y/n)? y

Serial console started. To stop, type #.

Note that the power-on sequence takes some time due to the power-on self tests

that are conducted. Once the Solaris OS boots for the first time, it will prompt you

for information including your language, time zone, and network settings.

Browser user interface setupThe other way to manage power and connect to the operating system console is

through the BUI. First power on the server by selecting the Remote Control tab and

then the Remote Power Control tab. Power on the server by selecting Power On from

the pop-up menu and clicking Save.


.

To connect to the server console, select the Remote Control tab and then select

Launch Redirection. You can bring up a serial console by selecting I Want To

See Serial Redirection. If you choose the serial console, you will need to use the

sequence of commands under “Command-line setup” above to turn the server power

on. You can bring up a graphical user interface by leaving the option unchecked.

Either approach will allow you to go through the sequence of setting up the system

identity.

Now that you have the Solaris OS initialized and on the network, you can connect to

the server via ssh, through the service console (serial port or ssh), or through the

BUI (graphical or serial console)

apply operating system patchesRefer to the Logical Domains Release Notes for a list of required operating system

patches. Download and apply the patches.

Download the Logical Domains softwareNow that your operating system and system firmware is up to date, download the

Logical Domains software. Visit http://www.sun.com/ldoms, click the Get It tab,

and download the current software. For Logical Domains 1.2 the file downloaded was

LDoms_Manager-1_2.zip.


.

install Logical Domains SoftwareNow connect to the server console to get a login prompt from the Solaris OS. You

can do this by ssh, or by connecting to the service processor (via serial port or ssh)

and then typing start /SP/console to connect to the server console.

Transfer the Logical Domains software to the server, and uncompress the zip file.

# unzip LDoms_Manager-1_2.zip

Navigate to the location of the install-ldm script and run it to install the Logical

Domains Manager software.

# cd LDoms_Manager-1_2# cd Install# ./install-ldm

The Logical Domains installation script will ask if you want to harden the server

operating system using the Solaris Security Toolkit. This is an excellent step to follow

for production systems as security of the control domain is important for protecting

guest domains. For the purpose of this quick install exercise, skip the hardening step

proceed with a standard Solaris OS configuration.

At this point you have a server with current firmware, an initialized Solaris operating

system instance, and running Logical Domains software. The next step is to convert

that OS instance into a control and I/O domain.

It is good security practice to prevent the

root user from directly logging into a server

via ssh, which is the default behavior.

In a laboratory setting, it is sometimes

expedient to override this default. To do so,

edit /etc/ssh/sshd_config. Change

the PermitRootLogin property from no

to yes. Then, restart the ssh daemon with

the command svcadm restart ssh.


.

Chapter 3

control and i/o Domain Setup

initial server stateA Sun CMT server supports the sun4v architecture, which is implemented in part

by a thin hypervisor layer that is always interposed between the operating system

and the hardware. The hypervisor is always present regardless of whether Logical

Domains software is used. The initial server state is illustrated in Figure 2.

Internal Storage

PCIeSlots

CPU Threads,MAUs, Memory

32 GB Memory

Built-inEthernet

1 128

Control & I/O Domain

PCI Buses

Primary

Hypervisor

Sun CMT Server

Solaris 10 OS andLogical Domains

Manager

128 CPU threads

16 MAUs

Figure 2. In the server’s initial state, a single control domain owns all of the system resources including CPU threads, MAUs, memory, and PCI buses. The control domain is also referred to as the primary domain.

The example server has two eight-core UltraSPARC T2 Plus processors. The server

has two PCI buses that connect to built-in Ethernet interfaces, internal disk drives,

and PCI Express slots for additional peripherals. Each CPU has 64 threads and eight

modular arithmetic units (MAUs) that accelerate encryption operations.

Verify the server’s initial state using the ldm list command. You’ll note that a

single domain primary owns 128 virtual CPUs (threads) and 64 GB of memory

# ldm listNAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIMEprimary active -n-c-- SP 128 65312M 0.2% 3d 15h 41m

The end state of the next several steps is illustrated in Figure 3 and consists of the

following:


.

A virtual disk server • primary-vds0 is ready to connect virtual disks in guest

domains to a service in the primary domain that serves data to them.

A virtual switch • primary-vsw0 is ready to connect virtual network devices in

guest domains to a virtual switch supported in the control and I/O domain. The

virtual switch is connected to a physical Ethernet port.

The control and I/O domain has given up most of its CPU, MAU, and memory •

resources into an unallocated pool that will provide resources to guest domains.

Built-inEthernet

Disk Storage Internal Resources

nxge0

4 CPU Threads

4 GB Memory

124 CPU Threads

16 MAUs

24 GB Memory

Unallocated Resources


Primary

Hypervisor

Sun CMT Server

Solaris 10 OS & LDM

primary-vsw0

primary-vsw0

primary-vds0

primary-vds0

1 124

Figure 3. The end state of this set of steps frees resources and creates virtual device infrastructure to support guest domains.

create a virtual disk serverThe virtual disk service primary-vds0 will support disk I/O in the guest domain;

actual sources of data will be attached to it later.

# ldm add-vdiskserver primary-vds0 primary

create a virtual switchUse the ifconfig command to determine the name of the physical network device

through which the primary domain communicates to the network. In this example,

the device is nxge0.


.

# ifconfig -alo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000nxge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.6.162.45 netmask fffffc00 broadcast 10.6.163.255 ether 0:14:4f:46:ee:98

Now use the knowledge of that interface to create a virtual switch primary-vsw0

that is connected to the physical interface.

# ldm add-vswitch net-dev=nxge0 primary-vsw0 primary

create a virtual console deviceA virtual console device allows the control and I/O domain to connect to the console

port of the guest domain. When connecting to the guest’s console, you will use the

command telnet localhost port, where port is in the range of 5000-5100

that the virtual console device will assign.

# ldm add-vconscon port-range=5000-5100 primary-vcc0 primary

Free cPU, maU, and memory resourcesThe following commands free CPU, MAU, and memory resources, which are currently

allocated to the primary domain. These resources are released into a free pool from

which the resources can be allocated to guest domains as we create them. The

commands leave the primary domain with four virtual CPUs, no MAU, and 4 GB of

main memory. Changing these resources puts the Logical Domains Manager into

delayed reconfiguration mode where the actual resources are changed only after the

primary domain is rebooted.

# ldm set-mau 0 primaryLDom primary is unable to dynamically reconfigure crypto units.Initiating delayed reconfigure operation on LDom primary. All configurationchanges for other LDoms are disabled until the LDom reboots, at which timethe new configuration for LDom primary will also take effect.# ldm set-vcpu 4 primary------------------------------------------------------------------------------Notice: LDom primary is in the process of a delayed reconfiguration.Any changes made to this LDom will only take effect after it reboots.------------------------------------------------------------------------------# ldm set-mem 4g primary------------------------------------------------------------------------------Notice: LDom primary is in the process of a delayed reconfiguration.Any changes made to this LDom will only take effect after it reboots.------------------------------------------------------------------------------


.

Save the configuration in the service processor

When making configuration changes, always remember to save the final state in the Note – service processor.

For the hypervisor to choose this new hardware configuration when the system

is powered on, the configuration must be stored in the service processor. In this

example, we store the changed state in the named location my-initial.

# ldm add-spconfig my-initial# ldm list-spconfigfactory-defaultmy-initial [current]

Now reboot the primary and I/O domain so that the delayed configuration takes

effect.

# shutdown -i6 -g0 -y


.

Chapter 4

Virtual Disk Backend Setup

Virtual disk optionsA guest domain needs to boot from a virtual disk, and the storage for that disk

is most easily allocated from an internal disk drive on the server. The storage for

a virtual disk is called a virtual disk backend, and it can store data in any of the

following locations:

A flat file stored on a UFS, Solaris ZFS, or other suitable file system•

A physical disk with or without underlying hardware RAID•

A partition on a physical disk•

There are significant advantages to using a flat file stored on a Solaris ZFS file

system. You can use snapshots to create a point-in-time image of a virtual disk for

use in backups or roll back a guest domain to a previous, known state. You can use

clones as a quick and space-efficient way to create multiple guest domains from the

same “golden master” Solaris OS installation. When more than one disk device is

dedicated to a ZFS file system pool, various levels of redundancy can be established

to further protect data.

This set of steps illustrates how you can create a ZFS pool from a single disk

partition. After creating the guest domain we will demonstrate how easy it is to

make a clone of your first guest domain for use in creating subsequent guests.

Locate or create a disk partitionWe have assumed that you have at least one internal disk drive on your server, and

that you have a factory-installed OS that is on a UFS file system. This sequence of

steps locates available disk space on one disk and creates a partition for use as a

Solaris ZFS file system.

Use the format command and choose the disk on which you wish to allocate a

partition as a virtual disk backend.

This example repartitions disk 0, where the Solaris OS is usually installed. Take Note – care when allocating unused space on this volume not to destroy your running Solaris OS installation for the control domain.

Depending on the underlying storage, this

configuration may create a single point of

failure. Techniques for increasing reliability

are discussed in the Sun BluePrints articles

“Data Reliability with Logical Domains” and

“Data Availability with Logical Domains.”


.

# formatSearching for disks...done

AVAILABLE DISK SELECTIONS: 0. c1t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848> /pci@400/pci@0/pci@8/scsi@0/sd@0,0 1. c1t1d0 <SEAGATE-ST914602SSUN146G-0603-136.73GB> /pci@400/pci@0/pci@8/scsi@0/sd@1,0

Specify disk (enter its number): 0

The format command will present a menu. Choose p to edit the partition table.

FORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels save - save new disk/partition definitions inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return quitformat> p

The format command presents the partition table for the selected disk. The swap

partition uses disk cylinders 0–402. The root partition uses cylinders 403–1912. The

backup partition represents the whole disk, which is 14087 cylinders. We can create

a partition from any of the unassigned partitions. We choose partition 6, starting the

partition at cylinder 1913 (one after the root partition) and make it 12174 cylinders

in size (cylinders 1913–14087). Select 6 to begin this partitioning process.

Current partition table (original):Total disk cylinders available: 14087 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks 0 root wm 403 - 1912 14.65GB (1510/0/0) 30731520 1 swap wu 0 - 402 3.91GB (403/0/0) 8201856 2 backup wm 0 - 14086 136.71GB (14087/0/0) 286698624 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

partition> 6


.

Allocate the remaining space with a partition that begins after the root partition

and extends to the end of the disk. Then choose l to label the disk and q to quit the

partition command.

Part Tag Flag Cylinders Size Blocks 6 unassigned wm 0 0 (0/0/0) 0

Enter partition id tag[unassigned]:Enter partition permission flags[wm]:Enter new starting cyl[0]: 1913Enter partition size[0b, 0c, 1913e, 0.00mb, 0.00gb]: 12174cpartition> pCurrent partition table (unnamed):Total disk cylinders available: 14087 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks 0 root wm 403 - 1912 14.65GB (1510/0/0) 30731520 1 swap wu 0 - 402 3.91GB (403/0/0) 8201856 2 backup wm 0 - 14086 136.71GB (14087/0/0) 286698624 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 1913 - 14086 118.14GB (12174/0/0) 247765248 7 unassigned wm 0 0 (0/0/0) 0

partition> lReady to label disk, continue? y

partition> q

Create a ZFS file system pool ldompool from the c1t0d0s6 partition, and create

the domain1 file system within the pool. By default, ZFS pools are mounted in the

root directory by the name of the pool, and ZFS file systems appear underneath

as directories. Create an empty file with the mkfile command to hold the guest

domain’s disk image. Using the -n option creates the file but does not allocate all of

its data. This example creates a 20 GB file.

# cd / # zpool create -f ldompool c1t0d0s6# zfs create ldompool/domain1# mkfile -n 20g /ldompool/domain1/diskimagefile


.

Chapter 5

guest Domain creation

allocate resources to a guest domainThe previous chapters have freed up and created resources that now can be allocated

to a guest domain easily. The end state reached in this chapter is illustrated in

Figure 4, and it has the following characteristics:

A guest domain • guest1 has four virtual CPUs and 4 GB memory allocated to it.

The domain in which it runs has the following:•

A virtual network device connecting it to the network•

A virtual disk connected to the virtual disk backend we created in •

the previous chapter

A virtual • cdrom device connected to a downloaded Solaris OS ISO image

Once the guest domain is started, it boots from the virtual cdrom device and begins

an installation of the Solaris OS.

Built-inEthernet

Disk Storage Internal Resources

nxge0

4 CPU Threads

4 GB Memory

4 CPU Threads

120 CPU Threads

16 MAUs

24 GB Memory4 GB Memory


Unallocated Resources

Solaris 10 OS


Primary Guest 1

Hypervisor

Sun CMT Server

Solaris 10 OS & LDM

/solaris-s10u6-dvd.iso

primary-vsw0

primary-vsw0

primary-vds0

cdromvdisk0

primary-vds0

ldompool/domain1/diskimagefile

vnet0

1 24

Figure 4. The end state of this quick-start guide is a guest domain with both physical and virtual resources connected to it.


.

create the guest and allocate resourcesCreate the guest domain with the ldm create command and allocate physical

resources including CPU threads (virtual CPUs) and memory. In this example, four

virtual CPUs and 4 GB of memory is allocated to the domain guest1.

# ldm create guest1# ldm set-vcpu 4 guest1# ldm set-mem 4g guest1

Create a virtual network device vnet0 in guest1 and connect it to the virtual

switch that was created in Chapter 3.

# ldm add-vnet vnet0 primary-vsw0 guest1

Attach the flat file /ldompool/domain1/diskimagefile to the virtual disk

service primary-vds0 with the name vol1. Then make that volume available to

the guest1 domain with the name vdisk0.

# ldm add-vdsdev /ldompool/domain1/diskimagefile vol1@primary-vds0# ldm add-vdisk vdisk0 vol1@primary-vds0 guest1

The Logical Domains Manager does not check to be sure that a file name you provide Note – is valid. The only indication that you have typed an invalid file name is a virtual disk that does not work in the guest domain.

Download a Solaris OS ISO image from http://www.sun.com/solaris/get.

Connect the downloaded ISO image to the virtual disk service primary-vds0 with

the name cdrom. Then make the volume available to the guest1 domain with the

name cdrom.

# ldm add-vdsdev /solaris-s10u6-dvd.iso cdrom@primary-vds0# ldm add-vdisk cdrom cdrom@primary-vds0 guest1

Each domain runs its own instance of OpenBoot™ software. You can set the domain’s

boot device so that it first tries to boot from a disk, and then from the virtual cdrom

device. On the first boot attempt, booting from the disk will fail and the domain

will boot from the cdrom device. Once the Solaris OS is installed, the next attempt

to boot from the disk will be successful. Note that you must boot from the cdrom

device’s f partition as shown in the following command.

# ldm set-variable boot-device="disk cdrom:f" guest1

Commit the resources that have been allocated using the bind command.

# ldm bind guest1


.

You can verify that the resources have been allocated using the

ldm list-bindings command.

# ldm list-bindings guest1NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIMEguest1 bound ------ 5000 4 4G

MAC 00:14:4f:fb:e3:4a

HOSTID 0x84fbe34a

VCPU VID PID UTIL STRAND 0 4 100% 1 5 100% 2 6 100% 3 7 100%

MEMORY RA PA SIZE 0xe000000 0x10e000000 4G

VARIABLES boot-device=cdrom:f

NETWORK NAME SERVICE DEVICE MAC MODE PVID VID vnet0 primary-vsw0@primary network@0 00:14:4f:fa:c0:ad 1 PEER MAC MODE PVID VID primary-vsw0@primary 00:14:4f:f9:6a:97 1

DISK NAME VOLUME TOUT DEVICE SERVER MPGROUP vdisk0 vol1@primary-vds0 disk@0 primary cdrom cdrom@primary-vds0 disk@1 primary

VCONS NAME SERVICE PORT guest1 primary-vcc0@primary 5000

Boot from the guest domain’s consoleUse the ldm list command to determine the port number for guest1’s console.

Observe that the console port is 5000.

# ldm listNAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIMEprimary active -n-cv- SP 4 4G 0.8% 5mguest1 bound ------ 5000 4 4G

Enable the virtual console service (you only need to do this one time), and then start

the guest domain.


.

# svcadm enable vntsd# ldm start guest1LDom guest1 started

Now telnet to guest1’s console. The domain should be in the process of booting

from the Solaris ISO image, and it should begin an interactive Solaris OS installation.

If you do not have automatic booting enabled, you can use commands such as

show-devs and show-nets at the OpenBoot software’s ok prompt to verify that

the devices you created are available to the guest domain.

A good practice is to use a separate ssh connection to the control domain for each Note – guest domain console connection you make. When you connect to the control domain through the service processor, only one connection can own the console, which limits the number of concurrent telnet sessions you can observe. When you create a new ssh connection to the control domain for each guest domain console connection, you don’t use a limited resource. Thus, response time is better because the connection is not limited by the console serial port speed.

# telnet localhost 5000Trying 127.0.0.1...Connected to localhost.Escape character is ‘^]’.

Connecting to console “guest1” in group “guest1” ....Press ~? for control options ..

Once the installation is complete, you have a guest domain that runs using its

own partition of the server’s physical resources, supported by the virtual resources

provided through the control and I/O domain. You can boot the domain, shut it

down, and interact with the OpenBoot software via the guest’s console.

Be sure to read on to the next chapter and save the current configuration to the

service processor.

Use Solaris ZFS snapshots and clonesSince you’ve created the guest’s virtual disk on a Solaris ZFS file system, you can take

advantage of the ability to take snapshots and make clones of the file system.

For example, suppose that you don’t want to go through the Solaris OS installation

process for every new guest domain that you create. You can create a golden master

virtual disk image that you can clone as many times as needed for each new logical

domain that you create. One of the benefits of using snapshots and clones in Solaris

ZFS file systems is that they are very space efficient. Every disk block in a clone is a

reference to the original golden master, and space is allocated only as needed using

copy-on-write semantics.

Note that, by default, the control domain

cannot send network traffic to guest

domains. Therefore, you cannot ping or

ssh from the control domain to the guest.

To enable networking between the control

and guest domains, refer to the instructions

“Enabling Networking Between the Control/

Service Domain and Other Domains” in the

Logical Domains Administration Guide.

Details on using Solaris ZFS technology

with Logical Domains are available in

Chapter 6 of the Sun BluePrints article

“Data Reliability with Logical Domains.”

The Logical Domains System Administration

Guide also makes suggestions in the section

“Using ZFS with Virtual Disks” in Chapter 6,

“Using Virtual Disks.”


.

To create a clone, first snapshot a Solaris ZFS file system and then make a clone of it.

The cloning process makes a permanent copy of the snapshot. The clone uses copy-

on-write semantics so that the clone is an independent, read/write copy.

Solaris ZFS file system snapshots operate on file systems, not files, so it is a good

practice to put only one virtual disk backend file in each Solaris ZFS file system.

In the following example, we create a snapshot of guest1’s file system named

initial. Then we clone the snapshot and create a new file system golden at the

same time.

# zfs snapshot ldompool/domain1@initial# zfs clone ldompool/domain1@initial ldompool/golden

The file /ldompool/golden/diskimagefile is a copy of /ldompool/

domain1/diskimagefile except that it occupies almost no disk space until blocks

in the file are changed. If you attach this golden master virtual disk backend to a

logical domain, however, it will have the identity and network settings of the guest1

logical domain you have already created.

You can remove the golden master’s identity with the sys-unconfig command,

however note that this will not remove many useful changes you can prepare in the

golden image, such as patches, applications, and configuration files. This will cause

the Solaris OS to go through the system identity setting prompts upon first boot so

that each clone of the golden master obtains its own identity. To accomplish this,

perform the following steps:

Shut down the 1. guest1 logical domain.

Prepare a new logical domain 2. guest2 using /ldompool/golden/

diskimagefile as its virtual disk backend.

Boot 3. guest2 and run the sys-unconfig command. The command will

unconfigure the system and halt it.

Snapshot the unconfigured state of the golden master file system.4.

# zfs snapshot ldompool/golden@goldenmaster

Now you can start 5. guest1 without any identity conflicts on the network. You

can start guest2 and give it an identity. To create new logical domains based

on the unconfigured golden master, use the zfs clone command using the

snapshot ldompool/golden@goldenmaster as the source.


.

Chapter 6

Housekeeping

Save configurations to the service processorWhenever you change a Logical Domains configuration, be sure to save the

configuration to the service processor. If you don’t do this, and you reboot the

control domain or cycle the power on the server, the service processor will only

re-instate the last saved configuration. This configuration may be as old as the

factory-default configuration. You can save up to eight named configurations

in the service processor. The last one saved is always the one used at the next boot

unless you explicitly set the desired configuration with the ldm set-spconfig

command. One way to save the configuration is to delete and re-create the

configuration that you have already saved, for example:

# ldm remove-spconfig my-initial# ldm add-spconfig my-initial

Logical Domains software includes a configuration auto-recovery mechanism in case

the configuration was not explicitly saved to the service processor.

Use proper shutdown proceduresThe Logical Domains Release Notes contain important information about how

to properly shut down a server with multiple logical domains. Note that it is not

sufficient to simply shut down the control domain as guest domains will continue to

run. The procedure is to do the following:

Shut down and unbind all non-I/O domains. In this example, you could use the 1.

command ldm stop guest1 to gracefully shutdown the guest domain, then

unbind with ldm unbind guest1.

Shut down and unbind any active I/O domains. In this example, there is no 2.

secondary I/O domain supporting redundant I/O paths. If there were, you would

use the two commands from Step 1.

Shut down and halt the primary domain. You can use the command 3. shutdown

-i5 -g0 -y. Following the shutdown the server will power off since there are

no longer any active domains running.

Removing guest domainsYou can use the following sequence of commands to shut down, unbind, and destroy

a logical domain if you no longer need it.


.

# ldm stop guest1LDom guest1 stopped# ldm unbind-domain guest1# ldm destroy guest1

Removing Logical Domains softwareIf you wish to restore your server to its original state, not running any logical

domains, you can execute the following sequence of commands.

# ldm rm-config my-initial# ldm stop-domain -aLDom guest1 stopped# ldm unbind-domain guest1# ldm destroy guest1# ldm set-config factory-default# svcadm disable ldmd# svcadm disable vntsd# pkgrm SUNWldm# shutdown -i5 -g0 -y

If you have enabled networking between the control domain and guests, you

may have to manually restore the previous hostname files (for example /etc/

hostname.nxge0) and remove the hostname files that plumbed virtual switches

into the control domain (for example /etc/hostname.vsw0).


.

Chapter 7

Summary

Once you have installed the correct server firmware and the Logical Domains

Manager software, it is simple and straightforward to create new logical domains

given a virtual disk image. As demonstrated in Chapter 1, starting with a populated

virtual disk image file, only seven commands are needed to create a guest domain.

The ldm create command creates the guest. The ldm set-vcpu and ldm set-

mem commands allocate virtual CPUs and memory to the guest. The ldm add-

vnet command gives the guest a network connection. The ldm-add-vdsdev

command attaches a virtual disk to the virtual disk service, and the ldm add-

vdisk command attaches the service to the guest. The ldm bind command

activates the resource allocation.

# ldm create guest1# ldm set-vcpu 4 guest1# ldm set-mem 4g guest1# ldm add-vnet vnet0 primary-vsw0 guest1# ldm add-vdsdev /ldompool/domain1/diskimagefile vol1@primary-vds0# ldm add-vdisk vdisk0 vol1@primary-vds0 guest1# ldm bind guest1

Sun Logical Domains provides an enterprise-class virtualization environment that can

provide many more powerful security, availability, and management tools than are

available on physical machines:

Security features such as virtual local-area networks (VLANs) translate directly into •

logical domains. In addition, virtual switches can be created to manage each of

the physical interfaces on a server, increasing security and network throughput.

Data reliability techniques, such as maintaining multiple copies of business-•

critical data, translate easily to Logical Domains environments.

Availability can be enhanced in the virtual world in a way similar to how it is done •

in the physical world: with multiple redundant paths to networks and disk storage

through independent I/O domains.

I/O performance can be enhanced by allowing guest domains to access I/O •

devices directly by assigning ownership of individual PCI buses to them (on servers

with more than one PCI bus). Application performance can be enhanced through

the ability to dynamically change the number of virtual CPUs and amount of

memory assigned to a domain without requiring a reboot.

Guest domains can be created quickly and easily using Solaris ZFS file system •

features including snapshots and clones, while using minimal space for new

virtual disks.


.

Guest domains can be moved between servers via warm or cold migration, •

simplifying datacenter workload management, and simplifying the process of

taking a physical server down for upgrades or maintenance.

Many of these features are described in Sun BluePrints articles available at

http://www.sun.com/blueprints, or in the LDoms Community Cookbook

available at http://wikis.sun.com/display/SolarisLogicalDomains/

LDoms+Community+Cookbook. With an initial guest domain up and running

on a Sun CMT server, an abundance of resources is available to support further

exploration and enterprise deployment.

about the authorsPeter A. Wilson has more than 16 years of industry experience, 14 of which have

been with Sun, serving in a wide variety of hardware, software, systems and product

marketing roles. Peter moved from the United Kingdom to the US in 2000 to lead

the customer tests of Sun’s Netra™ and fault-tolerant servers. Peter is currently a

technical marketing manager responsible for a team of Product Specialists covering

all of Sun’s hardware product portfolio. Peter holds a M.Eng (Master of Engineering)

degree in Microelectronics and Software Engineering from the University of

Newcastle-upon-Tyne, U.K.

Gary Combs is currently a SPARC Specialist in the Systems Technical Marketing Group.

He specializes in mid-range high-end SPARC servers, which includes the popular

M-Series as well as the higher end CMT servers, such as the Sun SPARC Enterprise

T5440 server. Gary also covers virtualization technologies that are implemented on

these platforms: Dynamic Domains, Logical Domains, and Solaris Containers. Gary

has over 15 years direct sales support experience as a systems engineer. For the

last 10 years, Gary has held marketing positions in product management, product

definition, and now technical marketing.

acknowledgmentsThe author would like to thank Steve Gaede, an independent technical writer and

engineer, for preparing this article. Thanks also to Alexandre Chartre, Cathleen

Reiher, and Pierre Reynes for giving the document a thorough review.

ordering Sun documentsThe SunDocsSM program provides more than 250 manuals from Sun Microsystems,

Inc. If you live in the United States, Canada, Europe, or Japan, you can purchase

documentation sets or individual manuals through this program.


.

accessing Sun documentation onlineThe docs.sun.com web site enables you to access Sun technical documentation

online. You can browse the docs.sun.com archive or search for a specific book title or

subject. The URL is

http://docs.sun.com/

To reference Sun BluePrints OnLine articles, visit the Sun BluePrints OnLine Web site:

http://www.sun.com/blueprints/online.html


Sun Microsystems, Inc. 4150 Network Circle, Santa Clara, CA 95054 USA Phone 1-650-960-1300 or 1-800-555-9SUN (9786) Web sun.com

Logical Domains Quick Start Guide

© 2008-2010 Sun Microsystems, Inc. All rights reserved. Sun, Sun Microsystems, the Sun logo, BluePrints, Java, OpenBoot, Solaris, SunDocs, and ZFS are trademarks or registered trademarks of Sun Microsystems, Inc. or its subsidiaries in the United States and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the US and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc. Information subject to change without notice. Printed in USA 1/10

sun logical domain quick start guide

Documents