event monitoring service version2

7/27/2019 Event Monitoring Service Version2

1/104

Using the Event Monitoring Service

Manufacturing Part Number: B7609-90022

E0603

C opyright 1997, 2003 H ewlett -P a cka rd Company .


2/104

2

Legal NoticesTh e in for m a t ion con ta i ned in th is d ocu m en t i s sub ject to ch a n ge w i th ou t

notice.

H ewl ett-Packard makes no warr anty of any ki nd wi th regard to thi s

manu al, in clu di ng, but not l i mi ted to, the im pl i ed w arr ant i es of

mer chan tabi l i ty and fitness for a par ti cul ar pur pose. Hewlett-Packardsha ll not be liable for errors cont a ined herein or direct, ind irect, s pecial,

incidental or consequential damages in connection with the furnishing,

performa nce, or use of this ma teria l.

Warranty

A copy of th e speci fi c w a r r a n ty ter m s a p plica b le to y ou r Hew let t -Pa cka r d

pr od uct a n d r epla cem en t pa r t s ca n be ob ta in ed f rom y ou r l oca l Sa l es a n d

Service Office.

Restricted Rights Legend

U se, duplicat ion or disclosure by t he U.S. G overnm ent is su bject t o

restr ictions a s set fort h in subpa ra gra ph (c) (1) (ii) of t he Right s in

Technical D a ta a nd C omputer S oftw a re clause a t DFARS 252.227-7013

for DOD a gencies, a nd s ubpa ra gra phs (c) (1) a nd (c) (2) of the

Commercial C omputer S oftwa re Restricted Rights clause a t FAR52.227-19 for ot her a gen cies.

H E WLE TT-PACK ARD DE VELOP ME NT COMP ANY, L.P.

2055 S.H . 249

H oust on, Texa s 77070

U se of this document a nd a ny support ing softw a re media supplied for

th is pa ck is restr icted to t his product only. Additiona l copies of th epr ogr a m s m a y be m a d e f or secu r it y a n d ba ck-u p pu r poses on ly. R esa le of

the programs, in t heir present form or with a lterat ions, is expressly

prohibited.


3/104

3

Copyright Notice

Copyright 1997-2003 Hew lett -Pa ckard Development Company, L.P. All

rights reserved. Reproduction, a da pta tion, or t ra nslat ion of this

document w ithout prior wr itt en permission is prohibited, except a s

allowed un der the copyright la ws.

High Ava i lab i li ty Monitors, Event Moni tor ing Service, HP OpenView, HP

OpenView I T/Opera tions, Ser viceGu a rd E xten sion for RAC, a nd

MC/ServiceGua rd a re products of Hewlett -P a cka rd D evelopment

Compan y L.P., and a ll a re protected by copyright .

Trademark Notices

X Window Syst em is a tr adema rk of the Massa chusetts I nstitut e of

Technology.

MS-DOS a nd Microsoft a re U.S. registered tr a dema rks of Microsoft

Corporation.OSF /Motif is a tr a dema rk of th e Open Softw a re Founda tion, Inc. in th e

U.S. an d other count ries.


4/104

4


5/104

Contents

5

1. Understanding the Event Monitoring Service

E vent Monitoring S ervice Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

E MS R equirement s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

E MS R esource Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Client a nd Ta rget Applica tions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

EMS with ServiceGua rd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

EMS G U I C lient Applicat ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

EMS CLI Client Applica tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

EMS a nd Ta rget Applica tions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Resource Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Inst a lling Resource Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Confi guring R esource Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Writin g Resource Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

EMS Fra mework Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

The E MS API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

The Registr a r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

The Resource Dictiona ry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2. Defining a Monitoring Request

St a rting a Monitoring Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Specifying w hen to send Event Notifi cat ions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Sett ing the P olling Int erval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Set ting Event Va lue Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Selecting P rotocols for S ending E vents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

The opcmsg (ITO) Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

tcp an d udp Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

The snm p Tra ps Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

The ema il Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

The console Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

The syslog Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

The text log Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Adding a Notifi cat ion C omment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3. Changing Monitoring Requests

Copying Monitoring Req uests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Modifying Monitoring Request s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Removing Monitoring Req uests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Viewing Monitoring Request s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48


6/104

Contents

6

4. Monitoring ServiceGuard Package Dependencies

5. Monitoring Network Interfaces

Netw ork Monitor Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Configur ing Network Monitoring Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

6. Selecting Resources to Monitor

St a rting t he Event Monitoring Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Selecting Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Viewing Resource Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

7. Monitoring System Resources

Sy stem Monitor Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Number of Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

J ob Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Filesystem Ava ilable Spa ce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Crea ting S ystem Resource Monitoring Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

8. Monitoring Cluster Resources

Clust er Monitor Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Cluster St a tus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Node St a tus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Pa cka ge St a tu s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Service Sta tus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Crea ting Clust er Monitoring Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

MIB Monitor C omma nd-Line Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

E MS D irectories an d Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Logging a nd Tra cing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

EMS Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Log File Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84EMS Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Debug logging of the E MS fra mew ork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

U sing resls to Check EMS Communica tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

U sing resls -s to Check the S ta tus of an E MS Resource. . . . . . . . . . . . . . . . . . . . . . . 88

EMS Pers istence Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

High Ava ilability Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89


7/104

Contents

7

EMS Tra cing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

P erforma nce Considera tions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Sy stem P erforma nce Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Network Performa nce Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Testing Monitor Request s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Testing C luster Monitor Request s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Testing Net w ork Monitor Requ ests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Testing S yst em Resource Monitor Request s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Making Sur e Monitors are Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

MIB Monitor Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

St eps to Obta in EMS D a ta to Reproduce an E MS P roblem . . . . . . . . . . . . . . . . . . . . . 96


8/104

Contents

8


9/104

9

Printing History

This edition document s ma teria l rela ted t o using th e Event Monitoring

Service to crea te monitoring req uests for syst em resources.

The printing da te chan ges wh en a n ew edition is printed. (Minor

corrections and updates which are incorporated at reprint do not causeth e d a te to ch a n ge. ) Th e pa r t n u mber i s r ev ised w h en ex ten s ive tech n ica l

changes are incorporated.

New editions of this ma nua l will incorpora te a ll mat erial updat ed since

th e previous edit ion.

HP P rint ing Division:

Business Critical Computing Business Unit (BCC)

Hewlett-Packard Co.

19111 Pruneridge Ave.

Cupertino, CA 95014

Table 1

Printing Date Part Number Edition

Ma rch 1999 B 7612-90009 E dit ion 1

November 1999 B 7612-90015 E dit ion 2

J une 2003 B 7609-90022 E dit ion 3


10/104

10


11/104

11

PrefaceTh is guid e d escr ibes h ow to u se th e Even t M on itor in g Serv ice (EM S) a n d

h ow to con fi gu r e th e M a n a gem en t I n for m a t ion Ba se (M I B) m on itor s. Th e

MIB monitors check a nd report sta tus on cluster, netw ork, a nd syst em

resources.

AudienceHP intends this manua l for system mana gers or a dministra tors

responsible for confi guring a nd m aint a ining the E vent Monitoring

S ervice (E MS ) on H P -U X 11i Versions 1 a nd 2.

This ma nua l is based on t he assumption tha t y ou ha ve:

An understanding of distr ibuted netw ork concepts and c lient-server

computing D em on st ra t ed kn ow led ge of UNIX

An un der st a n din g of E MS b a sics

EMS functions a t va rious levels independently a nd w ith:

H i gh a va i la b ili t y softw a r e such a s Ser viceGu a r d

Enterpr ise management products such as IT/O

The cont ents of this guide a re a s follow s:

U ndersta nding the E vent Monitoring S ervice

Selecting Resources to Monitor

Defi ning a Monitoring Request

Cha nging Monitoring Requests

Monitoring ServiceGuard Package Dependencies

Monitoring Clust er Resources

Monitoring Network Interfaces

Monitoring S yst em Resources

Dictionary File Command Line Options

Troubleshooting


12/104

12

Related

Publications

The follow ing document s cont a in a dditiona l relat ed informa tion:

Using H igh Avai l abi l i ty M oni tors

EM S Hardwar e M oni tors Users Guid e

M anaging M C/ Serv iceGuard

Configur in g OPS Clusters wi th Ser viceGuar d OPS Ed it ion

M anagin g Systems and Work groups

P eter Weyga nt , Clusters for H igh Avai l abi l i ty: A Pri mer of HP-UX

Solut ions(IS B N 0-13-089335-2). H P P ress: P rent ice Ha ll, In c., 2001

Tom Ma dell, Di sk and Fi le M anagement Tasks on H P-UX( ISBN

0-13-518861-X). H P P ress; P rent ice Ha ll, I nc., 1997

HP OpenVi ew I T/ Oper ati ons Admni str ators Refer ence(HP Pa r t

Nu mber B 6941-90001)

M anaging H ighly Avai lab le NFS

http://docs.hp.com Website for information about

Hewlett-Packards high-availability technologies where you can find

documents.Select HP-UX then High Availability.

www.software.hp.comWebsite for designing a nd building a n E MS

monitor. Select H igh Ava ilability t hen E vent Monitoring S ervice

Developers Kit.

ProblemReporting If you have a ny problems w ith t he softwa re or documenta tion, plea secontact your local Hewlett-Packard Sales Office or Customer Service

Center.


13/104

Understanding the Event Monitoring Service

Chapter 1 13

1 Understanding the EventMonitoring Service

The E vent Monitoring Service (E MS) is a fra mew ork for r esource

monitoring. EMS is used by syst em a dministra tors to monitor system

resources including confi guring, checking resource sta tus, a nd sendingnotificat ion w hen confi gured conditions a re met.

This cha pter conta ins t he following s ections:

E vent Monitoring S ervice Overview

EMS Requirements

EMS Resource Classes

Client a nd Ta rget Applica tions

Resource Monitors

EMS Fra mework Components


14/104


Event Monitoring Service Overview

Chapter 114


The E vent Monitoring Service (E MS) monitors syst em resources. U se

EMS to confi gure monitoring request s, check resource sta tu s, and send

notificat ion w hen confi gured conditions a re met.

EMS can work in a h igh ava ila bili ty environment. I t ca n report a loss of

redunda nt r esources. Ident ifying an d report ing single point s of failurehelps ma inta in a proactive approa ch in preventing t he loss of dat a an d

availability.

EMS only observes a system, a nd does not modify t he system. U se EMS

wit h a dditiona l softwa re to ta ke or specify an a ction.

The th ree bas ic components of EMS a re:

C lien t a n d Ta r get Appl ica t i on s

Sy stem a dminist ra tors use client a pplica tions to set, modify, or

remove monitoring requests. Syst em adm inistrat ors use ta rget

a pplica tions to receive event notifi cat ions a nd possibly ta ke actions.

Client a pplica tions include ServiceGua rd (MC/ServiceGua rd or

ServiceGua rd E xtension for RAC), the E vent Monitoring Service

(EMS) GU I (Gra phica l User In terf ace), the Event Monitor ing Service

(EMS ) CLI (Comma nd Line Int erface), or other a pplica tions tha t

comply with t he EMS API .

Th e ta r get a p pl ica t i on ca n be a n y a p plica t i on th a t suppor t s th e EM S

protocols. The s upported protocols a re:

TCP /IP or U DP /IP

This includes a ny a pplica tion th a t a ccepts t hese protocols a nd

follow s th e rules defin ed in the E MS D evelopers Kit .

opcmsgmethod (for ITO)

This opt ion is used for IT/Opera tions notifi cat ions.

SNMP t ra ps

Th is opt ion ca n be u sed w it h a n y a p plica t ion t h a t a ccept s S N MP

t r a ps, s uch a s N NM or I T/O. You n eed t o s et u p t h e a pplica t ion t o

recognize the SNMP tra ps generat ed.

ema il


15/104



Chapter 1 15

This option does not require a ny extr a ha ndling. Specify th e

email addr ess when t he monitoring request is creat ed.

syslog a nd textlog

This option does not requ ire an y extra ha ndling. Specify th e log

fi le when th e monitoring request is creat ed. Syslog notifi ca tions

go to the local syst em.

console

This option does not require a ny extr a ha ndling. Specify th econsole w hen th e monitoring request is crea ted. Notifi cat ions go

to the loca l system.

ServiceGuard

This option requires that the client and target application both

have S erviceGua rd running on the sa me local syst em.

R esour ce Mon it or s

Resource monitors observe designa ted r esources an d report ba ck

resource values or event s to th e Event Monitoring Service.

Hew lett -Pa ckard provides monitors w ith the High Availabili ty

Monitors pa cka ge an d wit h th e Event Monitoring Service. The

monitors ava ila ble through H ewlett-Pa ckard include: HA Dat a base

Monitor, H A Disk Monitor, H A Clust er Monitor, HA Netw ork

Int erface Monitor, a nd H A Sy stem R esource Monitor.

E v en t M on it or in g S er vice Fr a m ew or k

The EMS Fra mew ork provides th e interfa ce betw een the client

applications, monitors, and target applications.

The E MS Fra mework conta ins th e Applica tions P rogrammers

Int erface (AP I), registr a r, a nd t he Resource Dictiona ry.

Developers use the AP I t o creat e addit iona l monitors for use withclient a nd ta rget a pplicat ions, such a s the EMS G U I, EMS CLI or

ServiceGua rd. Monitor components t o be crea ted include: resource

dictiona ry, resource monitor bina ry fi le, man page (recommended),

a nd messa ge ca ta log (recommended).


16/104



Chapter 116

Figure 1-1 shows the r elat ionships betw een t he Event Monitoring

Service components.

Figure1-1 Event Monitoring Service Components

The process is a s follow s:

1. The system administra tor enters the cli ent appl ication, for example,

the E MS G U I, or the EMS CLI, t o begin t he discovery phase of

creat ing a monitoring request.

The discovery phase, includes identifying the resources to be

monitored and confi guring t he request. I t can be a ccomplished

thr ough m an y methods, including:

E MS G U I

E MS CLI

ServiceG ua rd

monconfig utili ty

resls or resdata commands
http://-/?-http://-/?-


17/104



Chapter 1 17

2. The EM S APIprovides the interface between the client request and

t he registrar. There is a one to one correspondence between the cl ientand the registrar.

3. The registrarrefers to the dictionary for a list of ava ilable resources

a nd relat ed monitors.

The resources listed in th e dictiona ry a re pass ed back to the client.

4. When a d iscovery req uest is ma de tha t exceeds the scope of th e

informa tion in the dictiona ry, the registrarlaunches the a ppropriat er esou r ce m on itor a p pl ica t i on , if i t is n ot a l rea d y r u nn in g, a n d pa sses

th e request on to th e monitor. Multiple registr a rs ma y a ccess the

sa me monitor.

5. The EM S APIprovides th e interface between the registra r a nd t he

monitor.

6. The monitoridentifi es t he resources. The list of resources is pa ssed

back through t he registra r t o the client requestor.

7. The system administra tor, through the clientapplication:

Cont inues to dr il l down t hrough the li st o f ava il ab le resources

supplied by th e registra r, dictiona ry, a nd m onitor

I d en t ifi es th e r esou r ces to m on itor

Completes the monitor ing request , defi nes condit ions of where

an d how t o send the event notificat ion

A completed monitoring requ est ident ifi es:

Wh a t res ou rces t o m on it or

Wh a t ev en t s t o w a t ch for a n d h ow oft en

Wh a t n ot ifi c a t ion s to sen d w h en a n even t occu r s

Wh er e t o s en d not ifi ca t ion s

Events ar e defined for either of tw o resource sta te t ypes:

Periodic checking against either thresholds or sta t e/value

changes

Cont inuous checking for a synchronous s ta te less events

8. The registrarpasses completed monitoring requ ests down t o th e

a ppropriat e resource monitor a pplicat ion.


18/104



Chapter 118

9. The monitorchecks th e resource as specifi ed in th e monitor req uest.

I t passes back to the EMS API wh ether the request is a ccepted orrejected.

10. The EM S APIprovides t he interface betw een t he monitor a nd t he

ta rget .

11. The monitor begins collecting da ta a s specifi ed in the monitoring

request.

12. The EM S APIinterpret s t he informa tion received from t he monitor,determines if an event occurred, and forwa rds t he notifi ca tion t o the

ta rget a pplica tions. The method of informing t he ta rget a pplica tion

of a critical resource va lue ca n va ry for different t a rget a pplica tions.

In the case of ServiceGua rd, th e client application and the t a rget

a pplica tion a re the sa me and reside on t he same syst em.


19/104


EMS Requirements

Chapter 1 19

EMS Requirements

The fol lowing are system requirements for the Event Monitor ing Service :

Al l ha rdware you in tend to monitor, such as d isks and LAN cards ,

ha ve been confi gured an d t ested prior t o confi guring E MS.

EMS must be ins ta l led on an H P 9000 Ser ies 700 or Ser ies 800

syst em run ning H P -U X version 10.20 or lat er.

NOTE For m or e in for m a t ion on sys tem r eq u ir em en ts, r ef er to th e EM S R elea seNotes.

When inst a lling one or more EMS component s, check tha t th e version

levels for t he other components a re compa tible. Refer t o the r ecentRelease Notes for each component, such a s E MS, H A Monitors, or

ServiceGuard.

1. G o to the Website,

http://docs.hp.com

2. From th e Website, select H P -U X, th en High Availability, then

scroll to ident ify an d select t he lat est Release Notes for eachcomponent.


20/104



Chapter 120


EMS groups resources into cla sses in a hierarchy similar to tha t of a

fi lesystem structure. Figure 1-2 is a n exa mple of a resource hiera rchy.

Figure1-2 Event Monitoring Service Resource Class Hierarchy
http://-/?-http://-/?-


21/104


Client and Target Applications

Chapter 1 21


This section describes some of th e client a nd t a rget a pplica tion options

a nd processes. Ta rget a pplica tions ca n be writ ten using t he EMS AP I.

EMS with ServiceGuard

ServiceGua rd can be confi gured with EMS to monitor the health ofselected resources, such as disks. B a sed on t he sta tus of th e resources,

Ser vi ceGu a r d ca n d ecid e to f a il pa cka ges over. Wh en w or kin g w i th EM S,

ServiceGua rd a cts a s both t he client a nd ta rget a pplica tion. EMS works

w ith MC/ServiceGua rd a nd ServiceGua rd E xtension for RAC.

Configur e EMS requests for use with ServiceGua rd packages by editing

the package confi gura tion ASCI I fi le.

In a ddition, i t is recommended tha t you a lso creat e requests thr oughEMS t o:

E n a ble a r ed un da n t not ifi ca t ion s ys t em

M on itor even t s tha t a f fect h igh a va i la b ili t y

E n a ble a ler t s du e t o a p a cka g e fa i lov er

ServiceGua rd ma y a lready be confi gured to monitor the hea lth of nodes,

ser vi ces , a n d subnet s , a n d to m a ke f a ilover d ecis ion s ba sed on th e s t a tu s

of these resources. Using E MS w ith S erviceGua rd a dds to the set of

failures or events t ha t trigger failover an d a ffect a vailabili ty.

EMS GUI Client Application

The EMS G U I client a pplica tion found in t he Resource Management

area of SAM is used to crea te monitoring request s for resources a ndta rgets. The EMS G U I sta rts from the gra phica l version of SAM. C lick

th rough a nd select from the va rious screens to defin e your monitoring

req uest . The options include:

1. Select t he resources to be monitored.

The full pat h of a r esource includes th e resource class hiera rchy a nd

inst a nce. An exa mple of a full resource pat h for th e physical volume

sta tus of th e device /dev/dsk/c0t1d2 belonging to volume groupvgDataBase, is /vg/vgDataBase/pv_pvlink/status/c0t1d2.


22/104



Chapter 122

2. Specify w hen t o collect t he va lue. Select eit her a nd/or all:

When value is ...

I f y ou a r e s et t in g u p a r eq u es t for a n a s yn ch ron ou s m on it or, t h is

is the only option a va ilable.

When value changes

At each interval

Select th is option t o send a n event periodica lly, rega rdless of thevalue.

Defi ne a polling interval t ha t is a ppropriat e to your syst em

performa nce a nd rea ction t ime needs. See St ep 3.

3. Specify a polling interva l for how often th e monitor checks th e

resource and reports t he value.

This appl ies only to non-asynchronous moni tors and goes wi th the At

each interval option in Step 2.

4. Specify how often t he monitor should check an d send notifi cat ion

a bout t he resource:

The options a re list ed below.

Initial option immediat ely checks a nd ret urns t he resource

va lue regar dless of threshold conditions

Repeat option checks a nd ret urns t he resource value at each

polling interva l if thr eshold condit ions h a ve been met

Return option checks a nd ret urns t he resource value aft er a

th reshold condition ha s been resolved and t he thr eshold

condition is no longer true.

5. Specify the notifi cat ion protocols:

opcmsg (IT/O), by s everit y or ma p severit y fr om va lues list ed

tcp or udp

snmp tr a p, by severity or map severity from values listed

email

console

syslog


23/104



Chapter 1 23

textlog

EMS CLI Client Application

emscli is a comma nd line util i ty tha t is used to confi gure and ma na ge

persistent m onitoring requ ests for Event Monitoring Service (E MS)

monitors, such as, HA Monitors, Hardware Monitors and Kernel

Monitors. The emscli utility ca n be us ed to a dd, modify, delete, list a nd

view monitoring requests a nd resources. I t allows the user t o generat e

scripts of the confi gured requests an d a lso ha s a help fea ture (emscli-h) to view t he usa ge of va rious options support ed by emscli.

For more informat ion on the command l ine opt ions suppor ted by emscli,

refer to th e emscli(1m)manpage.

NOTE emscli should be used on HP -U X versions w here the E MS G U I is not

available (HP-UX 11i Version 1.6 onwards).

EMS and Target Applications

Ta rget a pplicat ions receive notifi cat ion messa ges about t he monitored

resources.

To help confi gur e your Net w ork Node Man a ger a nd I T/Opera t ions orother syst em man agement softwa re for E MS, refer to the Wr i t i ng

M oni tors for t he Event M oni tori ng Ser vice (EM S) (H P Par t N umber

B7611-90016))a nd E vent Monitoring S ervice developers kit w eb page:


http://software.hp.com

2. From t he Website, select High Availability, then select EventM onit ori ng Ser vi ce Devel oper s K it.

3. Select Templates.


24/104


Resource Monitors

Chapter 124

Resource Monitors

Resource monitors ar e applicat ions w ritten to ga ther a nd report

informa tion about specifi c resources on th e system.

The r esource monitor:

P rov ides a l is t of resources tha t can be monitored

P r ov id es in for m a t ion a bou t th e r esou r ces

M on itor s the r esou r ces it su ppor t s

P r ov id es va l ues to th e EM S API n ot i fi ca t i on

Th e E MS fr a mew or k eva lu a tes t h e d a ta t o det er min e if a n even t h a s

occu r red . I f a n even t h a s occu r red , th e EM S API sen d s n ot i fi ca t i on in

the a ppropriat e forma t to the confi gured ta rget(s).

Installing Resource Monitors

To obta in ad ditional informa tion about inst a lled monitors:

1. Naviga te to the /etc/opt/resmon/dictionary directory.

E a ch m on it or r eg is t er ed w it h E M S h a s a d ict ion a r y fi l e t h a t is s t or ed

in this directory.2. View the monitor dictionary fi le.

E a ch dictiona ry fi le nam e is descriptive of its monitor. The fi le

extension is typica l ly .dict. For example, the mibmoni tor d ict ionary

fi lename is mibmond.dict.

Configuring Resource Monitors

To con fi gu r e y ou r HA M on itor s, u se eith er th e Even t M on itor in g Ser vice

(EMS ) th rough SAM, emscli or the packag e confi gura tion ar ea of

ServiceGua rd G U I or edit th e package confi gura tion a scii fi le.

U d t di th E t M it i S i


25/104


Resource Monitors

Chapter 1 25

Writing Resource Monitors

Th e E M S AP I pr ov id es a m et h od for w r it in g n ew r es ou rce m on it or s. To

creat e your own monitor, read t he Wri t in g M onit ors for the Event

M oni tori ng Ser vice (EM S)(H P P a rt Number B 7611-90016) ma nua l an d

insta ll the developers kit. B oth a re ava ilable at th e follow ing Website:


http://www.software.hp.com

2. From t he Website, select High Availability, then select t he Event

M onit ori ng Ser vi ce Devel oper s K it, w hich includes:

Wri t in g M oni tors for the Event M oni tor i ng Servi ce (EMS)(HP

P a rt Num ber B 7611-90016)

R ea d M e, I n st a l la t i on , a n d R elea seNotes

D eveloper s K it



26/104


EMS Framework Components

Chapter 126


This section describes th e EMS fra mew ork components.

The EMS API

The EMS AP I is th e interfa ce betw een the registr a r, client a pplicat ions,

ta rget a pplica tions, a nd resource monitors a s i l lustrat ed in Figure 1-1.The EMS AP I is provided as pa rt of the E MS product.

The EMS API ma na ges:

C lien t t o r eg ist r a r com mu nica t ion , w h ich pu ts clien ts in con t a ct w it h

th e appropria te monitor for discovery a nd regist ering monitor

requests.

R eg is tr a r t o m on it or com mu nica t ion , w h ich pa s ses clien t r eq uest s t oth e appropria te monitor.

Comparison between current resource va lues and pre-selected

threshold values.

Monitor ing of t a rget a pplica t ion communica t ion :

S en ds ev en t s t o con fi g ur ed t a r get s (pr e-exis t in g t a r get s or t a r get

you crea te)

Sends notifica t ions to target a pplica t ions when the resource

va lues meet event criteria

For exa m ple, a t a r get TC P a pplica t ion u ses t h e E M S AP I t o t r a n sla t e

TCP messa ges into EMS objects. This ena bles the fi elds to be rea l.

The ta rget a pplica tion then rea ds th e fields of th e EMS objects.

The Registrar

The registra r is a link betw een the client a pplica tions a nd t he resource

monitors. It commu nicat es wit h t he resource monitors on beha lf of the

client a pplica tions t o retrieve informa tion request ed by t he clients. The

registrar runs on t he sa me system as the r esource monitors. The

registrar is provided a s par t of the E MS product.

http://-/?-http://-/?-


27/104



Chapter 1 27

The registra r does not n eed to keep a ny st a te informa tion an d does not

need to be highly a va ilable. It does not n eed to be running w hile aresource is being monitored. The registr a r is needed only t o sta rt th e

monitors a nd t o provide commu nicat ion betw een clients a nd monitors.

One registra r process is sta rt ed each time a client a pplica tion ca lls

rm_client_connect(), so a r egistra r is a lwa ys connected to one client .

Depending on the requests sent by t he client, t he registra r ma y be

conn ected t o 0, 1, 2, or more r esource monit ors concurr ent ly. The

informa tion in the m essages conta in enough informa tion to allow the

registr a r to route the requ ests a nd replies correctly.

Figure 1-3 illustra tes how connections are m ade betw een clients a nd

registrars.

Figure1-3 Connections Among Clients and registrars

Ea ch t ime the registra r sta rts, i t rea ds th e resource dictiona ry,

exchanges internal version information with the client application, and

pr epa r es to r eceive clien t r eq u es t s. Wh en a r eq u es t a r r ives, th e r eg is t r a r

an a lyzes it t o determine if it is one that i t can reply to, or w hether it

needs to pass t he request to a resource monitor.

http://-/?-http://-/?-


28/104



Chapter 128

When th e registra r needs to pass t he request t o a r esource monitor, it

needs to determine if the resource monitor is current ly runn ing. If thea ppropriat e resource monitor process is not found, the registr a r sta rt s

th e process an d wa its unt il the resource monitor ca n commun ica te w ith

the registra r.

The Resource Dictionary

The resource dict ionary is the mechanism by which the resource moni tor

identifi es itself to EMS . The purpose of th e resource dictiona ry is t o givea prelimina ry pictu re of the resource structur e on a g iven system. It s

ma in function is to indicat e to th e registra r w hich resource monitors

should be cont a cted when informa tion is needed about a cert a in

resource. The resource dictiona ry defi nes r esources on th e local sys tem.

Defining a Monitoring Request


29/104

g g q

Chapter 2 29

2 Defining a Monitoring Request


St a rting a Monitoring Request

Specifying w hen t o send E vent Notifi cations

Sett ing the Polling Interva l

Setting Event Value Options

Selecting P rotocols for S ending E vents

Adding a Notifi ca tion C omment



30/104

g g q

Starting a Monitoring Request

Chapter 230

Starting a Monitoring Request

After you ha ve selected a resource to monitor, use t he Monitoring

Request Parameters screen t o specify wh en a nd how to send event

notificat ion (Figure 2-1). The follow ing s ections describe t he monit oring

par a meters a nd provide exa mples of common a pplica tions.

Figure2-1 Monitoring Request Parameters Screen

http://-/?-http://-/?-


31/104

Specifying when to send Event Notifications

Chapter 2 31


When you creat e a request , you specify th e conditions un der w hich you

wa nt to collect resource sta tu s va lues. While the monitor ma y be polling

disks every fi ve minutes, for exam ple, you ma y only wa nt t o be alerted

wh en something ha ppens tha t r equires your at tention. Specify these

condit ions in t he Notify area of the Monitoring Request Parameters

screen. H ere are t he terms un der wh ich you can be notifi ed:Table2-1 Options to specify when Event Notifications have to be sent

Wh en va lu e is ... You defi n e t he con dit ion s u nd er w hich you

w ish to be notifi ed for a pa rt icula r resource.

Choose a n operator(= , not equal, >, >= ,


32/104


Chapter 232

To set a n event tr igger:

Se lect f rom th e li st ed opt ion s in th e Notify area (When value

is..., When value changes, or At each interval).

Asynchronous m onitors a re event-driven, ra th er t ha n polled. They

gen er a te m essa ges a s even t s occu r. Th er ef or e, i f th e r eq u es t is f or a n

a synchronous m onitor, only the When value is... option is

available.

NOTE U pd a ted m on it or s m a y h a ve n ew s ta t us va lu es t ha t ch a ng e t h e m ea n in gof your monitoring requests, or generat e new a lerts. For exam ple,

assum e you ha ve a request for notifi cat ion, i f sta tus > 3, for a resource

wit h a values ra nge of 1 through 7, you will receive alerts ea ch t ime the

va lu e eq ua ls 4, 5, 6, or 7. I f t h e u pd a ted ver sion of t he m on it or h a s a n ew

sta tus va lue of 1 th rough 8, you will also see a lerts w hen t he resource

equals 8.You ca n use emscli to specify w hen event notifi cat ions h ave t o be sent

w hen a dding or modifying a m onitoring requ est. For more informa tion,

refer to th e emscl i (1m)manpage.


S i h P lli I l


33/104

Setting the Polling Interval

Chapter 2 33

Setting the Polling Interval

The polling int erva l specifi es how often th e resource monitor checks t he

resource value. The polling interva l is the ma ximum a mount of elapsed

time before a m onitor knows about a cha nge in sta tus for a pa rticula r

resource.

The shorter th e polling interva l, the m ore likely you ar e to ha ve recent

da ta . However, depending on the monitor, a short polling interva l mayuse too much CP U a nd syst em resources. You need to weigh t he

adva nta ges and disadva nta ges betw een being a ble to quickly respond to

events and maintaining good system performance. Some considerations

a re listed below. You may w a nt t o set:

A m in im um pollin g in t er va l (10 s econ ds or 30 s econ ds ) d epen din g on

the monitors ability to process quickly. For most resource monitors,

t h e m in im um is 30 s econ ds. D is k m on it or r eq ues t s ca n b e a s s hor t a s10 second s.

A short pol ling interval (30 seconds or less) when i t is cr i t ica l tha t

you ma ke a quick failover decision. MC /ServiceGua rd monitors

resources every few seconds.

A pol ling interval of 5 minutes or so for monitoring less cr i t ica l

resources.

A ver y lon g pollin g in ter va l (4 h our s) t o m on it or fa iled disks t ha t a re

n ot es sen t ia l t o t h e s ys t em , bu t w h ich s hou ld b e r epla ced in t h e n ext

few days.

Asynchronous monitors a re event -driven, n ot polled. They genera te

messa ges a s events occur. Therefore, if t he resource is a n a synchronous

monitor, t he Polling Interval fi eld display s n/a.

To set th e Polling I nt erva l:

1. Specify the qua ntit y of time in the numbered field.

2. S elect t he u nit of t im e fr om t h e unit of measure fi e ld lis t (seconds,

minutes, hours, day). The m a ximum va lue is one (1) da y.




34/104


Chapter 234


I f y ou s elect t h e When value is... fr om t h e list in t he Notify a r ea , t h e

Options a rea displays th ree choices. Select one or more of these th ree

options:

Init ia l U se t his opt ion to est a blish a ba seline w hen monitoring

resources such as available filesystem space or system

load. I t can also be used to test wh ether newlyrequested events a re being sent.

Repea t U se t his opt ion for urgent aler t s. The Repea t opt ion

sends an a lert a t ea ch polling interval a s long a s the

notify condit ion is met. U se this option w ith cau tion;

there is a risk of high CP U use or fi lling log fi les a nd

alert windows.

Ret urn U se t his opt ion t o t ra ck w hen emergency sit ua t ions

return t o norma l.

To set t he frequency of th e tr igger:

C lick on e or m or e but ton s on th e li st i n th e Options area (Initial,

Repeat, and Return).

Asynchronous monitors a re event -driven, n ot polled. They genera te

messages as events occur. Therefore if the resource is ana synchronous monitor, the va lues in the Options area defaul ts to

Repeat a nd cannot be cha nged.

The Options a rea w ill not be display ed if you ha ve selected When

value changes or At each interval from t he list in t he Notify

a rea . In th ese ca ses, the option default to Initial ca nnot be

changed.

You can use emscli to set event va lue options. For more informa tion,refer to th e emscl i (1m)manpage.


Selecting Protocols for Sending Events


35/104


Chapter 2 35


Through the Notify via a r ea , speci fy th e pr otocol y ou w a n t th e m on itor

to u se to sen d even t s. Th e opt ion s a r e d escr ibed i n th e f ollow i ng sect i on s.

The opcmsg (ITO) Option

This option sends messa ges to ITO a pplica tions via th e opcmsg daemon.For th is opt ion to display, IT Operation Managed Node Software 3.x or

4.x must be insta lled on the resource server running H P -U X version

10.20. This option is not currently a va ilable on syst ems run ning H P -U X

version 11.0.

The I TO messa ge severity options a re:

Map from value (th is is not a va ilable on a ll monitors)

Critical

Major

Minor

Warning

Normal

A specifi ed severity other th a n Normal is return ed under t he follow ingconditions:

When value is . . .: If th is notifi cat ion option is set, a

non-norma l severity occurs wh en th e value chan ges from FALSE t o

TRUE.

For exa m p le, if a d isk i s bein g m on itor ed , y ou w a n t n ot i fi ca t i on w h en

th e disk is down. The When value is condition is FALS E wh ile th e

disk is up a nd r unn ing correctly. The condit ion becomes TRU E ,

meaning a ction needs to be ta ken, when t he disk is down or not

operating correctly.

When value changes: I f t h is n ot ifi ca t ion opt ion is s et , a n on -n or ma l

severity occurs wh en th e current va lue does not m a tch th e previous

value.




36/104


Chapter 236

Cert a in monitors ca n ma p directly t o OP C severity levels. This is not

available with all monitors. Select Map from value option from t he listin the Severity area .

I f opcmsg is selected, EMS sets t he follow ing fi elds:

ITO application group: EMS (HP )

message group: H A

object: to t he full pat h of the r esource being monitored

S ee HP OpenView I T/ Operat ions Adm in istrators Task Guid e(Part

Num ber B 4249-90003) for m ore inform a t ion.

Templat es for confi guring IT/O a nd Netw ork Node Man a ger t o display

monitored events ca n be found on th e Hew lett-P a cka rd Webpage a t

http://www.software.hp.com. Click on High Availability, then

Event M onit ori ng Ser vi ce Devel oper s K i t.

To set t he opcmsg protocol for ITO, execut e th e following st eps:1. Specify the notifi cat ion ty pe from th e list in the Notify area .

2. Select the opcmsg (ITO) option from t he list in t he Notify area .

3. Select the severity level from the list in t he Severity area :

Map from value

Critical

Major

Minor

Warning

Normal

tcp and udp Options

Th is s en ds TC P or U D P en cod ed even t s t o t he t a r get h os t n a me a n d por t

indicat ed for th a t req uest. Thus, th e message can be directed t o a

user-w ritt en socket progra m.

To set t he TCP or U DP condit ions, execute t he follow ing st eps:

1. Select the TCP or UDP option, as a ppropriat e, from the list in the

Notify via area .




37/104


Chapter 2 37

2. Specify the ta rget host na me and t he port in t heir respective fields.

The snmp Traps Option

This sends messa ges to applica tions, such a s Netw ork Node Man a ger

tha t use SNMP tra ps. See HP OpenVi ew U sin g Netw ork N ode M anager

(P /N J 1169-90002) for more informa tion on confi gur ing S NMP t ra ps.

Ta ble 2-2 lists t ra ps used by EMS:

Table 2-2 SNMP Traps

Trap Name Trap Value Description

EMS_

ENTERPRISE_

OI D

"1.3.6.1.4.1.11.2.3.1.7"

EMS_NORMAL_

OI D

" 1.3.6.1.4.1.11.2.3.1.7.0.1" Norma l

Event

EMS_ABNORMAL

_

OI D

" 1.3.6.1.4.1.11.2.3.1.7.0.2" P roblem

Event

EMS_REBOOT_

OI D

" 1.3.6.1.4.1.11.2.3.1.7.0.3" Reboot E vent

EMS_RESTART_OI D

" 1.3.6.1.4.1.11.2.3.1.7.0.4" Rest a r tEvent

EMS_NORMAL_

SEV_OID

" 1.3.6.1.4.1.11.2.3.1.7.0.5" P roblem

Event

w /Norm a l

Severity

EMS_WARNING_SEV_OID " 1.3.6.1.4.1.11.2.3.1.7.0.6" P roblemEvent w /

Warning

Severity

EMS_MINOR_

SEV_OID

" 1.3.6.1.4.1.11.2.3.1.7.0.7" P roblem

Event

w /Minor

Severity


http://-/?-http://-/?-


38/104

g g

Chapter 238

The Severity ar ea options for SNMP tra ps are:

Map from value (this is not a vailable with a ll monitors)

Critical Major

Minor

Warning

Normal

A specifi ed severity oth er tha n Norma l is retur ned under t he follow ing

conditions:

When value is . . .: If th is notifi cat ion option is set, a


TRUE.


th e disk is down. The When value is condit ion is FALSE wh ile the

disk is up a nd r unn ing correctly. The condit ion becomesTRUE

,meaning a ction needs to be ta ken, when t he disk is down or not


When value changes: If th is notifi cat ion option is set, a

non-norma l severity occurs wh en the current va lue does not ma tch

th e previous va lue

Certa in SNMP monitoring requests can ma p directly t o severity levels.

Select the Map from value option from t he list in t he Severity area .

EMS _MAJ OR_

SEV_OID

" 1.3.6.1.4.1.11.2.3.1.7.0.8" P roblem

Event

w /Ma jor

Severity

EMS_CRITICAL_S

EV_OID

" 1.3.6.1.4.1.11.2.3.1.7.0.9" P roblem

Event w /Critical

Severity

Table2-2 SNMP Traps (Continued)

Trap Name Trap Value Description




39/104

Chapter 2 39

To set t he SNMP tr a p, execute t he follow ing st eps:

1. Specify the notifi cat ion ty pe from th e list in the Notify area .

2. Select t he SNMP tra p option from the list in the Notify via area .

3. Select the severity level from the list in t he Severity area :

Map from value

Critical

Major

Minor

Warning

Normal

The email Option

This sends event notifi cat ion to the e-ma il addr ess indica ted for tha t

request.

To set e-ma il notifi cat ion, execute th e following st eps:

1. Select the email option from the list in t he Notify via area .

2. Specify the full e-ma il address in t he email address field.

The console Option

This sends event notifi cat ion t o the syst em console.

To set console notifi cat ion, select t he console option from the list in t he

Notify via area .

The syslog OptionThis sends event notifi cat ion t o th e system log.

An a bnorma l event message (error) is retur ned und er t he follow ing

conditions:

When value is . . . - If t his notifi cat ion option is set, a


TRUE.




40/104

Chapter 240


th e disk is down. The When value is condit ion is FALSE wh ile thedisk is up a nd r unn ing correctly. The condit ion becomes TRUE,

meaning a ction needs to be ta ken, when t he disk is down or not


When value changes - I f th is n ot i fi ca t i on opt ion is set , a n on -n or m a l

severity occurs wh en th e current va lue does not m a tch th e previous

value

For a n a bnorma l event, a syst em error logging level w ill be associat edwit h t he logged message.

To set syst em log notifi cat ion, select th e syslog option from t he list in

the Notify via area .

The textlog Option

This sends event notifi cat ion t o a fi le you specify.

To set t extlog n otifi cat ion, execut e th e follow ing st eps:

1. Select the Textlog option from t he list in t he Notify via area .

2. Speci fy the filename and path in the File Path field.

The default pat h is /var/opt/resmon/log/event.log.

You can use emscli for select ing a protocol to send event s. For m ore

informa tion, refer to the emscl i (1m)manpage.


Adding a Notification Comment


41/104

Chapter 2 41


The notifi cat ion comment is useful for sending ta sk reminders t o

recipient s of an event. For example, you can confi gure a disk monitor

request tha t reports a n a lert w hen an entire mirror ha s failed. When

th a t event show s up in IT/Opera tions, you ma y wa nt a notifi cat ion

comment to include the na me of th e person to conta ct.

You ca n u se emscli t o a d d a n ot i fi ca t i on com m en t . For m or e in for m a t ion ,refer to th e emscli(1m)manpage.




42/104

Chapter 242

Changing Monitoring Requests


43/104

Chapter 3 43

3 Changing Monitoring Requests


Copying Monitoring Request s

Modifying Monitoring Request s

Removing Monitoring Requ ests

Viewing Monitoring Request s


Copying Monitoring Requests


44/104

Chapter 344


There a re tw o wa ys t o use th e copy function:

To cr ea t e r eq ues ts for mul t i p l eresources using t he samemonitoring

parameters .

To cr ea t e r eq u es t s for t he sameresource using differentmonitoring

parameters .To crea te r equests for mu ltiple resources using t he sa me monitoring

parameters :

1. From the Event Monitoring Service ma in screen, select t he

monitoring request w hose par a meters y ou wish t o copy.

You n eed t o h a ve con fi g ur ed a t lea s t on e s im ila r r eq u es t for a s im ila r

instance.

2. Select Actionsmenu: Copy option.

You see t he Add or Copy Monitoring Request screen.

3. From the Add or Copy Monitoring Request screen, select a

different r esource insta nce and click OK.

You see t he Monitoring Request Parameters screen.

4. Cl ick OK in the Monitoring Request Parameters screen.

You see a messa ge tha t indicates th e new request ha s been a dded.

You see t he Event Monitoring Service main screen.

To crea te r equests for t he sa me resource using different monitoring

parameters :


monitoring request w ith t he insta nce for w hich you wish t o havemult iple monitoring requests.

You need to have confi gured a t lea st one request for th e insta nce.

2. Select Actionsmenu: Copy option.

You see t he Add or Copy Monitoring Request screen.

3. Cl ick OK in the Add or Copy Monitoring Request screen.





45/104

Chapter 3 45

4. Modify the para meters as desired in the Monitoring Request

Parameters screen.

5. Cl ick OK.

A message is displayed to indica te th at the new request ha s been

a dded. The Event Monitoring Service ma in screen is displayed.

NOTE There is no option to copy monitoring requests using emscli.

H owever, you ca n extra ct a sh ell script from th e existing monitoringrequest tha t you may wa nt t o copy using the emscli -g option,

modify the shell script so that i t ha s modifi ed para meters or

resources, and th en run th e shell script. This procedure can a lso be

used t o copy monitor req uests from one syst em to a nother.


Modifying Monitoring Requests


46/104

Chapter 346

Modifying Monitoring Requests

To chang e the monitoring par a meters of a req uest:


monitoring request you wa nt to modify a nd either:

D ou ble click t he req ues t , or

Select Actionsmenu: Modify


2. Modify the pa ra meters as desired, by editing the fi elds in the

Monitoring Request Parameters screen.

3. Cl ick OK.

A message is displayed t o indica te t ha t the request ha s been

modified. The Event Monitoring Service main screen is d isplayed .

NOTE You can m odify a monitoring request using emscli -m option. FOrmore informa tion, refer t o th e emscli(1m)manpage.


Removing Monitoring Requests


47/104

Chapter 3 47

Removing Monitoring Requests

You ca n remove one or more monitoring r equests using t he Remove

Monitoring Requests opt ion. To remove monit oring req uest s, execut e

th e follow ing st eps:


monitoring request you wish t o remove.

To select cont iguous mu ltiple req uest s, hold the Shift key and click.

To select individua l mult iple requests, h old t he Ctrl key and click.

2. Select Actionsmenu: Remove option.

You see a Confirmation screen.

3. Cl ick OK on th e Confirmation screen.

You see a m essa ge tha t indicat es how m a ny requ est(s) ha ve beenremoved. You see t he Event Monitoring Service ma in screen.

4. To star t monitoring the resource aga in, you must recreat e the

request , eith er by copying a similar r equest for a s imilar r esource or

by re-entering the da ta .

NOTE You ca n r em ove a m on itor in g r eq u es t u sin g emscli -d option. For moreinforma tion, refer to the emscli(1m)manpage.


Viewing Monitoring Requests


48/104

Chapter 348

Viewing Monitoring Requests

To view t he para meters for a m onitoring req uest:


monitoring request you wish t o view.

2. Select Actionsmenu: View

You see th e View Monitoring Request Parameters s cr een w it h t h epara meters specified for t he monitoring request.

3. To modify the par a meters of this requ est, click the Modify

Monitoring Request option. You see t he Monitoring Request

Parameters screen. P roceed as described in Modifying Monitoring

Requests.

4. To close the View Monitoring Request screen, click OK.

NOTE You ca n v iew a m on itor in g r eq u es t u sin g emscli -v option. For moreinformation, refer to the emscli(1m)manpage.


49/104

Monitoring ServiceGuard Package Dependencies


50/104

Chapter 450

NOTE Crea te th e same requests on all nodes confi gured for a ServiceGua rdpackage.



51/104

Chapter 5 51

5 Monitoring Network Interfaces

The HA Network In terfa ce Monitor detects w heth er your LAN int erface

is UP or DOWN. It a llows y ou to send events t o a system ma na gement

interface as a n a lternat ive to looking in syslog for LAN sta tu s. The H A

Network Interface Monitor, lanmond, is part of the MIB Monitors

package.This cha pter conta ins t he following s ections:

Netw ork Monitor R eference

Configur ing Netw ork Monitoring Requests


Network Monitor Reference


52/104

Chapter 552


The HA Netw ork Int erface Monitor provides st a tus on the LAN

interfaces in a given node. I t monitors a ll the interfaces th at you see

when you run the lanscan comma nd on a syst em.

Th e HA Netw or k I n ter fa ce M on itor i s pa r t of th e M IB M on itor s pa cka ge.

Ta ble 5-1 lists the HA Network Interface Monitor.

Figure 5-1 illustrates the network monitored resources.

Figure5-1 Network Monitor Resource Class Hierarchy

The MIB-2 variable, ifOperStatus, provides th e LAN interface sta tus t o

th e m on itor. For I Pv6 en a b led sys tem s, ipv6ifOperStatus, pr ov id es th e

LAN interfa ce sta tus t o th e monitor. The monitor report s DOWN i f it r ead

th e MIB va lue of TES TING . See Ta ble 5-2.

Table5-1 HA Network Interface Monitor Names and Resources

MonitorResource

TypeResource

la nmond LAN Int erfa ce /net/interfaces/lan/status/


http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-


53/104

Chapter 5 53

To verify th e operat iona l sta tus of th e LAN interface, use the lanscan

(1M) or lanadmin(1M) comma nds.

The EMS product depends on TCP/IP (or UDP) to send events to ta rgets

such a s H P OpenView I T/Opera t ions or MC /S erviceGu a rd. I f a ll LAN

in t er fa ces on a s ub net fa il, n ot ifi ca t i on s m a y n ot b e r eceiv ed b y a r em ot e

ta rget .S t a n db y L AN s a r e r epor t ed a s D OWN u nles s t h ey h a ve b een a ct iv a t ed t o

replace a fa iled LAN int erface.

The min imum polling int erva l is 30 seconds.

Table 5-2 Interpreting LAN Interface Status

Resource Name: /net/interfaces/lan/status/LANname

Condition Value Interpretation

UP 1 The LAN int erfa ce is sending a nd receiving

packets.

DOWN 2 The LAN int erfa ce is not pa ssing opera t iona l

packets.


Configuring Network Monitoring Requests


54/104

Chapter 554

Configuring Network Monitoring Requests

Ta ble 5-3 recommends two moni tor ing requests for each node. With these

request s, you w ill see events w hen a LAN car d failed, and a ga in when it

cam e back up, an d you will see an event every hour t ha t t he LAN card

wa s down. You may elect t o chang e the polling interva l or not t o

confi gure a reminder at a ll .

Table 5-3 Recommended LAN Interface Requests

Resources to monitor

Monitoring Parameters

Notify Value OptionPollingInterval

/net/interfaces/lan/stat

us/LANname

when

va lue is

< UP RETURN 30 sec.

/net/interfaces/lan/statu

s/LANname

when

va lue is

= DOWN REPEAT 1 hour


6 Selecting Resources to Monitor
http://-/?-http://-/?-


55/104

Chapter 6 55

6 Selecting Resources to Monitor


St a rting t he Event Monitoring Service

Selecting Resources

Viewing Resource Descriptions


Starting the Event Monitoring Service


56/104

Chapter 656

Starting the Event Monitoring Service

To sta r t t he EMS G UI :

1. Log on as root t o the system with EMS a nd sta r t t he graphica l

version of SAM. At th e HP -U X prompt, ty pe:

sam

2. Double click the Resource Management icon.

3. Double click on t he Event Monitoring Service icon.

The ma in screen w ith t he Actions menu open, shown in Figure 6-1,

shows all requests confi gured on tha t syst em.

If th ere are no requests a nd Sent inel monitors are not insta lled:

A m essa ge is displa y ed on th e scr een :

Currently no resources are being monitored. Use the

action.

Th e fi e ld a r ea of th e m a i n scr een is em pty.

If Sent inel monitors a re insta lled, the screen is simply bla nk.

Figure6-1 Event Monitoring Service Main Screen

To start the E MS C LI, enter emscli with the relevant comma nd line

option at th e HP -U X comma nd line prompt. For exa mple, enter emscli

-a /system/numUsers to add a n ew monitoring request. Enter emscli

-h to view t he usa ge of va rious options support ed by emscli.


Selecting Resources
http://-/?-http://-/?-


57/104

Chapter 6 57

Selecting Resources

Resources a re divided in t o cla sses. To select a resource to monit or:

1. From the Event Monitoring Service ma in screen, click on t he

Actions menu.

A resource in th e E MS TU I ca n be selected by choosing F 1 +

keys for vt 100 termina ls an d F2 + keys for

hpterm.

Refer to t he section, St a rting t he Event Monitoring S ervice on

pa ge 72 for instructions on sta rting E MS.

2. Select the Add Monitoring Request menu item.

The top-level resource classes for all installed monitors are

dyna mica lly discovered a nd t hen listed as sh own in Figure 6-2.

Some Hewlett-Packard products include their own monitors within

t heir pr oduct h iera rchy. For exa mple, ATM Adapt er for H P /9000

Server s, HP O TS 9000 or STM (Su ppor t Tools M a n a ger ) for HP 9000

ha rdwa re monitoring. If this t ype of product is insta lled on the

syst em, th en its t op-level resource class a lso a ppea rs in t he Add or

Copy Monitoring Request screen.

NOTE After inst a llat ion, some monitors must be ena bled before theirresource clas ses appear in t he EMS Add or Copy Monitoring

Request screen. For exa mple, th e STM ha rdw a re monitors. Refer t o

th e documenta tion for your monitor for instru ctions on ena bling or

sta rting your monitor.

Similarly, top-level resource classes belonging to user-writtenmonitors, crea ted using t he procedures described in Wr i t i ng

M oni tors for t he Event M oni tori ng Ser vice (EM S) (HP Par t N um ber

B7611-90016), ar e discovered a nd display ed here.

To obtain a dditiona l informa tion about an y part icula r monitor:

R ev iew t h e m on it or d ict ion a r y fi le:


Selecting Resources

a . From the command line, go to the
http://install.pdf/http://install.pdf/http://-/?-http://-/?-http://install.pdf/http://install.pdf/


58/104

Chapter 658

g

/etc/opt/resmon/dictionary directory.

Informa tion about ea ch monitor ca n be found in the .dict

files.

b . View the monitor d ict ionary fi le.

The fi le name corresponds to i t s moni tor. The fi le extens ion is

.dict. For exam ple, the MIB Monitor dictiona ry fi lenam e is

mibmond.dict.

Review t he ma npa ge.

The ma npage na me can be found in th e dictionary fi le with t he

monitors na me. If a ma npage wa s creat ed, it is l isted in the

MONITOR entr y section of the dictiona ry fi le.

View t h e r es ou rce cla s s or in st a n ce des cr ipt ion t hr ou gh t he E MS

G U I .


Selecting Resources

The EMS Monitoring Request Parameters screen ha s a View


59/104

Chapter 6 59

Resource Description button that displays additional

informa tion supplied by th e selected r esource. See Figure 6-2.

Figure 6-2 EMS Monitoring Request Parameters Screen

3. Double click on a resource cla ss.

When y ou monitor a resource, you a ctua lly monitor one or m ore

specifi c insta nces of its resource cla ss. View th e resource insta nces

a ssociat ed wit h th e selected resource cla ss in the Resource

Instance fi eld. See Figure 6-3.

You can list a ll ava ilable resources with t he emscli -r option.


Selecting Resources

I f th e r esou r ce cl a ss h a s subcla s ses , th ose subcla s ses a r e lis t ed in th e
http://-/?-http://-/?-http://-/?-http://-/?-


60/104

Chapter 660

Resource Classes field.

Asynchronous m onitors a re event-driven, ra th er t ha n polled. They

generate m essages a s events occur, w ithout rega rd for relative

import a nce. Therefore, if th e resource inst a nce is an a synchronous

monitor, t he Current Value fi eld does not a pply, an d th e field

displays n/a.

Figure6-3 Add or Copy Monitoring Request Screen

4. Select a specifi c inst a nce or the wildcard (All Insta nces).


Selecting Resources

The (*) w ild ca r d is a con ven ien t w a y t o cr ea t e m a n y r eq u es t s a t on ce.

M h h di k k d d


61/104

Chapter 6 61

Most syst ems have more than one disk or netw ork card, a nd ma ny

ha ve severa l disks. To avoid ha ving to creat e a m onitor req uest foreach disk, select *(All Instances) in t he Resource Instance box.

The *(All Instances) l isting is alwa ys the fi rst i tem on the list .

See the fi gure above.

Wildca rds a re a vailable only w hen a ll instan ces of a subclass a re of

the same resource type and there are multiple instances. Selecting

t h e w ild ca r d a pplies t h e m on it or t o a ll t h e in st a n ces of t h a t r es ou rce

type. Wildcards are not ava i lab le for resource classes . For example, awildcar d is ava ila ble for t he sta tus inst a nces in the subclass,

/system/filesystem/availMb. A w ildca rd is not ava ilable for th e

entire volume group resource class, /vg.

5. Cl ick OK.

You will see t he Monitoring Request Parameters screen. S ee

Cha pter 4, Defi ning a Monitoring Request.

NOTE You can a dd a new m onitoring request using the emscli -a commandline option. For m ore informa tion, refer t o the emscl i (1m)manpage.


Viewing Resource Descriptions


62/104

Chapter 662

Viewing Resource DescriptionsResource clas s a nd resource inst a nce descriptions a re a va ilable for ea ch

resource.

To see a resource class descript ion, click the Show Class Description

button from t he Add or Copy Monitoring Request screen.

To see a resource inst a nce description, click eith er t he Show Instance

Description button from t he Add or Copy Monitoring Requestscr een or th e Show Instance Description but ton f rom th e Monitoring

Request Parameters screen.

NOTE You ca n v iew th e d escr ipt ion of a r esou r ce i ns t a n ce u sin g th e emscli -vcomma nd line option. For m ore informa tion, refer to th e emscl i (1m)

manpage.

Monitoring System Resources

7 Monitoring System Resources


63/104

Chapter 7 63

g y

The HA Syst em Resource Monitor sends event s a bout th e number of

users, ava ilable fi le system spa ce, an d job queues to help you

load-bala nce an d t une your system to keep it a vailable. I t is a n

alt ernat ive to rea ding syslog fi les to get t his informa tion. The HA

System Resource Monitor , pkgmond, is pa r t of t h e M IB M on it or s pa cka g e.


Sy stem Monitor Reference

Crea ting Syst em Resource Monitoring Requests


System Monitor Reference


64/104

Chapter 764

System Monitor ReferenceThe syst em monitor report s informa tion on the follow ing sys tem

resources:

/system/numUsers - num ber of users on a given n ode.

/system/jobQueue1Min, /system/jobQueue5Min, and

/system/jobQueue15Min - n u mb er of pr oces ses w a it in g for C P U a n d

performing disk I/O a s a n a vera ge over 1, 5, and 15 minut esrespectively. This is th e sam e as t he loa d a vera ges report ed by

uptime (1).

/system/filesystem/availMb/fsName - number of megabyt es

ava ilable for use in fi lesystem fsName.

The HA Syst em Resource Monitor is par t of the MI B Monitors pa ckage.

Ta ble 7-1 lists t he H A Sy stem R esource Monitors.

Table7-1 HA System Resource Monitor Names and Resources

MonitorResource

TypeResource

mibmond System /system/numUsers,

/system/jobQueue1Min,

/system/jobQueue5Min,/system/jobQueue15Min

fsmond Filesystem

Ava ilable S pace

/system/filesystem/availM



Figure 7-1 shows t he syst em resource monitor hierar chy. It ems in boxes

a re resource insta nces tha t can be monitored. The fsName in ita lics
http://-/?-http://-/?-http://-/?-http://-/?-


65/104

Chapter 7 65

cha nges depending on t he na mes of the fi le systems.

Figure7-1 System Resource Monitor Class Hierarchy

Number of Users

The number of users tells you how ma ny us ers a re logged in t o a given

system.

The MIB variables computerSystem, fileSystemBavail, and

fileSystemBsize from t he hp-unix MIB provides t he resource va lue to

th e m onitor.

To verify t he num ber of users on t he syst em, use th e uptime (1)

command.

Aler t s for n um ber of u ser s ca n be u sed t o d et er min e t he bes t t im e t o r un

ba ckups or other m a intena nce, for load-ba lan cing, or to disallow more

tha n a certa in number of users on a given system.

Table 7-2 Interpreting Number of Users

Resource NameValueRange

Interpretation

/system/numU sersinteger tota l number of users logged in to the node



The minimum polling interval is 30 seconds. A longer interval is

recommended. Short polling intervals may adversely affect system


66/104

Chapter 766

g

performance.



J ob Queues


67/104

Chapter 7 67

Th e job q u eu e m on itor ch ecks th e a ver a ge n u mber of pr ocesses th a t h a vebeen wa iting for CP U a nd performing disk I/O over th e last 1, 5, or 15

m in ut es. A va lu e of 4 in /system/jobQueue5Minm ea ns t ha t a t t he t ime

of polling there wa s a n a vera ge of 4 jobs in the q ueue over th e last 5

minutes.

The MIB variables computerSystemAvgJobs1,

computerSystemAvgJobs5, and computerSystemAvgJobs15 from th e

hp-unix MIB provides the resource value to the monitor.

To verify th e loa d a vera ges on t he syst em, use th e uptime (1) command.

The minimu m polling int erva l is 30 seconds. U nless your sy stem loa dtends t o fluctuat e wildly and need loa d-bala ncing a tt ention frequently,

s et a pollin g in t er va l g rea t er t h a n or eq ua l t o t h e job q ueu e in t er va l : 1, 5,

and 15 minutes, respectively.

Table7-3 Interpreting J ob Queues

Resource NameValueRange

Interpretation

/sy stem/jobQu eu e1M in i nteger a ver a ge n u mber of jobs in th e q u eu e in

the last minute

/sy stem/jobQu eu e5M in i nteger a ver a ge n u mber of jobs in th e q u eu e in

the last 5 minutes

/system/jobQueue15Min in teger average number of jobs in the queue in

the last 15 minutes



Filesystem Available Space


68/104

Chapter 768

The fi lesystem monitor checks the number of megabytes ava i lab le for usein ea ch fi l e s ys t em on t h e n od e. F ile s ys t em s m us t b e m ou nt ed a n d a ct iv e

to be monitored. File systems mounted over t he netw ork, such a s NFS

fi le systems, are not monitored.

The MIB variables fileSystemBavail, and fileSystemBsize from t he

h p-u nix MI B a r e u sed t o ca lcu la t e t h e n um ber of a va ila ble K B in t he fi l e

syst ems. The num ber is th en divided by 1024 to get t he num ber of

available MB.

You ca n h a ve m or e fi l e s ys t em s or d iffer en t n a m es. Th is d epen ds on h ow you have confi gured fi le systems on your system. When a fi le system

s t a rt s fi l lin g u p, clea n u p old fi l es or a d d d is k s pa ce, a n d r econ fi g ur e y ou r

fi le systems.

NOTE B e ca u se t h e / ch a r a ct er is n ot v a lid in a r es ou rce n a m e, it is r epla ced byth e _ cha ra cter. So the fi le system /tmp/users wil l appear a s the

resource nam e /system/filesystem/availMb/tmp_users. For t hesa me rea son, the root fi le system (/) is replaced by th e na me root.

The minimum polling interval is 30 seconds. A longer interval is

recommended; short polling intervals may adversely affect system

performa nce. When confi guring r equests t hrough E MS, a w ildca rd (*) is

ava ila ble to monitor all fi le systems on a system.

Table 7-4 Filesystem Available Space

Resource Name

Most common na mes ar e:

/system/filesystem/availMb/stand

/system/filesystem/availMb/root

/system/filesystem/availMb/home

/system/filesystem/availMb/opt

/system/filesystem/availMb/tmp_users

/system/filesystem/availMb/usr

/system/filesystem/availMb/var


Creating System Resource Monitoring Requests


69/104

Chapter 7 69

Creating System Resource MonitoringRequests

Ta ble 7-5 shows exa mples of how you can m onitor system r esources.

Table7-5 Examples of System Resource Requests

To be alertedwhen...

Resources tomonitor

Monitoring Parameters

Notify Value Option

fewer tha n 5 users

a r e on t h e n od e, for

running backups

/syst em/numU sers w hen

va lue is

< 5

m or e t h a n 20 u ser s

a r e on t h e n od e, forload ba lancing, and

w h en va lu e r etu r ns

to below 20 users

/syst em/numU sers w hen

va lue is

> 20 RETURN

syst em load is high /syst em/

jobQueue1Min

when

va lue is

> 7 INITIAL

/sy st em/

jobQueue5Min

when

va lue is

> 4 INITIAL

/sy st em/

jobQueue15Min

when

va lue is

> 3 INITIAL

fi le systems are

runn ing out of

space

/sy st em/fi lesy st em/

availMb for:

/hom e

/opt

/root

/st a nd

/tmp

/usr

/va r

when

va lue is

< 50 INITIAL
http://-/?-http://-/?-


70/104

Monitoring Cluster Resources

8 Monitoring Cluster Resources


71/104

Chapter 8 71

Th e HA C lu ste r M on itor sen d s even t s r ega r di ng th e s t a tu s of a clu ster. I t

is recommended tha t you use S erviceGua rd Ma na ger to monitor cluster

sta tus a nd receive cluster events. H A Clust er Monitor is prima rily for

use w ith non-OpenView sy stems, for exa mple, CA Un iCent er.


Clust er Monitor Reference

Crea ting C luster Monitoring Requests

Monitoring Cluster Resources

Cluster Monitor Reference

Cluster Monitor Reference


72/104

Chapter 872

Cluster Monitor ReferenceThe HA Clust er Monitor is useful in environment s not ru nning H P

OpenView wi th ServiceGua rd Manager. The HA Cluster Monitor repor ts

informa tion on t he sta tus of th e cluster t o which the loca l node belongs.

The resources monitored are:

/cluster/status/clustername, a su mm ar y of t he st a te of a ll n odes

in th e cluster clustername

/cluster/localNode/status/clustername, the sta tus o

event monitoring service version2

Documents