Application Performance Measurement

MIB

draft-waldbusser-rmonmib-apm-00.txt

Steve Waldbusser

Lucent Technologies

Solution

3 Fundamental Types of Applications

– Transaction Based

– Throughput Based

– Streaming

Transaction-Based Applications

Application Availability– The percentage of transactions that are successful

Application Responsiveness– The time that passes from when a user submits a request until when

they see the results (or a confirmation)

Throughput-Based Applications

Application Availability– The percentage of transactions that are successful

Application Responsiveness– The throughput (bytes/sec) seen from the beginning of the transaction

to the end

Streaming-based Applications

Application Availability– The percentage of transactions that are successful

Application Responsiveness– The percentage of time that the application delivered signal to the

user

– AKA: The percentage of time that the application kept the playback buffer from emptying

APM Metrics Model

Transaction-Oriented Transport-Oriented Streaming

Availability % Success % Success %Success

Responsiveness Response Time inTenths of seconds

Throughput inSeconds per Terabit

Percent of time with good signal, inParts per Million

Transaction-Oriented Transport-Oriented Streaming

Availability % Success % Success %Success

Responsiveness Response Time inTenths of seconds

Throughput inSeconds per Terabit

Percent of time with good signal, inParts per Million

Transaction Types

Some applications may look transaction-oriented sometimes and transport-oriented other times

Often classifying them by verbs will help

Sometimes you want to have both measurements available

MIB allows an implementer to choose which metrics are implemented per protocol

Transactions

Protocol Client Server Successful Responsiveness

HTTP Jim Amazon 1 6 sec.

SAP/R3 Jane SAP 1 17 sec.

HTTP Joe HR 0 -

FTP Jim ietf 1 212 Kbps

HTTP Joe HR 1 25 sec.

RealVideo Joe CNN 1 100.0%

HTTP Jane HR 1 5 sec.

HTTP Jim CNN 1 2 sec.

Flow-Based Aggregation

Protocol Client Server Availability Responsiveness

HTTP Jim Amazon 100% 6 sec.

SAP/R3 Jane SAP 100% 17 sec.

HTTP Joe HR 50% 25 sec.

FTP Jim IETF 100% 212 Kbps

RealVideo Joe CNN 100% 100.0%

HTTP Jane HR 100% 5 sec.

HTTP Jim CNN 100% 2 sec.

All transactions that share common protocol/server/client 3-tuples are aggregated together, resulting in a set of metrics for all such unique 3-tuples.

Client-Based Aggregation

Protocol Client Availability Responsiveness

HTTP Jim 100% 4 sec.

SAP/R3 Jane 100% 17 sec.

HTTP Joe 50% 25 sec.

FTP Jim 100% 212 Kbps

RealVideo Joe 100% 100.0%

HTTP Jane 100% 5 sec.

All transactions that share common protocol/client tuples areaggregated together, resulting in a set of metrics for allsuch unique tuples.

Server-Based Aggregation

Protocol Server Availability Responsiveness

HTTP Amazon 100% 6 sec.

SAP/R3 SAP 100% 17 sec.

HTTP HR 66% 15 sec.

FTP IETF 100% 212 Kbps

RealVideo CNN 100% 100.0%

HTTP CNN 100% 2 sec.

All transactions that share common protocol/server tuples are aggregated together, resulting in a set of metrics for all such uniquetuples.

Protocol-Based Aggregation

Protocol Availability Responsiveness

HTTP 80% 9.5 sec.

SAP/R3 100% 17 sec.

FTP 100% 212 Kbps

RealVideo 100% 100.0%

All transactions that share a common protocol are aggregatedtogether, resulting in a set of metrics for all such uniqueprotocols.

Can’t Aggregate Different Protocols

These transactions can be aggregated in several ways, providing

statistical summaries - for example summarizing all HTTP

transactions,or all HTTP transactions to the HR Server. Note that

data from different protocols may not be summarized because:

1. The performance characteristics of different protocols differ

widely enough to render statistical analysis meaningless.

2. The responsiveness metrics of different protocols may be different, making a statistical analysis impossible.

Aggregation Algorithms

TransactionCount– The total number of transactions during this period

SuccessfulTransactions– The total number of transactions that were successful

Aggregation Algorithms (cont.)

ResponsivenessMean– The average of the responsiveness metric for all aggregatedtransactions that completed

successfully

ResponsivenessMin– The minimum responsiveness metric for all aggregated transactions that completed

successfully

ResponsivenessMax– The maximum responsiveness metric for all aggregated transactions that completed

successfully

ResponsivenessBx– The count of successful transactions whose responsiveness metric fell into the range

specified for Bx. There are 7 buckets specified. Because the performance of different protocols varies widely, the bucket ranges are specified separately for each protocol (in the protocolDirExtTable) so that they may be tuned to typical performance of each protocol.

APM Report Table

APMReportControlTable

APMReport

APMReport

APMReport

APMReport

APMReport

APMReport Thu Mar 30 12-1PM

Joe CNN WEB stats

Jan POP POP3 stats

Jan POP SMTP stats

Bob HR PSOFT stats

...

...

Old

New

apmReportControlTable

apmReportControlIndex Integer32,

apmReportControlDataSource DataSource,

apmReportControlAggregationType INTEGER,

apmReportControlInterval Integer32,

apmReportControlRequestedSize Integer32,

apmReportControlGrantedSize Integer32,

apmReportControlRequestedReports Integer32,

apmReportControlGrantedReports Integer32,

apmReportControlStartTime TimeStamp,

apmReportControlReportNumber Integer32,

apmReportControlInsertsDenied Integer32,

apmReportControlDroppedFrames Counter32,

apmReportControlOwner OwnerString,

apmReportControlStatus RowStatus

apmReportControlTable

apmReportControlAggregationType OBJECT-TYPE

SYNTAX INTEGER {

flows(1), -- Least Aggregation

clients(2),

servers(3),

protocols(4) -- Most Aggregation

}

apmReportTable

apmReportIndex Integer32,

apmReportServerAddress OCTET STRING,

apmReportClientAddress OCTET STRING,

apmReportTransactionType INTEGER,

apmReportTransactionCount Integer32,

apmReportSuccessfulTransactions Integer32,

apmReportResponsivenessMean Integer32,

apmReportResponsivenessMin Integer32,

apmReportResponsivenessMax Integer32,

apmReportResponsivenessB1 Integer32,

apmReportResponsivenessB2 Integer32,

apmReportResponsivenessB3 Integer32,

apmReportResponsivenessB4 Integer32,

apmReportResponsivenessB5 Integer32,

apmReportResponsivenessB6 Integer32,

apmReportResponsivenessB7 Integer32

apmReportTable Indexing

Indexes– apmReportControlIndex– apmReportIndex,– protocolDirLocalIndex– protocolDirLocalIndex,– apmReportServerAddress– apmReportClientAddress– apmReportResponsivenessType

An example of the indexing of this entry is 3.15.34.18.4.128.2.6.7.4.128.2.6.6.1

Protocol Directory Extensions

protocolDirExtApmConfig INTEGER,

protocolDirExtApmResponsivenessType INTEGER,

protocolDirExtApmResponsivenessBoundary1 Integer32,

protocolDirExtApmResponsivenessBoundary2 Integer32,

protocolDirExtApmResponsivenessBoundary3 Integer32,

protocolDirExtApmResponsivenessBoundary4 Integer32,

protocolDirExtApmResponsivenessBoundary5 Integer32,

protocolDirExtApmResponsivenessBoundary6 Integer32

ApmExceptionEntry

Extends the ProtocolDirectory

Multiple threshold entries possible per protocol

ApmExceptionEntry ::= SEQUENCE {

apmExceptionIndex Integer32,

apmExceptionResponsivenessComparison INTEGER,

apmExceptionResponsivenessThreshold Integer32,

apmExceptionAvailabilityComparison INTEGER,

apmExceptionAvailabilityThreshold Integer32,

apmExceptionOwner OwnerString,

apmExceptionStatus RowStatus

}