tools, algorithms & system implementation for end-user performance monitoring

Download Tools, Algorithms & System Implementation for End-user  performance monitoring

Post on 23-Feb-2016




0 download

Embed Size (px)


Tools, Algorithms & System Implementation for End-user performance monitoring. dario.rossi. Dario Rossi . Agenda. Tools, algorithms System implementation End-user performance monitoring Two perspective: - PowerPoint PPT Presentation


Internet Traffic Classification

Tools, Algorithms & System Implementation for End-user performance monitoringdario.rossiDario Rossi AgendaTools, algorithmsSystem implementationEnd-user performance monitoring

Two perspective:Background (all available from my webpage)Foreground (open for collaboration)

BackgroundTools, AlgorithmsClassification (C45, SVM,..)Regression (ARMA,SVR,..)Statistical analysis (PCA, ANOVA,..)Inference (Apriori,)

Applied to:Traffic analysis & classificationSystem implementationTstatPassive flow-level sniffer, classifier, traffic analyzer

ModelNet-TEPacket-level emulator with Traffic Engineering capabilities

5Demonstration software at Sigcomm, Sigmetrics, Infocom, Globecom All available from SOFTWARE and DEMO categories at

End-user performance monitoringWebMethodology to infer, from TCP traffic, if a Web connection has been interruptedP2P-VoIPIn-depth black-box study of SkypeP2P-TV systemsAssessment of peer selection strategiesMore at Example: traffic classificationDeep Packet Inspection (DPI)Stochastic PacketInspection (KISS)Behavior analysis(Abacus)

GETMAIL FROM:BTSpecific KeywordApplication syntaxX M L TC S P TR S V PK G B XK G B XA P S TR S V PAlgorithm designEntropy of L7 header, Chi-square testContact weights CDFBhattaccharyya distance

Kiss vs Abacus algorithmsPPLiveTVAnts

Normalized c2 (first 14 header bytes)

Packets per sender peers pdf (5 sec intervals)

SopCast8 System implementation

ISP1HTTPYouTubeBitTorrentBitTorrent UDPOther UDPOther TCPeMuleISP59ForegroundInterestsVery high-speed implementation (>10Gbps)Monitoring & classification Federation of passive measurement pointsIncrease statistical relevance of measurementChallenging per seNew measures: Workload for CDN/ICNNew algorithms: Bufferbloat inferenceNew tools: Map-Reduce for traffic analysisSystem implementation (1/2)Wire-speed classification engines

Submitted to IMC12

System implementation (2/2)

ISP1ISP2Federation of passive measurement pointsAim: coalesce RRD data to increase statistical relevanceIncentive model: gain access to the aggregated dataImplementationStar topology: the root R fetch ISP1ISPn, aggregates on ISP* and redispatchChain: ISP2 aggregate ISP1 and ISP2, pass it to ISP3 and so on; chain ends at R that add its own data to ISP* and send it back P2P: structured vs unstructured? e.g., BitTorrent only to redispatch ISP*?13

ISPnSystem implementation (3/3)Exploit of (new) active measurement pointsCompare results between PlanetLab & e.g., BoincBoinc Aim: collaborative/volounteering computingUsed by: More than 295,000 worldwide locationIncentive to provide PCs: being on the top-100.Unexplored for network resources


End-user performance monitoring (1/2)Bufferbloat Large buffer size (128KB) + Narrow bw (1Mbps)= Queueing delay (1 sec)

Passive accurate methodto measure remote peers queue sizeIntegration on Dasu (BitTorrent plugin) to crowdsource ISP characterization ?

Submitted to IMC12


TCP AIMD fills the buffer!Nasty impact on interactive Web, VoIP, gaming trafficEnd-user performance monitoring (2/2)Workload for CDN/ICNGoal: assess the relevance of in-network cachingNeed: a relevant large-scale workloadChallengesCannot use Tier-1 backbone trace current dest. Server IP maps to CDN nodesCannot use DNS Caching => @root malformed > legitimate queries; frequencies avail at stub resolver, but impossible to get contemporary logs from many (>1000) of them Cannot use HTTPNot everything tunneled in HTTP; still, would need payload of Tier-1 backbone, with a large snaplen to get the full URLsSolution? In progress (=none so far)?? || //Backup slidesTraffic Classification TaxonomyApproachSubcategoryGranularityTimelinessComplexityCommentPayload Based[1,2] Deep Packet Inspection (DPI)Fine-grained individual applicationsEarly(first few packets).Access to packet payload of first few packets.Moderate costDeterministic technique; KISS[Ton10]Stochastic Packet InspectionFine-grained individual applicationsOnline (100s packets windows)Access to packet payload of several packets. High costRobust technique StatisticalAnalysis[4,5,6,7]Coarse-grained, class of applicationLate(after the flow end).Access to flow-level informationLightweight costPost-mortem analysis [8,9]Fine-grained individual applicationsEarly(first 5 packets)Access to first few packets Lightweight costOn the fly classificationBehavioralAnalysis[10,11]Coarse-grained, class of applicationLate(after the flow end).LightweightPost-mortem analysis Abacus [ComNet11]Fine-grained, individual P2P applicationsOnline (1s-5s seconds windows)LightweightOnline classificationLimited to P2POverviewDeep Packet Inspection (DPI)Stochastic PacketInspection (KISS)Behavior analysis(Abacus)

GETMAIL FROM:BTSpecific KeywordApplication syntaxX M L TC S P TR S V PK G B XK G B XA P S TR S V PAlgorithm design Y1 pkt1 cb d2 ... 02 60 Y1 pkt2 cc d5 ... 02 08 Y2 pkt1 01 da ... 02 65 Y1 pkt3 cd c0 ... 02 d9 Y2 pkt2 02 c1 ... 02 5c Y2 pkt3 03 dc ... 02 11 Y1 pkt4 ce cb ... 02 28 Y1 pkt5 cf d1 ... 02 8a Y1 pkt6 d0 ca ... 02 3a Y2 pkt4 04 c2 ... 02 b7 1) Extract the first N bytes of the payload from a window of W consecutive packets

2) Divide each byte in 2 chunks of 4 bits

3) Collect the frequency distribution Oi of the values assumed by each chunk

4) Compare the distribution to a uniform distribution Ei=/24 with a c2-like testcountersC||D = 3 bit fixedrandomdeterministicXY1Y2

measure the randomnessof each chunkKISS signature: [X1, X2, ... X2N]over W pkts KISS: Stochastic packet inspection

Header syntax is fixed, binary alphabet21Count the number of packets/bytes received in a fixed time window DT 2) Count the number of hosts sending a given number of packets/bytes (exponential binning) 3) Normalize the packet/bytewise counts to gather two probabilitymass functionsXY1Y22 4 8...Y3Y416

Y5Freq.Distribution = [1, 1, 3, 0]Signature = [0.2, 0.2, 0.6]Example using packets Abacus: Behavioral signatures

Applications implement different activities (signaling, data chunks) and tuning (chunk size)22

Kiss vs Abacus signaturesPPLiveTVAnts

Normalized c2 (first 14 header bytes)

Packets per sender peers pdf (5 sec intervals)

SopCast23Oops!Sorry, wrong key


View more >