Understanding Internet Dynamics from a Global View:Measurement Research in the WIDE Project

Kenjiro ChoIIJ/WIDE Project

November 19-20 2009

about meKenjiro Cho

I senior researcher at IIJ, a commercial ISP in JapanI a board member of WIDE, a research consortium in Japan

involved in traffic measurement research for the last 15 years

I data analysis of IIJ and WIDE trafficI data collection from other ISPsI publications/talks on Japanese residential traffic

international collaboration

I LIP6, ENS-Lyon, CAIDAI joined JST-CNRS collaboration between WIDE and LIP6

during 2006-2008I Serge Fdida, Kave Salamatian, Timur Friedman (LIP6),

Patrice Abry (ENS-Lyon)I community services

I ACM SIGCOMM2009 TPC, ACM CoNEXT steering committeeI OneLab2 advisory committeeI AINTEC steering committee, AsiaFI board member

2 / 26

hot topics in networking reseach

I future Internet researchI GENI/FIND in US, FIRE in EU, AsiaFI, Akari in Japan

I cloud computingI scale-merit, interoperability issues, network virtualization

I data center networkingI large layer-2 networks, power consumption, cooling

I content deliveryI shift of traffic from Tier-1 ISPs to gigantic service providersI peer-to-peer?

I effective use of whitespace

3 / 26

Understanding Internet Dynamics from a Global View

the Internet is an evolving open system

I no central point, no typical network or user

I complexity everywhere: topology, usage distribution, etc

I interferences among various components at different layers

I local optimizations often lead to a negative impact to theglobal system

I importance of global viewsI still, details matter (global impact of local mechanisms)

we still don’t have real science!

4 / 26

WIDE ProjectWIDE: Widely Integrated Distributed Environment

I a research consortium in Japan since 1988I about 100 sponsor companiesI 40 universities, 5 national research institutionsI 200 active members

I 1st decade (1988-1997): building the InternetI deployment, internationalization

I 2nd decade (1998-2007): for everyone, anytime, anywhereI IPv6, mobility, multimedia

I 3rd decade (2008-2017): beyond InternetI real-space networking, sensors, broadcasting

5 / 26

Research at WIDEmotto: research on our left hand, operation on our right hand.supporting social infrastructure with both hands.

I protocols and middlewareI KAME/USAGI, NEMO, MANET, DVTS

I testbedsI AI3, StarBED, JGN2plus, GLIF, WiMAX

I real space InternetI AutoID, Live E!, Locky, InternetCAR

I social interactionI SOI/SOI-Asia, Lens

6 / 26

characteristics of WIDE traffic

I WIDEI internet research through live network

I WIDE has its own backbone operated by membersI backbone includes

I international linksI IXesI root name serversI various link types up to 10GbE

I carrying both commodity traffic and experimentsI commodity: university traffic, WIDE membersI experiments: new products, our technologies under

developmentI IPv6 everywhereI events (including firedrills)

I not a typical internet but a showcase

7 / 26

traffic measurement and analysis in WIDE

I measurement activities across research groupsI broad perspectives

I tracking long-term trendsI analysis (with wide range of granularity)I operational tools (trouble-detection/shooting)I evaluation of new technologies

I emphasis onI wide-areaI multi-point

I measurement on backbone

I long-termI continuation by group effort

8 / 26

traffic measurement activities within WIDE

1. MAWI traffic repository

2. Residential Broadband Traffic Study

3. Regional AS Topology Structures

4. Anomaly Detection by Sketch and Non GaussianMultiresolution Statistical Detection Procedures

5. Longitudinal Statistical Analysis based on Robust Estimation

6. Host Clustering by Communication Patterns

9 / 26

international collaboration

I CAIDA (the Cooperative Association for Internet Data Analysis)

I collaboration since 2003 on DNS, topology, routing

I CASFI (Korean measurement effort)

I joint-workshops, data sharing

I CNRS

I measurement and modeling of emerging applications andsecurity threats

I other collaboration

I ISC OARC, USC/ISI, routeviews, RIPE, INRIA, AIT

I A day in the life of the Internet

I simultaneous measurement worldwide to promote researchbased on re-examinable data

I CAIDA, CASFI, WIDE, DNS root ops, etc.

10 / 26

MAWI Traffic Repository

I pcap packet traces from WIDE backbones

I anonymized traces publicly available

I many papers used MAWI traces

http://mawi.wide.ad.jp/mawi/

Kenjiro Cho, Koushirou Mitsuya and Akira Kato.Traffic Data Repository at the WIDE Project.USENIX FREENIX Track, San Diego, CA, June 2000.

11 / 26

Residential Broadband Traffic Study (1/2)key question: what is the macro level impact of video andother rich media content on traffic growth at the moment?

I traffic growth is one of the key factors driving research,development and investiment in technologies andinfrastructures

I crucial is the balance between demand and supply

I measurements: 2 data setsI aggregated SNMP data from 6 ISPs covering 42% of Japanese

trafficI Sampled NetFlow data from 1 ISP

K. Cho, K. Fukuda, H. Esaki, and A. Kato.Observing Slow Crustal Movement in Residential User Traffic.ACM CoNEXT2008, Madrid, Spain, Dec. 2008.

K. Cho, K. Fukuda, H. Esaki, and A. Kato.The impact and implications of the growth in residential user-to-user traffic.ACM SIGCOMM2006, Pisa, Italy, Aug. 2006.

12 / 26

Residential Broadband Traffic Study (2/2)daily traffic volume per user

I increase in download volume of client-type usersI out mode: from 32MB/day to 114MB/dayI in mode: from 3.5MB/day to 6MB/day

I while peer-type dist. isn’t growing much (mode:2GB/day)

104105 106 107

108 109 1010 1011

Daily traffic per user (bytes)

0

0.1

0.2

0.3

0.4

0.5

Prob

abili

ty d

ensi

ty

2005 (in)2005 (out)2009 (in)2009 (out)

changes in daily traffic per user (2005 vs. 2009)

13 / 26

Regional AS Topology Structures (1/2)

I study on topologies from regional viewsI Internet development affected by geographical constraintsI existing global topologies do not capture regionality

I goalsI understand similarities and differences in regional Internet

structuresI e.g., identifying regional hub ASes, hub cities

I methodologyI extract nodes in the region in traceroute data setsI by reverse DNS names, geo-IP mapping, etc

I visualization by CAIDA’s AS core map

Yohei Kuga, Kenjiro Cho, Osamu Nakamura.On inferring regional AS topologies.AINTEC2008. Bangkok, Thailand. Nov. 2008.

14 / 26

Regional AS Topology Structures (2/2)

regional topology visualization by AS Core Map

Europe (top left), Asia (top right), US-west (bottom left), US-east (bottom right)

15 / 26

Anomaly Detection by Sketch and Non GaussianMultiresolution Statistical Detection Procedures (1/2)

collaboration with Abry’s team at ENS-LyonI features

I generates self-reference from the target traffic, no trainingrequired

I can detect small hidden anomaliesI works with uni-directional data (applicable to backbone)

Guilaume Dewaele, Kensuke Fukuda, Pierre Borgnat, Patrice Abry, Kenjiro Cho.Extracting Hidden Anomalies using Sketch and Non Gaussian MultiresolutionStatistical Detection Procedures.SIGCOMM2007 LSAD Workshop. Kyoto Japan. August 2007.

16 / 26

Anomaly Detection by Sketch and Non GaussianMultiresolution Statistical Detection Procedures (2/2)

I sketch: divides packets into N groups by hashing

I for each group, extract statistical features from packet arrivaldistribution in multiple time resolutions

I compare normalized features among the groups, detectdeviations as anomalies

17 / 26

Longitudinal Statistical Analysis based on RobustEstimation (1/3)

the same sketch technique is used to extract typical behaviors forlong-term traffic analysis

I traffic statistics show huge variability

I median sketch as robust estimation to observe traffic evolution

I strong and persistent long range dependence found

Pierre Borgnat, Guillaume Dewaele, Kensuke Fukuda, Patrice Abry, Kenjiro Cho.Seven Years and One Day: Sketching the Evolution of Internet Traffic.INFOCOM2009. Rio de Janeiro, Brazil. April 2009.

18 / 26

Longitudinal Statistical Analysis based on RobustEstimation (2/3)

MiB/s

0s 150 300 450 600 750 900s0

0.5

1

1.5

2 LD for Byte count Hg=0.94

Hm

=0.88

2ms 16ms 128ms 1s 8s 64s

MiB/s

0s 150 300 450 600 750 900s0

0.5

1

1.5

2 LD for Byte count Hg=0.41

Hm

=0.80

2ms 16ms 128ms 1s 8s 64s

MiB/s

0s 150 300 450 600 750 900s0

0.5

1

1.5

2 LD for Byte count Hg=0.73

Hm

=0.79

2ms 16ms 128ms 1s 8s 64s

top: no congestion, middle: congestion, bottom: severe anomalies

left: sketch sub-trace time series, right: LDs for global, sketches, median-sketch

19 / 26

Longitudinal Statistical Analysis based on RobustEstimation (3/3)

I LRD over 7 years: global (gray) and median-sketch (black)estimates of H during 2001-2008

H (p

acke

ts) US2JpUS2Jp

2001 2 3 4 5 6 7 20080.40.60.8

11.2

H (b

ytes

)

US2JpUS2Jp

2001 2 3 4 5 6 7 20080.40.60.8

11.2

20 / 26

Host Clustering by Communication Patterns (1/3)

I profiling traffic at host level

I unsupervised statistical classification

I 9D features for clustering

I cross validation with existing tools

I visualization by graphlets

G. Dewaele, Y. Himura, P. Borgnat, K. Fukuda, P. Abry, O. Michel, R. Fontugne,K. Cho, H. Esaki.Unsupervised host behavior classification from connection patterns.submitted for publication.

21 / 26

Host Clustering by Communication Patterns (2/3)

9 features for clustering (S: entropy)I network connectivity

I # of dst addrsI # of src ports / # of dst addrsI # of dst ports / # of dst addrs

I connection dispersion in address spaceI S(IP2)/S(IP4)I S(IP3)/S(IP4)

I packet size distributionI mean # of packets/flowI % of small packets (≤ 144 bytes)I % of large packets (≥ 1392 bytes)I S(medium size packets)

22 / 26

Host Clustering by Communication Patterns (3/3)

Minimum Spanning Tree (MST) clustering

1. plot hosts in (reduced 2D) feature space, pick a random hostto start MST

2. remove edges longer than a threshold (dashed line)

3. dense clusters are divided further

(1) (2) (3)

MST clustering illustrated in 2D space

23 / 26

Host Connection Pattern Visualization

I goal: inspections of anomaly detection/host behaviorclustering results

I graphlets (BLINC[Karagiannis05]) to show src-based 5 tuplepatterns

I automation tool under development

sample graphlets: P2P (left) and scanning (right)

24 / 26

Summary

measurement actitivies at WIDE

I operational support

I tool development

I data sharing

I modeling and analysis

I visualization

I many more measurement related activities

WIDE is interested in open research collaboration

http://www.wide.ad.jp/

25 / 26

towards JST-ANR collaboration

I international collaboration is a common practice in networkingresearch

I shift from US-centric to global distribution in networks ofexcellence

I France-Japan: key players in EU-AsiaI a framework/funding is essential for progressI Japanese research tends to be domestic

I needs a global view/more exposure to the worldI France-Japan is an important step

I ambicious tight collaboration for top-level researchI impact to the realworld society

I broader research exchanges are also importantI especially for students/young researchers

26 / 26