a survey of tools for monitoring and visualization of network traffic

64
MASARYK UNIVERSITY FACULTY OF I NFORMATICS A Survey of Tools for Monitoring and Visualization of Network Traffic BACHELORS THESIS Jakub Šk ˚ urek Brno, Fall 2015

Upload: duongkhanh

Post on 14-Feb-2017

222 views

Category:

Documents


4 download

TRANSCRIPT

Page 1: A Survey of Tools for Monitoring and Visualization of Network Traffic

MASARYK UNIVERSITYFACULTY OF INFORMATICS

}w���������� ������������� !"#$%&'()+,-./012345<yA|A Survey of Tools for

Monitoring and Visualizationof Network Traffic

BACHELOR’S THESIS

Jakub Škurek

Brno, Fall 2015

Page 2: A Survey of Tools for Monitoring and Visualization of Network Traffic
Page 3: A Survey of Tools for Monitoring and Visualization of Network Traffic

Declaration

Hereby I declare, that this paper is my original authorial work, whichI have worked out by my own. All sources, references and literatureused or excerpted during elaboration of this work are properly citedand listed in complete reference to the due source.

Jakub Škurek

Advisor: doc. Ing. Jirí Sochor, CSc.

ii

Page 4: A Survey of Tools for Monitoring and Visualization of Network Traffic
Page 5: A Survey of Tools for Monitoring and Visualization of Network Traffic

Abstract

The main goal of this thesis is to survey and categorize existing toolsand applications for network traffic monitoring and visualization.This is accomplished by first dividing these into three main cate-gories, followed by additional subdivision into subcategories whilepresenting example tools and applications relevant to each category.

iii

Page 6: A Survey of Tools for Monitoring and Visualization of Network Traffic
Page 7: A Survey of Tools for Monitoring and Visualization of Network Traffic

Keywords

network monitoring, network visualization, survey, traffic, monitor-ing

iv

Page 8: A Survey of Tools for Monitoring and Visualization of Network Traffic
Page 9: A Survey of Tools for Monitoring and Visualization of Network Traffic

Acknowledgement

I would like to thank my advisor doc. Sochor for his advice and help-ful tips. A big thank you also goes to my family for the continuedsupport and kind words given, and mainly for being understandingof my reclusive behaviour during the writing process.

v

Page 10: A Survey of Tools for Monitoring and Visualization of Network Traffic
Page 11: A Survey of Tools for Monitoring and Visualization of Network Traffic

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Computer Network Monitoring and Visualization . . . . . 5

2.1 Simple Network Management Protocol . . . . . . . . . 52.2 NetFlow/IPFIX Protocol . . . . . . . . . . . . . . . . . . 62.3 Internet Control Message Protocol . . . . . . . . . . . . 62.4 Windows Management Instrumentation . . . . . . . . . 72.5 Packet Capture . . . . . . . . . . . . . . . . . . . . . . . 7

3 Tools for Local Network Visualization . . . . . . . . . . . . 93.1 Local Traffic and Performance Visualization . . . . . . . 9

3.1.1 NetGrok . . . . . . . . . . . . . . . . . . . . . . . 93.1.2 EtherApe . . . . . . . . . . . . . . . . . . . . . . 11

3.2 Packet Capture and Visualization . . . . . . . . . . . . . 133.2.1 Wireshark . . . . . . . . . . . . . . . . . . . . . . 13

3.3 Network Log Data Visualization . . . . . . . . . . . . . 163.3.1 Time-based Network Visualizer . . . . . . . . . 163.3.2 AfterGlow . . . . . . . . . . . . . . . . . . . . . . 18

3.4 Summary of presented tools and their features . . . . . 204 Tools for Global Network Visualization . . . . . . . . . . . 21

4.1 Global Traffic and Performance Visualization . . . . . . 214.1.1 Cacti . . . . . . . . . . . . . . . . . . . . . . . . . 214.1.2 PRTG Network Monitor . . . . . . . . . . . . . . 244.1.3 PhpWeatherMap . . . . . . . . . . . . . . . . . . 26

4.2 Network Topology Mapping . . . . . . . . . . . . . . . 274.2.1 The Dude . . . . . . . . . . . . . . . . . . . . . . 284.2.2 SolarWinds Network Topology Mapper . . . . . 29

4.3 Summary of presented tools and their features . . . . . 305 Tools for Anomaly Analysis and Intrusion Detection . . . . 32

5.1 Snort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335.2 Suricata . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365.3 Open Source HIDS SECurity . . . . . . . . . . . . . . . . 37

1

Page 12: A Survey of Tools for Monitoring and Visualization of Network Traffic

6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39A Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

A.1 Netgrok . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45A.2 EtherApe . . . . . . . . . . . . . . . . . . . . . . . . . . . 46A.3 Wireshark . . . . . . . . . . . . . . . . . . . . . . . . . . 47A.4 Time-based Network Visualizer . . . . . . . . . . . . . . 48A.5 PRTG Network Monitor . . . . . . . . . . . . . . . . . . 49A.6 PhpWeatherMap . . . . . . . . . . . . . . . . . . . . . . 50A.7 Solarwinds Network Topology Mapper . . . . . . . . . 51A.8 Suricata . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52A.9 Open Source HIDS SECurity . . . . . . . . . . . . . . . . 53

B Supplementary Data Files . . . . . . . . . . . . . . . . . . . . 54B.1 IRCFlood.pcap . . . . . . . . . . . . . . . . . . . . . . . . 54

2

Page 13: A Survey of Tools for Monitoring and Visualization of Network Traffic

Chapter 1

Introduction

The Internet became, in the last couple of years, an indispensablepart of many people’s lives. It is entangled in almost every cornerof our society and life without it is hard to fathom. Ease of accessto information, simplifying socialization or facilitation of commerce.Indispensable in everyday life, it represents an absolute must formany firms, companies and institutions. For these it is a key tool forcommunication, operation and advertisement. However, Internet’sbiggest strength is at the same time its biggest weakness—accessto it is available to anyone, even people with less than pure inten-tions. It then follows that being connected to the Internet means ex-posing oneself to risk of a wide range of potential malicious attacksaimed at retrieving one’s private and potentially valuable informa-tion. Chance of catching the culprit hiding behind the vast amountsof data produced by computer networks is without the proper in-struments almost a herculean task. This is where tools for monitoringand visualization of computer network traffic, whose categorizationand brief overview is this thesis dedicated to, prove invaluable. Pur-pose of these tools is to provide the user holistic and easily navigabledata about historical and actual happenings on a computer network,and assist him with comparing these events with those compatibleto a network attack or other malfunctions.

As briefly stated above, the aim of this thesis is to provide anoverview of popular currently in-use visualization tools and soft-ware for network monitoring and visualization and their differentapproaches towards depiction of network data. Dozens of such ap-

3

Page 14: A Survey of Tools for Monitoring and Visualization of Network Traffic

1. INTRODUCTION

plications exist, but no comprehensive overview has to date been cre-ated [2].

The tools are, for the purpose of this thesis, divided into threemain categories—local network monitoring, global network moni-toring, anomaly analysis and network intrusion detection—with achapter dedicated to each one. Tools selected to represent each cate-gory are based on their performance in said area and the uniquenessof their visualization technique. Focus is, for each tool, only on itsapplications in the category under which it is presented. Conclusionsto chapter three and four contain a summary table of all the featurestools covered in that given chapter possess.

The thesis, minus the introduction and conclusion, is divided intofour chapters. The first one gives a brief introduction into the areaof interest and provides a list of most common protocols and ap-proaches to gathering network data used for visualization. The nextthree chapters then deal with tools pertinent to them. A summary ofthe findings is given in the concluding sixth chapter.

The area of network visualization is still very young and chaotic.Big data and the problems connected with their visualization are avery recent phenomenon and adequate solutions are yet to be found[3]. This thesis is trying to act as a building block for a follow upresearch in this are by giving an overview of popular currently in-use visualization techniques. The final goal of this research wouldthen be systems capable of recognizing not only catalogued mali-cious software, but to also be able to prevent zero-day 1 attacks inreal-time.

1. A zero-day (also known as zero-hour or 0-day) vulnerability is an undisclosedcomputer application vulnerability that could be exploited to adversely affect thecomputer programs, data, additional computers or a network. It is known as a"zero-day" because once the flaw becomes known, the application author has zerodays in which to plan and advise any mitigation against its exploitation (by, forexample, advising workarounds or issuing patches) [4].

4

Page 15: A Survey of Tools for Monitoring and Visualization of Network Traffic

Chapter 2

Computer Network Monitoring and Visualiza-tion

Computer network monitoring and visualisation are areas closelytied together. Where network monitoring tools produce vast amountsof data, visualization tools give the data form that is concise, unclut-tered and easier for the user to digest.

This chapter aims to, before venturing forth towards categoriz-ing the various monitoring tools and applications, provide a briefoverview of the most common methods these tools use to gather datafrom the monitored network. Efficient data collection is an importantpart of the monitoring process. For this purpose many protocols andapproaches exist, are being developed, or refined.

2.1 Simple Network Management Protocol

Simple Network Managment Protocol (SMNP) is a protocol used fordata collection from, and configuration and management of, networkdevices such as routers, switches, servers and hubs on an InternetProtocol network. It uses variables known as object identifiers (OID)to allow remote data collection and device configuration. OIDs arestored in a hierarchy in Management information bases (MIB). Aprotocol query to a device produces a set of OIDs with accompa-nying data, showing, for example, interfaces available on a routeror a switch and traffic currently passing though them, which is thencross-referenced with an MIB and presented to the user.

To date, three versions of the protocol have been introduced. Ver-sion 1, also known as SNMPv1, is the initial and most widely sup-ported version of the protocol despite the criticism of its very poor

5

Page 16: A Survey of Tools for Monitoring and Visualization of Network Traffic

2. COMPUTER NETWORK MONITORING AND VISUALIZATION

security[5]. Version 2 (SNMPv2c) only expands upon the previousversion with the support of larger data types and additions to pro-tocol operation, leaving the security concerns unaddressed. Version3 (SNMPv3) is the most recent version implementing strong securityauthentication and data encryption, keeping its primary functional-ity unchanged.

2.2 NetFlow/IPFIX Protocol

NetFlow is a feature that was introduced on Cisco routers that pro-vides the ability to collect IP network traffic as it enters or exits aninterface. By analyzing the data provided by NetFlow, a user can de-termine things such as the source and destination of traffic, class ofservice, and the causes of congestion. A flow is a stream of data thatshares the network interface, source and destination IP addresses,ports and the same type of service header (for IPv4 transmissions)[6]. Once a flow is detected, a related flow record, aggregating infor-mation about all packets belonging to the same flow, is created.

A typical Netflow monitoring setup consists of a flow exporter,usually a Cisco router, that aggregates packets into flows and ex-ports flow records towards a flow collector that is responsible forrecord collection and storage and an analysis application that ana-lyzes received flow data in the context of intrusion detection or trafficprofiling.

Versions most commonly found today are Netflow 5, only re-stricted to IPv4, and Internet Protocol Flow Information eXport(IP-FIX), which is heavily based on Netflow 9 and standardised by theIETF[6].

2.3 Internet Control Message Protocol

The Internet Control Message Protocol (ICMP) is one of the mainprotocols of the Internet Protocol Suite. It is used by network devices,like routers, to send error messages indicating, for example, that arequested service is not available or that a host or router could notbe reached. ICMP can also be used to relay query messages [7].

6

Page 17: A Survey of Tools for Monitoring and Visualization of Network Traffic

2. COMPUTER NETWORK MONITORING AND VISUALIZATION

Although ICMP messages are contained within standard IP pack-ets, ICMP messages are usually processed as a special case, distin-guished from normal IP processing, rather than processed as a nor-mal sub-protocol of IP. In many cases, it is necessary to inspect thecontents of the ICMP message and deliver the appropriate error mes-sage to the application that generated the original IP packet, the onethat sent the packet that prompted the sending of the ICMP message.

Many network utilities commonly in use are based on ICMP mes-sages. The traceroute command can be implemented by transmittingIP datagrams with specially set IP TTL header fields, and looking forICMP Time to live exceeded in transit (above) and "Destination un-reachable" messages generated in response. The related ping utility isimplemented using the ICMP "Echo request" and "Echo reply" mes-sages. Network monitoring tools implement ICMP as an availabil-ity checker by querying a device and waiting for a reply signifyingavailability or unavailability of the device.

2.4 Windows Management Instrumentation

Windows Management Instrumentation (WMI) is the infrastructurefor management data and operations on Windows-based operatingsystems. Users can write WMI scripts or applications to automate ad-ministrative tasks on remote computers but WMI also supplies man-agement data to other parts of the operating system and products [8].In network monitoring software, WMI is used to access various con-figuration parameters and status values on Windows systems. Onedownside of this service is its relatively high impact on system per-formance [9].

2.5 Packet Capture

Packet capture (PCAP) refers to an API used for capture and loggingof network traffic. PCAP is implemented in the libpcap library forUnix-like system and the WinPcap library for Windows. Tools such

7

Page 18: A Survey of Tools for Monitoring and Visualization of Network Traffic

2. COMPUTER NETWORK MONITORING AND VISUALIZATION

as tcpdump1 and other monitoring software use this library to cap-ture packets traveling over a network and export them into packetcapture files (PCAPs). It can then be said that each PCAP file con-tains a time-bounded snapshot of activity on a given network inter-face. Such files are then used for archival purposes, further analysis,or as input files for packet analyzer2 software.

1. tcpdump is a common packet analyzer that runs under the command line. Itallows the user to display TCP/IP and other packets being transmitted or receivedover a network to which the computer is connected.2. Packet analyzer intercepts and logs traffic as it passes over a computer net-work. It can decode contents of individual packets and give alerts according tospecification.

8

Page 19: A Survey of Tools for Monitoring and Visualization of Network Traffic

Chapter 3

Tools for Local Network Visualization

Local network visualization encompasses all forms of monitoringand data gathering that can be done on a single network-connecteddevice. This category was for the purpose of this thesis divided intotraffic and performance visualization, packet capture and visualiza-tion of their contents, and log data visualization.

3.1 Local Traffic and Performance Visualization

Local traffic and performance visualization comprises all traffic andother network data that can be gathered and visualized on a localhost computer belonging to the network. There are very few toolsthat deal only with visualisation of local traffic, however, they arestill very popular and frequently in use. Two examples of such toolsare NetGrok and EtherApe, presented below.

3.1.1 NetGrok

NetGrok is a Java based tool for visualizing computer network us-age in real-time. It was created in spring 2008 during the Informa-tion Visualization course at the University of Maryland, College Parkwith the main goal of creating a tool that can capture traces froma live network interface, and filter the data set dynamically by band-width, number of connections, and time [10].

It supports two types of input – live listening on a selected net-work interface in promiscuous mode and packet capture files. Onceone of those conditions is satisfied NetGrok can begin its visualiza-tion process. NetGrok can then show the captured data in two dif-ferent ways—force-directed graph, treemap—which allow for differ-

9

Page 20: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

ent approaches in reading data. In addition to this is the edge tablethat lists all connections and data transferred between hosts capturedduring a session.

Figure 3.1 showcases the application’s Graph view. Nodes insidethe dashed blue circle are hosts on a local network, nodes outsidethe circle denote foreign hosts. Foreign hosts are laid out using ahashing algorithm paired with a conversion into planar coordinates,meaning they will always appear in the same place on any NetGrokinstallation and leading to easier orientation in larger networks[10].Color indicates bandwidth utilization with red denoting hosts withthe most bandwidth used and clear green denoting least bandwidthusage. White nodes surrounded by a dashed gray circle indicate zero-byte hosts 1. Node size represents number of connections propor-tional to other nodes. Connections between hosts can be seen bymousing over. Each node is labeled with its IP address, but a DNSlookup is possible.

TreeMap view transforms captured data into a 2D treemap [un-derline?] where local and foreign hosts are separated by a thick blackline. Again, hosts with the most connections are largest, those con-suming the most bandwidth red and least yellow. Treemap view isuseful when monitoring a large number of hosts that produce a verycluttered graph view.

Grouping of both foreign and local hosts is possible. Groupingis done in a groups.ini file that contains groups in the format Name-OfGroup = IPBlock1,IPBlock2,...,IPBlockn. Example of a groups.ini fileused in Figure 3.1:

[ l o c a l ]P r i v a t e 1 = 1 9 2 . 1 6 8 . 0 . 0 / 1 6P r i v a t e 2 = 1 7 2 . 1 6 . 0 . 0 / 1 2P r i v a t e 3 = 1 0 . 0 . 0 . 0 / 8

[ f o r e i g n ]Google = 2 1 6 . 2 3 9 . 0 . 0 / 1 6 , 6 4 . 2 3 3 . 0 . 0 / 1 6 , 6 4 . 6 8 . 0 . 0 / 1Last .FM= 8 7 . 1 1 7 . 2 2 9 . 0 / 2 4Wikimedia = 2 08 . 8 0 . 1 5 2 .0 / 2 4Valve = 1 6 2 . 2 5 4 . 0 . 0 / 1 6

1. The term Zero-byte host is defined as a host that receives traffic, but sends noneback [10].

10

Page 21: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

Figure 3.1: A snapshot of NetGrok’s Graph view. For a full image seethe appendix A.1.

Hosts grouped in this way show in close proximity to each otherin graph view, and nested inside a thick rectangle in treemap view.Timeline manipulation is also present, allowing filtering for connec-tions only active at the selected time. Additionally, search by IP ad-dress, bandwidth usage and number of connections is possible.

3.1.2 EtherApe

EtherApe is a graphical network monitoring software for Unix-likesystems developed and maintained by Juan Toledo et al. [11]. It isreleased as an open source software under the GNU General PublicLicense.

It features three modes of visualization. Link mode, showing traf-fic between nodes on a link layer, IP mode, showing nodes and ac-

11

Page 22: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

tive communications between them and TCP mode showing show-ing source and destination ports along with a used protocol.

All nodes are labeled with their IP address or a name if a DNSlookup is performed and visualized in a circular graph with linksshowing connections between them. Links are color coded accordingto the most used protocol and link size shows instantaneous trafficbetween two nodes.

Details about nodes exchanging traffic are found in the Nodespanel. Nodes can be filtered by name, the amount of instantaneousand accumulated traffic, number of packets and average packet sizeexchanged, and time since last communication occurred. Similar fil-tering is also available for individual protocols detected.

Figure 3.2: TCP traffic captured using EtherApe. For a full image seethe appendix A.2.

Figure 3.2 demonstrates the application’s TCP view, showing allnetwork and port traffic on a 10.0.0.0/24 home network, with toptalker being 10.0.0.4. All the ports with accompanying protocols areshown. Largest volume of traffic comes from 185.42.205.79, which isa twitch.tv streaming multicast server, over HTTP.

12

Page 23: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

EtherApe also includes the ability to filter captured traffic usingfilter syntax src net IPADDRESS dst net IPADDRESS, where IPAD-DRESS is the actual address or an address block to monitor. Addressblocks are defined by leaving the ranges defining a block blank in theIP definition. Multiple address blocks can be joined, excluded etc. us-ing boolean operators. Example filter syntax that shows only trafficon a 192.168.1.X network is then src net 192.168.1 dst net 192.168.1. Formore details about EtherApe and its functionality see [11].

3.2 Packet Capture and Visualization

A packet capturer is a computer program or piece of computer hard-ware that can intercept and log traffic that passes over a digital net-work or part of a network. As data streams flow across the network,the sniffer captures each packet and, if needed, decodes the packet’sraw data, showing the values of various fields in the packet, and ana-lyzes its content according to the appropriate RFC (Request for Com-ments) or other specifications.

Many packet capture applications are also packet analyzers andcan serve as network intrusion detection (NIDS) agents. For a de-tailed overview of NID systems see chapter 5.

3.2.1 Wireshark

Wireshark is one of the oldest packet capture tools in existence. Firstknown as Ethereal since its conception in 1998 until roughly mid2006, when it was renamed to Wireshark due to trademark issues[12].Wireshark is a network protocol analyzer that lets the user captureand interactively browse network traffic. It is developed and main-tained by The Wireshark team and freely available as open sourceunder GNU General Public License version 2.

Wireshark captures network traffic from a given network inter-face. It can do so locally, or globally by setting the interface, if it sup-ports it, into promiscuous mode. In some cases however, not evenpromiscuous mode assures total capture of all network traffic. If cap-turing on a port on a network switch, not all traffic through the switchis necessarily sent to the port where capture is done. This can be

13

Page 24: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

remedied either by port mirroring 2 or network taps 3 to extend cap-ture throughout the whole network.

Traffic captured by Wireshark is then shown in a table, with eachrow containing one packet, in the main application window (snap-shot in Figure 3.3). Packets are queued for inspection in the orderof capture, starting with the earliest. Each packet is also assigneda number alongside its timestamp for ease of orientation. Row back-ground is colored based on the packet protocol and font color basedon type — distinction is made, for example, between TCP/IP SYNACK packets and RST packet. Wireshark supports customizable colorfilters that can be changed on the fly and alter the way Wiresharkcolor codes packets, as well as temporary filters that can be appliedto parts of the data and cease to function once the capture sessionis over. Community-made color filters, highlighting TCP retransmis-sion, grouping WAP 802.11 packets, emphasizing and detecting er-rors in client/server communication, etc. are available for download[13]. Each packet can be further inspected to fully see both its headerand content.

In addition to color coding, filtering of irrelevant data by pro-tocol, source, destination, etc. is also possible. Wireshark containsdozens of built-in filters within the default installation, and also al-lows the creation of custom filters through the use of a built in filterlanguage. For a detailed introduction to custom filter creation see[14].

Example capture is shown in Figure 3.3. Highlighted along withfull header and content details is an ARP packet from 10.0.0.2 (MAC0c:d2:92:03:dd:5b) with a broadcast destination (MAC ff:ff:ff:ff:ff:ff),querying for the MAC of 10.0.0.138. Packet directly underneath is anARP response from 10.0.0.138.

Wireshark contains many additional feature. Since they are toonumerous to list and showcase fully, only those deemed most im-

2. Port mirroring is used on a network switch to send a copy of network packetsseen on one switch port a network monitoring connection on another switch port.3. A network tap is a hardware device which provides a way to access the dataflowing across a computer network. It has at least three ports A, B and a monitorport. A tap inserted between points A and allows network traffic to pass throughunimpede, but also copies all data to its monitor port, enabling third party listen-ing.

14

Page 25: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

Figure 3.3: Snapshot of a Wireshark interface showing manipulationwith captured network traffic. For a full image see the appendix A.3.

portant are mentioned: Ability to follow and decode conversationstreams between hosts, including voice over IP (VoIP) protocols thatcan be decoded and listened to (if captured under optimal circum-stances) [15]. Generation of packet length and IO packet statistics.Listing of all captured conversations with numbers of packets trans-ferred, total bandwidth consumed, conversation start and end times.Plugin support for protocols not supported in the default installa-tion.

15

Page 26: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

3.3 Network Log Data Visualization

Log data are a form of big data produced by applications and ser-vices in response to events and notable occurrences. In computer net-works, log data are most often produced by gateway routing rules,firewalls and tools monitoring traffic passing though those points.Many tools to visualize such data exist, two examples are presentedbelow.

3.3.1 Time-based Network Visualizer

Time-based Network Visualizer(TnV) depicts network traffic by vi-sualizing packets and links between local and foreign hosts. It is aJava based tool developed by John Goodall et al. and released underthe open-source MIT license.

TnV allows for two types of input – live capture from a networkinterface and PCAP packet capture files. In either case input data isparsed and stored in a MySQL database. No real-time visualizationis possible; live capture must end before the parsing process can be-gin. Captured data can be exported as PCAP files or stored in thedatabase for trend monitoring and analysis.

Main window of the application depicts foreign hosts on the leftside and local hosts in a reorderable matrix on the right side withlinks drawn between them. Matrix in the middle shows traffic foreach host on the local network. Background color of each cell showsaggregated packet activity. Packets are depicted as triangles with thepoint showing their directionality. Both packets and links are colorcoded to their respective protocols.

Selecting a cell within the matrix, representing a local host for acertain time period, the user can either show the packet details orthe port activity related to that host. Main mechanism for movingthrough the captured data is a scroll bar that sets the viewable sectionwith a bar graph showing overview of network activity. Sidebar onthe right side illustrates ports, both source and destination, involvedin the selected conversations.

Figure 3.4 shows the outlined functionality on a PCAP sample ofan Internet Relay Chat (IRC) flood attack, a typical example of a De-nial of Service attack with the goal of disconnecting the user from an

16

Page 27: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

Figure 3.4: Visualization of an IRC flood attack using TnV. For a fullimage see the appendix A.4.

IRC server by abusing the fact that the maximum number of mes-sages that can be sent in a given time frame is limited and controlledby the server. Flooding the user with messages thus causes the serverto disconnect him with a “Excess Flood” message. Highlighted pack-ets show communication between local host 192.168.248.105 and for-eign host 64.32.28.7 on port 5553, a non-standard IRC port, millisec-onds before a flood from 208.98.1.12 commences. In the next threeseconds local host is flooded with hundreds of packets until the com-munication ceases, signifying termination of the IRC session.

17

Page 28: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

3.3.2 AfterGlow

AfterGlow is a tool that facilitates the process of creating link graphvisualization of network traffic log data written in Perl created andmaintained by Raffael Marty. Many open source graphing libraries,Gephi4 or Pajek5, for example, require input in a very specific format,generally a graph description language. AfterGlow 1.6.X is designedto be run from the command line, lacking any graphical interface.This is expected to be remedied in AfterGlow 2.0..

The tool expects CSV files as input and generates an attributedgraph language file (DOT) that can be processed by tools like Pajekor Gephi (GraphViz). To help convert pcap and other log file formatsAfterGlow contains custom parsers that come with the installation.Most useful of which are the PCAP, sendmail log file and Snort alertlog parsers.

Figure 3.5: Visualization of an IRC flood attack using AfterGlow [16].

AfterGlow creates graphs with three distinct nodes; the source,which indicates the origin of a communication between two nodes,the event node, shaped as a rectangle, which indicates the targethost of a communication and the target node, indicating the targetedport. Node color may be customized using regular expressions in acolor.properties file that can be specified when running AfterGlow. Ex-

4. Gephi is an interactive visualization and exploration platform for all kinds ofnetworks and complex systems and hierarchical graphs gephi.github.io (cited2015-11-11).5. Pajek provides analysis tools for large networks and graph-drawing capabili-ties. http://mrvar.fdv.uni-lj.si/pajek/ (cited 2015-11-11).

18

Page 29: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

ample color.properties file used when generating DOT file for Figure3.5:

c o l o r . source=" yellow " i f ( $ f i e l d s [ 0 ] = ~ / ^ 1 9 2 \ . 1 6 8 \ . .∗/ ) ;c o l o r . source=" greenyellow " i f ( $ f i e l d s [ 0 ] = ~ / ^ 1 0 \ . .∗/ ) ;c o l o r . source=" yellow4 " i f ( $ f i e l d s [ 0 ] = ~ / ^ 1 7 2 \ . 1 6 \ . .∗/ ) ;c o l o r . source=" red "c o l o r . event=" yellow " i f ( $ f i e l d s [1 ]=~/^192\.168\. .∗/)c o l o r . event=" greenyellow " i f ( $ f i e l d s [1]=~/^10\. .∗/)c o l o r . event=" yellow4 " i f ( $ f i e l d s [1 ]=~/^172\.16\ . .∗/)c o l o r . event=" red "c o l o r . t a r g e t =" blue " i f ( $ f i e l d s [2 ] <1024)c o l o r . t a r g e t =" l i g h t b l u e "

Figure 3.5 shows a graph generated from DOT file created by Af-terGlow using the GraphViz library. The graph depicts the same IRCflood attack as Figure 3.5. It highlights the conversation between thelocal host 192.168.248.105 and foreign 64.32.28.7 over a non-standardIRC port 5553. The various hosts are also color coded according to theproperties file show above. Nodes belonging to the local network areyellow, foreign hosts are red, and ports grater than 1023 blue.

Another way to filter data are variables. AfterGlow allows theuser the define any variables using Perl expressions and use thosewhen allocating colors to nodes during DOT file generation [17].

19

Page 30: A Survey of Tools for Monitoring and Visualization of Network Traffic

3. TOOLS FOR LOCAL NETWORK VISUALIZATION

3.4 Summary of presented tools and their features

Figure 3.6 shows a full summary of all the functionality the user canexpect from tools covered in this chapter. The first horizontal linelists all the functionality covered in this chapter. LocalTraffVis standsfor local traffic visualization, GlobalTraffVis for global traffic visual-ization, PCAP for packet capture and visualization and LogVis forLog data visualiza tion. Check mark in a cell belonging to a givenapplication indicates that the application possesses the feature.

Functionality: LocalTraffVis GlobalTraffVis PCAP LogVisNetGrokEtherApeWireshark

TnVAfterGlow

Figure 3.6: Feature summary

20

Page 31: A Survey of Tools for Monitoring and Visualization of Network Traffic

Chapter 4

Tools for Global Network Visualization

4.1 Global Traffic and Performance Visualization

Global traffic is traffic travelling throughout, as well as inside andoutside of a network. Capture of such traffic is performed by multi-ple data gathering remote probes positioned at critical points, for ex-ample firewalls and switches, inside the network all connected to acentral data collection service. Visualization can be accomplished us-ing, for example, network topology maps with directed graphs rep-resenting traffic flows, regular bar or pie charts, or heatmaps.

Performance monitoring depicts the load various network de-vices are under. Similarly, charts and pie graphs, heatmaps, or bothcombined with network topology maps are the most common visu-alization approaches. Presented below are examples representing thegraph and topology map approach, respectively.

4.1.1 Cacti

Cacti is a multi-purpose network monitoring software being devel-oped by the Cacti Group and released free under the GNU GeneralPublic License. It leverages PHP, a custom polling1 client and RRD-tool2 to create near real-time graphs showing network traffic and per-formance. The basic Cacti installation consists of a web server with aMySQL database storing default graph, device and data source tem-plates. Newly added network devices get assigned a template, which

1. Polling, or polled operation, refers to actively periodically sampling the statusof an external device.2. Round-Robin Database tool is a high performance data logging and graphing sys-tem for time series data www.rrdtool.org/ (cited 2015-11-05).

21

Page 32: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

can be created, edited or further customized with a simple and intu-itive interface, describing what data sources should Cacti query for.Data sources can vary from simple wireless interface usage statis-tics for a router device to running processes and concurrent loggedin users on a server machine. Main method of data acquisition isthrough SNMP.

To retrieve network traffic and performance information an add-on polling program called Spine that comes with the default installa-tion is used. Spine periodically queries the monitored network overSNMP and saves the retrieved data to a round-robin database file(rrd). Each monitored data source–CPU load, memory utilization,disk space—has its own rrd file. These files then serve as data sourcesfor the traffic and performance graphing itself using RRDtool, whichoffers a set of commands for a wide range of individual graph cus-tomization. Selection of multiple data sources compared against eachother, color customization, rate of change, 95th percentile and manyothers are supported, leading to the creation of graphs the user re-quires. Cacti provides a built-in interface to easily write and storethese custom templates.

This functionality is demonstrated in Figure 4.1, depicting five in-dividual graphs generated by Cacti. First four are captures from a lo-cal network computer showing processor usage, bandwidth, dividedinto total bandwidth and unicast traffic, and used hard drive spaceover a period of seven hours. The last graph shows total incomingand outgoing wireless traffic over a router’s ppp1.1 interface.

Cacti can be further enhanced through many community-createdplug-ins made available for download on the official page 3. Popu-lar ones include PhpWeathermap, a plugin generating maps and dia-grams using data collected by Cacti or other sources, that also existsas a standalone installation (covered below), cereusreporting facilitat-ing instant PDF report creation, murlin, adding URL monitoring sup-port, etc.

The main strength of Cacti lies in its ability to pinpoint the ex-act time a notable event, such as a surge in network traffic, increasein connections, occurs. This allows the user to apply different tools,

3. Official Cacti plugin repository is located at http://docs.cacti.net/plugins (cited 2015-11-05).

22

Page 33: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

Figure 4.1: Traffic and performance graphs created using Cacti.

for example Snort, covered further on in this thesis, to analyze onlyrelevant data.

23

Page 34: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

4.1.2 PRTG Network Monitor

PRTG Network Monitor is a network traffic and performance moni-toring software developed by Paessler and building upon a previousversion known as the Paessler Router Traffic Grapher.

PRTG monitors the network by attaching sensors, called probes,to network-connected devices. These probes are objects that tell thetool what information to query for on a given device. Each PRTG in-stallation can support upwards of ten thousand probes dependingon sensor type and server hardware[18]. This number can, for largernetworks, be further increased by deploying Remote Probes, whichare additional installations of the service on computers within thenetwork. This splits the performance load of managing a high num-ber of sensors between multiple installations.

Probes are divided into four main categories: device availabilityand uptime, device status, network traffic and performance moni-toring, network traffic analysis. First category contains probes thatrange from a ping probe to monitor general device availability toprobes customized for monitoring common services. There are, forexample, probes that monitor folder and file content, count and theiravailability on a FTP server, IMAP and POP3 monitoring for mailservers including round trip probes reporting the time it takes foran e-mail to reach its destination after being sent, Oracle, MySQL,PostgreSQL database monitoring and virtual machine monitoring.

Probes in the second category report hardware status of moni-tored devices. CPU load, disk usage and general hardware make-upof a system as well as the status of anti-virus and other select secu-rity applications can be watched. Monitoring for Windows systemsis done mainly through WMI, other operating systems are monitoredthrough SNMP.

The network traffic and performance category houses probes vi-sualizing data flows on the network using Netflow/IPFIX and sib-ling protocols or SNMP. The resulting visualization can be filteredby incoming or outgoing traffic, unicast or multicast packets, errorsand discards or set to show only unknown protocol traffic. Exampleof such a sensor is shown in Figure 4.2.

Network traffic analysis contains pre-set and customisable packetsniffers that, when attached to an interface, monitor network traffic

24

Page 35: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

Figure 4.2: Traffic sensor deployed by PRTG [19]. For a full image seethe appendix A.5.

details and content. Customisable packet sniffers have filters that al-low the monitoring of only desirable traffic. Traffic can be filteredby protocols, source and destination IP or ports, MAC, IP version.Additional filtering options, interface, Actual-Sensor interface, areavailable for Netflow 5 and higher. Additional channels(graph datasources) can the be defined to further divide the filtered traffic.

Each probes triggers an alert or a notification whenever an event,for example a service becoming unavailable, occurs ,a custom thresh-old, bandwidth or CPU usage, is exceeded etc. Alerts depend onsensor type and are gathered on the dashboard, which provides anoverview of all devices on the network along with associated probesand their status.

PRTG is a highly customizable tool that provides the user withan overview of the whole network and alerts him when necessaryaccording to his rules.

25

Page 36: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

4.1.3 PhpWeatherMap

PhpWeatherMap (also known as Network Weathermap) is an opensource network visualization tool, that creates live network mapsfrom the network statistics collected by tools such as Cacti or RRD-tool.

Example of a live network map is shown in Figure 4.3. Depict-ing Switzerland’s SWITCHaai (Shibboleth)4 college network. Indi-vidual nodes depict whole networks with links between them colorcoded to show current bandwidth throughput. Hovering over a linkproduces an RRDtool bandwidth usage history graph. Image in thebackground can be freely customized, enabling the creation of pse-udo-topology maps. With enough available data, Network Weath-ermap can monitor networks down to individual links between de-vices.

Data is collected via plugins. Plugins are supplied for RRDtool,MRTG (RRD and old log-format), tab-delimited text files, SNMP, ex-ternal scripts, and Cacti-specific data. Usage o the RRDtool ensuresWeathermap has access to data from any application generating rrdfiles, including Cacti, Cricket, MRTG, and many more. Other sourcesare supported via plugins or external scripts.

Maps are created either through a web interface, or in a text filewith simple syntactic language. More information about this processcan be found in [21].

There is strong Cacti (covered above) integration in particular,leveraging its plugin architecture to provide a management user in-terface, and access control for maps using Cacti’s existing user data-base. Additional datasource plugins allow efficient access to datafrom Cacti’s poller directly.

4. Shibboleth is a single sign-on (log-in) system for computer networks and theInternet. It allows people to sign in using just one identity to various systems runby federations of different organizations or institutions. The federations are oftenuniversities or public service organizations.

26

Page 37: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

Figure 4.3: Network Weathermap of Switzerland university network.[20] For a full image see the appendix A.6

4.2 Network Topology Mapping

Network topology is the arrangement of the various elements (links,nodes, etc.) of a computer network. Essentially, it is the topologicalstructure of a network and may be depicted physically or logically.Physical topology is the placement of the various components of anetwork, including device location and cable installation, while logi-cal topology illustrates how data flows within a network, regardlessof its physical design. Distances between nodes, physical intercon-nections, transmission rates, or signal types may differ between twonetworks, yet their topologies may be identical.

An example is a local area network (LAN): Any given node inthe LAN has one or more physical links to other devices in the net-work; graphically mapping these links results in a geometric shapethat can be used to describe the physical topology of the network.

27

Page 38: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

Conversely, mapping the data flow between the components deter-mines the logical topology of the network.

Network topology mapping then refers to creation of these maps.Tools for network topology mapping create mainly logical maps. Ex-amples of two such tools are given below.

4.2.1 The Dude

The Dude is a free tool for network scanning and topology mappingbeing developed an maintained by MikroTik. It allows for scanninga selected range of IP addresses, detecting and sorting active hostsbased on available protocols, and creating a basic network layoutfrom retrieved data that can be further customized with user input.Dude then monitors all devices and traffic on such a network for anyservice outages.

The application window consists of a menu on the left side anda main window on the right. Bottom left corner contains a smalloverview of a currently selected monitored network. Network scanis started in the Discover submenu. There the user can specify thesubnet, services to search for, DNS, SMTP, NetBIOS, FTP, for exam-ple, and types of devices to discover, router, switch, SMTP server etc.Discovered devices are automatically added to the network map andcolor coded based on the availability of their services. Additional de-vices can be manually added. Relations between devices are depictedusing links. Three types of links exist—simple, SNMP, RouterOS—where simple shows an unknown link between two network com-ponents and SNMP with RouterOS link two devices over a selectedinterface. Ethernet links are straight, wireless links are displayed aslightning bolts. Hovering over each link produces a bandwidth us-age chart with variable zoom rate.

Network status is kept up-to-date by periodically polling all de-vices. Polling interval can be set between one second and one day.Figure 4.4 shows an example topology map of a home network withfive hosts connected to a single router constructed and laid out usingThe Dude. Green host indicates all detected services working and re-sponding, yellow shows a service outage and red complete unavail-ability. Each non-simple link also shows bandwidth usage detectedduring last poll.

28

Page 39: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

Figure 4.4: Topology map of a network created using The Dude.

The tool supports additional features such as custom chart gener-ation from existing or created data sources, custom functions editorfor use with chart generation, service outage and event logging withuser notification, MIB nodes panel showing all OID databases knownto Dude, and export of all tabular data to csv or pdf formats.

4.2.2 SolarWinds Network Topology Mapper

SolarWinds Network Topology Mapper (NTM) is a complex soft-ware solution whose main function resides in helping the user mapout his entire network, providing him with the means of discoveringwhich devices are connected to it. NTM is a commercially availableproduct developed and maintained by SolarWinds.

NTM discovers network devices according to given parameters—SNMP strings, WMI identification, VMware credentials, IP blocks

29

Page 40: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

and number of hops taken to reach the host. After device discoveryis complete, Physical and logical layouts are created. Those can befreely merged and customized. Default device layout is radial witha selectable radius. Devices without any detected connection to thenetwork are highlighted.

Created network maps can be further customized with backgrou-nd images, node images, link color coding etc. Example of a cus-tomized logical topology map generated with NTM is given in Fig-ure 4.5.

Figure 4.5: Example of a map generated and customized with Net-work Topology Mapper Trial version [23]. For a full image see theappendix A.7.

Generated networks can be exported as pdf files or to other simi-lar programs such as Microsoft Visio.

4.3 Summary of presented tools and their features

Figure 4.6 shows a full summary of all the functionality a user can ex-pect from the tools presented in this chapter. The first horizontal linelists all the functionality covered in this chapter. TraffVis stands for

30

Page 41: A Survey of Tools for Monitoring and Visualization of Network Traffic

4. TOOLS FOR GLOBAL NETWORK VISUALIZATION

traffic visualization, PerfVis for performance visualization, PCAP forpacket capture and visualization, LogVis for Log data visualizationand Topology the ability to visualize network topology.

Functionality: TraffVis PerfVis PCAP TopologyCactiPRTG

PhpWeatherMapThe Dude

SolarWinds NTM

Figure 4.6: Summary of features

31

Page 42: A Survey of Tools for Monitoring and Visualization of Network Traffic

Chapter 5

Tools for Anomaly Analysis and Intrusion De-tection

An intrusion detection system (IDS) is a device or software applica-tion that monitors network or system activities for malicious activ-ities and policy violations. Basic division of IDSes is into signature-based and anomaly-based. Signature-based systems match traffic toa known library of attacks whereas anomaly-based systems work offof heuristics and historical data to detect anomalies in normal net-work operations. Another division is between network based (NIDS)and host based (HIDS) intrusion detection systems. Some systemsmay attempt to stop an intrusion attempt but this is not required orexpected. Intrusion detection and prevention systems (IDPS) are pri-marily focused on identifying possible incidents, logging informa-tion about them, and reporting attempts.

Network Intrusion Detection Systems (NIDS) are placed at strate-gic points within the network to monitor traffic to and from all de-vices on the network. They perform an analysis of passing traffic onthe entire subnet, and match the traffic that is passed on the subnetsto the library of known attacks. These are known as signature-basedNIDS. Once an attack is identified, or abnormal behavior is sensed,the alert can be sent to the administrator. An example of an NIDSwould be installing it on the subnet where firewalls are located inorder to see if someone is trying to break into the firewall.

NID Systems are also capable of comparing signatures for sim-ilar packets to link and drop harmful detected packets which havea signature matching the records in the NIDS. There are two typesof NIDSes when classified based on interactivity: on-line and off-line. On-line NIDS deals with the network in real time. It analysescaptured live network traffic and decides, based on a given ruleset,

32

Page 43: A Survey of Tools for Monitoring and Visualization of Network Traffic

5. TOOLS FOR ANOMALY ANALYSIS AND INTRUSION DETECTION

whether it is an attack or not. Off-line NIDS does the same, but withalready captured and stored data.

Host-based Intrusion Detection Systems (HIDS) monitor the stateof a computing system. They watch over system and application re-source usage, monitor the state of the file system and log data andalert the user once any sign of tampering has been detected.

Two NIDS, Snort and Suricata, and one HIDS, OSSEC, are cov-ered below as examples with a brief description of their capabilities.

5.1 Snort

Snort is a free and open source NIDS with a voluntary subscription-based commercial model. It was created by Martin Roesch in 1998and is now developed and maintained by Sourcefire.

Snort’s NIDS has the ability to perform real-time traffic analysisand packet logging on IP networks. It performs protocol analysis,content searching and matching according to a given ruleset.

Snort allows the user to define custom rules. Snort rule are di-vided into two logical sections, the rule header and the rule options.The rule header contains the rule’s action, protocol, source and des-tination IP addresses and netmasks, and the source and destinationports information. The rule option section contains alert messagesand information on which parts of the packet should be inspected todetermine if the rule action should be taken [24]. Example snort rule:

alert tcp any any –> 192.168.1.0/24 111(content:|00 01 86 a5|; msG. “mountd access”;)

This rule generates an alert with the mountd access message whena TCP packet originating from any IP address and containing thespecified hexadecimal sequence is seen on the network and destinedfor any IP address on the 192.168.1.0 subnet on port 111. The text upto the first parenthesis is the rule header and the section enclosed inparenthesis contains the rule options. The words before the colonsin the rule options section are called option keywords. For more in-formation about custom rulesets and their definition see [25]. Many

33

Page 44: A Survey of Tools for Monitoring and Visualization of Network Traffic

5. TOOLS FOR ANOMALY ANALYSIS AND INTRUSION DETECTION

custom Snort rulesets are created and maintained by the commu-nity. One of the most well-known and popular being the EmergingThreats1 ruleset.

The program can also be used to detect probes or attacks, includ-ing operating system fingerprinting attempts, common gateway in-terface, buffer overflows, server message block probes, and stealthport scans using, for example, the network vulnerability scanner soft-ware Nmap [26].

Snort has three main modes of operation: packet analyzer, packetlogger, and network intrusion detection. In analyzer mode, the pro-gram will read network packets and display them on the console.In packet logger mode, the program will log packets to the disk. Inintrusion detection mode, the program will monitor network trafficand analyze it against the user’s ruleset and then perform tasks ac-cordingly.

Because Snort outputs everything into log files or the console,which makes it very hard to find relevant alerts and data, manythird-party security information and event management (SIEM) ap-plications, such as Snorby2 , Sguill3 , Aanval4 , interfacing with Snortexist. These serve as GUIs for Snort providing administrative, report-ing, performance and log analysis services. Example of a Snort GUISnorby is shown in Figure 5.1. Figure 5.2 shows detected Nmap scanon host 192.168.1.107.

1. Available from http://www.emergingthreats.net/ (cited 2015-11-17)2. Available from https://github.com/Snorby/snorby (cited 2015-11-17)3. Available from http://bammv.github.io/sguil (cited 2015-11-17)4. Available from https://www.aanval.com/ (cited 2015-11-17)

34

Page 45: A Survey of Tools for Monitoring and Visualization of Network Traffic

5. TOOLS FOR ANOMALY ANALYSIS AND INTRUSION DETECTION

Figure 5.1: Snort alerts gathered and displayed in Snorby, a SnortGUI. [27]

Figure 5.2: Nmap scan detected by Snort and displayed in Snorby.

35

Page 46: A Survey of Tools for Monitoring and Visualization of Network Traffic

5. TOOLS FOR ANOMALY ANALYSIS AND INTRUSION DETECTION

5.2 Suricata

Suricata is an open source-based IDS, IPS and Network Security Mon-itoring engine. It is developed and maintained by the Open Informa-tion Security Foundation (OISF).

Its NIDS capabilities are very similar to Snort. It analyzes live net-work traffic passing through key points on the network and providesalerts to the user based on the given ruleset. Suricata is capable ofusing the specialized Emerging Threats Suricata ruleset and the VRTruleset5. It provide some additional functionality over Snort, how-ever.

A single Suricata instance is capable of inspecting multi-gigabittraffic. The engine is built around a multi threaded and highly scal-able code base. Experimental GPU acceleration to offload CPU in-tensive tasks to the GPU is also possible. Suricata is also capable ofautomatic protocol detection, such as HTTP, on any port and apply-ing the proper detection and logging logic.

Suricata can also log HTTP requests, log and store TLS certifi-cates, extract files from flows and store them to disk. There is fullPCAP capture support for historic data analysis.

All of the event and alert output can be done through JSON files,which allows the usage of third-party GUIs like Kibana6 in additionto UIs like Sguill and Aanval also available for Snort.

It is difficult to provide graphical examples these applicationssince their setup is very individual and based mainly on the user’sneeds. Both Snort and Suricata can, because of the format of theiroutput use various third party applications for parsing and subse-quent visualization of results. Example in Figure 5.3 shows Suricataoutput in the Kibana GUI. All detected events are gathered on themain dashboard with visible statistics and trend information.

5. Available from https://snort.org/talos (cited 2015-11-19)6. Available from https://www.elastic.co/products/kibana (cited2015-11-19)

36

Page 47: A Survey of Tools for Monitoring and Visualization of Network Traffic

5. TOOLS FOR ANOMALY ANALYSIS AND INTRUSION DETECTION

Figure 5.3: Suricata output data displayed in the Kibana GUI [29].For a full image see the appendix A.8.

5.3 Open Source HIDS SECurity

OSSEC is a free, open source host-based intrusion detection system(HIDS). It performs log analysis, integrity checking, Windows reg-istry monitoring, rootkit detection, time-based alerting, and activeresponse. It provides intrusion detection for most operating systems,including Linux, OpenBSD, FreeBSD, OS X, Solaris and Windows. Itwas written by Daniel B. Cid and made public in 2004.

OSSEC uses a Client/Server based architecture, which is very im-portant in a HIDS system, since it could be potentially compromisedat the same time as the OS. Due to this distributed architecture, se-curity and forensic information leave the host to be stored elsewhereas soon as possible.This is to avoid any kind of tampering or obfus-cation from malicious software that would prevent detection.

It’s architecture design incorporates this strategy by deliveringalerts and logs to a centralized server where analysis and notifica-tion can occur even if the host system is taken offline or compro-mised. Alerts can also be sent to the user’s email. Another advantageof this architecture is the ability to centrally manage agents from asingle server. Since OSSEC supports deployment of up to thousands

37

Page 48: A Survey of Tools for Monitoring and Visualization of Network Traffic

5. TOOLS FOR ANOMALY ANALYSIS AND INTRUSION DETECTION

of agents, the ability to make changes en masse via a central server iscritical.

There are two types of agents within OSSEC: installable agentsand agentless agents. Installable agents are installed on hosts, andthey report back to a central OSSEC server via the OSSEC encryptedmessage protocol. Agentless agents require no installation on remotehosts. They are processes initiated from the OSSEC manager, whichgather information from remote systems, and use SNMP, WMI andsimilar protocols [31].

Similar to Snort and Suricata, OSSEC’s log output allows the us-age of many third party GUIs, some of which were mentioned above.It, however, also comes with it’s own GUI shown in Figure 5.4, de-picting OSSEC alerts from brute force SSH login attempts by a non-existent user.

Figure 5.4: OSSEC’s web UI depicting gathered alerts from a bruteforce SSH attack. [32]. For a full image see the appendix A.9

38

Page 49: A Survey of Tools for Monitoring and Visualization of Network Traffic

Chapter 6

Conclusion

This thesis sought to categorize and differentiate the multitude ofnetwork visualization tools in existence today. It did so by categoriz-ing them into three distinct categories—tools for local network mon-itoring, global network monitoring, anomaly analysis and networkintrusion prevention—and presenting examples from each one alongwith description of their methods of monitoring network traffic andintrusion.

Based on the feature summary table shown in Figure 3.6 of sec-tion 3.4 of chapter 3, dealing with tools for local network visualiza-tion, it can be concluded that very few tools are built to visualize onlylocally and even those that do so can still monitor the whole networkdue to traffic cloning. This, of course, does not mean that their visu-alization technique are particularly suited for such a purpose, or thatthe device providing it can handle the performance load. If, for ex-ample, NetGrok’s (section 3.1.1) method of visualization was to beapplied onto the whole network, it would be very difficult to acquireany meaningful data out of it. Similarly for EtherApe (section 3.1.2).From this it can be concluded that there are some visualization ap-proaches more suited to visualization of only parts of the network.

As a special case of local visualization were mentioned tools per-forming packet capture and tools enabling log data visualization.

Chapter 4 focused on global network visualization. Tools coveredthere provide much higher scalability for larger networks and, com-pared to tools from chapter 3, more presentation and customizationoptions.

Finally, chapter 5 provided examples of NIDS and HIDS systemsand their capabilities.

39

Page 50: A Survey of Tools for Monitoring and Visualization of Network Traffic

6. CONCLUSION

The aim of this thesis was to provide an overview of the mostpopular tools for network monitoring as selected by the author. Do-zens different tools exist with varying degrees of similarity. Provid-ing a complete categorization is outside the scope of this thesis, andwould be, at least according to the author’s experience, almost animpossible task.

40

Page 51: A Survey of Tools for Monitoring and Visualization of Network Traffic

Bibliography

[1] MARTY, Raffael. The Heatmap: Why is security visualisationso Hard?. In: SlideShare [online]. 2014 [cit. 2015-04-21]. Avail-able from: http://www.slideshare.net/zrlram/the-heatmap-why-is-security-visualization-so-hard.

[2] MARTY, Raffael. Cyber Security: How Visual Analytics Un-lock Insight. SlideShare [online]. 2013 [cit. 2015-04-21]. Availablefrom: http://www.slideshare.net/zrlram/kdd-2013-dm-challenges

[3] Five Big Data Challenges: And how to overcome them withvisual analytics. SAS: The power to know [online]. [cit. 2015-11-15]. Available from: https://www.sas.com/resources/asset/five-big-data-challenges-article.pdf

[4] What is a Zero-Day vulnerability. PcTools [online]. 2015 [cit.2015-11-10]. Available from: http://www.pctools.com/security-news/zero-day-vulnerability/

[5] AETHIS, Ubizen. Security in SNMPv3 versus SNMPv1 orv2c [online]. 2002 [cit. 2015-11-02]. Available from: http://text.123doc.org/document/1143814-security-in-snmpv3-versus-snmpv1-or-v2c-pdf.htm

[6] Introduction to Cisco IOS NetFlow: A Technical Overview[online]. 2011, May 2012 [cit. 2015-11-09]. Availablefrom: http://www.cisco.com/c/en/us/products/collateral/ios-nx-os-software/ios-netflow/prod_white_paper0900aecd80406232.html

[7] FOROUZAN, Behrouz A a Sophia Chung FEGAN. Data commu-nications and networking. 4th ed. New York: McGraw-Hill HigherEducation, c2007, xxxiv, 1134 p. ISBN 978-007-2967-753.

41

Page 52: A Survey of Tools for Monitoring and Visualization of Network Traffic

6. CONCLUSION

[8] Windows Management Instrumentation. Microsoft De-veloper Network [online]. 2015 [cit. 2015-11-10]. Avail-able from: https://msdn.microsoft.com/cs-cz/library/aa394582(v=vs.85).aspx

[9] Monitoring via WMI. Paessler: The Network Monitoring Company[online]. 1998 [cit. 2015-11-10]. Available from: https://www.paessler.com/manuals/prtg/wmi_monitoring

[10] BLUE, Ryan, Cody DUNNE, Adam FUCHS, Kyle KING aAaron SCHULMAN. Visualizing Real-Time Network ResourceUsage [online]. University of Maryland, College Park, 2008[cit. 2015-11-01]. Available from: http://www.cs.umd.edu/projects/netgrok/files/vizsec08-netgrok.pdf

[11] EtherApe: A graphical network monitor. SourceForge [online].SlashDot Media, 2015 [cit. 2015-11-09]. Available from: http://etherape.sourceforge.net/

[12] Wireshark: FAQ. Wireshark [online]. 1998 [cit. 2015-11-12]. Avail-able from: https://www.wireshark.org/faq.html#q1.2

[13] Wireshark: Coloring Rules. Wireshark [online]. 2015-04-16 [cit.2015-11-12]. Available from: https://wiki.wireshark.org/ColoringRules

[14] Wireshark: Building Display Filter Expresions. Wire-shark [online]. [cit. 2015-11-12]. Available from: https://www.wireshark.org/docs/wsug_html_chunked/ChWorkBuildDisplayFilterSection.html

[15] Wireshark: VoIP Calls. Wireshark [online]. [cit. 2015-11-12].Available from: https://wiki.wireshark.org/VoIP_calls

[16] SWITCH Traffic Weather Map. Linux Magazine [online].c©Linux New Media USA, LLC. [2015-11-11] Available

from: http://www.linux-magazine.com/var/linux_magazin/storage/images/linux-magazine.com/

42

Page 53: A Survey of Tools for Monitoring and Visualization of Network Traffic

6. CONCLUSION

issues/2009/106/pictures/figure-2/423148-1-eng-US/Figure-2_reference.png

[17] Advanced Network Graph Visualization with AfterGlow.Raffael Marty: Blog [online]. 2012, April 23, 2012 [cit. 2015-11-11]. Available from: http://raffy.ch/blog/2012/03/24/advanced-network-graph-visualization-with-afterglow/

[18] Planning Large Installations of PRTG Network Monitor. PaesslerKnowledge Base [online]. 2015, 2015-11-08 [cit. 2015-11-08]. Avail-able from: http://kb.paessler.com/en/topic/26383-planning-large-installations-of-prtg-network-monitor

[19] PRTG Network Monitor Traffic Sensor. Paessler [online]. c©2011Paessler AG. [2015-12-13] Available from: http://www.abload.de/img/prtgnetworkmonitorxpprdcyw.png

[20] SWITCH Traffic Weather Map. SWITCHaai [online]. c©2015SWITCH. [2015-11-13] Available from: https://traffic.lan.switch.ch/pub/international-map/index.html

[21] Network Weathermap 0.97b Manual. Network Weathermap [on-line]. 2004 [cit. 2015-11-13]. Available from: http://network-weathermap.com/manual/0.97b/

[22] Manual:The Dude [online]. [cit. 2015-11-03]. Available from:http://wiki.mikrotik.com/wiki/Manual:The_Dude

[23] SolarWinds Network Topology Mapper. Softpedia [online]. [cit.2015-11-13] Available from: http://i1-win.softpedia-static.com/screenshots/SolarWinds-Network-Topology-Mapper_1.png

[24] Writing Snort Rule: The Basics. Snort: Manual [online]. [cit.2015-11-17]. Available from: http://manual.snort.org/node28.html

[25] Writing Snort Rules. Snort: Manual [online]. [cit. 2015-11-17].Available from: http://manual.snort.org/node27.html

43

Page 54: A Survey of Tools for Monitoring and Visualization of Network Traffic

6. CONCLUSION

[26] KRISHNAMURTHY, Mohan. How to cheat at securing Linux. 1edition (30 Oct. 2007). Burlington, MA: Syngress, c2008, xvi, 415p. ISBN 978-159-7492-072.

[27] Snorby GUI. Everyday Is Zero Day [online]. [2015-11-13] Avail-able from: http://everydayiszeroday.blogspot.cz/2013/01/installing-snorby-on-ubuntu-1204.html

[28] Suricata: Documentation [online]. 2010 [cit. 2015-11-19]. Availablefrom: https://redmine.openinfosecfoundation.org/projects/suricata/wiki

[29] Suricata with Kibana GUI. Github [online]. [cit. 2015-11-19] Available from: http://mestizo.github.io/images/suricata.png

[30] OSSEC: Documentation [online]. 2010 [cit. 2015-11-21]. Availablefrom: http://ossec-docs.readthedocs.org/en/

[31] OSSEC Documentation: Agents [online]. 2010 [cit. 2015-11-21].Available from: http://ossec-docs.readthedocs.org/en/latest/manual/agent/index.html

[32] OSSEC SSH Brute Force Attack Detection raymii [online]. [cit.2015-11-21] Available from: https://raymii.org/s/inc/img/ossec/webui-brute-2.8.png

44

Page 55: A Survey of Tools for Monitoring and Visualization of Network Traffic

Appendix A

Images

A.1 Netgrok

45

Page 56: A Survey of Tools for Monitoring and Visualization of Network Traffic

A. IMAGES

A.2 EtherApe

46

Page 57: A Survey of Tools for Monitoring and Visualization of Network Traffic

A. IMAGES

A.3 Wireshark

47

Page 58: A Survey of Tools for Monitoring and Visualization of Network Traffic

A. IMAGES

A.4 Time-based Network Visualizer

48

Page 59: A Survey of Tools for Monitoring and Visualization of Network Traffic

A. IMAGES

A.5 PRTG Network Monitor

49

Page 60: A Survey of Tools for Monitoring and Visualization of Network Traffic

A. IMAGES

A.6 PhpWeatherMap

50

Page 61: A Survey of Tools for Monitoring and Visualization of Network Traffic

A. IMAGES

A.7 Solarwinds Network Topology Mapper

51

Page 62: A Survey of Tools for Monitoring and Visualization of Network Traffic

A. IMAGES

A.8 Suricata

52

Page 63: A Survey of Tools for Monitoring and Visualization of Network Traffic

A. IMAGES

A.9 Open Source HIDS SECurity

53

Page 64: A Survey of Tools for Monitoring and Visualization of Network Traffic

Appendix B

Supplementary Data Files

B.1 IRCFlood.pcap

Description:This accompanying packet capture file contains a captured sam-

ple of an Internet Relay Chat (IRC) flood attack. It was used whentesting visualization capabilities of Tnv and AfterGlow (section 3.3),and when creating graphical representations of their output for thisthesis.

54