intrusion detection systems (ids) · suricata intrusion detection system •suricata is a high...
TRANSCRIPT
Intrusion Detection Systems (IDS)
Adli Wahid
Role of Detection in Security
• Part of security monitoringoViolation of security policieso Indicators of compromiseo Threat drive or Vulnerability driven oWhat’s happening on the network?
• RulesoDetection is based on rules
• Action • What do we do when detection happens? • Alert and Investigate • Drop / Block
Perspective – Adversary Tactics and Techniques • Mitre Att&ck Framework• Tactics – what are the goals of the adversary?• Technique – how do they do it? • Subject to:
o Resourceso Platforms
• Can we used this knowledge for detection?o Observe Adversaries Behaviouro Techniques, Tactics and Procedures (TTPs)o Deploy in prevention, detection, response
https://attack.mitre.org
Your Adversaries
Your Assets Your Systems
MotivesTargets
InfrastructureBehaviour
Reference: https://published-prd.lanyonevents.com/published/rsaus19/sessionsFiles/13884/AIR-T07-ATT%26CK-in-Practice-A-Primer-to-Improve-Your-Cyber-Defense-FINAL.pdf
Reference: https://published-prd.lanyonevents.com/published/rsaus19/sessionsFiles/13884/AIR-T07-ATT%26CK-in-Practice-A-Primer-to-Improve-Your-Cyber-Defense-FINAL.pdf
Making Your Infrastructure Forensics Ready
• Detecting known or potentially malicious activities • Part of the incident response plan • If your infrastructure is compromised
oCan you answer the questions: what happened and since when?oCan we ‘go back in time’ and how far back?
• What information you you need to collect and secure? • Centralized logging
Intrusion Detection Systems
• An intrusion detection system (IDS) is a device or software application that monitors a network or systems for malicious activity or policy violations. Any malicious activity or violation is typically reported either to an administrator or collected centrally using a security information and event management (SIEM) system
Different types of Intrusion Detection Systems
• Host Based • Network Based
IDS Technology landscape
Preventive Real Time
Host Based IDS• A host-based IDS is capable of monitoring all or parts of the dynamic
behavior and the state of a computer system, based on how it is configured.owhich program accesses what resourceso state of a systemonot been changed by intruders
• Monitoring Dynamic Behaviour • Who is doing what in a system
• Monitoring State• Detect modifications
Host Based IDS (2)
• Techniques o System Integrity Check oAlertingoVulnerability Detection oConfiguration assessment oRootkit detection o Security PolicyoActive Response
• OpenSCAP• OpenSCAP is an OVAL (Open Vulnerability Assessment Language) and XCCDF
(Extensible Configuration Checklist Description Format) interpreter used to check system configurations and to detect vulnerable applications.
Source: https://wazuh.com
Examples • OSSEC
ohttps://www.ossec.net
• Wazuhohttps://www.wazuh.com
• Some other interesting projectsoOSQuery - https://www.osquery.io/o Loki - https://github.com/Neo23x0/Lokio Sysmon - https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon
oKey component – agent or log/data shipper
Network Based IDS • Network intrusion detection systems (NIDS) are placed at a strategic
point or points within the network to monitor traffic to and from all devices on the network.operforms an analysis of passing traffic on the entire subnet, and matches the
traffic that is passed on the subnets to the library of known attacks.
oDetection Method o Signature basedoAnomaly based
• Examples (Free / Open Source) o SNORTo Suricata o Zeek (Bro)
Limitations
• Noise • False Positives• Signature management
oOutdated o0-days
• Can’t compensate for weak authentication / identification • Encrypted packets
How to monitor the network?
• Network TAPso A network tap is a hardware device which provides a way to access
the data flowing across a computer network o The network tap has (at least) three ports: an A port, a B port, and a
monitor porto Network Taps are fully passive device
§ Pros§ Passive / Fail Safe§ Exact duplicate of network traffic
§ Cons§ Expensive § Require physical infrastructure
Port Mirroring / SPAN Port• Also known as SPAN (Switch Port Analyzer)• A SPAN is a dedicated port on a managed switch that takes a mirrored copy of network
traffic off the switch to be sent to a monitoring device • Port mirroring is used on a network switch to send a copy of network packets seen on
one switch port (or an entire VLAN) to a network monitoring connection on another switch port
• Pros• Low cost, easy to deploy • Feature available in most switch
• Cons• Potential packet loss • Utilise switch resources • Attacker can disable SPAN/Mirror Port
Caveats of IDS
• "Alert Fatigue", • can be a daunting task and quickly fill an analysts plate combing through false positives
trying to find that one good alert.• Administrators fail to keep alerts relevant
• IDS is seen as a system with many of false positives• No maintenance is devoted towards managing it, can be spotty coverage• Rules/signatures are not up to date
• Analysts fail to understand rules• Don't have proper training on how to validate rules• Are not kept in the loop on specific rules that are of high importance
• Organization can't respond to problems generated by IDS• Response policies are not in place• System administrators don't know where to look for issues• Security organization isn't empowered to respond to issues
Suricata
Suricata Intrusion Detection System
• Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine.
• It is open source and owned by a community-run non-profit foundation, the Open Information Security Foundation (OISF).
• Suricata is developed by the OISF• The Suricata source code is licensed under version 2 of the GNU
General Public License
Suricata - History
• Beta release – Dec 2009• First standard release – July 2010• Features
oMulti-threadingoAutomatic protocol detection o JSON standard outputso file matching, logging, extraction, md5 checksum calculationoDNS loggero etc
In a nutshell
• The Suricata engine is capable of real time intrusion detection (IDS), inline intrusion prevention (IPS), network security monitoring (NSM) and offline pcap processing
• Suricata inspects the network traffic using a powerful and extensive rules and signature language, and has powerful Lua scripting support for detection of complex threats
• With standard input and output formats like YAML and JSON integrations with tools like existing SIEMs, Splunk, Logstash/Elasticsearch, Kibana, and other database become effortless
Rules Management
• It is important to have rules that are up-to-date • Management of rules is being done by suricata-update• Within the configuration file there are variables for default-rules-path
and rule-files: • By default all rules are merged into a single file suricata.rules• Rules can be enabled and disabled
• /etc/suricata/enabled.conf• /etc/suricata/disabled.conf
Rules/Suricata
• Actions (i.e. alert or drop) are decided by rules • In most occasions people are using existing rulesets
• Emerging Threats • Talos/Cisco • https://github.com/suricata-rules/suricata-rules
Rules Format
• A rule/signature consists of the following:o The action, that determines what happens when the signature matcheso The header, defining the protocol, IP addresses, ports and direction of the
rule.o The rule options, defining the specifics of the rule
drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)
Rules – Action
• What happens if signature matches• Options
o Passo Drop (IPS mode)o Reject o Alert
alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)
Rules - Protocol
drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)• 4 protocols
o tcp (for tcp-traffic)oudpo icmpo ip (ip stands for ‘all’ or ‘any’)
• And some application layer protocols*o Dns, http, smb, ssh, smtp, imap, tls , etc
Rules - Source and destination
• drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)
• Source and Destination of traffic • IP address / Block• Domain names
• Can be set as: • Variables – defined in /etc/suricata.yaml• IP address (v4/v6) format • ‘any’• Negation i.e. ! can be used as well
Rules - Ports (source and destination)
• drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)
• Port number(s) can be applied to source and destination traffic • Port helps to determine which application is receiving data • Application that sends packet tend to be assigned random port
numbers • alert http $HOME_NET any -> $EXTERNAL_NET 80
• Variables can be set: • WEB_PORTs = [80, 443, 8080]
Rules – direction
drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)
• The direction tells in which way the signature has to match• Possible values:
-> : source -> destination<> : both directions
Rules – options
drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)
• Enclosed by parantheses and separated by • Format:
• keyword: settings; • keyword;
Rules – Keywords
• Meta keywords o They affect how Suricata report events o Many keywords!
• Examples: o msg: “some description related to alert”; o sid:1; <- signature ido rev:2; <- revision information o gid:1; <- group id for a set of rules o classtype: “information about classification of rules” (classification.config)o reference: type, ref
o reference:url, www.info.com or reference: cve, CVE-2014-1234o priority:1; highest priority will be examined first (1)
Rules – keywords (2)• The are also keywords for different categories i.e.
o IPo TCPo UDPo ICMPo Payloado HTTP o DNSo TLS o And many more!
alert ip $EXTERNAL_NET any -> $HOME_NET any (msg:”GPL MISC 0 ttl”; ttl:0; reference:url,support.microsoft.com/default.aspx?scid=kb#-#-EN-US#-#-q138268; reference:url,www.isi.edu/in-notes/rfc1122.txt; classtype:misc-activity; sid:2101321; rev:9;)o More information here:
o https://suricata.readthedocs.io/en/latest/rules/header-keywords.html
Rules – payload keywords
• Contento Content:”content/payload here”; o Take note of special characters, need to be represented in heximal notation i.e. ”
|22|o alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"Outdated Firefox on
Windows"; content:"User-Agent|3A| Mozilla/5.0 |28|Windows|3B| "; content:"Firefox/3."; distance:0; content:!"Firefox/3.6.13"; distance:-10; sid:9000000; rev:1;)
• Others: • nocase; • depth:• offset:• More here: https://suricata.readthedocs.io/en/latest/rules/payload-keywords.html
Thinking about rules
• Detect outbound SSH connections• Compare the two alerts
alert tcp $HOME_NET any -> $EXTERNAL_NET 22 (msg: ”Outbound SSH Detected"; sid:10; rev:1;)
alert ssh $HOME_NET any -> $EXTERNAL_NET any (msg: “Outbound SSH Detected”; sid:11; rev:1.1;)
Alerts Output
• Line based alerts logo fast.log10/05/10-10:08:59.667372 [**] [1:2009187:4] ET WEB_CLIENT ACTIVEX iDefense COMRaiderActiveX Control Arbitrary File Deletion [**] [Classification: Web Application Attack] [Priority: 3] {TCP} xx.xx.232.144:80 -> 192.168.1.4:56068
• Extensible Event Format (EVE) JSON Outputo Alerts, metadata, file info and protocol specific records through JSONo Enabled by default, very verbose
• Binary format, compatible with unified2o Can be processed by tool such as Barnyard2
• Other line based alerts – http.log, dns.log, • Pcap - full packet capture format
{"timestamp": "2017-11-
09T15:35:16.120665+0000","flow_id": 1736687033191953,"pcap_cnt": 55,"event_type": "alert","src_ip": "192.185.57.176","src_port": 80,"dest_ip": "10.11.9.101","dest_port": 49167,"proto": "TCP","metadata": {
"flowbits": ["min.gethttp","ET.http.binary"
]},
{"tx_id": 0,"alert": {"action": "allowed","gid": 1,"signature_id": 2018959,"rev": 4,"signature": "ET POLICY PE EXE or DLL
Windows file download HTTP","category": "Potential Corporate
Privacy Violation","severity": 1,"metadata": {"updated_at": ["2017_02_01"
],"created_at": ["2014_08_19"
],"former_category": ["POLICY"
]}}
eve.json
Deployment Considerations
• Distributed sensor for visibilityo Where should you tap?
• Playbooks? o Policyo Threat Model o Rulesets
• Collect and store logs for analysis & visualization (SIEM)o Space for storage o Sending logs securely to centralised locationo Suporting datasets
• Incident Response Plan o What to do when malicious activities / attacks are detected?
• Privacy o Handling user privacy / trust
o Managing alertso Check out threshold, eent_filter and suppress in threshold.conf and can also be defined within rules
• Tuning
Viewing alerts • A couple of GUI options
o Snorbyo Eveboxo SguiloMoloch (with full packet capture)
• SIEM integration • Elasticstack and Splunk• Prepackaged in distributions like
SecurityOnion or SELKS
Snorby
Links: 1. Snorby: https://github.com/Snorby/snorby2. Evebox: https://evebox.org/ 3. https://bammv.github.io/sguil/index.html4. SecurityOnion: https://securityonion.net/
5. SELKS: https://www.stamus-networks.com/open-source/
Evebox
Where Should We Place our sensors?
Source: The Practice of Network Security Monitoring, Richard Bejtlich (2013)
• Consider different sets of IP block for the network
• Considerations • IP Blocks • Network Address Translations
• B,G,HoAbility to see true source and
destination IP address
Other features
• Extract file within traffic to disk • http, smtp, imap
• File extraction o - file-store:o enabled: no # set to yes to enableo log-dir: files # directory to store the fileso force-magic: no # force logging magic on all stored files
JA3 fingerprinting
• Monitoring / detection based on TLS fingerprinting ohttps://github.com/salesforce/ja3o app-layer.protocols.tls.ja3-fingerprints’ to ‘yes’
• Parses multiple fields set in TLS client hello packet sent over during SSL handshake
• Exampleo SSLVersion,Cipher,SSLExtension,EllipticCurve,EllipticCurvePointFormato769,49172-49171-53-47-49162-49161-56-50-10-19-5-4,0-5-10-11-65281,23-
24-25,0o1eede9d19dc45c2cb66d2f5c6849e843
ja3 -j Poseidon.pcap*[ { "destination_ip": "185.67.0.108", "destination_port": 443, "ja3": "769,49172-49171-53-47-49162-49161-56-50-10-19-5-4,0-5-10-11-65281,23-24-25,0", "ja3_digest": "1eede9d19dc45c2cb66d2f5c6849e843", "source_ip": "192.168.56.101", "source_port": 49161, "timestamp": 1527008276.377147 }
https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/inspecting-encrypted-network-traffic-with-ja3/
* Poseidon point-of-sales malware
ja3 - rules
alert tls $HOME_NET any -> $EXTERNAL_NET any (msg:"match JA3 hash"; \flow:established,to_server; \ ja3_hash; content:"1eede9d19dc45c2cb66d2f5c6849e843 "; \ sid:100001; rev:1; )
Labs
1. Setup and run Suricata on the honeypot 2. Analyze Suricata Logs with jq
a. JSON log format
3. Cowrie log with jq
sensoX.honeynet.asia
cowrie
Suricata
Discussion & Recap
• Firewall• ID• Security Monitoring