using netzob for protocol reverse engineering
TRANSCRIPT
Using Netzob for protocol reverse engineering
Onur CATAKOGLU , Bastien DROUOT,Paul GERMOUTY, Florent TARDIF
Advisor: Frederic GUIHERY
November 3, 2017
Onur, Bastien, Paul, Florent November 3, 2017 1 / 27
Objective
Adopt an active approach to protocol reverse engineering
Attack Modelno previous knowledge of the protocol (no documentation)access to a client implementationpassive monitoring / active querying of the server
Onur, Bastien, Paul, Florent Introduction November 3, 2017 2 / 27
Netzob
A framework helping the reverser to discover the protocol
abstraction of requests with a symbolic representation with fieldsmethods to send messages to the serverfunctions to split messages in fields:
→ splitStatic: use static messages parts→ splitAligned: detect moving reccurrent parts→ splitDelimiter: use manually entered delimiters
Experimentation on Snap7 (S7) protocol,used in SCADA communication
Onur, Bastien, Paul, Florent Introduction November 3, 2017 3 / 27
Example 1
Analyse with SplitAligned
Field-0 | Field-1 | Field-2 | Field-3 | Field-4---------- | ---------------------------- | ------- | ----------------------- | ------’\x03\x00’ | b’\x00\x16\x11\xe0\x00’ | ’\x00’ | ’’ | ’\x00’’\x03\x00’ | b’\x00\x16\x11\xd0’ | ’\x00’ | ’\x01’ | ’\x00’’\x03\x00’ | b’\x00\x19\x02\xf0\x802\x01’ | ’\x00’ | ’\x00(\x00\x00\x08\x00’ | ’\x00’’\x03\x00’ | b’\x00\x1b\x02\xf0\x802\x03’ | ’\x00’ | ’\x00(\x00\x00\x08\x00’ | ’\x00’’\x03\x00’ | b’\x00!\x02\xf0\x802\x07’ | ’\x00’ | ’\x00)\x00’ | ’\x00’’\x03\x00’ | b’\x00\x99\x02\xf0\x802\x07’ | ’\x00’ | ’\x00)\x00’ | ’\x00’’\x03\x00’ | b’\x00!\x02\xf0\x802\x07’ | ’\x00’ | ’\x00*\x00’ | ’\x00’’\x03\x00’ | b’\x01}\x02\xf0\x802\x07’ | ’\x00’ | ’\x00*’ | ’\x00’’\x03\x00’ | b’\x00!\x02\xf0\x802\x07’ | ’\x00’ | ’\x00+\x00\x00\x08’ | ’\x00’’\x03\x00’ | b’\x00Q\x02\xf0\x802\x07’ | ’\x00’ | ’\x00+\x00\x00\x0c’ | ’\x00’’\x03\x00’ | b’\x00\x1f\x02\xf0\x802\x01’ | ’\x00’ | ’\x00,\x00\x00\x0e’ | ’\x00’’\x03\x00’ | b’\x00.\x02\xf0\x802\x03’ | ’\x00’ | ’\x00,\x00\x00\x02’ | ’\x00’’\x03\x00’ | b’\x008\x02\xf0\x802\x01’ | ’\x00’ | ’\x00-\x00’ | ’\x00’’\x03\x00’ | b’\x00\x16\x02\xf0\x802\x03’ | ’\x00’ | ’\x00-\x00\x00\x02’ | ’\x00’’\x03\x00’ | b’\x00\x1f\x02\xf0\x802\x01’ | ’\x00’ | ’\x00.\x00\x00\x0e’ | ’\x00’’\x03\x00’ | b’\x00.\x02\xf0\x802\x03’ | ’\x00’ | ’\x00.\x00\x00\x02’ | ’\x00’’\x03\x00’ | b’\x00\x1f\x02\xf0\x802\x01’ | ’\x00’ | ’\x00/\x00\x00\x0e’ | ’\x00’’\x03\x00’ | b’\x00\xeb\x02\xf0\x802\x03’ | ’\x00’ | ’\x00/\x00\x00\x02’ | ’\x00’’\x03\x00’ | b’\x00\xf5\x02\xf0\x802\x01’ | ’\x00’ | ’\x000\x00’ | ’\x00’’\x03\x00’ | b’\x00\x16\x02\xf0\x802\x03’ | ’\x00’ | ’\x000\x00\x00\x02’ | ’\x00’---------- | ---------------------------- | ------- | ----------------------- | ------
Onur, Bastien, Paul, Florent Introduction November 3, 2017 4 / 27
Example 2
Analyse with SplitStatic
Field-0 | Field-1 | Field-2 | Field-3---------- | ----------------------------------- | ------- | --------------------’\x03\x00’ | b’\x00\x16\x11\xe0\x00\x00\x00\x01’ | ’\x00’ | b’\xc0\x01\n\xc1 ...’\x03\x00’ | b’\x00\x16\x11\xd0\x00\x01\x00\x01’ | ’\x00’ | b’\xc0\x01\n\xc1 ...’\x03\x00’ | b’\x00\x19\x02\xf0\x802\x01\x00’ | ’\x00’ | b’(\x00\x00\x08 ...’\x03\x00’ | b’\x00\x1b\x02\xf0\x802\x03\x00’ | ’\x00’ | b’(\x00\x00\x08 ...’\x03\x00’ | b’\x00!\x02\xf0\x802\x07\x00’ | ’\x00’ | b’)\x00\x00\x08 ...’\x03\x00’ | b’\x00\x99\x02\xf0\x802\x07\x00’ | ’\x00’ | b’)\x00\x00\x0c ...’\x03\x00’ | b’\x00!\x02\xf0\x802\x07\x00’ | ’\x00’ | b’*\x00\x00\x08 ...’\x03\x00’ | b’\x01}\x02\xf0\x802\x07\x00’ | ’\x00’ | b’*\x00\x00\x0c ...’\x03\x00’ | b’\x00\x1f\x02\xf0\x802\x01\x00’ | ’\x00’ | b’/\x00\x00\x0e ...’\x03\x00’ | b’\x00\xeb\x02\xf0\x802\x03\x00’ | ’\x00’ | b’/\x00\x00\x02 ...’\x03\x00’ | b’\x00\xf5\x02\xf0\x802\x01\x00’ | ’\x00’ | b’0\x00\x00\x0e ...’\x03\x00’ | b’\x00\x16\x02\xf0\x802\x03\x00’ | ’\x00’ | b’0\x00\x00\x02 ...---------- | ----------------------------------- | ------- | --------------------
Onur, Bastien, Paul, Florent Introduction November 3, 2017 5 / 27
Example 3
Analyse with SplitDelimiter with delimiter \x02
Field-0 | Field-sep-02 | Field-2-------------------------------------------------------------| ------------ | -------------------b’\x03\x00\x00\x16\x11\xe0\x00\x00\x00\x01\x00\xc0\x01\n\xc1’| ’\x02’ | b’\x01\x00\xc2’b’\x03\x00\x00\x16\x11\xd0\x00\x01\x00\x01\x00\xc0\x01\n\xc1’| ’\x02’ | b’\x01\x00\xc2’’\x03\x00\x00\x19’ | ’\x02’ | b’\xf0\x802\x01\x00’\x03\x00\x00\x1b’ | ’\x02’ | b’\xf0\x802\x03\x00’\x03\x00\x00!’ | ’\x02’ | b’\xf0\x802\x07\x00b’\x03\x00\x00\x99’ | ’\x02’ | b’\xf0\x802\x07\x00’\x03\x00\x00!’ | ’\x02’ | b’\xf0\x802\x07\x00’\x03\x00\x01}’ | ’\x02’ | b’\xf0\x802\x07\x00’\x03\x00\x00\x16’ | ’\x02’ | b’\xf0\x802\x03\x00’\x03\x00\x00\x1f’ | ’\x02’ | b’\xf0\x802\x01\x00’\x03\x00\x00.’ | ’\x02’ | b’\xf0\x802\x03\x00’\x03\x00\x00\x1f’ | ’\x02’ | b’\xf0\x802\x01\x00b’\x03\x00\x00\xeb’ | ’\x02’ | b’\xf0\x802\x03\x00b’\x03\x00\x00\xf5’ | ’\x02’ | b’\xf0\x802\x01\x00’\x03\x00\x00\x16’ | ’\x02’ | b’\xf0\x802\x03\x00-------------------------------------------------------------| ------------ | -------------------
Onur, Bastien, Paul, Florent Introduction November 3, 2017 6 / 27
Outline
Input
Select messages
Delimit fields
Characterize fields
Assess the impact of the fields
Demonstration
Onur, Bastien, Paul, Florent Outline November 3, 2017 7 / 27
Select messages
Input
Select messages
Delimit fields
Characterize fields
Assess the impact of the fields
Demonstration
Onur, Bastien, Paul, Florent Outline November 3, 2017 8 / 27
Select messages
Onur, Bastien, Paul, Florent Outline November 3, 2017 9 / 27
Pcap definition
Onur, Bastien, Paul, Florent Outline November 3, 2017 10 / 27
Delimit fields
Input
Select messages
Delimit fields
Characterize fields
Assess the impact of the fields
Demonstration
Onur, Bastien, Paul, Florent Outline November 3, 2017 11 / 27
Fields search
Field search with delimiters (Delim)
Improved aligned search (Smartaligned)
Automated aligned search
Onur, Bastien, Paul, Florent Outline November 3, 2017 12 / 27
Using delimiters
Analyse with Delim
candidatesb’\x0f’ b’\xf0’ b’2’ b’\x80’ b’\x02’ b’\x00\x04’b’\x02\xf0’ b’\x04\x01’ b’\xf0\x80’ b’\x0f\x00’ b’\x802’b’\x00\x0f’ b’\x02\x00’ b’\xf0\x802’ b’\x02\xf0\x80’b’\x00\x04\x01’ b’\x00\x00\x04’ b’\x00\x00\x0f’b’\x0f\x00\x00’ b’\x00\x0f\x00’b’\x00\x0f\x00\x00’ b’\x00\x00\x04\x01’b’\x00\x00\x0f\x00’ b’\x02\xf0\x802’
Crucials[b’\x04\x01’, b’\xf0\x802’]
SplitDelimiterField-0 | Field-sep-0---------------------------------------------------------------------------- | -----------b’\x03\x00\x00\x1f\x02\xf0\x802\x01\x00\x00\x0f\x00\x00\x0e\x00\x00’ | ’\x04\x01’b’\x03\x00\x00\x1a\x02\xf0\x802\x03\x00\x00\x0f\x00\x00\x02\x00\x05\x00\x00’ | ’\x04\x01’---------------------------------------------------------------------------- | -----------Field-0 | Field-sep-f08032 | Field-2---------------------- | ---------------- | ----------------------------------------------’\x03\x00\x00\x1f\x02’ | b’\xf0\x802’ | b’\x01\x00\x00\x0f\x00\x00\x0e\x00\x00\x04\x01’\x03\x00\x00\x1a\x02’ | b’\xf0\x802’ | b’\x03\x00\x00\x0f\x00\x00\x02\x00\x05\x00\x00---------------------- | ---------------- | ----------------------------------------------
Onur, Bastien, Paul, Florent Outline November 3, 2017 13 / 27
SmartAligned
Analyse with SmartAligned
Field | Field | Field | Field | Field | Field |-------------- | ------------------------| ---------- | ----- | ------ | ----------------------- |’\x03\x00\x00’ | b’\x16\x11\xe0’ | ’\x00\x00’ | ’’ | ’\x00’ | ’\x01’ |’\x03\x00\x00’ | b’\x19\x02\xf0\x802\x01’| ’\x00\x00’ | ’,’ | ’\x00’ | b’\x00\x08\x00\x00\xf0’ |’\x03\x00\x00’ | b’!\x02\xf0\x802\x07’ | ’\x00\x00’ | ’-’ | ’\x00’ | ’\x00\x08\x00\x08’ |’\x03\x00\x00’ | b’!\x02\xf0\x802\x07’ | ’\x00\x00’ | ’.’ | ’\x00’ | ’\x00\x08\x00\x08’ |’\x03\x00\x00’ | b’!\x02\xf0\x802\x07’ | ’\x00\x00’ | ’/’ | ’\x00’ | ’\x00\x08\x00\x08’ |-------------- | ------------------------| ---------- | ----- | ------ | ----------------------- |
Field | Field | Field-----------------------------------------| ----- | -----------------------------------------------b’\x03\x00\x00!\x02\xf0\x802\x07\x00\x00’| ’-’ | b’\x00\x00\x08\x00\x08\x00\x01\x12\x04\x11D ...b’\x03\x00\x00!\x02\xf0\x802\x07\x00\x00’| ’.’ | b’\x00\x00\x08\x00\x08\x00\x01\x12\x04\x11D ...b’\x03\x00\x00!\x02\xf0\x802\x07\x00\x00’| ’/’ | b’\x00\x00\x08\x00\x08\x00\x01\x12\x04\x11D ...-----------------------------------------| ----- | -----------------------------------------------
Onur, Bastien, Paul, Florent Outline November 3, 2017 14 / 27
Automated SmartAligned
Directly cluster messages by similarity
Apply a DBSCAN on a set of (not selected) messagesGet several clusters of messages grouped by similarity→ probably same type of messages in each cluster
(ex. HTTP GET, HTTP POST)
Apply SplitAligned on each cluster
But: need a large number of valid distincts messages
Onur, Bastien, Paul, Florent Outline November 3, 2017 15 / 27
Characterizing fields
Input
Select messages
Delimit fields
Characterize fields
Assess the impact of the fields
Demonstration
Onur, Bastien, Paul, Florent Outline November 3, 2017 16 / 27
Characterizing fields
Static fields / Dynamic fields
Types: TEXT / BINARY / INTEGER
Change a static field→ if error/timeout ⇒ confirm static field (possibly a keyword)
Onur, Bastien, Paul, Florent Outline November 3, 2017 17 / 27
Assess the impact of the fields
Input
Select messages
Delimit fields
Characterize fields
Assess the impact of the fields
Demonstration
Onur, Bastien, Paul, Florent Outline November 3, 2017 18 / 27
Fields order importance
Onur, Bastien, Paul, Florent Outline November 3, 2017 19 / 27
Find increment
Onur, Bastien, Paul, Florent Outline November 3, 2017 20 / 27
Stateless vs. stateful
Stateful messages:depend on a global state (ex. authentication)require understanding of the session mechanism
Stateless messages:return the same responsesdo not have any session
Detectionget a valid messagereplay the message several time
same messages ?YES ⇒ statelessNO ⇒ stateful
Onur, Bastien, Paul, Florent Outline November 3, 2017 21 / 27
Fuzzing
Impact of modifications in the messages
identify the type of the fields (static or dynamic)take a message with a dynamic fieldquery the server with both the original and modified messagecalculate the difference between the two
Onur, Bastien, Paul, Florent Outline November 3, 2017 22 / 27
Demonstration
Input
Select messages
Delimit fields
Characterize fields
Assess the impact of the fields
Demonstration
Onur, Bastien, Paul, Florent Outline November 3, 2017 23 / 27
Demonstration
Demonstration
Onur, Bastien, Paul, Florent Outline November 3, 2017 24 / 27
Future work
Dependency of fields
Determinate boundariesIncrease number of generated messages to apply clusteringtechnique
Onur, Bastien, Paul, Florent Future Work November 3, 2017 25 / 27
Conclusion
Used and improved the methods of Netzob (ex. delimiters)Adopted an ”active” strategy to study the behavior of the fields inmessages (ex. characterization of fields)Conducted experiments with a real world protocol (S7)Explored the relationship between dymamic fields and responsemessages
Onur, Bastien, Paul, Florent Future Work November 3, 2017 26 / 27
Thank you for your attention
5813 lines of code
156 commits
4 contributors
54h sur 4 jours
Onur, Bastien, Paul, Florent Future Work November 3, 2017 27 / 27