embedded systems security: the need for a holistic approach · embedded security must learn lessons...
TRANSCRIPT
![Page 1: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/1.jpg)
Embedded Systems Security: The Need for a Holistic Approach
Stephen Checkoway!Johns Hopkins University!Department of Computer Science
1
![Page 2: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/2.jpg)
Computers are everywhere
2
![Page 3: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/3.jpg)
Computers are everywhere
2
![Page 4: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/4.jpg)
Trends
✤ Mechanical systems replaced by software-controlled embedded systems!✤ Elevators!✤ Slot machines!✤ Planes, trains, and automobiles!✤ Etc.!
✤ Embedded systems gain external connectivity!✤ Wi-fi!✤ Bluetooth!✤ Ethernet!✤ “Sneakernet” 3
![Page 5: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/5.jpg)
PC security is hard (a timeline)
4
Prehistory
Key
Very vulnerable
Somewhat vulnerable
Not vulnerable
Internet usage becomes common
2000 2015…
Miscreants realize they can make money!
All PCs are very vulnerable
![Page 6: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/6.jpg)
Attacks on embedded systems
5Steel mill hack!Germany 2014
Tram hack!Poland 2008
Stuxnet!Iran 2010
![Page 7: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/7.jpg)
But can miscreants make money?
✤ Linux.Darlloz worm!✤ Targets Linux on x86, PowerPC, MIPS, and ARM!✤ Mines cryptocurrencies: Mincoin, Dogecoin
6
![Page 8: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/8.jpg)
Embedded systems I’ve examined
✤ Electronic voting machines!
✤ Automobile computers!
✤ Webcams in laptops!
✤ X-ray scanners used in airports!
✤ Computers used in general aviation
7
![Page 9: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/9.jpg)
Thesis
8
Embedded systems are insecure because we fail to evaluate the systems both adversarially and
holistically.
![Page 10: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/10.jpg)
Talk outline
✤ Introduction!✤ Controlling your car from afar!✤ Defeating your airport security!✤ Conclusions
9
![Page 11: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/11.jpg)
Automobiles
✤ Cars are cyberphysical systems: software controlling the physical world!
✤ Vulnerabilities in automotive systems can be life-threatening
Checkoway, McCoy, Kantor, Anderson, Shacham, Savage, Koscher, Czeskis, Roesner, Kohno. Comprehensive Experimental Analyses of Automotive Attack Surfaces. USENIX Security, 2011.
Koscher, Czeskis, Roesner, Patel, Kohno, Checkoway, McCoy, Kantor, Anderson, Shacham, and Savage. Experimental Security Analysis of a Modern Automobile. IEEE Symposium on Security
and Privacy (“Oakland”), 2010
10
![Page 12: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/12.jpg)
The Evolution of the Automobile
Air/Fuel Mix
Exhaust
Transmission
Brake Line
11
![Page 13: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/13.jpg)
The Evolution of the AutomobileExhaust
Engine Control Unit
Transmission
Brake Line
11
![Page 14: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/14.jpg)
The Evolution of the AutomobileExhaust
Engine Control Unit
TCU
Transmission
Brake LineABS
Airbag Control Unit
Body Controller!Locks/Lights!
Anti-Theft
Keyless Entry
Radio HVAC
11
![Page 15: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/15.jpg)
The Evolution of the AutomobileExhaust
Engine Control Unit
TCU
Transmission
Brake LineABS
Radio
Keyless Entry
Anti-Theft
Body Controller!Locks/Lights!
Airbag Control Unit
HVAC
12
![Page 16: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/16.jpg)
The Evolution of the AutomobileExhaust
Engine Control Unit
TCU
Transmission
Brake LineABS
Radio
Keyless Entry
Anti-Theft
Body Controller!Locks/Lights!
Airbag Control Unit
HVAC
12
![Page 17: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/17.jpg)
The Evolution of the AutomobileExhaust
Engine Control Unit
TCU
Transmission
Brake LineABS
Radio
Telematics _
Internet/!PSTN
Keyless Entry
Anti-Theft
Body Controller!Locks/Lights!
Airbag Control Unit
HVAC
12
![Page 18: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/18.jpg)
The Evolution of the AutomobileExhaust
Engine Control Unit
TCU
Transmission
Brake LineABS
Radio
Telematics _
Internet/!PSTN
Keyless Entry
Anti-Theft
Body Controller!Locks/Lights!
Airbag Control Unit
HVAC
12
![Page 19: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/19.jpg)
✤ Engine on/off!
✤ Brakes on/off!
✤ Horn!
✤ Locks!
✤ Lights!
✤ HVAC!
✤ Telematics!
✤ Instrument panel!
✤ Wipers!
✤ Antitheft measures!
✤ Car alarm!
✤ Starter motor!
✤ Radio!
✤ Etc.
Car components under attacker control
13
![Page 20: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/20.jpg)
✤ Engine on/off!
✤ Brakes on/off!
✤ Horn!
✤ Locks!
✤ Lights!
✤ HVAC!
✤ Telematics!
✤ Instrument panel!
✤ Wipers!
✤ Antitheft measures!
✤ Car alarm!
✤ Starter motor!
✤ Radio!
✤ Etc.
Car components under attacker control
13
![Page 21: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/21.jpg)
✤ Engine on/off!
✤ Brakes on/off!
✤ Horn!
✤ Locks!
✤ Lights!
✤ HVAC!
✤ Telematics!
✤ Instrument panel!
✤ Wipers!
✤ Antitheft measures!
✤ Car alarm!
✤ Starter motor!
✤ Radio!
✤ Etc.
Car components under attacker control
13
![Page 22: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/22.jpg)
✤ Engine on/off!
✤ Brakes on/off!
✤ Horn!
✤ Locks!
✤ Lights!
✤ HVAC!
✤ Telematics!
✤ Instrument panel!
✤ Wipers!
✤ Antitheft measures!
✤ Car alarm!
✤ Starter motor!
✤ Radio!
✤ Etc.
Reflash most ECUs!(even while driving)
Car components under attacker control
13
![Page 23: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/23.jpg)
Security assumption
Physical access to the car is required to tamper with its
computer systems
14
![Page 24: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/24.jpg)
Indirect physical
✤ Definition:!✤ Attacks over physical interfaces!✤ Constrained: Adversary may not directly access the physical
interfaces herself!✤ Extends attack surface
to that of the device
15
![Page 25: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/25.jpg)
Short-range wireless
Definition: Attacks via short-range wireless communications (meters range or less)
16
![Page 26: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/26.jpg)
Long-range wireless
Definition: Attacks via long-range wireless communications (miles, global-scale)
17
![Page 27: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/27.jpg)
Attack vectors explored in depth
✤ Components we compromised!✤ Indirect physical: diagnostic tool!✤ Indirect physical: media player!✤ Short-range wireless: Bluetooth !✤ Long-range wireless: cellular !!
✤ Every attack vector leads to complete car compromise
18
![Page 28: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/28.jpg)
Insert a CD, take over the car
✤ Attack 1: Vestigial radio reflash from CD code!✤ Attack 2: WMA parsing bug; tricky overflow
19
![Page 29: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/29.jpg)
Telematics networking stack
20
![Page 30: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/30.jpg)
3G
PPP
SSL
Tele-matics
Telematics networking stack
20
![Page 31: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/31.jpg)
Telematics networking stack
20
![Page 32: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/32.jpg)
3G
PPP
SSL
Tele-matics
Cell phone
Voice channel
Software modem
Tele-matics
Telematics networking stack
20
![Page 33: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/33.jpg)
Cell phone
Voice channel
Software modem
Tele-matics
Dest
Src
Boundrary
memcpy()
Telematics networking stack
20
![Page 34: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/34.jpg)
Call the car, take over the car
✤ Call telematics unit!
✤ Transmit malicious payload!
✤ Instantiation 1. Implement modem protocol!
✤ Instantiation 2. Play MP3 into phone
21
![Page 35: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/35.jpg)
Call the car, take over the car
✤ Call telematics unit!
✤ Transmit malicious payload!
✤ Instantiation 1. Implement modem protocol!
✤ Instantiation 2. Play MP3 into phone
21
![Page 36: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/36.jpg)
Post-compromise control
✤ External connectivity enables additional command and control!✤ IRC chat client on the telematics unit enables controlling multiple
cars simultaneously
22
![Page 37: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/37.jpg)
Car theft
23
![Page 38: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/38.jpg)
✤ Compromise car
Car theft
23
![Page 39: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/39.jpg)
✤ Compromise car
✤ Locate car (via GPS)
Car theft
23
![Page 40: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/40.jpg)
✤ Compromise car
✤ Locate car (via GPS)
✤ Unlock doors
Car theft
23
![Page 41: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/41.jpg)
✤ Compromise car
✤ Locate car (via GPS)
✤ Unlock doors
✤ Start engine
Car theft
23
![Page 42: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/42.jpg)
✤ Compromise car
✤ Locate car (via GPS)
✤ Unlock doors
✤ Start engine
✤ Bypass anti-theft
Car theft
23
![Page 43: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/43.jpg)
Surveillance
24
![Page 44: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/44.jpg)
Surveillance
24
✤ Compromise car
![Page 45: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/45.jpg)
Surveillance
24
✤ Compromise car
✤ Continuously report GPS coordinates
![Page 46: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/46.jpg)
Surveillance
24
✤ Compromise car
✤ Continuously report GPS coordinates
✤ Stream audio recorded from the in-cabin mic
![Page 47: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/47.jpg)
What went wrong with the car?
✤ Lack of adversarial pressure (this has started to change)!
✤ Subsystems evaluated in isolation, not holistically!
✤ Manufacturers are really integrators
25
![Page 48: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/48.jpg)
No adversarial testing
✤ Manufacturers provide vendors incomplete functional specifications!
✤ Minimal conformance testing!✤ Spec says “on input A, perform action X”; test that!✤ Spec says nothing about input B; no tests!
✤ All computers on the bus implicitly trusted!
✤ No notion of an adversary
26
![Page 49: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/49.jpg)
Isolated evaluation
✤ Heterogeneous, distributed, multi-vendor system!✤ Internals of components frequently opaque!✤ Incorrect assumptions between different suppliers!✤ Almost all bugs found at component boundaries!
✤ Formerly disconnected systems now connected!✤ No analysis of implications
27
![Page 50: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/50.jpg)
Talk outline
✤ Introduction!✤ Controlling your car from afar!✤ Defeating your airport security!✤ Conclusions
28
![Page 51: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/51.jpg)
Full-body, X-ray Scanners
✤ Another cyberphysical system!
✤ Uses X-rays to produce naked images of subjects
29
Mowery, Wustrow, Wypych, Singleton, Comfort, Rescorla, Checkoway, Halderman, and Shacham Security Analysis of a Full-body Scanner. USENIX Security, 2014.
![Page 52: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/52.jpg)
30
Warning: NudityThis section shows unmodified scanner
images to demonstrate the privacy implications of full body scanning.
![Page 53: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/53.jpg)
Full-body scanners
31I M A G E : R A P I S C A N C O R P. , L - 3 C O M M U N I C AT I O N S
![Page 54: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/54.jpg)
Full-body scanner deployment
32I M A G E : R A P I S C A N C O R P. , L - 3 C O M M U N I C AT I O N S
2008 2009 2010 2011 2012 2013 20142007
![Page 55: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/55.jpg)
Full-body scanner deployment
32I M A G E : R A P I S C A N C O R P. , L - 3 C O M M U N I C AT I O N S
Feb 2007: TSA introduces FBSs as ‘secondary screening’
2008 2009 2010 2011 2012 2013 20142007
![Page 56: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/56.jpg)
Full-body scanner deployment
32I M A G E : R A P I S C A N C O R P. , L - 3 C O M M U N I C AT I O N S
Feb 2007: TSA introduces FBSs as ‘secondary screening’
Dec 2009: Failed bombing of Transatlantic flight
2008 2009 2010 2011 2012 2013 20142007
![Page 57: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/57.jpg)
Full-body scanner deployment
32I M A G E : R A P I S C A N C O R P. , L - 3 C O M M U N I C AT I O N S
Feb 2007: TSA introduces FBSs as ‘secondary screening’
Dec 2009: Failed bombing of Transatlantic flight
Dec 2009: TSA moves FBSs to primary screening
2008 2009 2010 2011 2012 2013 20142007
![Page 58: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/58.jpg)
Full-body scanner deployment
32I M A G E : R A P I S C A N C O R P. , L - 3 C O M M U N I C AT I O N S
Feb 2007: TSA introduces FBSs as ‘secondary screening’
Dec 2009: Failed bombing of Transatlantic flight
Dec 2009: TSA moves FBSs to primary screening
Nov 2012: Secure 1000 arrives at our lab
2008 2009 2010 2011 2012 2013 20142007
![Page 59: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/59.jpg)
Full-body scanner deployment
32I M A G E : R A P I S C A N C O R P. , L - 3 C O M M U N I C AT I O N S
Feb 2007: TSA introduces FBSs as ‘secondary screening’
Dec 2009: Failed bombing of Transatlantic flight
Dec 2009: TSA moves FBSs to primary screening
Nov 2012: Secure 1000 arrives at our lab
May 2013: TSA retires Secure 1000
2008 2009 2010 2011 2012 2013 20142007
![Page 60: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/60.jpg)
Public debate
33
![Page 61: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/61.jpg)
Public debate
33
Radiological Safety?
“ … T H E D O S E T O T H E S K I N M AY B E D A N G E R O U S LY H I G H . ”
— UC San Francisco
![Page 62: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/62.jpg)
Public debate
33
Privacy?
Radiological Safety?
“ … T H E D O S E T O T H E S K I N M AY B E D A N G E R O U S LY H I G H . ”
— UC San Francisco
![Page 63: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/63.jpg)
Public debate
33
Privacy?
Contraband!Detection?
Radiological Safety?
“ … T H E D O S E T O T H E S K I N M AY B E D A N G E R O U S LY H I G H . ”
— UC San Francisco
![Page 64: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/64.jpg)
TSA response
34
![Page 65: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/65.jpg)
Acquisition
35
![Page 66: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/66.jpg)
Our contribution: The facts
1. Is the Secure 1000 radiologically safe?!
2. What privacy safeguards exist?!
3. How effective is it at detecting contraband?
36
![Page 67: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/67.jpg)
Inside the Secure 1000
37
![Page 68: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/68.jpg)
38
![Page 69: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/69.jpg)
X-ray physics 101
39
Photoelectric Effect!(X-ray absorbed)
Incoming Photon
Electron
PhotoelectronIncoming Photon
Electron
Recoil electron
Scattered Photon
Compton Scattering!(X-ray scattered)
Dominant effect depends on material’s “effective atomic number”
![Page 70: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/70.jpg)
Secure 1000 X-ray hardware
F I G U R E A D A P T E D F R O M U . S . PAT E N T 8 , 1 9 9 , 9 9 6 !R . H U G H E S , J U N E 2 0 1 2 40
![Page 71: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/71.jpg)
Secure 1000
✤ Chopper spins!
✤ Head assembly moves vertically
41
![Page 72: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/72.jpg)
Secure 1000
✤ Chopper spins!
✤ Head assembly moves vertically
41
![Page 73: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/73.jpg)
Secure 1000 X-ray hardware
42
![Page 74: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/74.jpg)
Image production
43
![Page 75: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/75.jpg)
The results
44
![Page 76: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/76.jpg)
Radiation safety
45
![Page 77: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/77.jpg)
Radiation safety
✤ X-ray energy: 50 KeV at 5 mA!✤ Dose per scan: 70-80 nSv!
✤ ~24 minutes of background exposure!✤ Similar results by AAPM (2013)
46
![Page 78: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/78.jpg)
Cyberphysical radiation safety
✤ Safety controls on radiological output!✤ Not security controls!!
✤ Simple, modular design!✤ Cannot over-irradiate scan subject without ROM replacement
47
![Page 79: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/79.jpg)
Privacy
48
![Page 80: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/80.jpg)
External PMT reconstruction
✤ X-rays backscatter in all directions!
✤ Allows nearby adversary to capture images
49
![Page 81: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/81.jpg)
External PMT reconstruction
✤ X-rays backscatter in all directions!
✤ Allows nearby adversary to capture images
49
![Page 82: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/82.jpg)
External PMT reconstruction
✤ This is a small PMT!
✤ The larger the PMT, the more detailed
50
![Page 83: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/83.jpg)
Efficacy
51
![Page 84: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/84.jpg)
Operator software
52
![Page 85: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/85.jpg)
Console malware
53“Secret knock” Visible light X-ray
![Page 86: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/86.jpg)
Console malware
53“Secret knock” Visible light X-ray
Operator’s View
![Page 87: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/87.jpg)
Adversarial physics
54
![Page 88: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/88.jpg)
Adversarial physics
54
![Page 89: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/89.jpg)
Firearms
✤ Subject is carrying a .380 ACP pistol
55
![Page 90: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/90.jpg)
Firearms
✤ Subject is carrying a .380 ACP pistol
55
![Page 91: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/91.jpg)
Folding knife
✤ Subject is carrying a folding knife
56
![Page 92: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/92.jpg)
Folding knife
✤ Subject is carrying a folding knife
56
![Page 93: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/93.jpg)
Plastic explosives
Q U O T E : H T T P : / / A B C N E W S . G O . C O M / B L O G S / P O L I T I C S / 2 0 1 3 / 0 8 / O U T G O I N G - D H S - S E C R E TA RY- J A N E T-N A P O L I TA N O - WA R N S - O F - S E R I O U S - C Y B E R - AT TA C K U N P R E C E D E N T E D - N AT U R A L - D I S A S T E R /
57
![Page 94: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/94.jpg)
Sandia: C4 detection (1992)
58R E P R O D U C E D F R O M “ E VA L U AT I O N T E S T S O F T H E S E C U R E 1 0 0 0 S C A N N I N G S Y S T E M ” !
T E C H N I C A L R E P O RT S A N D 9 1 - 2 4 8 8 , U C - 8 3 0 , S A N D I A N AT I O N A L L A B O R AT O R I E S , A P R . 1 9 9 2 .
![Page 95: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/95.jpg)
Sandia: C4 detection (1992)
58R E P R O D U C E D F R O M “ E VA L U AT I O N T E S T S O F T H E S E C U R E 1 0 0 0 S C A N N I N G S Y S T E M ” !
T E C H N I C A L R E P O RT S A N D 9 1 - 2 4 8 8 , U C - 8 3 0 , S A N D I A N AT I O N A L L A B O R AT O R I E S , A P R . 1 9 9 2 .
![Page 96: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/96.jpg)
Think adversarially!
59
![Page 97: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/97.jpg)
Think adversarially!
59Plastic!
![Page 98: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/98.jpg)
Plastic explosives
60
No contraband
vs.
Subject carrying 200+ g of C-4simulant
![Page 99: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/99.jpg)
Plastic explosives
60
No contraband
vs.
Subject carrying 200+ g of C-4simulant
![Page 100: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/100.jpg)
Plastic explosives
60
No contraband
vs.
Subject carrying 200+ g of C-4simulant
![Page 101: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/101.jpg)
Efficacy results
✤ Our results imply adversaries can conceal:!✤ Knives!✤ Firearms!✤ Plastic explosive & detonators!!
✤ Access to Secure 1000 allows attack refinement
61
![Page 102: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/102.jpg)
What went wrong with the scanner?
✤ Limited threat model!✤ Assumes naïve adversary/nonadaptive!✤ Doesn’t consider insiders!!
✤ Didn’t evaluate holistically!✤ Didn’t consider limitations of X-ray physics
62
![Page 103: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/103.jpg)
Talk outline
✤ Introduction!✤ Controlling your car from afar!✤ Defeating your airport security!✤ Conclusions
63
![Page 104: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/104.jpg)
How did we get here?
✤ Embedded systems not designed with a security mindset!✤ Basic flaws (e.g., buffer overflows)!✤ Few technologically-enforced access controls!✤ Insiders not considered!
✤ Components not designed with connectivity in mind!✤ Failure to evaluate systems holistically
64
![Page 105: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/105.jpg)
What should we do about it?
✤ Embedded security must learn lessons from the PC world or it will repeat the mistakes!
✤ Embedded systems can implement defenses deemed to unacceptably degrade PC performance!
✤ Construct and use realistic threat models!
✤ Systems should be designed and audited as a whole!
✤ Updates should be pushed to devices
65
![Page 106: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/106.jpg)
Design choices
✤ Move from federated to integrated (e.g., in aircraft avionics)!✤ Modular design with narrow data interfaces!
✤ Simplifies security analysis!✤ Limits damage from compromised components!✤ E.g., car vs. scanner!
✤ Car: modular design but ECU could be completely reprogrammed from the bus!
✤ Scanner: modular design with constrained interface (HOME, SU, SD, …)
66
![Page 107: Embedded Systems Security: The Need for a Holistic Approach · Embedded security must learn lessons from the PC world or it will repeat the mistakes! Embedded systems can implement](https://reader035.vdocuments.mx/reader035/viewer/2022062602/5e79102fcdcf0330747655fa/html5/thumbnails/107.jpg)
Thank you!
Fin67