test and verification solutionshow to avoid the internet ... · • most iot products have security...

Mar, 2017

Test and Verification SolutionsHow to avoid the Internet of

Insecure Things

Delivering Tailored Solutions for

Hardware Verification and Software Testing

Index

✓ Why IoT Testing

✓ IoT Challenges and Recommendations

✓ Standards Body and Value addition

Copyright T&VS Limited | Private & Confidential |

IoT headlines – lack of consumer trust


IoT Solution


Testing of IoT Stack

Application

Process & Platform

Analytics

Network & Connectivity

Gateways DevicesS

ecu

rity

Perf

orm

an

ce

Fu

ncti

on

al

Inte

gra

tio

n

Pro

toc

ol

UI


Nest Smoke Alarm

▪ Product - Nest Protect smoke + CO alarm

▪ Desc - Industrial-grade smoke sensor, can be silenced from

your phone, tests itself automatically and lasts for up to 10years. It also tells you what's wrong and even alert your phone.

▪ Connectivity Requirements:• Wi-Fi connection

• Free Nest account

• Phone or tablet with iOS 8 or later, or Android 4 or later


Example - IoT issues

▪ (1) Nest Protect smoke alarm fault in 2014. The alarm could bedeactivated by waving at the device putting it into sleep mode.• Fix – users had to disable wave gesture feature, and a patch was made available via wifi.

▪ (2) Nest home thermostat recent fault meant the heating woulddeactivate and could not be turned back on.• Fix – a manual reset or 9 step procedure.

▪ (3) Nest Cam and Dropcam frequent outages on service for live homestreaming – potential baby monitoring.• Fix – no fix yet, was a service outage on the live video streams.

▪ All could have been caught with better testing• Also – see crowd testing later


Example of an encryption bug: Logjam

▪ Logjam attack against the TLS protocol.• The Logjam attack allows a man-in-the-middle attacker to downgrade vulnerable TLS

connections to 512-bit export-grade cryptography.

• Due to a flaw in the TLS protocol rather than an implementation vulnerability

• Attacks a Diffie-Hellman key exchange rather than an RSA key exchange

▪ Discovered May 2015• By a group of academics

• It had been present since 1990!

• Affected all modern web browsers

▪ The advice given• If you have a web or mail server

• You should disable support for export cipher suites and use a 2048-bit Diffie-Hellman group.

• If you use a browser…• Use the most recent version of your browser installed, and check for updates frequently.

• If you’re a sysadmin or developer …• Make sure any TLS libraries you use are up-to-date ...

▪ Advice for testing …• It is IMPOSSIBLE to find these bugs in testing

• Make sure you keep up with the relevant websites that report such issues

• And make sure you follow the advice

Protocol Vulnerable to Logjam

HTTPS — Top 1 Million Domains 8.4%

HTTPS — Browser Trusted Sites 3.4%

SMTP+StartTLS — IPv4 Address Space 14.8%

POP3S — IPv4 Address Space 8.9%

IMAPS — IPv4 Address Space 8.4%


What is a DDoS on DNS Services?

▪ DDoS = Distributed Denial of Service

▪ DNS = Domain Name System: translates website names into Internet Protocol (IP) addresses, and locate resources on the Internet.


Botnet DNS attack

▪ Hackers hijacked millions of IoT devices

▪ Sent vast amounts of junk traffic at DNS services operated by US company Dyn

▪ Popular websites inaccessible.

Two things are clear, however: the

freewheeling idiots of the Internet of Things

business need the fear of regulation put

into them – and so do network owners and

operators.


Technical Details

▪ Mirai is malware that turns computer systems running Linux into remotely controlled “botnets” • It primarily targets online consumer devices such as remote cameras and home routers

▪ Mirai spreads by logging into devices using their default, factory-set passwords• Mirai takes over routers, CCTV cameras, digital video recorders, etc.

Zombie = IoT device

running Mirai malware

▪ Advice for testing• Again – need to know which version of the

software is used on these devices and track reports regarding known faults

• Add tests to check if the device can be used with default passwords (force user to change


Why are IoT devices so vulnerable?

Quality

Assurance

Security

Connectivity

standards

• Nest home thermostat had a fault where the heating

would deactivate and not be turned back on

• Petnet smart pet feeder recent incident saw a third-party

server service failure, causing pet feeds to be missed.

• Most IoT products have security measures that are 10

years out of date

• HP: 70% of the IoT devices and sensors examined were

susceptible to the vulnerabilities in the OWASP IoT Top 10

Connected devices create an increased

level of intrusion, generating new types and

unprecedented quantities of data, raising

potential quality and security issues.

onem2mOpen Interconnection

ConsortiumWireless IoT forum

IETF ZigBee AllianceIndustrial Internet

Consortium

ITUAllSeen

GSMAAlliance

IEEE AllJoyn Thread


Patching IoT devices – the device

▪ IoT Devices are “constrained” and typically have limitations :• ROM/Flash: Puts limits on software or firmware code size

• RAM: Puts limits on the size of state and buffers available to executing software or firmware

• Processing: Puts limits on the amount of computation that can be performed in a given period of time.

• Cryptography: May have limited or no cryptographic abilities

• Energy: Limitations on the energy it has available to perform processing.

• Temperature: Temperature range under which it can operate safely.

• Interface: Puts limits on accessibility or what can be updated (e.g. software, firmware or security keys)

• Determinism: Patching mechanism must therefore be to alert other system components that this node will be off-line for a period of time

• Monolithic code: Most constrained devices have limited memory protection. Hence isolation of code components and updates of modules must be addressed via enhanced system design.


Patching IoT devices – the connection

▪ Patching also requires consideration of the network connection• Low achievable bitrate/throughput (including limits on duty cycle),

• High packet loss and high variability of packet loss (delivery rate),

• Highly asymmetric link characteristics,

• Severe penalties for using larger packets (e.g., high packet loss due to link-layer fragmentation),

• Limits on reachability over time (a substantial number of devices may power off at any point in time but periodically "wake up" and can communicate for brief periods of time), and

• Lack of (or severe constraints on) advanced services such as IP multicast.


Implications for testing

▪ Patching MUST be tested in advance within realistic situations

▪ Device level testing• Can the device be taken offline for the duration of the patch?

• Is the patching tested under all expected network conditions?

• What happens if the network condition is lost in the middle?(NOTE: many mobile phones have been “bricked” because the network connection was lost in the middle of a patch)

▪ System level testing• What is the effect of off lining a device for patching?

• How do you protect against a malicious patch?

• Have malicious patch protections been tested?


A view on IoT testing


Testing challenges – mass interoperability

▪ Many Communication protocols:• Mobile Z-Wave • Wifi 6LowPAN• Bluetooth Thread • Zigbee NFC

▪ Simulate wide range of Networking conditions:• RF testing• cell handovers• low signal strength• protocol analysis• moving between 2G, 3G & LTE or wifi

▪ Test scenarios to consider:• Moving between networks

• Losing power on upgrade

• Low bandwidth

• Simulate signal loss (going through a tunnel)

• Patching the device


Crowd testing of the IoT

▪ What is crowd testing?• Crowdsourced testing is an emerging trend in software testing which exploits the benefits,

effectiveness, and efficiency of crowdsourcing and the cloud platform.

• It differs from traditional testing methods in that the testing is carried out by a number of different testers from different places, and not by hired consultants and professionals.

• The software is put to test under diverse realistic platforms

• Crowdsource testing allows for remote usability testing because specific target groups can be recruited through the crowd

▪ Example – Smart metering• Recruit a crowd of testers who have smart meters

• Upload latest versions of software just to their smart meters

• Get their feedback

▪ Example – Nest products• Install Nest products in selected crowds of testers


Communication protocols - scenarios

1 Device registers to network and data connection is successfully established

2 Verify the data transferred from device to IoT platform.

3 IoT device can transfer/move between network connection types (if applicable.)

4 Device Application “stores and forwards” data to minimise the number of network

connections made by the device.

5 IoT Device Application uses dynamic polling intervals.

6 Check IoT Device Application behaviour in situations when network

communication requests fail

7 Check IoT Device Application reports power failure

8 Check IoT Device Application’s use of “off-peak’ communication

9 Check behaviour of IoT Device Application when resetting the Communications

Module after any communication failures or error conditions

10 Upgrade testing – verify post upgrade the comms unit is functioning correctly

11 Check the IoT Communications Module does not send unsolicited messages

12 Check the IoT Communications Module sends only a AAAA DNS Query. IPV6


Security testing – OWASP TOP 10

1.) Insecure web interface

2.) Insufficient authentication/authorization

3.) Insecure network services

4.) Lack of transport encryption

5.) Privacy concerns

6.) Insecure cloud interface

7.) Insecure mobile interface

8.) Insufficient security configurability

9.) Insecure software/firmware

10.) Poor physical security


OWASP TOP first 5 scenarios:

1. Insecure web interface• Check weak passwords• Product setup prompts for username and password• SQL Injections and sharing credential without encryption over the network.

2. Insufficient authentication/authorization• Password Complexity and two factor authentication• Password recovery and insecure mechanism• Lack of Role definition for access.

3. Insecure network services• Insecure web-services are subjected to Buffer overflow and DoS attacks• Open Ports and exploitable UDP services

4. Lack of transport encryption• Unencrypted services• Poorly implemented SSL / TSL

5. Privacy concerns• Poor data protection• Unwanted personal data collection

https://www.owasp.org/images/7/71/Internet_of_Things_Top_Ten_2014-OWASP.pdf



OWASP TOP last 5 scenarios:

6. Insecure cloud interface• Account Enumeration

• No Account Lockout

• Credentials Exposed in Network Traffic

7. Insecure mobile interface

8. Insufficient security configurability• Poorly defined User Role and Permissions

• Lack of Password Security

• Open Security monitoring and Logging

9. Insecure software/firmware• Encryption not used to fetch Updates

• Unencrypted Updates file

• Sensitive Information in firmware

10. Poor physical security• Access to USB ports

• Removal of Storage Media




NMI - IoTSF

▪ September 2015 NMI launched the Internet of ThingsSecurity Foundation (IoTSF), a non-profit organisationestablished to drive security excellence.

▪ Collaborative, vendor-neutral and international initiative; theIoTSF is a dedicated expert resource to drive securityexcellence by sharing knowledge, best practice and advice.

▪ 5 working groups:• 1: Self-Certification Scheme

• 2: Connected Consumer Products

• 3: Patching Constrained Devices

• 4: Framework for Vulnerability Disclosure

• 5: IoT Security Landscape

https://iotsecurityfoundation.org/


Standards bodies – building TRUST

▪ The BSI (British Standards Institute) attempts to build TRUST with consumers• Can we build standards that guarantee some level of confidence

▪ Do we need different levels of confidence?• Autonomous car vs. smoke detector vs. pet feeder

• In safety systems we start with a hazard analysis

• From that we can set an integrity level

• And that implies different levels of development practises

▪ The NMI prefers levels of sign off• Self- certification

• External certification

• Independent certification

• Full certification against industry standards


How we are organized

Members

Plenary Group

Executive

Steering Board

Working Groups

Working Group 1: Self-Certification

Working Group 2: Connected Consumer /

Home

Working Group 3: Patching Constrained

devices

Working Group 4: Vulnerability Disclosure

Working Group 5: IoT Landscape

2016 Priority Working

Groups

Chaired

by:

Mike

Bartley,

T&VS


Working Group 1: Self-

Certification Scheme

The objective of this working group is to determine appropriaterequirements for a low-cost, accessible and fit-for-purpose systemof self-certification in order to improve the quality andpervasiveness of security in IoT products.

26Confidential & copyright © IoTSF 2016

Self-certificate

Is this the way forward?


Marking the IoT Supply Chain of Trust for Complex products

Device

Hardware

Sensor

Actuator

TPM

Comms

module

Firmware

Encryption

keys


Complex supply chain

ODM –

Develops

and

makes

device

Software

developer

Software

developers

Software

developer

Chip

vendor

Software

developer

Comms

module

vendor

“Brand

Owner” –

markets

and

supports

service

Users

Softwa

re

develo

per

IP

vendor


Trusted supply chain

ODM –

Develops

and

makes

device

Software

developer

Software

developer

s

Software

developer

Chip

vendor

Software

developer

Comms

module

vendor

“Brand

Owner” –

markets

and

supports

service

Users

Softwa

re

develo

per

IP

vendor

OTS

RTOS

= IoTSF stamp of approval

= not approved, requires separate

audit


IoTSF members…

▪ Follow the security guidelines for the

relevant device class

▪ Complete WG1 questionnaire with all

questions answered

▪ Assemble evidence of conformance - think

“Technical Construction File”

▪ Are entitled to use the Foundation Trustmark

for the product (possibly subject to audit)


Auditing…

▪ The customer takes self-certification for

granted

▪ The customer audits either as due-diligence

or when there is a breach

▪ Or a trusted third party (“T3P”) may audit as

due-diligence

▪ Or IoTSF may audit (possibly via T3P) and

license Trustmark usage


IoT Kitemark Model

IoT Network connectivity – (1) IoT End 2 end security – (2)

Purpose ensure IoT solutions verified against a wide range of networking connection / connectivity protocols

ensure IoT solutions verified against a wide range of security conditions and scenarios

Standards • GSMA IoT connection efficiencyguidelines

• onem2m connection standards

• GSMA IoT security standards• Onem2m security standards• OWASP Internet of Things Top 10• Online Trust Alliance’s IoT Trust

Framework

Example scenarios

1.) minimize the number of network connections. 2.) cope with variances in network data speed and latency considering 3.) communication requests fail.4.) Communication retry mechanisms implemented verified.

1.) Authentication / authorisation eginterfaces disallows weak passwords.2.) Encryption model eg HTTPS.3.) Cloud interface has account lockout4.) Software / firmware. Eg Ensure alldevices operate with a minimalnumber of network ports active.


Summary

▪ Increased regulation

▪ Focus on QA & security

▪ IoT ongoing maintenance

▪ IoT Kitemark model

▪ Rebuild consumer trust

Unless these issues are addressed the only winners in the IoT wild west will be the hackers.

Mar, 2017

Test and Verification Solutions

Thank you

Delivering Tailored Solutions for

Hardware Verification and Software Testing

“IoT Device Testing: can we provide

assurance in the new wild west?”

Gaurav MaheshwariPrashant Jain

[email protected]

test and verification solutionshow to avoid the internet ... · • most iot products have security...

Documents