docker 101 @kacst saudi hpc 2016
TRANSCRIPT
![Page 1: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/1.jpg)
Saudi HPC 2016 Tutorials@WALIDSHAARI21st April 2016
![Page 2: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/2.jpg)
§ KACST to giving us this opportunity
§ Docker Inc. For allowing us use some of their graphics from docker slideshare
§ Christian Qnib @CQnib for the valuable HPC insights, and slide ideas for CNAME
§ All for attending
![Page 3: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/3.jpg)
§ vim/Emacs
§ Linux
§ Git
§ vagrant
§ Windows
§ Development/Operations/other
§ Devops
§ Configuration management tool
§ Containers
§ Schedulers
§ Containers management tools, or distributions
§ Cluster management tools
![Page 4: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/4.jpg)
0.0.3 March 2013
1.0 June 2014
1.1 July 2014
1.2 August 2014
1.3 October 2014
1.4 December
2014
1.5 February
2015
1.6 April 2015
1.7 June 2015
1.8 August 2015
1.9 November
2015
1.10 Feburary
2016
1.11 April 2016
![Page 5: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/5.jpg)
![Page 6: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/6.jpg)
Linux features?
Name spaces?cgroups?Lxc?
Union file systems
Configuration management ?
Virtualization technology?
npm
jar
Packaging ?
rpm
deb
tar.gz
Virtual/environment management ?
![Page 7: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/7.jpg)
Manualconfiguration
Traditional VMs
Less PortableMinimal overhead
Most PortableLots of overhead
Configuration Management tools
Containers
Docker
Intel ClearContainers
![Page 8: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/8.jpg)
![Page 9: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/9.jpg)
§ There are other containers § Shifter
§ Didn’t like it, it made wrong assumptions about containers, and meant for CRAY, it exploits the current Docker registry system to build its own chrooted squashefscontainers in loopback devices, does not have the full Docker advantages.
§ Singularity§ Rocket
§ Back to the Linux/Unix philosophy, developed by CoreOS team
§ Intel clear containers§ One possible solution to Container security
![Page 10: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/10.jpg)
![Page 11: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/11.jpg)
Container encapsulate an application completely with all of its software dependencies into a standardized unit for software portable across different platforms*.
https://www.docker.com/what-docker
![Page 12: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/12.jpg)
Bin/Libs
APP C
Docker Engine
Docker capable OS
Bin/Libs
App B
Bin/Libs
APP A
![Page 13: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/13.jpg)
Bin/Libs
Database
Docker Engine
Docker capable OS
Bin/Libs
Load balancer
Bin/Libs
Web ServerWeb Server
Bin/Libs
![Page 14: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/14.jpg)
•Open Source Devops platform • empowers developer to build and package services and
applications• ship containerized applications as independent building unit
“Lego”• Enables continuous delivery, and deployment• Portability, reproducibility• Behaves the same way in production, or other env.
• Run standalone or distributed
![Page 15: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/15.jpg)
§ Docker Core Components:§ Engine : Portable (light/heavy) weight run time, and packaging tool§ Registry : Central repos to manage sharing of packaged Docker images, and
enabled (build, ship, release, run) workflows ( the default registry Docker HUB)
§ Containers Eco System§ Modular
§ Check Schedulers§ Compliant
§ OCI§ Holistic
§ End to end workflows
![Page 16: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/16.jpg)
![Page 17: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/17.jpg)
§ cgroups by Google
§ namespace by IBM
§ selinux by NSA
§ iptables
§ Docker in March 2013 was a lightweight runtime and robust tooling to build and ship the encapsulated operating environment to run virtual containers on Linux for your distributed applications.
![Page 18: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/18.jpg)
![Page 19: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/19.jpg)
§ Image:The basis of a Docker container
§ Docker Container: The standard unit in which the application service resides "running image instance"
§ Docker Engine: Runs on Linux, Creates, ships and runs Docker containers deployable on physical or virtual host locally, in a datacenter or cloud service provider
§ Docker Machine:Docker Machine is a tool that lets you install Docker Engine on virtual hosts, and manage the hosts with docker- machine commands. You can use Machine to create Docker hosts on your local Mac or Windows box, on your company network, in your data center, or on cloud providers like AWS or Digital Ocean.
§ Docker Registry: On-premises registry for image storing and collaboration
![Page 20: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/20.jpg)
![Page 21: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/21.jpg)
![Page 22: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/22.jpg)
![Page 23: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/23.jpg)
Containers
• Packages up software binaries and dependencies "minimal"• Isolates software from each other• Container is a standard format• Easily portable across environment• Allows ecosystem to develop around its standard Container
![Page 24: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/24.jpg)
![Page 25: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/25.jpg)
§ Deploy faster, and more often
§ Shorter time between deploys
§ Resource usage improved too (2x-10x)
§ Portability Freedom: Move containers around (laptop to workstation, laptop to servers, colo-to-cloud, cloud-to-cloud, cloud-to-colo, ...)
![Page 26: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/26.jpg)
Network Storage I/OMemoryCPU
CGROUP 1
CGROUP 2
CGROUP 3
CGROUP 4
CGROUP 5
CGROUP 6 CGROUP 8
CGROUP 9
CGROUP 7
- Other subsystems:• DEVICE• HUGETLB• CPUSET
![Page 27: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/27.jpg)
https://github.com/yadutaf/ctop
![Page 28: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/28.jpg)
Host
Isolation is achieved via Linux kernel namespaces
Container 3pbs_serve
rpbs_sched
sshd
Container 2
mpi_task(s)
pbs_mom
sshd
Container 1
mpi_task(s)
pbs_mom
sshd
Mount IPC UTSNetworkPID USER
![Page 29: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/29.jpg)
Container have several name spaces
Host
Container 2
mpi_task(s)
pbs_mom
sshd
Container 1
mpi_task(s)
pbs_mom
sshd
Mount IPC UTSNetworkPID
Container 3pbs_serve
rpbs_sched
sshd
USER
![Page 30: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/30.jpg)
Network namespace can be shared between one or more container
Host
Container 2
mpi_task(s)
pbs_mom
sshd
Container 1
mpi_task(s)
pbs_mom
sshd
Mount IPC UTSNetworkPID
Container 3pbs_serve
rpbs_sched
sshd
USER
![Page 31: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/31.jpg)
![Page 32: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/32.jpg)
![Page 33: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/33.jpg)
https://docs.docker.com/
![Page 34: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/34.jpg)
1- Go to https://docs.docker.com/windows/2- then start installation https://docs.docker.com/windows/step_one/3- Ensure git bash terminal is installed4- invoke a git bash terminal5- verify the docker-machine
![Page 35: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/35.jpg)
1- Initialize the Docker machine environment2- Verify the Docker-machine
docker-machine ls
docker-machine env default
3- You might need to create a virtual default machine
![Page 36: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/36.jpg)
You might need to create a virtual default machinedocker-machine create --driver virtual box default
![Page 37: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/37.jpg)
eval `docker-machine env default`docker run hello-world
![Page 38: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/38.jpg)
docker search hello-world
![Page 39: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/39.jpg)
docker search –s 20 alpine
curl -s -S 'https://registry.hub.docker.com/v2/repositories/library/alpine/tags/' | jq '."results"[]["name"]' |sort
![Page 40: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/40.jpg)
docker pull docker.io/seqvence/static-sitedocker pull mhart/alpine-nodedocker pull python:2.7-alpinedocker pull manomarks/workerdocker pull redis:alpinedocker pull postgres:9.4docker history syed/linpack
![Page 41: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/41.jpg)
$docker pull alpine:2.6$docker images
![Page 42: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/42.jpg)
https://medium.com/@mccode/the-misunderstood-docker-tag-latest-af3babfd6375#.2rn04fqc9
$docker run alpine:latest cat /etc/alpine-release$docker run alpine:2.6 cat /etc/alpine-release$docker run alpine:2.6 uptime$docker run alpine:latest uptime
![Page 43: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/43.jpg)
$docker history alpine:latest$docker history alpine:2.6$docker history syed/linpack
![Page 44: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/44.jpg)
![Page 45: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/45.jpg)
docker images
docker ps -a
![Page 46: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/46.jpg)
![Page 47: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/47.jpg)
-I interactive-t
![Page 48: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/48.jpg)
for i in {1..10}; do docker run alpine uptime; done
docker ps -a
man docker-rm
docker ps –a
docker rm container [container...]
Or:
docker rm `docker ps --all -q`
What if you want to remove:
volumes associated with the container?
Links associated with the container?
Running container(s)?
![Page 49: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/49.jpg)
docker stats
docker ps -a
![Page 50: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/50.jpg)
![Page 51: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/51.jpg)
[walid@silverII ~] $ docker run seqvence/static-site
![Page 52: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/52.jpg)
How to pass a signal to Kill?
![Page 53: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/53.jpg)
docker run --name static-site -e AUTHOR="Walid Shaari" -d -P seqvence/static-site
![Page 54: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/54.jpg)
![Page 55: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/55.jpg)
§ From the docker-machine§ $ docker-machine ip default
docker run --name static-site-2 -e AUTHOR="Your Name" -d -p 8888:80 seqvence/static-site
![Page 56: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/56.jpg)
§ Base
§ Child
§ Official
§ User
![Page 57: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/57.jpg)
§ https://github.com/docker/docker-birthday-3
§ In your homedir§ git clone https://github.com/docker/docker-birthday-3§ cd docker-birthday-3/example-voting-app/§ Follow https://github.com/docker/docker-birthday-3 instruction for compose, and swarm§ Check example enteries
§ https://blog.docker.com/2016/04/docker-birthday-app-challenge-winners/
![Page 58: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/58.jpg)
https://github.com/yadutaf/infilter
To share the host network, PID, and IPC namespaces with the container,
run the following command:$ docker run -it --net=host --pid=host --ipc=host alpine sh
![Page 59: Docker 101 @KACST Saudi HPC 2016](https://reader033.vdocuments.mx/reader033/viewer/2022051404/58e4ac311a28abbb038b591d/html5/thumbnails/59.jpg)
• There is more• Schedulers
• Nomad• Kubernetes• Mesos
• Storage engines• ZFS• BTRFS• CEPH
• Network overlays• And definitely there is more to be awesome!