containers and kubernetes put a lid on it: securing for ... · put a lid on it: securing containers...
TRANSCRIPT
#vmworld
Put a Lid on It: Securing Containers and Kubernetes
on vSphere and AWSSteve Hoenisch, VMware, Inc.Nolan Karpinski, VMware, Inc.
CNA1656BE
#CNA1656BE
VMworld 2018 Content: Not for publication or distribution
Disclaimer
2©2018 VMware, Inc.
This presentation may contain product features orfunctionality that are currently under development.
This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery.
Pricing and packaging for any new features/functionality/technology discussed or presented, have not been determined.
VMworld 2018 Content: Not for publication or distribution
Agenda
3©2018 VMware, Inc.
The Cloud-Native Stack
Container Security Risks and Threats
Security in Pivotal Container Service on vSphere
Security in VMware Cloud PKS on AWS
Security with AppDefense and Demo
Summary
VMworld 2018 Content: Not for publication or distribution
4©2018 VMware, Inc.
The Cloud-Native StackLayers and Processes Requiring Security
Applications
Infrastructure
Orchestration Platform
Container Registry
Container Runtime
Container Host
Container Images
Container Management
VMworld 2018 Content: Not for publication or distribution
5©2018 VMware, Inc.
Container LifecycleSecurity for Developers Takes Place in this Workflow
DeployBuild Run
VMworld 2018 Content: Not for publication or distribution
6©2018 VMware, Inc.
Risks and ThreatsAttack Vectors throughout the Stack Identified by NIST
VMworld 2018 Content: Not for publication or distribution
7©2018 VMware, Inc.
Just like traditional apps, containerized apps are vulnerable to typical flaws.
Vulnerability can be exploited.
Containers give you a prescribed way to apply security principles with depth and breadth.
Core protection strategies help secure cloud workloads.
The Application
Applications
Infrastructure
Orchestration Platform
Container Registry
Container Runtime
Container Host
Container Images
Container Management
VMworld 2018 Content: Not for publication or distribution
8©2018 VMware, Inc.
Images with vulnerabilities, malware, embedded secrets, etc.
Images that are outdated or unpatched or of unknown origin
Insecure connectivity
Inadequate access control
Insecure configuration (running as privileged user or with SSH)
The rogue containers of hasty developers
Operating systems with a large attack surface, shared kernel, or package vulnerabilities
Container ManagementPortability and Reuse Heighten Risks
Applications
Infrastructure
Orchestration Platform
Container Registry
Container Runtime
Container Host
Container Images
Container Management
VMworld 2018 Content: Not for publication or distribution
9©2018 VMware, Inc.
Unsecured components, like the Kubernetes Dashboard, on the Internet
Lack of standard directory service and RBAC
Unencrypted data
Apps sharing the same virtual networks
Mixing workloads with different sensitivity levels and threat postures
Risks to the Orchestration SystemUnsecure Components, Lack of Access Control, and Mixed Workloads Pose Threats
Applications
Infrastructure
Orchestration Platform
Container Registry
Container Runtime
Container Host
Container Images
Container Management
VMworld 2018 Content: Not for publication or distribution
10©2018 VMware, Inc.
Unprotected data in transit and in storage heightens risks of a breach
Lack of logging, monitoring, and visibility can make it hard to identify intrusions and respond quickly
Infrastructure
Applications
Infrastructure
Orchestration Platform
Container Registry
Container Runtime
Container Host
Container Images
Container Management
VMworld 2018 Content: Not for publication or distribution
11©2018 VMware, Inc.
Security in VMwarePivotal Container Service Container Images
Access Control
Micro-Segmentation
Logging and Monitoring
VMworld 2018 Content: Not for publication or distribution
12©2018 VMware, Inc.
Architecture
VMworld 2018 Content: Not for publication or distribution
13©2018 VMware, Inc.
Harbor: Image Registry Imposes CountermeasuresScans, Secures, and Signs Images and then Prescribes Use with Policies
DevelopmentTeam
RBAC
UAA AUTH REPLImage Pull
K8s Cluster deployed by PKS
Build Image Push Image Scan ImageFor CVEs
Sign Trusted Image
kubectl run
ImageRegistry
Image Scanning
Image Signing
HarborProjects
AUDIT LOGGING
VMworld 2018 Content: Not for publication or distribution
14©2018 VMware, Inc.
Identity and Access ManagementControlling Access to Kubernetes Clusters
Operator admin
Developers Developers
Kubernetes
Namespace Namespace
Namespace NamespaceKubernetes
Namespace Namespace
Namespace Namespace
PKS Broker
UAA PKS API
BOSH CredHub
Authentication and RBAC for PKS CLI and Kubernetes API
Centralized credential generation and management with CredHub
PKS secures the Kubernetes Dashboard by default with user authentication.
VMworld 2018 Content: Not for publication or distribution
15©2018 VMware, Inc.
PKS with NSX-T Provides Strong Isolation for Kubernetes ClustersEach Cluster Isolated on its own Network Segment
Virtual Switch
K8 Worker
K8 Worker
K8 Worker
Kubernetes Cluster
T1 Router
Virtual Switch
K8 Worker
K8 Worker
K8 Worker
Kubernetes Cluster
T1 Router
T0 RouterLB LB
Cloud Provider Uplinks Easily create clusters and then use a cluster as a unit of tenancy in a multi-tenant context.
Tenants get separate clusters. NSX ensures isolation is
logically enforced in the network.
NSX isolates orchestrator traffic from workload traffic.
Cluster Networking
VMworld 2018 Content: Not for publication or distribution
16©2018 VMware, Inc.
NSX-T Isolates Namespaces with Logical Switches and RoutersProvides Stronger Isolation than Default K8s Namespaces
admin@k8s-master:~$ kubectl create namespace foonamespace ”foo" created
admin@k8s-master:~$ kubectl create namespace barnamespace ”bar" created
admin@k8s-master:~$ kubectl run nginx-foo --image=nginx -n foodeployment "nginx-foo" created
admin@k8s-master:~$ kubectl run nginx-bar --image=nginx -n bardeployment "nginx-bar" created
Namespace: foo Namespace: bar
K8s Topology with NSX
10.24.0.0/24 10.24.1.0/24 10.24.2.0/24
NAT boundary
K8s nodesK8s Masters
• Within each cluster, each namespace gets its own network segment for strong isolation.
VMworld 2018 Content: Not for publication or distribution
17©2018 VMware, Inc.
NSX-T Can Use K8s Network Policies for Micro-SegmentationSegments Pods by Applying Dynamic Security Groups and Policies
admin@k8s-master:~$ vim nsx-demo-policy.yamlapiVersion: extensions/v1beta1kind: NetworkPolicymetadata:name: nsx-demo-policy
spec:podSelector:matchLabels:app: web
ingress:- from:
- namespaceSelector:matchLabels:ncp/project: db
ports:- port: 80protocol: TCP
admin@k8s-master:~$ kubectl create -f nsx-demo-policy.yaml
Namespace: foo Namespace: bar
NSX / K8s topology
10.24.0.0/24 10.24.1.0/24 10.24.2.0/24
NAT boundary
DBLabel: app=db
WebLabel: app=web
• Assign policies in Kubernetes that get translated into to NSX policies and firewall rules.
• Policies are defined as part of app’s deployment for portability.
VMworld 2018 Content: Not for publication or distribution
18©2018 VMware, Inc.
Pod Micro-SegmentationNSX Uses Network Policy to Dynamically Create Security Groups and Policy
$ kubectl create -f nsx-demo-policy.yaml
Dynamic Creation of Security Groups
Dynamic Creation of Security Policy based on K8s Network PolicyVMworld 2018 Content: Not for publication or distribution
19©2018 VMware, Inc.
Getting Visibility Across the StackTraceflow with NSX-T
Monitor network traffic
Trace packets from containers to physical networks
Visualize traffic flows across the stack
Other tools:
Port Mirroring
Port Connection Tool
Spoofguard
Syslog
Port Counters
IPFIX
NSX-T Traceflow
VMworld 2018 Content: Not for publication or distribution
20©2018 VMware, Inc.
vRealize Log Insight and vRealize Operations
Gain comprehensive visibility across apps and infrastructure
Analyze logs for suspicious activity
Monitoring, Logging, and AnalyticsOperational Visibility and Auditing
vRLI vROPS
Logs Metrics
VMworld 2018 Content: Not for publication or distribution
21©2018 VMware, Inc.
Lifecycle Management Maintains SecurityBOSH Patches and Repairs Nodes
VMware GCP
AvailabilityZone
master
etcd
worker
etcd etcd
master
worker worker
master
etcd
worker
etcd etcd
master
worker worker
AvailabilityZone
AvailabilityZone
BOSH
Health Monitor
Health Monitor
worker workerPatch
K8sNewVer
Release
Repair
Deploy
ScaleUpgradePatchRepair
Day 1
Day 2
workerworker
PKS
Con
trol
Pla
ne
CVE
VMworld 2018 Content: Not for publication or distribution
22©2018 VMware, Inc.
Operator admin
PKS Security Overview
InfrastructureStorageCompute Networking
Container Mgmt.
Namespace
vSphere Google Cloud Platform
Hybrid
Namespace
Namespace Namespace
Container Mgmt.
Namespace Namespace
Namespace
Developers
IAM
Events & Monitoring
Build Pipelines
Namespace
Platform LCM
Kubernetes Cluster Kubernetes Cluster
Apps Apps
Micro-service
Micro-service
Micro-service
Micro-service
Micro-service
Micro-service
Micro-service
Micro-service
Platform LCM
Container Management
Infrastructure
LDAP or AD
IAM
VNFM
vRealizeOperations
Monitoring
Logging
vRealize Log Insight
VMworld 2018 Content: Not for publication or distribution
23©2018 VMware, Inc.
Security in VMware Cloud PKS
Overview
Access Control Policies (Lightwave)
Linux Container Host (Photon OS)
VMworld 2018 Content: Not for publication or distribution
24©2018 VMware, Inc.
VMware Cloud PKS OverviewHighly Secure and Available Kubernetes Service on AWS
Fully managed for you by VMware
Smart Clusters Elastic cluster size HA configuration Automatic recovery from
failure Granular multi-tenant access
policies Multi-cloud ready Differences between
VMware Cloud PKS and PKS
US west
US east
Europe West
VMworld 2018 Content: Not for publication or distribution
25©2018 VMware, Inc.
Manage user identities centrally in VMware Cloud Services
Access VMware Cloud Services, VMware Cloud PKS, and all your Kubernetes clusters with your identity
Single Sign-On with Unified Identity at cloud.vmware.com
25
Part of VMware Cloud Services
VMworld 2018 Content: Not for publication or distribution
26©2018 VMware, Inc.
Applications
Infrastructure
Orchestration Platform
Container Registry
Container Runtime
Container Host
Container Images
Container Management
Access Control Policies
VMworld 2018 Content: Not for publication or distribution
27©2018 VMware, Inc.
Multi-Tenant Access PoliciesAccess Control Encompasses Entire Service
ENGINEERING FOLDER QUOTAPOLICY
DEV/TEST PROJECT
SMART CLUSTER
NAME SPACE
SMART CLUSTER
PRODUCTION PROJECT
SMART CLUSTER SMART CLUSTER
AI PROJECT
SMART CLUSTER
FINANCE FOLDER
EU-West-1US-West-2 US-East-1 US-West-2
SMART CLUSTER
EU-West-1
ACCESS POLICIES
ACCESS POLICIES
QUOTAPOLICY
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
NAME SPACE
VMworld 2018 Content: Not for publication or distribution
28©2018 VMware, Inc.
Role-Based Access Control (RBAC) Tenants can organize clusters
with folders and projects.
Roles grant a configurable set of permissions, such as administering or viewing a cluster or a namespace.
Access polices are sets of roles bound to users or groups.
Roles are inherited down the tree.
VMware Cloud PKS pushes policies to Kubernetes and translates them into Kubernetes RBAC.
Policies can apply two key principles: Separation of dutiesand principle of least privilege.
Organization
Folder 1 Folder 2
Project 1 Project 2
Cluster 1 Cluster 2
Namespace 1 Namespace 2
SmartCluster.Admin: ClusterAdmin1
SmartCluster.Admin: ClusterAdmin1
SmartCluster.Admin: ClusterAdmin2SmartCluster.Admin: ClusterAdmin1
SmartCluster.Admin: ClusterAdmin1
SmartCluster.Admin: ClusterAdmin1
Project 1
Cluster 1
VMworld 2018 Content: Not for publication or distribution
31©2018 VMware, Inc.
Lightwave in VMware Cloud PKS Directory Service, Certificate Authority, Secure Token Service
31
LIGHTWAVEDIRECTORY SERVICE
LW D
C1
LW D
C2 LW
DC
1
LW D
C2
ESXiHOST
ESXiHOST
ESXiHOST
ESXiHOST
ESXiHOST
C
K8SK8S
NNN
NNN
Cluster
NN
NN
Cluster
NNN
NNN
Cluster
T1T4 T2N
NN
Cluster
T3
K8SK8S
AWS US-Region
N
K8S K8S
N N N
N N
Cluster
N N
N N
Cluster
N N N
N N N
Cluster
T1 T4T2 N
N N
Cluster
T3
K8S K8S
CC
1 NSXManager
CC
2
CC
3
CC
3NSXManager
CC
2
CC
1
AWS EU-RegionLW
DC
3
LW D
C3
N
Photon OS
https://github.com/vmware/lightwaveVMworld 2018 Content: Not for publication or distribution
32©2018 VMware, Inc.
Minimalist: The number of packages is limited to the minimum necessary for hosting containers.
Security-hardened Linux: The kernel is configured according to the recommendations of the Kernel Self-Protection Project (KSPP).
Curated packages and repositories: Packages are built with hardened security flags.
Advanced lifecycle management: There are timely security patches and updates to container packages, such as Docker and Kubernetes.
Project Lightwave integration: Lightwave clients are installed on Photon OS by default, which let it join to Lightwave domain and be managed by Lightwave consistently
Photon OSA Linux Container-Optimized Operating System
https://github.com/vmware/photonVMworld 2018 Content: Not for publication or distribution
33©2018 VMware, Inc.
Security Managed for You
Data encryption at rest (EBS volumes are encrypted; keys are managed by Amazon)
Data encryption in motion (TLS)
Encrypted K8s secrets in etcd
Data Encryption
Kubernetes
Automatic secure OS updates on Kubernetes nodes
Patches and Upgrades
VMworld 2018 Content: Not for publication or distribution
34©2018 VMware, Inc.
Each Organization is mapped to an AWS account managed by VMware Cloud PKS
Each production Smart Cluster is deployed on a separate Network segment
Infrastructure Isolation on AWS
VMware Cloud PKS
Acme Globex
user@Acme user@Globex
VPC VPC VPC
AWS cloud
AWS Account(vke.tenant.acme) AWS Account(vke.tenant.globex)
Smart Cluster Smart Cluster Smart Cluster
VMworld 2018 Content: Not for publication or distribution
35©2018 VMware, Inc.
Container Security at RuntimeAppDefense and Aqua Security
VMworld 2018 Content: Not for publication or distribution
36©2018 VMware, Inc.
Applications
Infrastructure
Orchestration Platform
Container Registry
Container Runtime
Container Host
Container Images
Container Management
Application Layer
VMworld 2018 Content: Not for publication or distribution
37©2018 VMware, Inc.
Opportunity to Focus on Core Protection StrategiesGartner Market Guide for Cloud Workload Protection Framework
AV
Deception
HIPS withVulnerability Shielding
Server Workload EDRBehavioral Monitoring
IaaS Data at Rest Encryption
Exploit Prevention / Memory Protection
Application Control / Whitelisting
System Integrity Monitoring / Management
Network Firewalling, Segmentation and Visibility
Hardening, Configuration and Vulnerability ManagementFoundational
Less Critical
Optional Protection Strategies
Core Protection Strategies
Important, but often provided outside of CWPP
Figure 1. Cloud Workload Protection Controls Hierarchy, © 2018 Gartner, Inc.
Source: Gartner, Market Guide for Cloud Workload Protection Platforms, Neil MacDonald, March 26th 2018. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document.
VMworld 2018 Content: Not for publication or distribution
38©2018 VMware, Inc.
Gartner Market Guide for Cloud Workload Protection Framework
AV
Deception
HIPS withVulnerability Shielding
Server Workload EDRBehavioral Monitoring
IaaS Data at Rest Encryption
Exploit Prevention / Memory Protection
Application Control / Whitelisting
System Integrity Monitoring / Management
Network Firewalling, Segmentation and Visibility
Hardening, Configuration and Vulnerability ManagementFoundational
Less Critical
Optional Protection Strategies
Core Protection Strategies
Important, but often provided outside of CWPP
Figure 1. Cloud Workload Protection Controls Hierarchy, © 2018 Gartner, Inc.
Source: Gartner, Market Guide for Cloud Workload Protection Platforms, Neil MacDonald, March 26th 2018. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document.
Post-deployment Controls
Opportunity to Focus on Core Protection Strategies
VMworld 2018 Content: Not for publication or distribution
39©2018 VMware, Inc.
Cyber ThreatsResidual Risk
Apps Data
Cyber HygieneAttack Surface
EncryptionMicro-Segmentation
RepavingLeastPrivilege
Multi-FactorAuthentication
VMworld 2018 Content: Not for publication or distribution
40
vSphere
AppDefense Manager
AppDefenseAppliance
vCenter
SaaS
Guest OS
Guest Module
Host Module
Customer Data Center
Guest OS
Guest Module
Guest OS
Guest Module
1. Application Context2. Central Intelligence
Architecture
ContainerContainerContainerContainerContainerContainer
ContainerContainerContainer
VMworld 2018 Content: Not for publication or distribution
41©2018 VMware, Inc.
1001010101001100101001010101010110100101010100110010100101010101011010010101010011001010010101010101101001010101001100101001010101010110100101010100110010100101010101011010010101010011001010010101010101101001010101001100101001010101010110100101010100110010100101010101011010010101010011001010010101010101101001010101001100101001010101010110100101010100110010100101010101011010010101010011001010010010101011010010101010011001010010101010101101001010101001100101001010101010110100101010100110010100101010101011010010101010011001010010101010101010010101010011001010010101010101101
Changing the Application Security ModelFrom chasing bad to ensuring good
Pro
cess
es
Pro
cess
es
Pro
cess
es
OSXX,000,000 XX
Chasing Bad Ensuring Good
VMworld 2018 Content: Not for publication or distribution
42©2018 VMware, Inc.
Ensuring Known Good
Detect Deviations RespondCapture & Analyze
Learn Intended State Protect Intended State
ManifestApp ManifestApp
VMworld 2018 Content: Not for publication or distribution
43©2018 VMware, Inc.
LearnCapture the purpose and intended state of a container
Intended State Engine
vCenter IaaS/PaaS
AppScope
Off-the-shelf images Custom imagesCI/CD pipeline
Machine Learning!
Learn Protect
Detect RespondCapture & Analyze
ManifestApp
ManifestApp
ManifestApp
VMworld 2018 Content: Not for publication or distribution
44©2018 VMware, Inc.
DetectRuntime application attestation and secure manifest store
Co
ntai
ner
Co
ntai
ner
Co
ntai
ner
OS
AppDefenseMonitor
Co
ntai
ner
Co
ntai
ner
Co
ntai
ner
OS
AppDefenseMonitor
Co
ntai
ner
Co
ntai
ner
Co
ntai
ner
OS
AppDefenseMonitor
Protected zone
Learn Protect
Detect RespondCapture & Analyze
ManifestApp ManifestAppManifestApp
VMworld 2018 Content: Not for publication or distribution
45©2018 VMware, Inc.
RespondOrchestrated incident response routines for the SOC
Secureinfrastructure
IntegratedEcosystem
Learn Protect
DetectRespond
Capture & Analyze
Add Behavior Block/Alarm
Quarantine Repave!
VMworld 2018 Content: Not for publication or distribution
Integrated Solution Architecture
46
ContainerContainerContainer
Hypervisor
AppDefense Manager
Aqua Manager
Aqua Agent
SaaS
Guest OS
Container Host
Integration Container
VMworld 2018 Content: Not for publication or distribution
47©2018 VMware, Inc.
Demo: AppDefense with Aqua
VMworld 2018 Content: Not for publication or distribution
48©2018 VMware, Inc.
Applications
Infrastructure
Orchestration Platform
Container Registry
Container Runtime
Container Host
Container Images
Container Management
Summary
VMworld 2018 Content: Not for publication or distribution
49©2018 VMware, Inc.
NIST Application Container Security Guide
NIST Security Assurance Requirements for Linux Application Container Deployments
Cloud-Native Stack Security: How VMware Pivotal Container Service Secures Containers and Kubernetes
Control Access with VMware Cloud PKS
Containers on Virtual Machines or Bare Metal? Deploying and Securely Managing Containerized Applications at Scale
Glossary of Cloud-Native Terms
Securing Cloud Platforms with Project Lightwave
Photon OS: A Linux Container-Optimized Operating System
A Dash of Security: Locking Down Kubernetes Admin Access
References and Other ResourcesGuides and White Papers
VMworld 2018 Content: Not for publication or distribution
50©2018 VMware, Inc.
Summary: Best PracticesImplement Container-Specific Countermeasures
Integrate countermeasures into life cycle and pipeline
Monitor containers thru life cycle & stack for full visibility
Enforce security with policies for RBAC and image use
Use only the latest patched, scanned, and signed images
Run images as non-privileged, immutable containers without SSH and manage through the orchestrator
Securely store secrets, encrypted, in the orchestrator, not the image
Connect to registries, dashboards over secure channels
Control access to registries, orchestrators, and dashboards with RBAC using principles of least privilege and separation of duties
Provide single sign-on with a single, standard directory
Log, monitor, and audit access to registry and orchestrator
Encrypt data at rest using container-specific methods; see NIST 800-111
Segment orchestrator traffic into discrete virtual networks by sensitivity level
Only mix workloads of the same sensitivity level and threat posture on the same host
Use a patched, up-to-date, CVE-monitored runtime
Constrain network access from containers
Profile and protect apps at runtime to ensure known good
Use an up-to-date, minimalist container OS to narrow the attack surface; see NIST SP 800-123
Set the root file system to read-only
Limit, log, and audit host OS access to detect anomalies and privileged operations
VMworld 2018 Content: Not for publication or distribution
DON’T FORGET TO FILL OUT YOUR SURVEY.
#vmworld #CNA1656BE
VMworld 2018 Content: Not for publication or distribution
THANK YOU!
#vmworld #CNA1656BE
VMworld 2018 Content: Not for publication or distribution