armv8 port of the jailhouse hypervisor · huawei technologies duesseldorf gmbh armv8 port of the...
TRANSCRIPT
Security Level:
Huawei Technologies Duesseldorf GmbH
ARMv8 port of the Jailhouse hypervisor
Antonios Motakis [email protected]
Version: V1.0(20160321)
Huawei Technologies Duesseldorf GmbH 2
Acknowledgements
Jan Kiszka, SIEMENS (Upstream project)
Jean-Philippe Brucker, ARM (ARM32 port)
Huawei ERC Munich team
Huawei Technologies Duesseldorf GmbH 3
Introduction
Why a new hypervisor?
Why Jailhouse on ARM64 / ARMv8?
Huawei Technologies Duesseldorf GmbH 4
Modern Virtualization
Hypervisor
A portable abstraction of a machine
VM
CPU CPU CPU CPU CPU CPU CPU CPU
vCPUv vCPU
Hypervisor
Huawei Technologies Duesseldorf GmbH 5
Modern Virtualization
Rich in features
CPU CPU CPU CPU
Hypervisor
CPU CPU CPU CPU
VM
vCPUv vCPU
Hypervisor
Huawei Technologies Duesseldorf GmbH 7
Jailhouse: the hypervisor for safety
Partitioning of hardware resources
Certifiable
Safe and secure
Simple
Less than 10k lines of codeCPU CPU CPU CPU
Cell Cell Cell
RTOS Bare metal
Huawei Technologies Duesseldorf GmbH 8
Jail… what?
Cell => guest in other hypervisors
Root cell => host in KVM, Dom0 in Xen
Inmate => guest software
Huawei Technologies Duesseldorf GmbH 9
Advantages
Real time
Safety and isolation
Low overheads (close to bare metal)
Huawei Technologies Duesseldorf GmbH 10
Safety critical applications
Industrial control
Mixed criticality
Automotive
Huawei Technologies Duesseldorf GmbH 11
Beyond safety critical systems
Linux based system + bare metal data plane
Secure
Predictable latencies
Low overhead
Scalability concerns in large multi-core systems
Huawei Technologies Duesseldorf GmbH 12
Beyond safety critical systems
Data plane / control plane
separation
Huawei Technologies Duesseldorf GmbH 13
Why Jailhouse on ARM64
64 bit instruction set for ARM
Core count keeps increasing
Huawei Technologies Duesseldorf GmbH 14
Partitioning a system 101
Our building blocks
Linux module
Jailhouse firmware
Root cell configuration
Inmate cells configuration
+ inmate binaries
Huawei Technologies Duesseldorf GmbH 15
Partitioning a system 101
Our building blocks
Linux module
Jailhouse firmware
Root cell configuration
Inmate cells configuration
+ inmate binaries
Load Jailhouse
Interface with Jailhouse
Huawei Technologies Duesseldorf GmbH 16
Partitioning a system 101
Our building blocks
Linux module
Jailhouse firmware
Root cell configuration
Inmate cells configuration
+ inmate binaries
Higher privilege level than Linux
All the interesting stuff
Huawei Technologies Duesseldorf GmbH 17
Partitioning a system 101
Our building blocks
Linux driver
Jailhouse firmware
Root cell configuration
Inmate cells configuration
+ inmate binaries
All hardware resources (initially)
assigned to the host system
Huawei Technologies Duesseldorf GmbH 18
Root cell configuration
.cell binary built from C
Hypervisor configuration
struct {
struct jailhouse_system header;
...
} __attribute__((packed)) config = {
.header = {
.signature = JAILHOUSE_SYSTEM_SIGNATURE,
.hypervisor_memory = {
.phys_start = 0x82fc000000,
.size = 0x4000000,
},
...
.root_cell = {
.name = "amd-seattle",
...
Huawei Technologies Duesseldorf GmbH 19
Root cell configuration
Hardware resources
.mem_regions = {
/* gpio */ {
.phys_start = 0xe0030000,
.virt_start = 0xe0030000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_IO,
},
/* gpio */ {
.phys_start = 0xe0080000,
.virt_start = 0xe0080000,
...
Huawei Technologies Duesseldorf GmbH 20
Root cell configuration
On x86 can be automatically generated
On ARM: write it yourself
Device tree information
/proc/iomem
Provided configurations
ARMv8 Foundation model (simulation)
A real hardware target
Huawei Technologies Duesseldorf GmbH 21
Partitioning a system 101
Our building blocks
Linux module
Jailhouse firmware
Root cell configuration
Inmate cells configuration
+ inmate binaries
Resources assigned to a cell
Provided examples
Binary built to be run from within a
cell
Huawei Technologies Duesseldorf GmbH 22
Video demo
CPU CPU CPU CPU
Root cell Cell Linux cell
Demoapp
CPUCPU CPU CPU
Huawei Technologies Duesseldorf GmbH 23
How to port a Hypervisor
…to ARM 64 bit processors
Hardware virtualization support
Portability of Jailhouse
Huawei Technologies Duesseldorf GmbH 24
How to port a Hypervisor
Currently supported
Any ARM 64 bit core with virtualization extensions
GICv2 interrupt controller
Huawei Technologies Duesseldorf GmbH 25
ARM64 Privilege Levels
EL0 UserApplications
EL1 KernelRoot cell (Linux)
Inmate cells
EL2 HypervisorJailhouse
EL3 Monitor
Huawei Technologies Duesseldorf GmbH 28
Hypervisor initialization
static int jailhouse_cmd_enable(struct jailhouse_system __user *arg)
{
...
on_each_cpu(enter_hypervisor, header, 0);
/* Entry point for Linux loader module on JAILHOUSE_ENABLE */
.text
.globl arch_entry
arch_entry:
...
hypervisor/arch/arm64/entry.S
Huawei Technologies Duesseldorf GmbH 29
Hypervisor initialization
/* Entry point for Linux loader module on JAILHOUSE_ENABLE */
.text
.globl arch_entry
arch_entry:
...
hypervisor/arch/arm64/entry.S
Initialize a stack
Call generic entry() function
Huawei Technologies Duesseldorf GmbH 30
Initialization overview
hypervisor
setup.c
control.c
paging.c
…
arm
paging.c
…
arm64
entry.S
setup.c
control.c
…
arch_entry and friends
Huawei Technologies Duesseldorf GmbH 31
Initialization overview
hypervisor
setup.c
control.c
paging.c
…
arm
paging.c
…
arm64
entry.S
setup.c
control.c
…
entry
init_early
Huawei Technologies Duesseldorf GmbH 32
Initialization overview
hypervisor
setup.c
control.c
paging.c
…
arm
paging.c
…
arm64
entry.S
setup.c
control.c
…
Page table generating
code
page_alloc and friends
for the hypervisor
Shared infrastructure
with ARM32
Huawei Technologies Duesseldorf GmbH 33
Hypervisor initialization
One challenge!
Jailhouse is a
statically linked
binary!
arch_entry• Entry in
Linux context!
entry• …still in
Linux context
early_init
paging_init• Init
page tables
Huawei Technologies Duesseldorf GmbH 34
Jailhouse entry (x86, ARM32)Linux VA
address space
Kernel VA
range
JailhouseJAILHOUSE_BASE
Jailhouse VA address space
Jailhouse
0x0
Kernel base1. kernel driver
loads Jailhouse
firmware
2. Jailhouse
initialization
starts in Linux
VA space
3. Jailhouse
switches to own VA
space during init
0x0
JAILHOUSE_BASE
Huawei Technologies Duesseldorf GmbH 35
…Jailhouse piggybacks on the Linux page tables during
initialization!!!
Huawei Technologies Duesseldorf GmbH 36
Jailhouse entry on ARM64?Linux VA
address space
Kernel VA
range
JailhouseJAILHOUSE_BASE
Jailhouse VA address space
Jailhouse
0x0(TTBR0_EL1)
Kernel base0xffff000000000000
(TTBR1_EL1) 1. kernel driver
loads Jailhouse
firmware
2. Jailhouse
initialization
starts in Linux
VA space3. Ooops! The
same VA range is
not mapable in EL2
0x0(TTBR0_EL2)
JAILHOUSE_BASE
Huawei Technologies Duesseldorf GmbH 37
Main challenge summary
Jailhouse on other platforms
Linux loads Jailhouse at JAILHOUSE_BASE
Jailhouse linked to run JAILHOUSE_BASE
Early init relies on this; shared MMU context
Jailhouse on ARM64
Linux loads Jailhouse anywhere
Jailhouse linked to run from JAILHOUSE_BASE (!= anywhere)
Huawei Technologies Duesseldorf GmbH 38
Possible solutions
Start with the MMU off
No unaligned memory accesses
Caches are being bypassed
Position independent binary
Need a linker during arch_entry…
Generate early bootstrap page tables
Huawei Technologies Duesseldorf GmbH 39
Solutions
Start with the MMU off
No unaligned memory accesses
Caches are being bypassed
Position independent binary
Need a linker during arch_entry…
Generate early bootstrap page tables
Huawei Technologies Duesseldorf GmbH 40
Initialization (finally)
hypervisor
setup.c
control.c
paging.c
…
arm
paging.c
…
arm64
entry.S
setup.c
control.c
…
Initialize the hypervisor
Restore the host, as a
root cell
Huawei Technologies Duesseldorf GmbH 41
Lifetime control of Jailhouse
hypervisor
setup.c
control.c
paging.c
…
arm
paging.c
…
arm64
entry.S
setup.c
control.c
…
Receive hypercalls from
the root cell
Create, destroy cells
Huawei Technologies Duesseldorf GmbH 42
Shared with ARM32 port
MMU / page tables generation
Extended for up to 48 bit address space, 4 level page tables
Huawei Technologies Duesseldorf GmbH 43
Shared with ARM32 port
GIC (ARM Generic Interrupt Controller) handling
Huawei Technologies Duesseldorf GmbH 44
Shared with ARM32 port
PSCI implementation (SMP support)
Low level PSCI operations in assembly
Huawei Technologies Duesseldorf GmbH 45
Jailhouse cell demos
Bare metal demo applications using the GIC, timers, and UART
Ported from ARM32
Huawei Technologies Duesseldorf GmbH 46
Shared components with ARM32
Extended for AArch64 support• 48 bit support• 4 level page tables• 64 bit PSCI operations
Shared drivers• GICv2• PL011 UART
Huawei Technologies Duesseldorf GmbH 47
Linux as an inmate
Allows for fully working Linux inmate, alongside the root cell
On the floor demo we can demonstrate a Linux inmate using
the second NIC of the platform
Huawei Technologies Duesseldorf GmbH 48
Floor demo
Only one UART port
No SMMU support (yet)
For demonstration, assign the second NIC using identity mapping
However, we lose the security guarantees offered by the SMMU
Huawei Technologies Duesseldorf GmbH 49
Conclusion
ARM64 processor core count increasing
Ideal for safety & real time
Addresses scalability concerns
Upstreaming in progress
https://github.com/siemens/jailhouse branch wip/arm64
Huawei Technologies Duesseldorf GmbH 50
Future directions
Upstreaming of the ARM64 port
More hardware support
GICv3
Targets with uncommon interrupt controller or SMMU
ARM SMMU support
Test and benchmark under more real world scenarios
Copyright©2014 Huawei Technologies Duesseldorf GmbH. All Rights Reserved.
The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating
results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither
an offer nor an acceptance. Huawei may change the information at any time without notice.