Your NameLine 1 of Your AffiliationLine 2 of Your Affiliation
A tour of F9 microkernel and BitSec hypervisor
Louie Lu ()
11/24 @ ARRC Tainan
Content
From 2nd gen to 3nd gen microkenrel
F9 microkernel feature
BitSec: 3rd generation IoT Hypervisor
From 2nd Gen to 3rd Gen microkernel
From 2nd gen to 3nd gen microkenrel
2nd Generation microkernelL3Qualitative improvement of IPC performance
22 times faster then Mach
1993 [Jochen Liedkte]
L4 familystart from 1995, V2 ABI about L3
From 2nd gen to 3nd gen microkenrel
2nd Generation microkernelL3Qualitative improvement of IPC performance
22 times faster then Mach
1993 [Jochen Liedkte]
L4 familystart from 1995, V2 ABI about L3
Fiasco, the first high level language (C++) and open source L4 microkernel
L4Ka::Pistachio,
OKL4, from OKLabs
From 2nd gen to 3nd gen microkenrel
2nd Generation microkernelL3Qualitative improvement of IPC performance
22 times faster then Mach
1993 [Jochen Liedkte]
L4 familystart from 1995, V2 ABI about L3
Fiasco, the first high level language (C++) and open source L4 microkernel
L4Ka::Pistachio,
OKL4, from OKLabs
From 2nd gen to 3nd gen microkenrel
2nd Generation microkernelIssues about Ad-hocGlobal thread name space => covert channel
Threads as IPC target => insufficient encapsulation
Single kernel memory pool => DOS attack
Insufficient delegation of authority
From 2nd gen to 3nd gen microkenrel
3rd Generation microkernel, focus on...Capability Model
Less system calls (seL4 only have 3 syscalls)
Clear semantics (Async EndPoint ...etc)
Formal Verification
From 2nd gen to 3nd gen microkenrel
Capability ModelCnode, Caps in seL4
fine-graint permission control
Less system calls (seL4 only have 3 syscalls)
Clear semantics (Async EndPoint ...etc)
Formal Verification
From 2nd gen to 3nd gen microkenrel
Capability ModelCnode, Caps in seL4
Less system calls (seL4 only have 3 syscalls)
Clear semantics (Async EndPoint ...etc)
Formal VerificationseL4 [SOSO'09]
Verve [PLDI'10]
FSCQ [SOSP'15]
mCertiKOS ['POPL'15]
CoGENT [ASPLOS'16]
From 2nd gen to 3nd gen microkenrel
Formal VerificationseL4 [SOSP'09]
mCertiKOS [POPL'15]
verified sequential kernels
From 2nd gen to 3nd gen microkenrel
Formal VerificationIroncald [OSDI'14]
verified software stack
From 2nd gen to 3nd gen microkenrel
Formal VerificationFSCQ [SOSP'15]
CoGENT [ASPLOS'16]
verified sequential file systems
From 2nd gen to 3nd gen microkenrel
Formal Verification Concurrent Memory kernel ?
From 2nd gen to 3nd gen microkenrel
Formal Verification Concurrent Memory kernel ?
from certikos-osdi16
From 2nd gen to 3nd gen microkenrel
Formal Verification Concurrent Memory kernel ?
from certikos-osdi16
From 2nd gen to 3nd gen microkenrel
Formal Verification Concurrent Memory kernel ?CertiKOS [OSDI'16]
Solve the problemglobal changes
I/O concurrency
fine-grained lock
liveness
compiler
cost...
From 2nd gen to 3nd gen microkenrel
The still unsolved problem (From L3 to seL4 20y)Principled, policy-free control of CPU time.
Handling of multicore processors in the age of verification.
F9 microkernel
F9 microkernel
Memory Management
Debugging Tools
The Bug
Unsolved problems
F9 microkernel
Memory ManagementMPU (Memory Protection Unit)
Fpage
Address Space
F9 microkernel - MPU
Protected Memory System Architecture (PMSAv7)ARMv7-M supports PMSAv7
MPUSupport 8 memory region for setting
Provide access premission control
Provide memory execute setting
F9 microkernel - MPU
MPU RegisterMPU_TYPE - Type Register
MPU_CTRL - Control Regiser
MPU_RNR- Region Number Register
MPU_RBAR - Region Base Address Register
MPU_RASR- Region Attribute and Size Register
F9 microkernel - MPU
MPU Region Number RegisterSelect the region to read or write
affect with RBAR, RASR
F9 microkernel - MPU
MPU Region Base Address RegisterSetting base address of the region
Setting region is valid or not
Setting which region to writesSo you may just change this value without RNR
F9 microkernel - MPU
MPU Region Attribute and Size Register
F9 microkernel - MPU
MPU region sizeCan only set region sizewith 2 ** n
F9 microkernel - MPU
MPU AttributeeXecute Never
access permissions (user (unprivileged) and privileged)
cache policy
device type and cacheability
F9 microkernel - MPU
MPU AttributeeXecute Never
access permissions (user (unprivileged) and privileged)
cache policy
device type and cacheability
F9 microkernel - MPU
MPU AttributeeXecute Never
access permissions (user (unprivileged) and privileged)
cache policy
device type and cacheability
F9 microkernel - MPU
MPU mpu_setup_region
F9 microkernel - Fpage
Fpagefundamental element for address space
F9 microkernel - Fpage
Fpagefundamental element for address space
F9 microkernel - Fpage
Fpagefundamental element for address space
assign_fpage
map_fpage
unmap_fpage
split_fpage
F9 microkernel - Fpage
Fpagefundamental element for address space
assign_fpage
map_fpage
unmap_fpage
split_fpage
F9 microkernel - Fpage
Fpagefundamental element for address space
assign_fpage
map_fpage
unmap_fpage
split_fpage
F9 microkernel - Fpage
Fpagefundamental element for address space
assign_fpage
map_fpage
unmap_fpage
split_fpage
F9 microkernel Address Space
Address Space (as)Construct by fpage list
Contain one thread's fpage information
5 API
F9 microkenrel Address Space
Address Space APIas_t *as_create(uint32_t as_spaceid);
void as_destroy(as_t *as);
void as_setup_mpu(as_t *as, memptr_t sp, memptr_t pc, memptr_t stack_base, size_t stack_size);
void as_map_user(as_t *as);
void as_map_ktext(as_t *as);
F9 microkenrel Address Space
Address Space APIas_t *as_create(uint32_t as_spaceid);
call by:
F9 microkenrel Address Space
Address Space APIas_t *as_create(uint32_t as_spaceid);
code:
F9 microkenrel Address Space
Address Space APIvoid as_map_user(as_t *as);Map user text, data and hardware device memory
void as_map_ktext(as_t *as);Map kernel text, data
F9 microkenrel Address Space
Address Space APIvoid as_setup_mpu(as_t *as, memptr_t sp, memptr_t pc, memptr_t stack_base, size_t stack_size);
F9 microkenrel Address Space
Address Space APIas_setup_mpuprotect stack fpages first
thenPC
always-mapped fpages
other
F9 microkernel Memory management
System calls are provided to manage address spaces:Grant: The memory page is granted to a new user and cannot be used anymore by its former user.
Map: This implements shared memory the memory page is passed to another task but can be used by both tasks.
Flush: The memory page that has been mapped to other users will be flushed out of their address space.
F9 microkernel Memory management
System calls are provided to manage address spaces:Grant: The memory page is granted to a new user and cannot be used anymore by its former user.
Map: This implements shared memory the memory page is passed to another task but can be used by both tasks.
Flush: The memory page that has been mapped to other users will be flushed out of their address space.
F9 microkernel Memory management
System calls are provided to manage address spaces:Grant: The memory page is granted to a new user and cannot be used anymore by its former user.
Map: This implements shared memory the memory page is passed to another task but can be used by both tasks. (as_map_user/ktext)
Flush: The memory page that has been mapped to other users will be flushed out of their address space.
F9 microkernel Memory management
System calls are provided to manage address spaces:Grant: The memory page is granted to a new user and cannot be used anymore by its former user.
Map: This implements shared memory the memory page is passed to another task but can be used by both tasks.
Flush: The memory page that has been mapped to other users will
be flushed out of their address space.
(unmap_fpage)
F9 microkernel Memory management
Recursive Address Space:Initial address space controlled by first process.Controls all available memory
Other address spaces empty at boot
In F9 microkernel, root thread was the init address space holder
F9 microkernel Memory management
Recursive Address Space:Initial address space controlled by first process.Controls all available memory
Other address spaces empty at boot
In F9 microkernel, root thread was the init address space holder
Other processes (threads) obtain memory pages from first or from their other processes that got page from parent
F9 microkernel Memory management
Recursive Address Space:Initial address space controlled by first process.Controls all available memory
Other address spaces empty at boot
In F9 microkernel, root thread was the init address space holder
Other processes (threads) obtain memory pages from first or from their other processes that got page from parent
F9 microkernel Memory management
Recursive Address Space:Initial address space controlled by first process.Controls all available memory
Other address spaces empty at boot
In F9 microkernel, root thread was the init address space holder
Other processes (threads) obtain memory pages from first or from their other processes that got page from parent
F9 microkernel
Debugging Toolskdb
kproble
openocd + gdb
F9 microkernel Debugging Tools
KDBIn-kernel debugger is built and run at boot by default
support multiple useful command
easy to adding new command
F9 microkernel Debugging Tools
KDBIn-kernel debugger is built and run at boot by default
support multiple useful command
F9 microkernel Debugging Tools
kprobeLinux-style kprobe
support banding by symbol name
F9 microkernel Debugging Tools
openocd + gdbPowerful then st-util
useful command:http://openocd.org/doc/html/General-Commands.html
F9 microkernel Debugging Tools
openocd + gdbPowerful then st-util
useful command:http://openocd.org/doc/html/General-Commands.html
F9 microkernel Debugging Tools
openocd + gdbuseful command:http://openocd.org/doc/html/General-Commands.html
memory accessmdw / mdh / mdb Display contents of address addr
mww / mwh /mwb Writes the specified value at addr
F9 microkernel Debugging Tools
openocd + gdbuseful command:http://openocd.org/doc/html/General-Commands.html
memory accessmdw / mdh / mdb Display contents of address addr
mww / mwh /mwb Writes the specified value at addr
access register valuereg dump all register value
reg [number | name] [value] just OP
F9 microkernel The Bug
The BugReally matter one
F9 microkernel The Bug
The BugReally matter one
Adding new feature for l4test (a test suite from pisachio)
IPC testing via Local destination Id
F9 microkernel The Bug
The BugReally matter one
Adding new feature for l4test (a test suite from pisachio)
IPC testing via Local destination Id
F9 microkernel The Bug
The BugReally matter one
Adding new feature for l4test (a test suite from pisachio)
IPC testing via Local destination Id
F9 microkernel The Bug
The BugReally matter one
Adding new feature for l4test (a test suite from pisachio)
IPC testing via Local destination Id
F9 microkernel The Bug
The Bug - Sender
F9 microkernel The Bug
The Bug - Reciver
F9 microkernel The Bug
The Bug
F9 microkernel The Bug
The Bug
F9 microkernel Unsolved problems
Unsolved problemsTime-slice scheduleUser space busy loop hang system
POSIX layer support
BitSec
BitSec
BitSec is......BitSec is a microkernel intended for high-reliability embedded applications on ARMv7-M architecture processors, such as the Cortex-M4.
BitSec is a third-generation microkernel. Its design is heavily inspired by EROS/KeyKOS, MINIX 3, and the L4 family, particularly seL4.
BitSec
BitSec, like other third-generation microkernelFocuses on minimality and security,
Expresses all authority through explicit capabilities,
Moves other mechanisms with security implications outside the kernel,
Blurs the line between a traditional microkernel and a hypervisor, and
Targets a very small kernel codebase (in BitSecs case, less than 2500 sloccount lines of code).
BitSec
BitSec, like other third-generation microkernelFocuses on minimality and security,
Expresses all authority through explicit capabilities,
Moves other mechanisms with security implications outside the kernel,
Blurs the line between a traditional microkernel and a hypervisor, and
Targets a very small kernel codebase (in BitSecs case, less than 2500 sloccount lines of code).
BitSec
BitSec is not...A complete operating system
BitSec
BitSec is not...A complete operating system
By analogy: putting the Linux kernel on a computer doesnt allow one to run Firefox. Linux is only a kernel; theres a whole lot of operating system that must be added to make a complete system.
This is even more true of BitSec than of Linux, because BitSecs design is so minimal. Like other third-generation microkernels, BitSec doesnt even include hardware drivers in the kernel.
BitSec
BitSec is not...A complete operating system
BitSec
BitSec is not...A complete operating system
By analogy: putting the Linux kernel on a computer doesnt allow one to run Firefox. Linux is only a kernel; theres a whole lot of operating system that must be added to make a complete system.
This is even more true of BitSec than of Linux, because BitSecs design is so minimal. Like other third-generation microkernels, BitSec doesnt even include hardware drivers in the kernel.
BitSec - Key
BitSec capability: KeyConceptually, Key combinesA reference to a kernel object
A set of operations on that object that the key enables
BitSec - Key
BitSec capability: KeyImplementation, a key is a kernel data structure built from three partA brand, which distinguishes kines of keys to particular object from one another
A generation, which distinguishes keys to successive objects, created in the same location, from one another.
A pointer to a kernel object.
BitSec - Key
BitSec capability: Keystruct
BitSec - Key
BitSec capability: Keystruct
keys can be freely copied but not directly created
mint the key from object table
BitSec - Key
BitSec capability: KeyBrands64 bit value chosen when the key was create (minted)
indelible, cannot be alter except by minting a new key
program cannot discover the brand of keys they hold
when a key is copied, the copies bear the same brand
Brands allow two keys to the same object to behave differently. The meaning of a brand is entirely up to the object designated by the key. Kernel objects specify the meaning of brands so that the System can mint appropriate keys,but transparent objects (particularly Gates) pass the brand up to programs for interpretation. It may encode access permissions, unique client identifiers, etc.
BitSec - Key
BitSec capability: Keykeys can be freely copied but not directly created
mint the key from object table
program can only perform three operations on a keyCopy the key into a different key register
Send a message to the object designated by the key
Receive a message from the object designated by the key
BitSec - Key
BitSec capability: Keykeys can be freely copied but not directly created
mint the key from object table
program can not do following without holding addi authDetermine the kind of kernel object designated by a key
Determine if two keys designate the same kernel object
Determine if two keys give the same rights on their designated objects
BitSec - Key
BitSec capability: Keykeys can be freely copied but not directly created
mint the key from object table
program can not do following without holding addi authDetermine the kind of kernel object designated by a key
Determine if two keys designate the same kernel object
Determine if two keys give the same rights on their designated objects
BitSec - Key
BitSec capability: KeyGenerations and Revocationused to provide efficient revocation
When program invokes a keykernel compares the key's generation to a object generation
If two match, proceeds normally
If not match, object has been invalidated
BitSec - Key
BitSec capability: KeyGenerations and Revocationused to provide efficient revocation
When program invokes a keykernel compares the key's generation to a object generation
If two match, proceeds normally
If not match, object has been invalidated
BitSec Object Oriented
BitSec design: Object Oriented Using c++11 standard
Every resource and service that the system can see in the kernel is represented as an objectbundling together state and operations on that state.
BitSec Object Oriented
BitSec design: Object Oriented Using c++11 standard
Every resource and service that the system can see in the kernel is represented as an objectbundling together state and operations on that state.
For example: the Keys
BitSec Object Oriented
BitSec design: Object Oriented Every resource and service that the system can see in the kernel is represented as an objectbundling together state and operations on that state.
For example: the Keys
BitSec Object Oriented
BitSec design: Object Oriented Using c++11 standard
Every resource and service that the system can see in the kernel is represented as an objectbundling together state and operations on that state.
For example: the Keys
BitSec Object Oriented
BitSec design: Object OrientedUsing c++11 standard
Every resource and service that the system can see in the kernel is represented as an objectbundling together state and operations on that state.
For example: the Keys
And can be usable from C
BitSec Object Oriented
BitSec design: Object OrientedAnd can be usable from C
BitSec is implemented in C++11, but was designed to be usable from legacy languages suchas C and assembler.
As FreeRTOS/BitSec is entirely C (and C90 at that), this seems to have worked.
BitSec Future work
BitSec future workEnhancement of the build systems
porting guest OS to BitSec
(FreeRTOS, Linux, or maybe from stratch)
Formal Verify by CoGENT
Adding network stack ?
F9 / BitSec
Demo
on stm32f429
A tour of F9 microkernel and BitSec hypervisor
Q&A
A tour of F9 microkernel and BitSec hypervisor
Thanks
References
CertiKOS: An Extensible Architecture for Building Certified
Concurrent OS Kernels
http://flint.cs.yale.edu/certikos/publications/certikos-osdi16.pdf
F9 microkernel & ARM hypervisor
https://hackmd.io/s/r16AUMbJe
F9: A secure microkernel for MMUless embedded systems
https://github.com/grapherd/f9-report
F9: A secure and Efficient Microkernel Built for Deeply Embedded
Systems
http://www.slideshare.net/jserv/f9-microkernel
Technical Report on the BitSec Kernel
OpenOCD general command
http://openocd.org/doc/html/General-Commands.html