A tour of F9 microkernel and BitSec hypervisor

Your NameLine 1 of Your AffiliationLine 2 of Your Affiliation

A tour of F9 microkernel and BitSec hypervisor

Louie Lu ()

11/24 @ ARRC Tainan

Content

From 2nd gen to 3nd gen microkenrel

F9 microkernel feature

BitSec: 3rd generation IoT Hypervisor

From 2nd Gen to 3rd Gen microkernel


2nd Generation microkernelL3Qualitative improvement of IPC performance

22 times faster then Mach

1993 [Jochen Liedkte]

L4 familystart from 1995, V2 ABI about L3






Fiasco, the first high level language (C++) and open source L4 microkernel

L4Ka::Pistachio,

OKL4, from OKLabs






Fiasco, the first high level language (C++) and open source L4 microkernel

L4Ka::Pistachio,

OKL4, from OKLabs


2nd Generation microkernelIssues about Ad-hocGlobal thread name space => covert channel

Threads as IPC target => insufficient encapsulation

Single kernel memory pool => DOS attack

Insufficient delegation of authority


3rd Generation microkernel, focus on...Capability Model

Less system calls (seL4 only have 3 syscalls)

Clear semantics (Async EndPoint ...etc)

Formal Verification


Capability ModelCnode, Caps in seL4

fine-graint permission control



Formal Verification


Capability ModelCnode, Caps in seL4



Formal VerificationseL4 [SOSO'09]

Verve [PLDI'10]

FSCQ [SOSP'15]

mCertiKOS ['POPL'15]

CoGENT [ASPLOS'16]


Formal VerificationseL4 [SOSP'09]

mCertiKOS [POPL'15]

verified sequential kernels


Formal VerificationIroncald [OSDI'14]

verified software stack


Formal VerificationFSCQ [SOSP'15]

CoGENT [ASPLOS'16]

verified sequential file systems


Formal Verification Concurrent Memory kernel ?



from certikos-osdi16



from certikos-osdi16


Formal Verification Concurrent Memory kernel ?CertiKOS [OSDI'16]

Solve the problemglobal changes

I/O concurrency

fine-grained lock

liveness

compiler

cost...


The still unsolved problem (From L3 to seL4 20y)Principled, policy-free control of CPU time.

Handling of multicore processors in the age of verification.

F9 microkernel

F9 microkernel

Memory Management

Debugging Tools

The Bug

Unsolved problems

F9 microkernel

Memory ManagementMPU (Memory Protection Unit)

Fpage

Address Space

F9 microkernel - MPU

Protected Memory System Architecture (PMSAv7)ARMv7-M supports PMSAv7

MPUSupport 8 memory region for setting

Provide access premission control

Provide memory execute setting


MPU RegisterMPU_TYPE - Type Register

MPU_CTRL - Control Regiser

MPU_RNR- Region Number Register

MPU_RBAR - Region Base Address Register

MPU_RASR- Region Attribute and Size Register


MPU Region Number RegisterSelect the region to read or write

affect with RBAR, RASR


MPU Region Base Address RegisterSetting base address of the region

Setting region is valid or not

Setting which region to writesSo you may just change this value without RNR


MPU Region Attribute and Size Register


MPU region sizeCan only set region sizewith 2 ** n


MPU AttributeeXecute Never

access permissions (user (unprivileged) and privileged)

cache policy

device type and cacheability




cache policy





cache policy



MPU mpu_setup_region

F9 microkernel - Fpage

Fpagefundamental element for address space





assign_fpage

map_fpage

unmap_fpage

split_fpage



assign_fpage

map_fpage

unmap_fpage

split_fpage



assign_fpage

map_fpage

unmap_fpage

split_fpage



assign_fpage

map_fpage

unmap_fpage

split_fpage

F9 microkernel Address Space

Address Space (as)Construct by fpage list

Contain one thread's fpage information

5 API

F9 microkenrel Address Space

Address Space APIas_t *as_create(uint32_t as_spaceid);

void as_destroy(as_t *as);

void as_setup_mpu(as_t *as, memptr_t sp, memptr_t pc, memptr_t stack_base, size_t stack_size);

void as_map_user(as_t *as);

void as_map_ktext(as_t *as);



call by:



code:


Address Space APIvoid as_map_user(as_t *as);Map user text, data and hardware device memory

void as_map_ktext(as_t *as);Map kernel text, data


Address Space APIvoid as_setup_mpu(as_t *as, memptr_t sp, memptr_t pc, memptr_t stack_base, size_t stack_size);


Address Space APIas_setup_mpuprotect stack fpages first

thenPC

always-mapped fpages

other

F9 microkernel Memory management

System calls are provided to manage address spaces:Grant: The memory page is granted to a new user and cannot be used anymore by its former user.

Map: This implements shared memory the memory page is passed to another task but can be used by both tasks.

Flush: The memory page that has been mapped to other users will be flushed out of their address space.







Map: This implements shared memory the memory page is passed to another task but can be used by both tasks. (as_map_user/ktext)





(unmap_fpage)


Recursive Address Space:Initial address space controlled by first process.Controls all available memory

Other address spaces empty at boot

In F9 microkernel, root thread was the init address space holder





Other processes (threads) obtain memory pages from first or from their other processes that got page from parent











F9 microkernel

Debugging Toolskdb

kproble

openocd + gdb

F9 microkernel Debugging Tools

KDBIn-kernel debugger is built and run at boot by default

support multiple useful command

easy to adding new command


KDBIn-kernel debugger is built and run at boot by default

support multiple useful command


kprobeLinux-style kprobe

support banding by symbol name


openocd + gdbPowerful then st-util

useful command:http://openocd.org/doc/html/General-Commands.html


openocd + gdbPowerful then st-util

useful command:http://openocd.org/doc/html/General-Commands.html


openocd + gdbuseful command:http://openocd.org/doc/html/General-Commands.html

memory accessmdw / mdh / mdb Display contents of address addr

mww / mwh /mwb Writes the specified value at addr


openocd + gdbuseful command:http://openocd.org/doc/html/General-Commands.html

memory accessmdw / mdh / mdb Display contents of address addr

mww / mwh /mwb Writes the specified value at addr

access register valuereg dump all register value

reg [number | name] [value] just OP

F9 microkernel The Bug

The BugReally matter one



Adding new feature for l4test (a test suite from pisachio)

IPC testing via Local destination Id














The Bug - Sender


The Bug - Reciver


The Bug


The Bug

F9 microkernel Unsolved problems

Unsolved problemsTime-slice scheduleUser space busy loop hang system

POSIX layer support

BitSec

BitSec

BitSec is......BitSec is a microkernel intended for high-reliability embedded applications on ARMv7-M architecture processors, such as the Cortex-M4.

BitSec is a third-generation microkernel. Its design is heavily inspired by EROS/KeyKOS, MINIX 3, and the L4 family, particularly seL4.

BitSec

BitSec, like other third-generation microkernelFocuses on minimality and security,

Expresses all authority through explicit capabilities,

Moves other mechanisms with security implications outside the kernel,

Blurs the line between a traditional microkernel and a hypervisor, and

Targets a very small kernel codebase (in BitSecs case, less than 2500 sloccount lines of code).

BitSec

BitSec, like other third-generation microkernelFocuses on minimality and security,

Expresses all authority through explicit capabilities,

Moves other mechanisms with security implications outside the kernel,

Blurs the line between a traditional microkernel and a hypervisor, and

Targets a very small kernel codebase (in BitSecs case, less than 2500 sloccount lines of code).

BitSec

BitSec is not...A complete operating system

BitSec


By analogy: putting the Linux kernel on a computer doesnt allow one to run Firefox. Linux is only a kernel; theres a whole lot of operating system that must be added to make a complete system.

This is even more true of BitSec than of Linux, because BitSecs design is so minimal. Like other third-generation microkernels, BitSec doesnt even include hardware drivers in the kernel.

BitSec


BitSec


By analogy: putting the Linux kernel on a computer doesnt allow one to run Firefox. Linux is only a kernel; theres a whole lot of operating system that must be added to make a complete system.

This is even more true of BitSec than of Linux, because BitSecs design is so minimal. Like other third-generation microkernels, BitSec doesnt even include hardware drivers in the kernel.

BitSec - Key

BitSec capability: KeyConceptually, Key combinesA reference to a kernel object

A set of operations on that object that the key enables

BitSec - Key

BitSec capability: KeyImplementation, a key is a kernel data structure built from three partA brand, which distinguishes kines of keys to particular object from one another

A generation, which distinguishes keys to successive objects, created in the same location, from one another.

A pointer to a kernel object.

BitSec - Key

BitSec capability: Keystruct

BitSec - Key

BitSec capability: Keystruct

keys can be freely copied but not directly created

mint the key from object table

BitSec - Key

BitSec capability: KeyBrands64 bit value chosen when the key was create (minted)

indelible, cannot be alter except by minting a new key

program cannot discover the brand of keys they hold

when a key is copied, the copies bear the same brand

Brands allow two keys to the same object to behave differently. The meaning of a brand is entirely up to the object designated by the key. Kernel objects specify the meaning of brands so that the System can mint appropriate keys,but transparent objects (particularly Gates) pass the brand up to programs for interpretation. It may encode access permissions, unique client identifiers, etc.

BitSec - Key

BitSec capability: Keykeys can be freely copied but not directly created


program can only perform three operations on a keyCopy the key into a different key register

Send a message to the object designated by the key

Receive a message from the object designated by the key

BitSec - Key



program can not do following without holding addi authDetermine the kind of kernel object designated by a key

Determine if two keys designate the same kernel object

Determine if two keys give the same rights on their designated objects

BitSec - Key



program can not do following without holding addi authDetermine the kind of kernel object designated by a key

Determine if two keys designate the same kernel object

Determine if two keys give the same rights on their designated objects

BitSec - Key

BitSec capability: KeyGenerations and Revocationused to provide efficient revocation

When program invokes a keykernel compares the key's generation to a object generation

If two match, proceeds normally

If not match, object has been invalidated

BitSec - Key

BitSec capability: KeyGenerations and Revocationused to provide efficient revocation

When program invokes a keykernel compares the key's generation to a object generation

If two match, proceeds normally

If not match, object has been invalidated

BitSec Object Oriented

BitSec design: Object Oriented Using c++11 standard

Every resource and service that the system can see in the kernel is represented as an objectbundling together state and operations on that state.




For example: the Keys


BitSec design: Object Oriented Every resource and service that the system can see in the kernel is represented as an objectbundling together state and operations on that state.







BitSec design: Object OrientedUsing c++11 standard



And can be usable from C


BitSec design: Object OrientedAnd can be usable from C

BitSec is implemented in C++11, but was designed to be usable from legacy languages suchas C and assembler.

As FreeRTOS/BitSec is entirely C (and C90 at that), this seems to have worked.

BitSec Future work

BitSec future workEnhancement of the build systems

porting guest OS to BitSec
(FreeRTOS, Linux, or maybe from stratch)

Formal Verify by CoGENT

Adding network stack ?

F9 / BitSec

Demo
on stm32f429


Q&A


Thanks

References

CertiKOS: An Extensible Architecture for Building Certified Concurrent OS Kernels
http://flint.cs.yale.edu/certikos/publications/certikos-osdi16.pdf

F9 microkernel & ARM hypervisor
https://hackmd.io/s/r16AUMbJe

F9: A secure microkernel for MMUless embedded systems
https://github.com/grapherd/f9-report

F9: A secure and Efficient Microkernel Built for Deeply Embedded Systems
http://www.slideshare.net/jserv/f9-microkernel

Technical Report on the BitSec Kernel

OpenOCD general command
http://openocd.org/doc/html/General-Commands.html

Technology

A tour of F9 microkernel and BitSec hypervisor