Using New Trusted Pools

Capability in Folsom Release

Gang Wei

2

Agenda

Trusted Pools

• Concept

• Implementation & Usage

Trusted Launch with Trusted Boot (Tboot)

Remote Attestation with OpenAttestation (OAT)

More on Trusted Pools

• Patches

• Deployment & Configuration

Summary

3

Trusted Pools - Concept

Trusted Pools is also called

• Trusted Computing Pools (TCP)

Trusted Pools relies on:

• Trusted Launch

• Remote Attestation Internet

Compliance Hardware support for compliance reporting enhances auditability of cloud environment

Trusted Launch Verified platform integrity reduces malware threat

Trusted Pools Control VMs based on platform trust to better protect data

4

Trusted Pools - Implementation

Attestation

Service

Scheduler

EC

2 A

PI

OS

AP

I

Query API

User specifies :: Mem > 2G Disk > 50G GPGPU=Intel trusted_host=trusted HW/TXT

Hypervisor / tboot

OS

App App

App

OS

App App

App Host agent

Attestation Server

Privacy CA

Appraiser

Whitelist DB

Whitelist API

Ho

st Ag

en

t AP

I

Qu

ery

AP

I

OpenStack

TrustedFilter Create

Atte

st

Rep

ort

Qu

ery

tru

ste

d/

u

ntr

uste

d

Create VM

OAT-Based

Tboot-Enabled

5

Using Trusted Pools

Create a trusted flavor(instance type)

• Create a new flavor ‘m1.trusted’

• Add a ‘trusted_host=trusted’ property in flavor extra spec

Create a trusted instance

• Issue a request to start a new instance and specify a trusted flavor like`m1.trusted‘

• The filter scheduler call the trusted filter for each node in the system.

• The trusted filter query the attestation service to get the trust level for each of those nodes.

• Only those nodes that have a trust level as ‘trusted’ will be schedulable, all others will be ignored.

6

Agenda

Trusted Pools

• Concept

• Implementation & Usage

Trusted Launch with Trusted Boot (Tboot)

Remote Attestation with OpenAttestation (OAT)

More on Trusted Pools

• Patches

• Deployment & Configuration

Summary

7

Trusted Execution Technology extensions for measured

launch & memory protection (SMX)

VT-d chipset feature blocks device access

(e.g DMA) to protected memory pages

3rd party Trusted Platform Module(TPM)

stores and reports trusted environment

measurements

TPM

CPU

Chipset

Intel Authenticated Software

SINIT AC Module BIOS AC Module

Memory

3rd party Software VMM/OS uses TXT

mechanisms to establish a measured launch

environment

Processor contains hardware to authenticate AC Modules and perform measurements

Intel® Trusted Execution Technology (TXT)

BIOS / Flash BIOS AC Module and platform initialization

8

Trusted Boot (Tboot) Project http://sourceforge.net/projects/tboot

Open source, pre-kernel/VMM module, BSD licensed

Uses Intel TXT to perform verified launch of OS kernel/VMM

• Supports ELF and Linux file formats

• Extends LCP to verify VMM / kernel

Mercurial repo http://tboot.hg.sourceforge.net:8000/hgroot/tboot/tboot

Project also contains tools for policy creation and provisioning

• Intel TXT Launch Control Policy (LCP)

• Tboot Verified Launch policy

Distributions containing tboot package (Xen 3.4+, Linux 2.6.35+):

• Fedora 14+, RHEL 6.1+, SLE11 SP2, Ubuntu 11.10+

9

Trusted Launch with Tboot

tboot AP

join

All threads

participating

tboot starts

APs

tboot TXT

pre-launch

time

BIOS loads and

starts bootloader

All

Threads

BIOS

boot

GRUB SINIT

SENTER

Event

SENTER Bootstrap

Processor (BSP)

Application

Processor (AP)

SMP bringup wakes

APs

tboot post-

launch

VMM / kernel

ops

verify & prepare SINIT starts

tboot

put APs in

wait-for-SIPI

VMM/kernel

starts

GRUB loads

tboot + VMM / kernel + SINIT

and starts tboot

Extend

PCR 17

Extend

PCR 18 Extend PCR

17/18/19/…

* PCR – Platform Configuration Register in TPM

10

Agenda

Trusted Pools

• Concept

• Implementation & Usage

Trusted Launch with Trusted Boot (Tboot)

Remote Attestation with OpenAttestation (OAT)

More on Trusted Pools

• Patches

• Deployment & Configuration

Summary

11

OpenAttestation Project https://github.com/OpenAttestation/OpenAttestation.git

SDK for managing host integrity verification using Trust Computing

Group (TCG) defined remote attestation protocol

• Targeted at cloud and enterprise management tools

Key features:

• Supports major Linux host OS’s

• PCR-based report schema and policy rules

• RESTful based Query API

• Reference web portal/GUI implementation

– Historical PCRs data tracking/comparison

– Whitelist management

• Flexible access control to attestation server

– Supports Tomcat 2-way SSL/TLS for Query APIs

– Hook for ISVs to implement custom access control

* Whitelist –known good PCR values

12

SDK Architecture

Code base is from National Information Assurance Research Lab

(NIARL) of NSA

– Privacy Certificate Authority(Privacy CA), Appraiser, Host Agent are Java

– Host Agent accesses TPM through TrouSerS

Attestation Server (Tomcat)

Privacy CA

Appraiser

Whitelist API

Ho

st Ag

en

t AP

I

Qu

ery

AP

I SDK Components

DB(mysql)

whitelist table

hosts table

Portal reference code

HW/TXT

Hypervisor / tboot

OS

App App

App

OS

App App

App Host agent

Installation and

provisioning scripts

Hibernate

13

A Example for Query

Synchronically request host state from server

• Post and wait for hosts trustworthiness to return

POST OpenAttestationWebServices/V1.0/PollHosts

Host: Attestation.ras.com:8443

Context-Type: application/json

Accept: application/json

Auth_blob: authenticationBlob

Content-length: 39

{

“count”:1,

“hosts”: [host1.compute.com]

}

HTTP/1.1 200 OK

Server: BaseHTTP/0.3 Python/2.7.1+

Date: Wed, 24 Aug 2011 03:19:56 GMT

Context-Type: application/json

Content-length: 112

{

“count”:1,

“hosts”:[{“host_name”:“host1.compute.com”,

“trust_lvl”:“trusted”,

“vtime”: “Wed Aug 24 03:19:56 2011”}]

}

Request Response

14

• HTTPS Query API access control, setup/operated by Cloud

Provider, is thru. Tomcat Truststore by verifying both Server and

Client Certificates

• ISV specific Auth_blob is included in all request headers

• Opaque to Attestation SDK

• ISV to implement authentication hook per its access control requirement

Command Input parameters

Output parameters

Comment

POST https://server/PostHosts

Auth_blob, SelectedPCRs bitmask, {HostNames…}

RequestId Request to Attestation server for Hosts trust state and selected PCR values asynchronously

GET https://server/PostedHosts

Auth_blob, RequestId Hosts’ trust state data & Selected PCR values

Retrieve previously posted result

POST https://server/PollHosts

Auth_blob, SelectedPCRs bitmask, {HostNames…}

Hosts’ trust state data & Selected PCR values

Poll and wait for Attestation server to retrieve Hosts trust state and selected PCR values synchronously

Query API – Query Hosts’ Trust State

15

HTTPS access with both Server and Client Certificates verified through Tomcat

Truststore

ISV specific Auth_blob included in all request headers

• ISV to implement verification hook per access control requirement

Command w/ input parameters

Output parameters Comment

PUT /PCR Entry Index Create a new PCR entry for update (PCRindex, PCRvalue, PCRdesc)

UPDATE /PCR?Index=n N/A Update specific entry data

DELETE /PCR?Index=n N/A Delete specific entry data

GET /PCR PCRindex,PCRvalue,PCRdesc entries

Display all the entries

GET /PCR?Index=n PCRindex,PCRvalue,PCRdesc Retrieve a specific entry

GET /PCR?PCRindex=n PCRindex,PCRvalue,PCRdesc entries

Retrieve all the entries w/ PCRindex=n

GET /PCR?PCRdesc=desc PCRindex,PCRvalue,PCRdesc entries

Retrieve all the entries w/ PCRdesc=secription

GET /PCR?PCRindex=n&PCRdesc=desc

PCRindex,PCRvalue,PCRdesc Retrieve the entry with matched specification

WhiteList Data API – Add/Delete good/known WhiteList entries

16

TPM

Attesting Hosts Appraiser

Load AIK Send Nonce and requested PCRs

HostName, Quote

Retrieve AIK Certificate base HostName Verify AIK Certificate base on PrivacyCA.cert

Validate PCR

Quote = Sign(Requested PCR, Nonce)AIKpriv

Verify HostName and nonce

Verify Quote signature thru AIK Cert

Request appraisal

Create random nonce and get PCR_SELECT mask

*

*

* AIK – Attestation Identity Key

Attestation Flow in OpenAttestation – HostAgent to Server

17

Agenda

Trusted Pools

• Concept

• Implementation & Usage

Trusted Launch with Trusted Boot (Tboot)

Remote Attestation with OpenAttestation (OAT)

More on Trusted Pools

• Patches

• Deployment & Configuration

Summary

18

TrustedFilter

TrustedFilter

• Select current host as a candidate if

– trusted_host property not exist

– Or trusted_host property have a same value as trust level of current host got

via AttestationService

AttestationService

• Provide access wrapper to attestation server to get integrity report.

commit 14c01e09b68b367d708c6ddd6f3d4e440687727c Author: Don Dugger <donald.d.dugger@intel.com> Date: Tue May 8 18:30:57 2012 -0600 Add scheduler filter for trustedness of a host Implements blueprint trusted-computing-pools

19

Set Flavor Extra Specs

TrustedFilter requires a ‘trusted_host’ property in flavor extra spec

4 ways to set flavor extra specs:

• Access database directly

– mysql -u$MYSQL_USER -p$MYSQL_PASSWORD nova -e 'insert into

instance_type_extra_specs (`deleted`,`instance_type_id`,`key`,`value`)

values (0,6,"trusted_host",“trusted");‘

• Enhance nova-manage to set flavor extra specs

– nova-manage instance_type add_key m1.trusted trusted_host trusted

• Enhance nova-client to set flavor extra specs

• Enhance Dashboard(Horizon) to set flavor extra specs

commit 8644584eb6daf4d2870cee9bba5b849bc37e36d0 Author: Yunhong, Jiang <yunhong.jiang@intel.com> Date: Wed Jul 18 14:32:36 2012 +0800 Enhance nova-manage to set flavor extra specs blueprint update-flavor-key-value

20

Trusted Pools Deployment & Configuration

Steps:

• Deploy normal Nova controller & compute nodes

• Deploy OAT based attestation service

• Enable TPM & TXT in BIOS on compute nodes

• Install Host Agent on compute nodes

• Install tboot and enable trusted launch on compute nodes

• Configure attestation service and provision White List

• Configure Nova controller for Trusted Pools

21

Deploy OAT Based Attestation Service

Future approach: Install package(s) shipped with Linux distributions

Current approach: Build and install from source code.

• Build: https://github.com/OpenAttestation/OpenAttestation/raw/master/docs/Build.pdf

– Build system could be Ubuntu/SuSE/Fedora/RHEL

– Download & install required tools/libraries

– Build package with scripts

• Install: https://github.com/OpenAttestation/OpenAttestation/raw/master/docs/Installation.pdf

– Support Ubuntu/SuSE/Fedora/RHEL

– Install required modules

– Install the package generated in previous step

– Verify with accessing http://localhost/OAT/ in browser

22

Install Host Agent

System must have TPM 1.2 compliant device with driver installed, and

TPM/TXT enabled in BIOS.

Steps: https://github.com/OpenAttestation/OpenAttestation/raw/master/docs/Installation.pdf

• Install dependent packages

• Download Client Installation Package from OAT server:

– http://<server.domain>/ClientInstaller.html

• Unzip & run general-install.sh to install package

• Verify the Host Agent is registered into OAT service

– http://<server.domain>/OAT/reports.php

• There are hints for how to setup two way SSL/TLS auth

23

Install Tboot and Enable Trusted Launch

Install with tboot package in Linux distributions

• For ubuntu1204, apt-get install tboot

• For Fedora17/RHEL6.3/SLES11sp2, yum install tboot, then manually change grub.conf or.cfg.

Install from source

• Get source code from either upstream repo or released src package on sourceforge

• Install trousers/trousers-devel/libtsp package

• Make & make install with root priviledge

• Change grub.conf or .cfg

Refer to README of tboot project for more information

24

Configure Attestation Service & Provision White List Service Configuration: https://github.com/OpenAttestation/OpenAttestation/raw/master/docs/Installation.pdf

• in /usr/lib/apache-tomcat-6.0.29/webapps/ HisWebServices/WEB-INF/classes/OAT.properties

– PCR_SELECT=FFFFFF --- Include pcr 0~23 in integrity reports

– ALERT_MASK_CSV=0,17,18 --- Verify PCR0, 17, 18 to report trust level

White List provisioning:

• Get desired PCR value for PCRs specified in ALERT_MASK_CSV

• Create White List entry

– With Admin Console https://<server.domain>:8443/OpenAttestationAdminConsole/PCRManifest.jsp

– Or via invoking White List API through app or tools like curl

25

Configure Nova Controller

/etc/nova/nova.conf

[default]

compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler

scheduler_default_filters=TrustedFilter

[trusted_computing]

server=aa.bb.com --- attestation server http

server_ca_file=/a/b/c.cer --- attestation server Cert file for Identity verification

port=8443 --- attestation server port

api_url=/OpenAttestationWebServices/V1.0

--- attestation web API URL

auth_blob=xxxx --- attestation authorization blob - optional

26

Agenda

Trusted Pools

• Concept

• Implementation & Usage

Trusted Launch with Trusted Boot (Tboot)

Remote Attestation with OpenAttestation (OAT)

More on Trusted Pools

• Patches

• Deployment & Configuration

Summary

27

Summay

Trusted Pools feature in OpenStack was implemented and pushed into

Nova for next Folsom release.

The implementation is based on the Query API of attestation services

deployed using SDK provided by OpenAttestation (OAT) project.

It is strongly recommended to enable Trusted Boot (tboot) for each

compute node to take advantage of Intel TXT technology to involve

OS/VMM integrity into the host trust level judgment.

Call for Action:

• Try Trusted Pools Capability, seeking chances to do optimization.

28

Notices and Disclaimers

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS.

NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY

INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS

PROVIDED IN INTEL’S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL

ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED

WARRANTY, RELATING TO SALE AND/OR USE OF INTEL® PRODUCTS INCLUDING LIABILITY

OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE,

MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER

INTELLECTUAL PROPERTY RIGHT. INTEL PRODUCTS ARE NOT INTENDED FOR USE IN

MEDICAL, LIFE SAVING, OR LIFE SUSTAINING APPLICATIONS.

Intel may make changes to specifications and product descriptions at any time, without notice.

All products, dates, and figures specified are preliminary based on current expectations, and are subject to change without

notice.

Intel, processors, chipsets, and desktop boards may contain design defects or errors known as errata, which may cause

the product to deviate from published specifications. Current characterized errata are available on request.

Intel, and Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States

and other countries.

*Other names and brands may be claimed as the property of others.

Copyright © 2012 Intel Corporation. All rights are protected.