A Dynamic Aspect-oriented System for Data-driven Profiling of OS Kernels

OS2000e.g. ULESMP (FreeBSD), O(1)CFS (Linux)e.g. NetBSDFreeBSDSee Also: http://bulk.fefe.de/scalability/ (syscall )e.g. KSE (FreeBSD)NPTL (Linux)

int inode_change_ok(){ if ((ia_valid & ATTR_UID) && attr->ia_uid != inode->i_uid) goto error;

if ((ia_valid & ATTR_GID) && }Linux (fs/attr.c)struct timeval tv;do_gettimeofday(&tv) ;print_tv(tv);printk(%ld, inode->i_uid);

(1)) SystemTAP [Vara 05]DTrace [Bryan 04]

(2)) Kerninst [Ariel 99], GILK [David 02]Linux

Kerninst:

: Kerninst kmgr.createInstPointAtAddr(inst_ptr, &insert_point);kmgr.findModule(profiler, &kmod);kmod.findFunction(print_log, &pf);hook = kapi_call_expr(pf.getEntryAddr(), args);Kmgr.insertSnippet(hook,insert_point);void print_log() { __asm__ (movl %%ebp, %0 : =r(ebp)); uid = ((struct inode*)ebp[11])->i_uid; /* ebp[11] is inode */

(AOP) ()call/executionset/getn_timeIptime(){ getmicrotime(&atv); t = return (htonl(t));}

(DAOP) () AOP or e.g. PROSE [Andrei 02]Wool [Yoshiki 03]e.g. Steamloom [Christoph 04]

DAOP ()CDAOP (): :

DAOPCDAOPJavaDAOPJavaDAOPOSCCDAOPOS

OSe.g. (bottom half) (top half)

Ccf. InlineOS

CDAOPTOSKANA [Michael 05]TinyC2 [Charles 03] DAC++ [Sufyan 05]Arachne [Remi 05]Dyner [Marc 03]

DAOP(1) access [GPCE 06]()OS(2) xflow [PRO 07]()

(1) Access pointcut and KLASY

KLASY: Kernel Level Aspect-oriented SYstemAccess pointcutOSSource-based binary-level dynamic weavingKerninst

KLASY

linux/time.h access(inode.i_uid) AND within_function(inode_change_ok) AND target(inode_value) struct inode *i = inode_value; struct timeval tv; do_gettimeofday(&tv); print(i->i_uid, tv.tv_sec, tv.tv_usec);

pointcutadviceint inode_change_ok(){ if ((ia_valid & ATTR_UID) && attr->ia_uid != inode->i_uid) goto error;

if ((ia_valid & ATTR_GID) && }Linux (fs/attr.c)

KLASYaccessexecutionbeforeafter

targetaccesslocal_varaccessargumentexecution

KLASYSource-based binary-level dynamic weavingGNU C (gcc):()

KerninstCXML

OSOS OSKLASYgccinsmod

KLASY gccKLASY gcc:gcc

(-g)

OSOS OSKLASYgccinsmod

Kerninst

KLASYOSKerninst

linux/time.h access(inode.i_uid) AND within_function(inode_change_ok) AND target(inode_value) struct inode *i = inode_value; struct timeval tv; do_gettimeofday(&tv); print(i->i_uid, tv.tv_sec, tv.tv_usec);

int inode_change_ok(){ if ((ia_valid & ATTR_UID) && attr->ia_uid != inode->i_uid) goto error;

if ((ia_valid & ATTR_GID) && }selectedLinux (fs/attr.c)

KLASY gccKLASY gcc) inode.length&inode inode_ptr->lengthinode_ptr Gcc

UnixBenchLinux: gcc-fomit-frame-pointerKLASY: KLASY gcc-fomit-frame-pointer -fomit-frame-pointerCPU: AthlonXP 1800+, Mem: 1GB, Linux 2.6.10Kerninst 2.1.1, gcc 3.3.3, Intel Ethernet PRO/1000

dhry2reg: drystonesyscall: pipe: pipeexecl: execlcontext:

Sheet1

LinuxKLASYnormal

dhry2reg380380380

syscall830820770

pipe790750720

execl740770740

context720780780

diffdiff

dhry2reg000

syscall10060

pipe106060

execl301030

context404050

Graph2

3803800000

830820101000

79075010106060

74077030301010

72078040404040

Linux

KLASY

Sheet2

Sheet3

: I/OLinux Sk_buffSk_buffLinuxSk_buffKLASY

access(sk_buff.%) AND target(arg0) struct sk_buff *skb = arg0; unsigned long timestamp; if (skb->protocol != ETH_P_ARP) { STORE_DATA($pc$); STORE_DATA(skb); DO_RDTSC(timestamp); STORE_DATA(timestamp); }

ARP sk_buffskb->protocol

Skb_copy_datagram_iovecskb_copy_datagram_iovec1000_clean_rx_irqnetif_receive_skbip_rcvip_rcv_finishtcp_v4_do_rcvtcp_v4_rcvtcp_rcv_established__kfree_skb

Graph2

0.340.722.371.42.013.993.411.71.21

0.3912.961.442.264.163.22688.333.49

1000_clean_rx_irq

netif_receive_skb

ip_rcv

ip_rcv_finish

tcp_v4_rcv

tcp_v4_do_rcv

tcp_rcv_established

skb_copy_datagram_iovec

__kfree_skb

Elapsed time

Time scale of packet arrival

trace_diff

functionpacket 1packet 2packet 2

1000_clean_rx_irq0.340.390.34

netif_receive_skb0.7210.7

ip_rcv2.372.962.52

ip_rcv_finish1.41.441.58

tcp_v4_rcv2.012.261.97

tcp_v4_do_rcv3.994.164.43

tcp_rcv_established3.43.222.4

skb_copy_datagram_iovec11.7688.33428.48

__kfree_skb1.213.493.14

000

0.340.390.34

1.061.391.04

3.434.353.56

4.835.795.14

6.848.057.11

10.8312.2111.54

14.2315.4313.94

25.93703.76442.42

27.14707.25445.56

trace_diff

packet 1

packet 2

packet 2

accessSk_buffaccessLinuxexecution76(21)InlineOS) Linux -Osstatic inline6

1accessSource-based binary-level dynamic weaving

I/Oaccess

(2) Xflow pointcut and XenLASY XenLASY = KLASY + xflow + VM

I/OI/OI/O

() U0I/O

(ID)

VMI/OOSI/O0VMI/O

) XenI/O

XenLASYXenI/O xflowXen: xflow(, , ID)ID

KLASYKLASY [Yanagisawa 06] KLASYaccessSource-based binary-level dynamic weavingXenxflow@

XenLASY

linux/time.h access(sk_buff.%) AND target(skb) AND xflow(netflow, skb, id) long long tsc; DO_RDTSC(tsc); STORE_DATA3($pc$, tsc, id);

sk_buff netflow pcid

netflow: I/O

xflownetflowXenI/Ostart: transit: skb_clonequit:

access(sk_buff.head) AND within_function(skb_clone)

access(sk_buff.%) AND within_function(__kfree_skb)

access(sk_buff.data) AND within_function(alloc_skb_from_cache)

start/quit (/)ID

accessselectDB/action


int id = new_flowid();

transit ()moveskb: IDn: IDskbIDcopy


transitxin_movexout_moveIDIDID0DomUID

transitlinUlin0IDxin_moveskbIDtxflags412xout_movexin_moveIDskbxin_movename@tx->flags12bit4bit

access(netif_tx_request.flags) AND within_file(drivers/../netback.c@lin0)

access(netif_tx_request.flags) AND within_file(drivers/../netfront.c@linU)

xflow: start/quit

access(sk_buff.data) AND target(skb) AND within_function(alloc_skb_from_cache)

id = get_new_flowid(); register_flowid(netflow, id, skb);


quittarget DBIDalloc_skb_from_cacheskbIDDB

xflow: transit

access(sk_buff.head) AND within_function(skb_clone) AND local_var(skb, skbp) AND local_var(n, np)

void *skb = *((void **)skbp); void *n = *((void **)np); int id = get_flowid(netflow, skb); if (id != 0) { register_flowid(netflow, id, n); remove_flowid(netflow, skb);}

skbIDnlocal_var


skb_cloneskbIDn

xflow: transit

access(netif_tx_request.flags) AND within_file(drivers/.../netfront.c@linU) AND local_var(tx, txp) AND local_var(skb, skbp)

struct netif_tx_request *tx = txp; void *skb = *((void **)skbp); id = get_flowid(skb); if (id != 0) { tx->flags |= id > 12; } remove_flowid(netflow, skb);

ID XenI/OUnetif_tx_requestID

access(netif_tx_request.flags) AND within_file(drivers/../netfront.c@linU)

local_var

: KerninstXenKerninst [Tamches 99]KerninstU0XenKerninstring0

xflow 2000CPU: AMD Athlon 64 3500+: 2GB0: 256MB U: 128MB0

I/OU0xflowU0sk_buffID

skb_cloneTCPTCP0 netif_receive_skballoc_skb_from_cachetcp_sendmsgnetwork_start_xmitnetbk_fill_flagsnetif_rxnetif_receive_skb__br_forwardSkGeXmitFreeTxDescriptors

Graph2

3.4127.4751.712.42415.541.8110.432.9

()

trace_diff

function

alloc_skb_from_cache0

3.43.4

127.4130.8

751.7882.5

12.4894.9

2415.53310.4

41.83352.2

110.43462.6

FreeTX32.93495.5

xflowxflowxflowApacheBench3001060%

Dflow pointcut [Masuhara 03]dflowDJcutter [Nishizawa 04]DAC++ [Almajali 05]Causeway [Chanda 05]FreeBSDI/O

2xflowKLASYI/O0

DAOPDAOP(1) access(2) xflowCDAOP

x86IDVMMOSVMMDAOPUI

AccessOS KLASY,:, vol. 48, no. SIG 10 (PRO33)A Dynamic Aspect-oriented System for OS Kernels, GPCE06XflowXenLASY: XenI/O,:, PRO35

, , : XenLASY: XenI/O, 65 (PRO-2007-2), , 200781 - 3 ((PRO35)), , , : OS KLASY, :vol. 48, no. SIG 10 (PRO33)pp.176-188June 2007. (19)Yoshisato Yanagisawa, Kenichi Kourai, Shigeru Chiba, and Rei Ishikawa: A Dynamic Aspect-oriented System for OS Kernels, In Proc. of the 5th Int'l Conf. on Generative Programming and Component Engineering (GPCE'06), pp.69-78. Portland, Oregon, USA. October 22-26, 2006. ,,,: SAccessor: PC,19 ComSys 2007, , 20071127, , : , 11 (SPA 2007), , 2007 9 3 - 5: OS, 10 (SPA X), , 2006 8 28 - 30 , , : , 99 , , 2005 525-27, 2005-OS-99, pp.149-156Yoshisato YANAGISAWA, Shigeru CHIBA, Kenichi KOURAI: A Source-level Kernel Profiler based on Dynamic Aspect-Orientation, Dynamic Aspects Workshop (DAW05) Chicago, USA. March 15th, 2005 (held in conjunction with AOSD 2005): , // (SwoPP2004) 2004-OS-97, pp.33-40, 20048.: ,//(SWoPP '03) 94OS,pp.83-8920038., , , : KLAS,9 (SPA 2006), , 2006 3 5 - 7 . ()Yoshisato YANAGISAWA, Shigeru CHIBA, Kenichi KOURAI: KLAS: Kernel Level Aspect-oriented System Source-based Binary-level Dynamic Weaving -, AOSD 2005 (poster), Chicago, Illinois, USA, March 16th, 2005. (Best Poster Award)11 (SPA 2007)

Documents

A Dynamic Aspect-oriented System for Data-driven Profiling of OS Kernels