Networking Laboratory

  • View
    213

  • Download
    1

Embed Size (px)

Transcript

  • NetworkingLaboratory1/

    Sungkyunkwan University

    Copyright2000-2016NetworkingLaboratory

    Recovering Device Drivers

    Michael M. Swift, Muthukaruppan Annamalai, Brian N. Bershad, and Henry M. Levy

    University of Washington, USASymposium on Operating Systems Design and Implementation, 2004

    2016-05-16, ,

  • OperatingSystemsDesignNetworkingLaboratory2/44

    Introduction

    Shadow driver

    Evaluation

    Conclusion

  • OperatingSystemsDesignNetworkingLaboratory3/44

    Introduction system failure device driver failure.

    Driver failure

    Failure-isolation system driver fail kernelapplication

    failed driver kernel driver Failed driver application state application

    Device driver fail application

    Shadow Drivers

  • OperatingSystemsDesignNetworkingLaboratory4/44

    Device Drivers fail? Device driver failure

    . Deterministic

    IO requests generic tools

    Transient input

    Fail-stop Failed driverOS,

  • OperatingSystemsDesignNetworkingLaboratory5/44

    Shadow Drivers(1/6)shadow drivers?

    Shadow driver Taps. Kernel Driver failure shadow driver failed driver failed driver

    3 Isolation service Redirection mechanism Object tracking service Nooks

  • OperatingSystemsDesignNetworkingLaboratory6/44

    Shadow Drivers(2/6)Shadow drivers

    Passive mode Kernel device driver

  • OperatingSystemsDesignNetworkingLaboratory7/44

    Shadow Drivers(3/6)Shadow drivers

    Active mode Driver fail Failed driver kernel call Failed driver kernel

    driver call

  • OperatingSystemsDesignNetworkingLaboratory8/44

    Shadow Drivers(4/6)Shadow drivers

    Passive mode IO request

    Connection-oriented driver : active connection Request-oriented driver:

    Kernel device driver

    Persistent log

  • OperatingSystemsDesignNetworkingLaboratory9/44

    Shadow Drivers(5/6)Shadow drivers

    Active mode Failed driver fail

  • OperatingSystemsDesignNetworkingLaboratory10/44

    Shadow Drivers(6/6)

  • OperatingSystemsDesignNetworkingLaboratory11/44

    Evaluation(1/4)

    [] []

  • OperatingSystemsDesignNetworkingLaboratory12/44

    failure-isolation shadow driver overhead

    Evaluation(2/4)performance

  • OperatingSystemsDesignNetworkingLaboratory13/44

    Driver fail

    Evaluation(3/4)Fault-Tolerance

  • OperatingSystemsDesignNetworkingLaboratory14/44

    Evaluation(4/4)Limitations

    Non-fail-stop

  • OperatingSystemsDesignNetworkingLaboratory15/44

    Conclusion(1/3) Rx

    Software failure checkpoint rollback

  • OperatingSystemsDesignNetworkingLaboratory16/44

    Conclusion(2/3) Shadow driver

    wrapper fault write

  • OperatingSystemsDesignNetworkingLaboratory17/44

    ..

    Fault HAL driver fault rate Arch

    fault source code

    fault tool

    Conclusion(3/3)

  • NetworkingLaboratory18/

    Sungkyunkwan University

    Copyright2000-2016NetworkingLaboratory

    IX : A Protected Dataplane Operating System for High Throughput and Low Latency

    2016-05-16Mincheol Cha / Daecheon Kim / HeeJin Kim

    Sungkyunkwan University

  • OperatingSystemsDesignNetworkingLaboratory19/44

    Contents

    Introduction

    Terminology

    IX

    Evaluation

    Discussion & Conclusion

  • OperatingSystemsDesignNetworkingLaboratory20/44

    IntroductionDatacenter Applications (1/2) Microsecond tail latency

    RPC latency request latency node 99% request latency

    High packet rates Datacenter application request reply connection counts app packet concurrent churn

  • OperatingSystemsDesignNetworkingLaboratory21/44

    IntroductionDatacenter Applications (2/2) Protection

    Kernel or hypervisor networking stackServer service app isolation

    Resource efficiency user traffic datacenter app load service node resource packet rate tail latency

  • OperatingSystemsDesignNetworkingLaboratory22/44

    IntroductionThe Hardware OS hardware Mismatch

    The wealth of hardware resources but poor OS latency packet rateHardware OS

  • OperatingSystemsDesignNetworkingLaboratory23/44

    IntroductionThe Hardware Alternative Approaches

    User-space networking stacks Kernel overhead user-space networking stack Kernel packet processing packet rate latency tradeoff

    TCP UDP -> user level

    POSIX API Mega pipe : software overhead packet rate kernel-based networking stack

    OS Tuning kernel-based stacks system Epoll

  • OperatingSystemsDesignNetworkingLaboratory24/44

    TerminologyModern system(1/3)

    Zero-copy API : context switch copy data network

  • OperatingSystemsDesignNetworkingLaboratory25/44

    TerminologyModern system(2/3)

    mTCP : multicore system User-level TCP Stack

  • OperatingSystemsDesignNetworkingLaboratory26/44

    TerminologyModern system(3/3)

    Hierarchical ring(protection ring) : faultsmalicious behaviordata

  • OperatingSystemsDesignNetworkingLaboratory27/44

    IXDesign of IX

    controlplane and dataplane Controlplane : resource configuration, provisioning, and so on(simple kernel mode) Dataplane: networking stack app logic

    adaptive batching RTC Data instruction cache locality

    flow control zero-copy API

    Flow , -free processing

  • OperatingSystemsDesignNetworkingLaboratory28/44

    IXIX implementation The IX dataplane operating system

    Control Data Plane Adaptive Batching RTC (network stack)

    DataplaneControlplane

  • OperatingSystemsDesignNetworkingLaboratory29/44

    IXIX Parts, Hierarchical rings IX Rings

    FullLinuxKernelDune

    VMXrootring0

    =

    Dataplane VMXnon-rootring0OS.(guestkernel)

    VMXNon- rootring3

    App App1. Dataplane pagetables,exceptions2. Dataplane ->NIC3. Controlplane,dataplanes, appcode

    Full3-wayprotection

  • OperatingSystemsDesignNetworkingLaboratory30/44

    IXIX Parts, Controlplane IX Controlplane

    IXControlplane

    FullLinuxKernelIXCP(Userlevelprogram)

    1. PCIe(Lan card)2. (,,)

    1. Resource,dataplane2.

    --*dataplane

    tradeoff* app

    ( load)

    Controlplane dataplane

    Coarsh-grainedmanner resource

  • OperatingSystemsDesignNetworkingLaboratory31/44

    IXIX Parts, Dataplane(1/4) Dataplane Thread

    IXDataplane 1. HighperformancenetworkIO2. app(libraryOS)+memoryisolation3. kernel-levelservices

    Single,multithreadappOS

    ThreadElasticthread:dataplane,network I/OBackgroundthread

    = threadPOSIXsystemcallissue

    Elasticthread->blockingcall( packetprossesing network) latency dataplane hardwarethread core

    Backgroundthread hardwarethread

    Controlplane elasticthread =exokernel

  • OperatingSystemsDesignNetworkingLaboratory32/44

    IXIX Parts, Dataplane(2/4) Dataplane memory management

    Hardwarethread

    Memorypooldataobject

    Hot-path 1. objectarray2. Page-sizedblock

    1. Internalmemory2. MemoryUp

    *Object->Freeobjects->Allocationroutines ()*Single*kernel pages spervisor bit*Swappablememory()

  • OperatingSystemsDesignNetworkingLaboratory33/44

    IXIX Parts, Dataplane(3/4) Dataplane

    Virtual address translation = nested paging OS 2MB large pages : 1. address translation overhead

    2. memory

    Mbuf : packet

    Hierarchical timing wheel : TCP timer ) High-resolution timeout (TCP incast congestion)

  • OperatingSystemsDesignNetworkingLaboratory34/44

    IXIX Parts, Dataplane(4/4) Dataplane

    Dune,VT-x

    ARM,SPARC,Power porting

    Intel82599chipsetNIC()

    39kSLOC

    RFC-Compliant:UDP,ARP,ICMP

    LwIP :()->multicorescalability,fine-grainedtimermanagement

    DPDK41%IntelNICdevicedriver

    LwIP 26% DUNELib16%

    NewCode7kSLOC

  • OperatingSystemsDesignNetworkingLaboratory35/44

    IXStructure of dataplane batched System Calls The IX dataplane system call event condition API

    Elastic Threads batch system calls event conditions

  • OperatingSystemsDesignNetworkingLaboratory36/44

    EvaluationExperimental Methodology general evaluation

    NetPIPE : message sizes system software IX Linux, mTCP

    SPEC. Quanta/Cumulus 48X10GBe switch Mix of Xeon E5-2637 Xeon E5-2665 server 256GB of DRAM 8 cores and 16 hyperthreads Intel x520 10GBE NICs

  • OperatingSystemsDesignNetworkingLaboratory37/44

    EvaluationLatency Single-flow Bandwidth (1/4)

    Multicore Scalability for Short Connections

  • OperatingSystemsDesignNetworkingLaboratory38/44

    EvaluationLatency Single-flow Bandwidth (2/4)

    N round-trips per connection

  • OperatingSystemsDesignNetworkingLaboratory39/44

    EvaluationLatency Single-flow Bandwidth (3/4)

    Different sizes s (n=1)

  • OperatingSystemsDesignNetworkingLaboratory40/44

    EvaluationLatency Single-flow Bandwidth (4/4)

    Connection

  • OperatingSystemsDesignNetworkingLaboratory41/44

    EvaluationThroughput Scalability

    2 memcached

  • OperatingSystemsDesignNetworkingLaboratory42/44

    Discussion & ConclusionDiscussion

    IX?

    adaptive batching

    current prototype

  • OperatingSystemsDesignNetworkingLaboratory43/44

    Discussion & ConclusionDiscussion

    IX vs Arrakis

    Arrakis :IX per-app networkstack(batchinghandling)

    IX Arrakis

  • OperatingSystemsDesignNetworkingLaboratory44/44

    Dis