Nesting Containers: Real Life Observations

Andre MorugaDirector of Program ManagementOdin

Virtualization and infrastructure platform for service providers

Secure VMs, containers, and virtualized storageIntroduced Linux container virtualization in 2001, over 1+ million

production instancesOwner and/or major contributor to numerous open source efforts,

including…

2

Why Go Nested?Docker in Docker

Useful in Docker developmentWorks since Docker 0.6 Requires “privileged” mode for the “parent” containerThere is a Docker image that does it - dind

Docker in System Containers

System containers are popular platform for Virtual Private Servers (VPS)Implementations: LXC, LXD, OpenVZ, VirtuozzoUsage scenarios are same Virtual MachinesDocker is expected to work

Benefits of Nested Containers (versus VMs)Efficiency

No hardware virtualization overheadElasticity

No downtime resize anythingNo hard partitioning

Tenant isolationSolid security (15 years in hosting)Resources caps on groups of containersExtra isolation layer

5

What Does it Take to Run Nested Containers in Linux?

Nested Namespaces

Nested cgroups

Nested storage drivers

6

What’s Wrong with Storage?Docker relies upon layered storageStorage support is implemented via storage “graph” driversAvailable options

Relies on block device mount and not allowed in

containers due to security restrictions

Slow, space consuming, and

inefficient

AUFS, Device Mapper, BTRFS, Overlay VFS

7

Better Storage for Nested Containers“Extpoint” extensibility for storage drivers was developed to Docker mainstream, likely candidate for Docker version 1.10It allows moving storage driver functionality outside of Docker

Docker Stor

age

Extp

oint Storage

Driver

Docker Storage

System Container

Host

What’s Next?Storage Extpoint is a framework to build a solution, but not a

solution yetProblems to solve

Storage driver “Server” piece implementationImages need to be protected so container owner could not modify them

directlyStorage space management: cannot modify but can remove when no longer

needed

8

More Bumps Down the RoadCgroups and Namespaces: Virtualized does not mean identical to rootDocker monitoring and management software may not work properly (e.g. cAdvisor)Further compatibility is likely to break as Docker evolves

9

SummaryIt worksIt is arguably good enough for “light” useStorage extpoint and improved storage driver will make it good enough for straightforward usage scenariosFor advanced configurations, better to wait until wider adoption

10

Thank you!Visit Odin at Booth #24Andre Moruga, Odinamoruga@odin.com