Kubernetes spravujte linuxové kontejnery jako jednotný systém

spravujte linuxové kontejnery jako jednotný systém

● Základní rozdíly mezi samotným řešením Docker (Docker Compose) a Kubernetes.● Jakým způsobem řešit sítě v kontejnerech a jejich srovnání (Flannel, Calico, Contrail, Weave).● Způsoby používání storage v kontejnerech případně integrace s CEPH.● Diskuze na téma nasazení Kubernetes ve vaší společnosti (Use Case).● Živá ukázka jednoduchého aplikačního clusteru.● Příklady využití v rámci CI/CD systému (rolling updates, testování, validace, automatizace).

Agenda

Basics Recap

Containers

What is container

Control groups (cgroups)Virtualize by sharing and limiting access for:• CPU• Memory• Disk IO• Network IO

NamespacesVirtualize by isolating the:• User IDs

Kernel

App

1

App

1

App

2

App

2

App

2program, libraries, config files

• Processes isolated from the host and (optionally) other containers • Share the same underlying Kernel• Virtual network interfaces / addresses (maybe host NAT’d)• Files and optional (shared) mounts from the host filesystem

Each Linux container...

• Process IDs & tree• Filesystem mounts• Network interfaces

Security:• SELinux policy and enforcement control over

all resources• AppArmor to restrict a program’s abilities• Linux capabilities etc.

Containers have similar resource isolation and allocation benefits as virtual machines but a different architectural approach allows them to be much more portable and efficient

(more specifics on Docker)

vs

App

1A

pp1

App

2A

pp2

App

2program, libraries,

config files

Guest OS

Hypervisor

Host OS Host OS

App

1A

pp1

App

2A

pp2

App

2

program, libraries, config files

Container Engine

Very obvious:No Guest OSin a container

Not so obvious:Containers sharethe Host OS kernel;engine doesn’t reallysit beneath containers

Not so obvious:Container's appsand files can have a whole OS filesystemlike Ubuntu

How it is different from VMs

• Docker Engine: The engine that runs on the host OS to manage container images and instances

• Docker Registry: Docker Hub is the free public registry-aaS, others: quay.io, Google Cntr Reg, private ones

• Docker Tools: cli, GUI, container compose, container swarm clustering

Docker Components

App

1A

pp1

App

2A

pp2

App

2program, libraries, config

files

Any Linux distro…• Ubuntu, Debian, Red Hat, Fedora, Centos, SUSE, etc.

And with some caveats:• Boot2docker VM or DockerMachine on Windows or OSX• Windows Server 2016!

Where Docker Works…

What is Docker

Docker Images• Can be saved from a running container (docker commit)• Can be built from a Dockerfile manifest (docker build)• Are downloaded into local cache (docker pull)• Are uploaded to your own repository in the registry (docker

push)

Container Management Made Easy

Layered format Dockerfile• Base image contains an operating system (e.g. Debian or

Ubuntu)• Additional layers add applications and configuration files• Docker tracks and creates just the difference in

running/stored images• Uses a copy-on-write and union file system to optimize

footprint

Optimized Container Format

referenceparent image

• Fast deployment and restart

• Minimum overhead• Easy to migrate to any

Linux

Understanding Docker Containers

● Instantiate an image as a running container (docker run)● Attach to a running container (docker attach)● Pause a container (docker stop) or restart (docker restart)● Destroy a container instance (docker rm)●Best Practice: Run a single process in a single container● Containers can talk through normal networking or assisted

by linking

Docker Basics

Docker Engine● Uses its own libcontainer to control the kernel (no longer

uses LXC)○ Runc is its plumbing that was standardized in the

Open Container Format by the OCI● Usually looks like a whole system to the process inside of

the container

How It Works

How to scale:

Follow the Unix adage,several decoupled simple

components are better than a single complicated one

Docker Container Lifecycle

Using Docker: Summary

Kubernetes Intro

Kubernetes is an open-source system for automating deployment, operations, and scaling of containerized applications.

What Kubernetes try to solve?

High Available Container ClusterNetworking for containers

Storage for containersScheduling

Application Lifecycle

•

Kubernetes tools

Master Components Node components

kube-apiserver

etcd

kube-controller-manager

These controllers include:

● Node Controller● Responsible for noticing & responding when nodes go down.● Replication Controller● Responsible for maintaining the correct number of pods for every

replication controller object in the system.● Endpoints Controller● Populates the Endpoints object (i.e., join Services & Pods).● Service Account & Token Controllers● Create default accounts and API access tokens for new namespaces.● … and others.

kubelet

kubelet is the primary node agent. It: * Watches for pods that have been assigned to its node (either by apiserver or via local configuration file) and: * Mounts the pod’s required volumes * Downloads the pod’s secrets * Run the pod’s containers via docker (or, experimentally, rkt). * Periodically executes any requested container liveness probes. * Reports the status of the pod back to the rest of the system, by creating a “mirror pod” if necessary. * Reports the status of the node back to the rest of the system.

kube-proxy

Kubernetes tools

Master Components Node components

kube-scheduler

addons

Addons are pods and services that implement cluster features. They don’t run on the master VM, but currently the default setup scripts that make the API calls to create these pods and services does run on the master VM.

DNS

User interface

Container Resource Monitoring

Cluster-level Logging

docker

rkt

supervisord

fluentd.

Kubernetes Intro

What actually is Kubernetes?!

Kubernetes Workload

Networking

Ports vs SDN

Container network config

● Map one or more host address:port to container’s assigned IP:port

● Easily learn services of other containers with Docker linking

Libnetwork

● Like libcontainer, a separate library and API used by Docker Engine

● Manages the networking objects: network, endpoint, and sandbox/cntr

● This was separated out after Docker’s Socket Plane acquisition● Delegates implementation to a driver

Bridge (docker0) Driver

● docker0 is a Linux bridge shared by the host interfaces and containers

● Each container gets a veth-pair “tunnel” with one end called eth0 inside the container and the other end bound to the docker0’s bridge

● Uses iptables for NAT

How Does Docker network?

Kubernetes Networking Model

● All containers can communicate with all other containers without NAT● All nodes can communicate with all containers (and vice-versa) without NAT● The IP that a container sees itself as is the same IP that others see it as

OR plugins/backends

● Flannel● Calico● Weave● OpenContrail

Flannel

Calico

● Scale to millions of workloads with minimal CPU and network overhead

● What is happening is “obvious” – traceroute, ping, etc., work as expected; routing and ACL rules tell you everything you need to know

● Path from workload to non-virtualized device is just a route● Physical fabric is state-light (standard IP forwarding only)● External connectivity is achieved by assigning a public IP● Equal Cost Multi-Path (ECMP) any Anycast just work,

enabling scalable resilience and full utilization of physical links

● Traffic between data centers is natively L3 routedBasic IP networking knowledge only required

Weave

● Weave creates a virtual network that connects Docker containers deployed across multiple hosts and enables their automatic discovery.

● Applications use the network just as if the containers were all plugged into the same network switch, with no need to configure port mappings, links, etc…

● Weave can traverse firewalls and operate in partially connected networks.● Weave routers establish TCP connections to each other, over which they perform a protocol

handshake and subsequently exchange topology information.● Weave creates a network bridge on the host. Each container is connected to that bridge via a

veth pair, the container side of which is given an IP address & netmask supplied either by the user or Weave’s IP address allocator.

Full-featured SDN implementation

● Run controller as usual on a few servers or VM● On nodes running Docker Engine

○ Contrail vRouter replaces the docker0 bridge○ Every container still has a veth-pair but one end is

bound to a VRF in the vRouter○ Containers get Contrail IPAM & DHCP○ Containers are reachable from the outside using

floating IPs○ Containers are securely isolated using

microsegmentation policies

OpenStack nova-docker driver

● Container orchestration by OpenStack like VMs● Easily networked by Contrail Networking like VMs

OpenContrail

Networking Comparison

Flannel Calico OpenContrail

Overlay VxLAN No MPLSoverGRE, VxLAN

Multi-tenancy No Yes Yes

Openstack Federation

No No Yes

Network Policy No Beta release Yes - native

BGP/ L3VPN/EVPN No No Yes

Storage

Local vs Shared

● emptyDir● hostPath● gcePersistentDisk● awsElasticBlockStore● nfs● iscsi● flocker● glusterfs● rbd● gitRepo● secret● persistentVolumeClaim● downwardAPI● azureFileVolume● CinderVolume

•

Backend Types

An rbd volume allows a Rados Block Device volume to be mounted into your pod. Unlike emptyDir, which is erased when a Pod is removed, the contents of a rbd volume are preserved and the volume is merely unmounted. This means that a RBD volume can be pre-populated with data, and that data can be “handed off” between pods.

A feature of RBD is that it can be mounted as read-only by multiple consumers simultaneously. This means that you can pre-populate a volume with your dataset and then serve it in parallel from as many pods as you need. Unfortunately, RBD volumes can only be mounted by a single consumer in read-write mode - no simultaneous writers allowed.

•

Ceph Backend

Cinder Backend

Use Case

Smart City IoT

Available at https://pisek.urbandata.cz/en/data-samples/Graphite metrics at http://graphite-server.iot.tcpcloud.eu/

Live Show

How simple is that?

Hybrid Environment

Live Demo

Adam Skotnickýadam.skotnicky@tcpcloud.eu@ada_sko

Jakub Pavlíkjakub.pavlik@tcpcloud.eu@JakubPav

Thank you!