Kubernetes IntroductionAdvanced Technology Group (ATG) for Open Source & Cloud
August 2016
2
What is Kubernetes?
Κυβερνήτης — Greek: A nautical term meaning “helmsman” or “pilot”
“K8s”
3
Kubernetes“Open Source Container Cluster Manager”
• Google — Architect and creator.
• Borg — Google’s internal cluster management software. Kubernetes – complete rewrite, (in Go).
• Google partnered with Linux Foundation to form: Cloud Native Computing Foundation (CNCF) offered Kubernetes as a seed technology
4
Kubernetes History
2013 2014 2015 2016
Apr 2015Tectonic formed (commercial support)
Apr 2015The Borg Paper is published
Sep 2014Kubernetes announced in Wired magazine
Jun 2014Kubernetes 1st GitHub commit
Mar 2013Docker initial release
Aug 2014CoreOS introduces Flannel networking
Oct 2013CoreOS initial release
2008 …2006
2006Google starts work on “Process Containers”(renamed “cgroups”)
Jan 2008cgroups merged into Linux (2.6.24)
2007
July 2015CNCF Formed, K8s v1.0 released, donated to CNCF
Borg development inside Google
5
Kubernetes Tech Specs
Features
• μService Architecture
• Automatic Workload Placement (efficient)
• Auto Remediating (self healing)
• Horizontal Scaling
• Load Balanced
• Declarative Deployment
• Service Discovery included
• A/B & Canary Deployments (testing)
Surrounding Ecosystem
Docker – the container “engine” on each host. etcd (from CoreOS) – distributed K/V store.
CoreOS – the platform. Flannel – overlay networking.
Hosted Service: Google Container Platform GKE is the abbreviation.
6
Network
Client
μService Programming Model — Cloud Native
proxy
μS
…μS
μS
proxy
μS
…μS
μS
proxy
μS
…μS
μS
proxy
μS
…μS
μS
proxy
μS
…μS
μS
proxy
μS
…μS
μS
(HTTP
) Route / P
roxy
Optional
(nginx)
Pod(container)
Service
“Load Balancer”
7
Kubernetes – Programming Model
• Filesystem – that the program uses.• Persistent – how state is saved beyond run-time.• Persistent Volumes are attached and live outside of the
K8s cluster.
Volumes & Persistent Volumes
Pod• One (or more) containers “grouped”• Network (IP address): shared• Volumes: shared
Service• Common API (behavior) replicated across the cluster.• Well Known Endpoint – a consistent IP address,
regardless of changes in specific Pods underneath.
Service
proxy
Host (“node” in K8s)
Pod – different μS
Pod
Container(s)
proxy
Host (“node” in K8s)
Pod
Container(s)
Volume,external
to K8s
Abstract
(Common IP)
8
Kubernetes – Framework Architecture
Client
ControlPlane
Workload
*https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/architecture.md
9
Kubernetes – Framework Architecture
• K8s is extensible• Storage Plugin(s)
- NFS / iSCSI- AWS-EC2 / Google GCE- Ceph (RBD/CephFS) / Gluster- Cinder (OpenStack)
• Other Extension Points- Logging- Access & Auth- Scheduler
Control Plane Worker Node(s) Client
Extension Points
kubelet: local, control plane agent. Pod management using docker-engine.
kube-proxy: internal service routing (i.e. TCP/UDP stream forwarding)
docker-engine: container execution
kube-apiserver: Client’s API access point. Routes requests to appropriate, internal components.
kube-controller-manager: Embeds the core control loops.
• Replication controller• Endpoints controller (proxies)• Namespace controller
kube-scheduler: Workload (Pod) placement. Sophisticated, configurable, globally aware.
etcd (from CoreOS): Distributed, watchable storage The k8s system state
kubectl: CLI into K8s
HTTP — RESTful protocol.
Kubernetes – Deployment ModelA Declarative Model
10
Manifest File(s)
Labels
PodSpec clause – within most descriptors
Replication Controller descriptor
• Optional only in trivial cases. • (trivial = CLI only possible)
• YAML (or JSON) format.
• Key/Value “tags” – placed on any deployable object.
• Selectable – by actions and other declarations.• Configuration Flexibility
• Labeled• allows versioning • other constraint application
• Container(s)• very Dockerfile / docker-compose like.• Image location, (including image version)• Volume requirements• Ports exposed
• “template/spec” clause declares PodSpec configuration.• “replica” clause declares sizing of the service.• Rolling-updates & canary deploys are a supported
pattern.
Descriptor Types (partial list)
• Replication Controller• Deployment
• Pod• Job
• Service
11
Running a Kubernetes Cluster
“There’s more than one way to do it”
– Larry Wall
12
Kubernetes in Public Cloud
Hosted Solution — Google Cloud Platform
Google Container Engine (GKE)
• Kubernetes Getting Started Guide “101”• Hello World Walkthrough
https://cloud.google.com/container-engine/
http://kubernetes.io/docs/hellonode/
Turn-key Solutions
Amazon Web Services (AWS) EC2 http://kubernetes.io/docs/getting-started-guides/aws/
Azure http://kubernetes.io/docs/getting-started-guides/azure/
Free Trial —60 days
$300 credit
13
Kubernetes Run Locally
On a Laptop / Desktop
Minikube• K8s recommended method for single node deploy http://kubernetes.io/docs/getting-started-guides/minikube/
Vagrant — superseded by Minikube, still usable. http://kubernetes.io/docs/getting-started-guides/vagrant/
kube-up.sh — another previous “#1” method by k8s http://containertutorials.com/get_started_kubernetes/index.html
Easy Kubernetes Cluster for macOS• Recently discovered and recommended by our team (ATG). https://github.com/TheNewNormal/kube-cluster-osx
Multi-host / LabCoreOS w/ Fleet • https://github.com/CaptTofu/kubernetes-cluster-fleet
• https://github.com/coreos/coreos-vagrant• https://github.com/mhamrah/kubernetes-coreos-units
14
A Kubernetes Application
15
Kubernetes Application – minimalist application –
1. Construct • Create a standard Docker application, a μService.• Package it as a Docker Image.
2. Deploy • Deploy the Docker Image to a Docker Repository.
3. Run • kubectl run … --image=<Image-Repository-Path>
16
K8s App — Constructapp.py*
from flask import Flaskapp = Flask(__name__)
@app.route('/')def hello_world(): return '-- Hello Flask Dockerized --\n'
if __name__ == '__main__': app.run(debug=True, host='0.0.0.0')
Dockerfile*
FROM ubuntu:latestRUN apt-get update -yRUN apt-get install -y python-pip python-dev build-essentialCOPY . /aptWORKDIR /aptRUN pip install -r requirements.txtENTRYPOINT ["python"]CMD ["app.py"]
*https://github.com/egustafson/ex-py-docker-flask
Build
Run
Verify (in a separate console)
# docker build –t ex-py-docker-flask . ... ...<many lines of output> ...Successfully built 0fb21b16f3dd#
# docker run –p 5000:5000 ex-py-docker-flask * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger pin code: 236-035-556
# curl http://localhost:5000-- Hello Flask Dockerized –-#
run outside localhost(default port: 5000)
17
K8s App — Deploy
Hosted K8s – Google Container EngineLocal “laptop” – Minikube ... (from the construct stage … mostly) ...
# docker build –t gcr.io/<my-proj-id>/ex-py-flask:v1 . ...# gcloud docker push gcr.io/<my-proj-id>/ex-py-flask:v1
# minikube startStarting local Kubernetes cluster...Kubernetes is available at https://192.168.99.100:8443.Kubectl is now configured to use the cluster.# eval $(minikube docker-env)# docker build –t library/ex-py-docker-flask .
Caveat: the method used above is a bit of a “hack”. Using the ‘docker-env’ combined with ‘docker build’ works because Minikube only deploys into a single host. As a consequence the Docker image will be available in the local Docker repository. If Minikube ran across two or more hosts then the node Kubernetes choses to run the Pod (container) on may not match where it was built.
*http://kubernetes.io/docs/hellonode/
GCR Convention
(alternate)
18
K8s App — Run
Hosted K8s – Google Container Engine Local “laptop” – Minikube
# kubectl run flask-node \ -–image=gcr.io/<my-proj-id>/ex-py-flask:v1 \ --port=5000Deployment “flask-node” created# kubectl get podsNAME READY STATUS RESTARTS AGEflask-node-714049816-ztzrb 1/1 Running 0 6m# kubectl expose deployment flask-node -–type=“LoadBalancer”# kubectl get services flask-nodeNAME CLUSTER_IP EXTERNAL_IP PORT(S) AGEhello-node 10.3.246.12 23.251.159.72 5000/TCP 2m
Run
Verify
Run
Verify
# curl http://23.251.159.72:5000-- Hello Flask Dockerized –#
1.
2.
3.
4.
# kubectl run flask-node \ -–image=library/ex-py-docker-flask \ --port=5000Deployment “flask-node” created# kubectl get podsNAME READY STATUS RESTARTS AGEflask-node-714049816-ztzrb 1/1 Running 0 6m# kubectl expose deployment flask-node -–type=“NodePort”
1.
2.
3.
# minikube service flask-node –-urlhttp://192.168.99.100:31992# curl $(minikube service flask-node –-url)-- Hello Flask Dockerized –#
19
Getting Involved
Community http://kubernetes.io/community/
GitHub http://github.com/kubernetes
Project Page & Documents http://kubernetes.io
Slack (chat) (sign-up: http://slack.k8s.io/) https://kubernetes.slack.com
Special Interest Groups (SIGs) (+20 topics)
Community Page SIGs (https://github.com/kubernetes/community/blob/master/README.md#special-interest-groups-sig)
20
Demo
https://github.com/egustafson/ex-gke-webdrop https://github.com/egustafson/webdrop-py
21
Thank youAdvanced Technology Group for Open Source and Cloud
Eric Gustafson [email protected] Galbraith [email protected] Springer [email protected]
22
Backup Slides(Kubernetes Introduction)
Advanced Technology Group for Open Source & CloudHPE's Advanced Technology Group for Open Source & Cloud embraces a vision that is two steps ahead of today's solutions. We use this vision to drive product adoption and incubate technologies to advance HPE. Through open source initiatives we foster collaboration across HPE and beyond.
23
Patrick [email protected]://patg.net/
Interests: Kubernetes, Ansible, MySQL projects
New Hampshire, USA
Eric [email protected]://egustafson.github.io/
Interests: Monitoring, Networking, Embedded/IoT
Colorado, USA
Brian Aker, Fellow
Yazz Atlas, Principle Engineer
Hillary Cirimele, Executive Assistant
Matt Farina, Principle Engineer
Patrick Galbraith, Principle Engineer
Eric Gustafson, Principle Engineer
Clare Springer, Program Manager
24
References – Kubernetes Introduction
• “Large-scale cluster management at Google with Borg”• https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43438.pdf
• “Omega: flexible, scalable schedulers for large compute clusters”• https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41684.pdf
• “Borg, Omega, and Kubernetes”• https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44843.pdf
• “Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network”• http://conferences.sigcomm.org/sigcomm/2015/pdf/papers/p183.pdf