Docker¾Kubernetes¾Neutron 5 · Docker 5 - 5O² & d b

www.chinastor.com中国存储网

Docker¾Kubernetes¾Neutron �5�®º2

UnitedStack�� 5�


http://www.chinastor.com/a/jishu/docker/0301230552016.html

Docker �5 - 5O² & d�b<U - libnetwork

Kubernetes �5 - 5O² & d�b<U - service & kube-proxy - pause`�

Neutron �5 - z?¦² - ML2 & d�b<U

�h*�:�


Docker��I - �I K & +:)�#

4

Linux Bridge


Docker��I - �I K & +:)�#

5

`�J5 or J5`�

`�`�


Docker��I - �I K & +:)�#

6

��¯ Docker�Ã

[root@dev)~]#)systemctl)status))docker.service)docker.service)7)Docker)Application)Container)Engine))))Loaded:)loaded)(/usr/lib/systemd/system/docker.service;)disabled)))))Active:)inactive)(dead)))))))Docs:)http://docs.docker.com)))[root@dev)~]#)brctl)show)bridge)name)bridge)id)))))))STP)enabled)interfaces)virbr0))))))8000.52540027e8bc)))yes)))))virbr07nic)

[root@dev)~]#)iptables7save)#)Generated)by)iptables7save)v1.4.21)on)Thu)Aug))6)13:40:20)2015)*nat):PREROUTING)ACCEPT)[13:997]):INPUT)ACCEPT)[0:0]):OUTPUT)ACCEPT)[0:0]):POSTROUTING)ACCEPT)[0:0])COMMIT)#)Completed)on)Thu)Aug))6)13:40:20)2015)#)Generated)by)iptables7save)v1.4.21)on)Thu)Aug))6)13:40:20)2015)*filter):INPUT)ACCEPT)[0:0]):FORWARD)ACCEPT)[0:0]):OUTPUT)ACCEPT)[44:5440])7A)INPUT)7m)state)77state)RELATED,ESTABLISHED)7j)ACCEPT)7A)INPUT)7p)icmp)7j)ACCEPT)7A)INPUT)7i)lo)7j)ACCEPT)7A)INPUT)7p)tcp)7m)state)77state)NEW)7m)tcp)77dport)22)7j)ACCEPT)7A)INPUT)7j)REJECT)77reject7with)icmp7host7prohibited)7A)FORWARD)7j)REJECT)77reject7with)icmp7host7prohibited)COMMIT)#)Completed)on)Thu)Aug))6)13:40:20)2015


Docker��I

7

Docker��I - �I K & +:)�#

7

¯ Docker�Ã

[root@dev)~]#)service)docker)start)Redirecting)to)/bin/systemctl)start))docker.service)[root@dev)~]#)brctl)show)bridge)name)bridge)id)))))))STP)enabled)interfaces)docker0(((((8000.56847afe9799(((no()))))virbr0))))))8000.52540027e8bc)))yes)))))virbr07nic)[root@dev)~]#)ip)l)show)dev)docker0)8:)docker0:)<NO7CARRIER,BROADCAST,MULTICAST,UP>)mtu)1500)qdisc)noqueue)state)DOWN)mode)DEFAULT)))))link/ether)56:84:7a:fe:97:99)brd)ff:ff:ff:ff:ff:ff

[root@dev)~]#)iptables7save)#)Generated)by)iptables7save)v1.4.21)on)Thu)Aug))6)13:41:52)2015)*nat):PREROUTING)ACCEPT)[22:1779]):INPUT)ACCEPT)[0:0]):OUTPUT)ACCEPT)[0:0]):POSTROUTING)ACCEPT)[0:0]):DOCKER(:([0:0](:A(PREROUTING(:m(addrtype(::dst:type(LOCAL(:j(DOCKER(:A(OUTPUT(!(:d(127.0.0.0/8(:m(addrtype(::dst:type(LOCAL(:j(DOCKER(:A(POSTROUTING(:s(172.17.0.0/16(!(:o(docker0(:j(MASQUERADE(COMMIT)#)Completed)on)Thu)Aug))6)13:41:52)2015)#)Generated)by)iptables7save)v1.4.21)on)Thu)Aug))6)13:41:52)2015)*filter):INPUT)ACCEPT)[0:0]):FORWARD)ACCEPT)[0:0]):OUTPUT)ACCEPT)[18:2944]):DOCKER(:([0:0](7A)INPUT)7m)state)77state)RELATED,ESTABLISHED)7j)ACCEPT)7A)INPUT)7p)icmp)7j)ACCEPT)7A)INPUT)7i)lo)7j)ACCEPT)7A)INPUT)7p)tcp)7m)state)77state)NEW)7m)tcp)77dport)22)7j)ACCEPT)7A)INPUT)7j)REJECT)77reject7with)icmp7host7prohibited):A(FORWARD(:o(docker0(:j(DOCKER(:A(FORWARD(:o(docker0(:m(conntrack(::ctstate(RELATED,ESTABLISHED(:j(ACCEPT(:A(FORWARD(:i(docker0(!(:o(docker0(:j(ACCEPT(:A(FORWARD(:i(docker0(:o(docker0(:j(ACCEPT(7A)FORWARD)7j)REJECT)77reject7with)icmp7host7prohibited)COMMIT)#)Completed)on)Thu)Aug))6)13:41:52)2015

�� mangle-PREROUTING -> nat-PREROUTING -> mangle-INPUT -> filter-INPUT

�� mangle-OUTPUT -> nat-OUTPUT -> filter-OUTPUT -> mangle-POSTROUTING -> nat-POSTROUTING

�� mangle-PREROUTING -> nat-PREROUTING -> mangle-FORWARD -> filter-FORWARD -> mangle-POSTROUTING -> nat-POSTROUTING


Docker��I - �I K & +:)�#

8

[root@dev)~]#)docker)run)7dit)7p)8888:80)77name)test7os2)docker.io/centos)/bin/bash

[root@dev)~]#)iptables7save)#)Generated)by)iptables7save)v1.4.21)on)Thu)Aug))6)14:29:18)2015)*mangle):PREROUTING)ACCEPT)[2644:222950]):INPUT)ACCEPT)[2588:218246]):FORWARD)ACCEPT)[56:4704]):OUTPUT)ACCEPT)[393:75016]):POSTROUTING)ACCEPT)[449:79720])COMMIT)#)Completed)on)Thu)Aug))6)14:29:18)2015)#)Generated)by)iptables7save)v1.4.21)on)Thu)Aug))6)14:29:18)2015)*nat):PREROUTING)ACCEPT)[38:2854]):INPUT)ACCEPT)[0:0]):OUTPUT)ACCEPT)[1:55]):POSTROUTING)ACCEPT)[1:55]):DOCKER)7)[0:0]):A(PREROUTING(:m(addrtype(::dst:type(LOCAL(:j(DOCKER(7A)OUTPUT)!)7d)127.0.0.0/8)7m)addrtype)77dst7type)LOCAL)7j)DOCKER)7A)POSTROUTING)7s)172.17.0.0/16)!)7o)docker0)7j)MASQUERADE)7A)POSTROUTING)7s)172.17.0.6/32)7d)172.17.0.6/32)7p)tcp)7m)tcp)77dport)80)7j)MASQUERADE):A(DOCKER(!(:i(docker0(:p(tcp(:m(tcp(::dport(8888(:j(DNAT(::to:destination(172.17.0.6:80(COMMIT)#)Completed)on)Thu)Aug))6)14:29:18)2015)#)Generated)by)iptables7save)v1.4.21)on)Thu)Aug))6)14:29:18)2015)*filter):INPUT)ACCEPT)[0:0]):FORWARD)ACCEPT)[0:0]):OUTPUT)ACCEPT)[29:2822]):DOCKER)7)[0:0])7A)INPUT)7m)state)77state)RELATED,ESTABLISHED)7j)ACCEPT)7A)INPUT)7p)icmp)7j)ACCEPT)7A)INPUT)7i)lo)7j)ACCEPT)7A)INPUT)7p)tcp)7m)state)77state)NEW)7m)tcp)77dport)22)7j)ACCEPT)7A)INPUT)7j)REJECT)77reject7with)icmp7host7prohibited)7A)FORWARD)7o)docker0)7j)DOCKER)7A)FORWARD)7o)docker0)7m)conntrack)77ctstate)RELATED,ESTABLISHED)7j)ACCEPT)7A)FORWARD)7i)docker0)!)7o)docker0)7j)ACCEPT)7A)FORWARD)7i)docker0)7o)docker0)7j)ACCEPT)7A)FORWARD)7j)REJECT)77reject7with)icmp7host7prohibited):A(DOCKER(:d(172.17.0.6/32(!(:i(docker0(:o(docker0(:p(tcp(:m(tcp(::dport(80(:j(ACCEPT(COMMIT)#)Completed)on)Thu)Aug))6)14:29:18)2015

�� mangle-PREROUTING -> nat-PREROUTING -> mangle-INPUT -> filter-INPUT

�� mangle-OUTPUT -> nat-OUTPUT -> filter-OUTPUT -> mangle-POSTROUTING -> nat-POSTROUTING

�� mangle-PREROUTING -> nat-PREROUTING -> mangle-FORWARD -> filter-FORWARD -> mangle-POSTROUTING -> nat-POSTROUTING


Docker��I - libnetwork

9

��package

��³vÃ Sandbox¾Endpoint¾Network

e�±��ÀO²?ÁÃ NetworkController¾Driver¾Network¾Endpoint¾Sandbox

[root@dev)src]#)cat)bingo/bingo.go)package)main)import)()))))"fmt")))))"libnetwork")))func)main()){)))))controller,)err):=)libnetwork.New())))))if)err)!=)nil){)))))))))fmt.Println(err))))))})))))fmt.Println(controller))}



10

[root@dev drivers]# pwd /root/projects/libnetwork/drivers [root@dev drivers]# ll F�K 8 drwxr-xr-x. 2 root root 4096 7G 29 14:49 bridge drwxr-xr-x. 2 root root 39 7G 23 14:41 host drwxr-xr-x. 2 root root 39 7G 22 11:33 null drwxr-xr-x. 2 root root 4096 7G 29 14:13 overlay drwxr-xr-x. 2 root root 61 7G 22 11:33 remote drwxr-xr-x. 2 root root 23 7G 20 12:13 windows

// Driver is an interface that every plugin driver needs to implement. type Driver interface { // Push driver specific config to the driver Config(options map[string]interface{}) error

// CreateNetwork invokes the driver method to create a network passing // the network id and network specific config. The config mechanism will // eventually be replaced with labels which are yet to be introduced. CreateNetwork(nid types.UUID, options map[string]interface{}) error

// DeleteNetwork invokes the driver method to delete network passing // the network id. DeleteNetwork(nid types.UUID) error

// CreateEndpoint invokes the driver method to create an endpoint // passing the network id, endpoint id endpoint information and driver // specific config. The endpoint information can be either consumed by // the driver or populated by the driver. The config mechanism will // eventually be replaced with labels which are yet to be introduced. CreateEndpoint(nid, eid types.UUID, epInfo EndpointInfo, options map[string]interface{}) error

// DeleteEndpoint invokes the driver method to delete an endpoint // passing the network id and endpoint id. DeleteEndpoint(nid, eid types.UUID) error

// EndpointOperInfo retrieves from the driver the operational data related to the specified endpoint EndpointOperInfo(nid, eid types.UUID) (map[string]interface{}, error)

// Join method is invoked when a Sandbox is attached to an endpoint. Join(nid, eid types.UUID, sboxKey string, jinfo JoinInfo, options map[string]interface{}) error

// Leave method is invoked when a Sandbox detaches from an endpoint. Leave(nid, eid types.UUID) error

// Type returns the the type of this driver, the network type this driver manages Type() string }

o/f«�Driver

Driver�TV0�

CreateNetwork -> ioctlCreateBridge -> syscall.Syscall(..., ioctlBrAdd, ...)



11

1.daemon��[gkl d.netController, err = initNetworkController(config) if err != nil { return nil, fmt.Errorf("Error initializing network controller: %v", err) }

2.CMDA\ [root@dev client]# pwd /root/projects/docker/api/client [root@dev client]# grep Cmd ./* | grep func | cut -f4 -d' ' | cut -f1 -d'(' | sort | uniq CmdNetwork …… import ( "os" nwclient "github.com/docker/libnetwork/client" ) func (cli *DockerCli) CmdNetwork(args ...string) error { nCli := nwclient.NewNetworkCli(cli.out, cli.err, nwclient.CallFunc(cli.callWrapper)) args = append([]string{"network"}, args...) return nCli.Cmd(os.Args[0], args...) }

Docker �y0Nlibnetworkkl

3. API� 1�^ func createRouter(s *Server) *mux.Router { r := mux.NewRouter() if os.Getenv("DEBUG") != "" { ProfilerSetup(r, "/debug/") } m := map[string]map[string]HttpApiFunc{ "GET": { "/_ping": s.ping, "/events": s.getEvents, "/info": s.getInfo, "/version": s.getVersion,

…………


Kubernetes��I - �I K & +:)�#

12


Kubernetes��I - �I K & +:)�#

13

vxlan½�

vxlan½�

172.17.10.1/24

172.17.20.1/24

172.17.30.1/24

172.17.20.100

172.17.30.100

10.10.10.10

10.10.10.11

10.10.10.12

10.10.10.13

�UOS��"��s¼Ãhttp://bingotree.cn/?p=828

j)Cm��

ping 172.17.20.100


http://bingotree.cn/?p=828

Kubernetes��I - service & kube-proxy

14

serviceÃ�,f«VIP kube-proxyÃ¢¤service9;�iptables��f«proxy

pC1

pC2VIP:Port -> proxy:random port�DNAT

X�1��Dping VIP


Kubernetes��I - pause(=

15

�Podf«��5�Namespace¿#�Pod��¿ Dockerfile pause©&(E1�


Neutron��I - 3�FK

16

Agent!�Ã DHCP¾L3¾OVS¾SR-IOV¾i!SDN Agent……


Neutron��I - ML2 & +:)�#

17

The Modular Layer 2 (ml2) plugin ��CORE PLUGINÂ~¡NeutronPluginBaseV2 8R75��TYPE(VLAN/VXLAN/GRE...)��¬Â4Zxc�!Mechanism Driver

17



1818

��

��

VXLAN�@��

ªE$¥´P�@��

{�%

nL%

�H%

£L%



191919

n-cpu,q-agt

�»�E¾W�

n-cpu,q-agt,q-sriov-agt

q-dhcp,q-l3,q-agt



202020

n-cpu,q-agt

�»�E¾W�

n-cpu,q-agt,q-sriov-agt

q-dhcp,q-l3,q-agt

.·w¹�5d�<U


�-��9

212121

�.� �M0 !��D� �� ,�I/

�'L81*��

��EH 5 �"

Docker�I 7�$�&.�

A 1* NA �1* J&�OL8;>�B

�1* &�

K8s�I � A 1* 1* �1* 1*OVSNCalico�

�41* !$PaaS%2

Neutron � � 1* 1* 1* 1*OVS�?C�E6CG "�E

1*(<�ML2)

IaaS6<�IaaS��@%2

6/�}i]+=5$-��¨:�Ã


�-��9

222222

DockerÃ - f«libnetworkÂqw YTu�libnetworkB&59;��

Kubernetes: - �,�|t°uW�("etcd/zookeeeper')f«�Eµ¶ - �|t°uW�.�a�Qd�

Neutron: - d�¸a�Qd�Âd�¸�� - �,rMB&�E�µ¶��S


�-��9

232323

Docker

Kubernetes

Neutron……

iptables/NATroute

linux bridge/ovs……vlan/vxlan/i!tag......……

�Y�>� 3��>�

§_/uI� + �»�E

namespace




Documents

Docker¾Kubernetes¾Neutron 5 · Docker 5 - 5O² & d b