参考官方部署文档
节点规划:
kube-master:192.168.14.131
kube-node1:192.168.14.132
kube-node2:192.168.14.133

[root@hill-test ~]# cat /etc/redhat-release 
CentOS Linux release 7.3.1611 (Core) 

部署前环境准备

修改hosts

[root@hill-test ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.14.128 kube-master
192.168.14.129 kube-node1
192.168.14.130 kube-node2

关闭防火墙

[root@hill-test ~]# systemctl stop firewalld.service && systemctl disable firewalld.service
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.

关闭SElinux

[root@hill-test ~]# setenforce 0
setenforce: SELinux is disabled
[root@hill-test ~]# getenforce 
Disabled

配置yum源

也可以使用kubernetes提供的yum源

[virt7-docker-common-release]
name=virt7-docker-common-release
baseurl=http://cbs.centos.org/repos/virt7-docker-common-release/x86_64/os/
gpgcheck=0

kubernetes架构

kubernetes结构,图片来自这篇好文章,建议阅读。

510038228291.png

部署kubernetes

on master

安装组件kubernetes etcd flannel

安装kubernetes etcd flannel组件

  • Kubernetes软件包提供了一些服务:kube-apiserver,kube-scheduler, kube-controller-manager, kubelet, kube-proxy。这些服务由systemd管理,配置集中保存在/etc/kubernetes目录下
  • etcd是一个高可用的键值存储系统,主要用于共享配置和服务发现。etcd是由CoreOS开发并维护的,灵感来自于 ZooKeeper 和 Doozer,它使用Go语言编写,并通过Raft一致性算法处理日志复制以保证强一致性。
  • flannel是CoreOS团队针对Kubernetes设计的一个网络规划服务,简单来说,它的功能是让集群中的不同节点主机创建的Docker容器都具有全集群唯一的虚拟IP地址。Flannel的设计目的就是为集群中的所有节点重新规划IP地址的使用规则,从而使得不同节点上的容器能够获得“同属一个内网”且”不重复的”IP地址,并让属于不同节点上的容器能够直接通过内网IP通信。
[root@hill-test ~]# yum -y install --enablerepo=virt7-docker-common-release kubernetes etcd flannel

Installed:
  etcd.x86_64 0:3.2.7-1.el7                                     flannel.x86_64 0:0.7.1-2.el7                                     kubernetes.x86_64 0:1.5.2-0.7.git269f928.el7                                    

Dependency Installed:
  audit-libs-python.x86_64 0:2.7.6-3.el7                               checkpolicy.x86_64 0:2.5-4.el7                                        conntrack-tools.x86_64 0:1.4.4-3.el7_3                              
  container-selinux.noarch 2:2.28-1.git85ce147.el7                     container-storage-setup.noarch 0:0.7.0-1.git4ca59c5.el7               docker.x86_64 2:1.12.6-61.git85d7426.el7.centos                     
  docker-client.x86_64 2:1.12.6-61.git85d7426.el7.centos               docker-common.x86_64 2:1.12.6-61.git85d7426.el7.centos                kubernetes-client.x86_64 0:1.5.2-0.7.git269f928.el7                 
  kubernetes-master.x86_64 0:1.5.2-0.7.git269f928.el7                  kubernetes-node.x86_64 0:1.5.2-0.7.git269f928.el7                     libcgroup.x86_64 0:0.41-13.el7                                      
  libnetfilter_cthelper.x86_64 0:1.0.0-9.el7                           libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7                           libnetfilter_queue.x86_64 0:1.0.2-2.el7_2                           
  libseccomp.x86_64 0:2.3.1-3.el7                                      libsemanage-python.x86_64 0:2.5-8.el7                                 oci-register-machine.x86_64 1:0-3.13.gitcd1e331.el7                 
  oci-systemd-hook.x86_64 1:0.1.14-1.git1ba44c6.el7                    oci-umount.x86_64 2:2.0.0-1.git299e781.el7                            policycoreutils-python.x86_64 0:2.5-17.1.el7                        
  python-IPy.noarch 0:0.75-6.el7                                       setools-libs.x86_64 0:3.3.8-1.1.el7                                   skopeo-containers.x86_64 1:0.1.24-1.dev.git28d4e08.el7              
  socat.x86_64 0:1.7.3.2-2.el7                                         yajl.x86_64 0:2.0.4-4.el7                                            

Updated:
  dracut.x86_64 0:033-502.el7                                 selinux-policy-targeted.noarch 0:3.13.1-166.el7_4.5                                 systemd.x86_64 0:219-42.el7_4.4                                

Dependency Updated:
  audit.x86_64 0:2.7.6-3.el7                     audit-libs.x86_64 0:2.7.6-3.el7        dracut-config-rescue.x86_64 0:033-502.el7   dracut-network.x86_64 0:033-502.el7   libgudev1.x86_64 0:219-42.el7_4.4    
  libnetfilter_conntrack.x86_64 0:1.0.6-1.el7_3  libselinux.x86_64 0:2.5-11.el7         libselinux-python.x86_64 0:2.5-11.el7       libselinux-utils.x86_64 0:2.5-11.el7  libsemanage.x86_64 0:2.5-8.el7       
  libsepol.x86_64 0:2.5-6.el7                    policycoreutils.x86_64 0:2.5-17.1.el7  selinux-policy.noarch 0:3.13.1-166.el7_4.5  systemd-libs.x86_64 0:219-42.el7_4.4  systemd-sysv.x86_64 0:219-42.el7_4.4 

Complete!

配置kubernetes

编辑/etc/kubernetes/config ,修改master监听端口

[root@hill-test kubernetes]# cat  config 
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
#   kube-apiserver.service
#   kube-controller-manager.service
#   kube-scheduler.service
#   kubelet.service
#   kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://kube-master:8080"

配置etcd

编辑/etc/etcd/etcd.conf,修改所有IP监听

[root@hill-test etcd]# cat /etc/etcd/etcd.conf 
# [member]
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"

#[cluster]
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"

配置etcd保存network overlay配置

[root@hill-test ~]# etcdctl mkdir /kube-centos/network
[root@hill-test ~]# etcdctl mk /kube-centos/network/config "{ \"Network\": \"172.30.0.0/16\", \"SubnetLen\": 24, \"Backend\": { \"Type\": \"vxlan\" } }"
{ "Network": "172.30.0.0/16", "SubnetLen": 24, "Backend": { "Type": "vxlan" } }

配置kube-apiserver

编辑/etc/kubernetes/apiserver,修改绑定地址和etcd监听地址

[root@hill-test etcd]# cat /etc/kubernetes/apiserver 
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
KUBE_API_PORT="--port=8080"
KUBELET_PORT="--kubelet-port=10250"
KUBE_ETCD_SERVERS="--etcd-servers=http:// 0.0.0.0:2379"
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
KUBE_API_ARGS=""

配置flannel

编辑/etc/sysconfig/flanneld

[root@hill-test etcd]# cat /etc/sysconfig/flanneld 
FLANNEL_ETCD="http://kube-master:2379"
FLANNEL_ETCD_KEY="/kube-centos/network

配置kube-controller-manager

编辑/etc/kubernetes/controller-manager

启动所有组件

for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler flanneld; do
    systemctl restart $SERVICES
    systemctl enable $SERVICES
    systemctl status $SERVICES
done

on slaves/nodes

安装组件kubernetes flannel

master的一样:

[root@hill-test ~]# yum -y install --enablerepo=virt7-docker-common-release kubernetes flannel

Installed:
  etcd.x86_64 0:3.2.7-1.el7                                     flannel.x86_64 0:0.7.1-2.el7                                     kubernetes.x86_64 0:1.5.2-0.7.git269f928.el7                                    

Dependency Installed:
  audit-libs-python.x86_64 0:2.7.6-3.el7                               checkpolicy.x86_64 0:2.5-4.el7                                        conntrack-tools.x86_64 0:1.4.4-3.el7_3                              
  container-selinux.noarch 2:2.28-1.git85ce147.el7                     container-storage-setup.noarch 0:0.7.0-1.git4ca59c5.el7               docker.x86_64 2:1.12.6-61.git85d7426.el7.centos                     
  docker-client.x86_64 2:1.12.6-61.git85d7426.el7.centos               docker-common.x86_64 2:1.12.6-61.git85d7426.el7.centos                kubernetes-client.x86_64 0:1.5.2-0.7.git269f928.el7                 
  kubernetes-master.x86_64 0:1.5.2-0.7.git269f928.el7                  kubernetes-node.x86_64 0:1.5.2-0.7.git269f928.el7                     libcgroup.x86_64 0:0.41-13.el7                                      
  libnetfilter_cthelper.x86_64 0:1.0.0-9.el7                           libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7                           libnetfilter_queue.x86_64 0:1.0.2-2.el7_2                           
  libseccomp.x86_64 0:2.3.1-3.el7                                      libsemanage-python.x86_64 0:2.5-8.el7                                 oci-register-machine.x86_64 1:0-3.13.gitcd1e331.el7                 
  oci-systemd-hook.x86_64 1:0.1.14-1.git1ba44c6.el7                    oci-umount.x86_64 2:2.0.0-1.git299e781.el7                            policycoreutils-python.x86_64 0:2.5-17.1.el7                        
  python-IPy.noarch 0:0.75-6.el7                                       setools-libs.x86_64 0:3.3.8-1.1.el7                                   skopeo-containers.x86_64 1:0.1.24-1.dev.git28d4e08.el7              
  socat.x86_64 0:1.7.3.2-2.el7                                         yajl.x86_64 0:2.0.4-4.el7                                            

Updated:
  dracut.x86_64 0:033-502.el7                                 selinux-policy-targeted.noarch 0:3.13.1-166.el7_4.5                                 systemd.x86_64 0:219-42.el7_4.4                                

Dependency Updated:
  audit.x86_64 0:2.7.6-3.el7                     audit-libs.x86_64 0:2.7.6-3.el7        dracut-config-rescue.x86_64 0:033-502.el7   dracut-network.x86_64 0:033-502.el7   libgudev1.x86_64 0:219-42.el7_4.4    
  libnetfilter_conntrack.x86_64 0:1.0.6-1.el7_3  libselinux.x86_64 0:2.5-11.el7         libselinux-python.x86_64 0:2.5-11.el7       libselinux-utils.x86_64 0:2.5-11.el7  libsemanage.x86_64 0:2.5-8.el7       
  libsepol.x86_64 0:2.5-6.el7                    policycoreutils.x86_64 0:2.5-17.1.el7  selinux-policy.noarch 0:3.13.1-166.el7_4.5  systemd-libs.x86_64 0:219-42.el7_4.4  systemd-sysv.x86_64 0:219-42.el7_4.4 

Complete!

配置kubernetes

编辑/etc/kubernetes/config ,修改master监听端口

[root@hill-test kubernetes]# cat  config 
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
#   kube-apiserver.service
#   kube-controller-manager.service
#   kube-scheduler.service
#   kubelet.service
#   kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://kube-master:8080"

配置kubelet

编辑配置文件/etc/kubernetes/kubelet

[root@hill-test ~]# cat /etc/kubernetes/kubelet 
KUBELET_ADDRESS="--address=0.0.0.0"
KUBELET_PORT="--port=10250"
# hostname-override即工作节点主机名,如kube-nodeN。
KUBELET_HOSTNAME="--hostname-override=kube-node1"
KUBELET_API_SERVER="--api-servers=http://kube-master:8080"
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
KUBELET_ARGS=""

配置flannel

编辑/etc/sysconfig/flanneld

[root@hill-test etcd]# cat /etc/sysconfig/flanneld 
FLANNEL_ETCD="http://kube-master:2379"
FLANNEL_ETCD_KEY="/kube-centos/network

启动组件服务

for SERVICES in kube-proxy kubelet flanneld docker; do
    systemctl restart $SERVICES
    systemctl enable $SERVICES
    systemctl status $SERVICES
done

配置kubectl将node加入集群

[root@hill-test ~]# kubectl config set-cluster default-cluster --server=http://kube-master:8080
Cluster "default-cluster" set.
[root@hill-test ~]# kubectl config set-context default-context --cluster=default-cluster --user=default-admin
Context "default-context" set.
[root@hill-test ~]# kubectl config use-context default-context
Switched to context "default-context".

查看node信息

[root@hill-test ~]# kubectl get nodes
NAME         STATUS    AGE
kube-node1   Ready     51m
kube-node2   Ready     3m

集群已经搭建完成。