Kubeadm init流程

1.引导前检查（pre-flight checks）

kubeadm init执行之后，首先会进行pre-flight checks检查，确保master节点可以满足master组件可以安装的所有条件

error级别的检查：

kubeadm版本要与kubernetes版本的对比检查，kubeadm版本不小于kubernetes版本
kubernetes安装的系统需求检查，内核版本需大于3.10以上，4.x以上，是否设置了cgroups子系统，后端服务是否正常工作
其他检查：用户、主机、端口、swap、工具等

2.生成私钥和数字证书

kubeadm会为整个集群生成多组私钥和数字证书，如果不指定外部的证书授权机构，kubeadm init会自建证书授权机构，并生成ca的私钥（ca.key）、自签署的公钥数字证书（ca.crt），用于签发后续kubernetes需要的其他公钥证书

kubeadm init生成的证书都在/etc/kubernetes/pki目录下

自建CA

如果不指定外部的证书授权机构，kubeadm init会自建证书授权机构，并生成ca的私钥（ca.key）、自签署的公钥数字证书（ca.crt），用于签发后续kubernetes需要的其他公钥证书

查看ca.crt证书内容

[root@k8s-master pki]# openssl x509 -in ca.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 0 (0x0)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=kubernetes
        Validity
            Not Before: Jan 14 02:37:42 2021 GMT
            Not After : Jan 12 02:37:42 2031 GMT
        Subject: CN=kubernetes
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:e1:9d:a8:43:d5:8f:74:99:40:21:df:88:a9:7b:
                    cf:37:24:00:fc:87:73:3d:29:0c:5c:f1:53:46:40:
                    56:16:97:fb:a4:c8:fb:2e:23:74:bc:18:e0:c6:c8:
                    7e:d5:35:49:c6:23:5b:1a:82:80:ed:90:be:c2:2b:
                    0f:94:fc:27:d3:7f:f7:3f:80:be:f9:e4:6e:64:d4:
                    dc:1a:49:d0:f2:64:4b:45:6f:a3:9e:e3:c1:c9:a5:
                    a0:20:51:65:b2:78:85:09:00:49:f6:ae:07:3a:e0:
                    e1:c4:b5:10:fd:7b:fc:e3:c5:83:86:77:22:f8:34:
                    26:df:c6:8d:7e:51:b4:cf:24:3e:08:9f:1e:6d:dd:
                    85:99:16:6f:3f:f0:ea:78:6c:4f:15:e7:4a:40:03:
                    0b:ab:a0:6f:c5:e1:b9:3b:43:69:94:44:09:88:fe:
                    7d:48:6e:6c:ec:8e:7b:31:4a:e3:dd:c0:1d:48:33:
                    05:93:5b:ec:9b:18:e9:a0:d1:bd:31:7b:60:0b:4e:
                    6f:74:d6:5d:5c:7b:70:ce:43:ba:cd:7c:4c:93:32:
                    db:e4:58:3d:73:11:bc:66:53:6c:a7:57:17:66:95:
                    6e:12:90:5e:5f:5b:f6:a6:9e:9c:bc:45:40:e6:50:
                    5c:b4:56:6b:a1:e8:55:e7:2f:29:20:fe:37:74:d0:
                    fa:b7
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment, Certificate Sign
            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
         88:d5:b9:ed:94:25:bd:57:a8:f0:da:eb:ec:98:6b:c0:f0:ee:
         0b:27:a9:bd:52:3a:2e:35:78:ee:35:a1:a0:a3:a7:52:8a:f6:
         52:bd:a2:b4:f7:bc:2b:39:2c:a3:a8:9d:be:02:3a:fd:79:cc:
         32:aa:a4:44:fc:85:91:d2:89:88:20:fd:8f:e0:de:19:23:78:
         cd:f5:7c:ae:fd:41:c6:bd:e3:e1:27:43:b5:74:00:f7:ae:c6:
         a5:25:7d:f9:d6:e4:4e:c6:7c:94:9b:f0:38:f3:53:be:8b:35:
         68:20:7e:0e:b7:ad:96:47:3b:6b:e3:4b:10:47:df:b3:c3:44:
         4e:58:91:82:e0:4d:b3:b2:6d:49:44:d0:cc:2d:8b:66:63:dc:
         37:67:44:0c:ca:51:ea:ca:ac:bf:8b:60:dc:b3:d8:fd:7b:4f:
         8a:c9:9f:4b:32:37:bb:83:5c:af:8b:59:f5:32:35:b0:3d:82:
         c3:50:af:5f:6c:9b:79:98:7c:1d:07:21:57:dc:fb:92:6a:54:
         47:24:e6:2c:24:c4:80:17:21:a5:ac:90:e3:f4:54:ea:d0:2f:
         6c:44:84:9e:51:62:66:76:e9:91:32:c4:a8:cc:12:ff:43:43:
         4e:16:71:22:d6:c2:5d:7d:bb:f1:ff:3c:a7:ce:cc:38:29:ca:
         fb:ca:44:98

apiserver私钥和公钥证书

kubeadm会生成apiserver的私钥文件（apiserver.key），用ca来签发apiserver的公钥数字证书（apiserver.crt）

查看apiserver.crt证书内容

[root@k8s-master pki]# openssl x509 -in apiserver.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 6548373687643854637 (0x5ae07f2c9486d72d)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=kubernetes
        Validity
            Not Before: Jan 14 02:37:42 2021 GMT
            Not After : Jan 14 02:37:43 2022 GMT
        Subject: CN=kube-apiserver
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:cd:8c:87:bf:18:0a:61:b4:51:d3:22:20:e3:72:
                    42:4e:ac:32:cc:d2:0a:41:3c:f0:70:eb:34:3c:09:
                    0a:68:a4:f5:6d:c6:0a:8d:5f:93:dc:dd:9c:9f:68:
                    21:97:f9:3d:95:da:4a:2f:69:82:ae:35:f1:e2:c4:
                    cd:78:3a:98:55:9a:0f:24:d5:ae:19:92:c6:d7:8b:
                    ff:5f:81:99:a6:52:0c:c3:9c:eb:d3:e9:91:47:f9:
                    cf:51:1e:6b:a5:83:0d:c0:37:e0:61:3d:b7:ee:09:
                    02:e9:25:b4:ef:2d:58:9b:e7:2a:b5:74:bf:e8:a1:
                    62:f4:8c:ae:58:25:b3:bc:a8:c3:af:58:db:25:e6:
                    85:14:2e:39:70:86:1f:c4:27:70:50:6c:05:85:aa:
                    e5:8e:9b:1c:55:b0:95:96:25:e7:df:f2:f6:42:be:
                    6e:ed:85:46:72:c7:db:64:e2:23:d7:ca:38:e9:9a:
                    24:7b:de:1b:ca:e3:b4:ac:c2:41:fa:bc:af:88:7e:
                    0c:15:72:f7:f7:2b:3a:a9:fd:7a:57:1d:95:2c:c9:
                    61:c6:48:ac:cf:2d:db:62:4d:41:be:e6:69:01:a0:
                    97:5f:0f:66:b7:c3:3d:2f:12:84:3f:78:fc:87:40:
                    8c:74:7e:23:4c:c0:f0:c8:f0:9e:1a:4f:1b:7e:a5:
                    db:3f
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Subject Alternative Name: 
                DNS:k8s-master, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, IP Address:10.1.0.1, IP Address:192.168.83.128
    Signature Algorithm: sha256WithRSAEncryption
         41:04:f1:f1:ec:03:61:30:1e:57:4e:bf:d5:f1:29:0e:ab:99:
         68:3d:51:0e:d4:e4:98:d3:87:4f:9d:4c:d5:3a:50:73:0a:87:
         49:a6:fe:a6:99:34:4f:24:40:ca:ee:a8:37:bf:09:3c:c3:d8:
         36:f4:f6:9f:29:ad:e8:71:90:aa:1d:cf:fc:db:4a:50:9b:3d:
         c4:3f:e5:f8:07:4f:d3:9a:c4:7f:50:ca:2f:92:65:42:e5:b4:
         27:7e:41:bb:87:17:5a:1c:01:e0:f7:70:38:59:a6:f3:e5:03:
         07:a6:c7:de:03:b9:32:e7:f7:79:8d:ed:d6:3d:0f:6e:e2:79:
         a4:22:a7:50:95:d4:83:8e:42:fa:59:10:02:cd:82:80:27:16:
         80:e0:45:99:b0:85:60:76:d7:e9:07:ed:cc:68:c5:80:b0:82:
         6a:b9:24:b9:3c:b9:4d:5e:88:a6:54:7a:45:79:35:46:5e:68:
         7b:77:f2:78:5c:3c:60:79:63:de:96:79:c3:c6:85:3b:ef:d2:
         3f:02:de:70:d7:f6:5c:4d:64:fb:ad:f9:31:62:ae:1b:ae:8c:
         b6:b6:cf:9e:cc:ac:a9:3a:b5:f4:63:6d:80:95:c8:79:e8:d9:
         8f:88:1d:80:15:97:58:c6:65:ce:98:4b:b1:6c:1b:29:ec:60:
         8c:39:65:b5

apiserver访问kubelet使用的客户端私钥与证书

apiserver需要向各个node的kubelet主动发起连接，kubelet会通过client端的ssl证书，校验apiserver建立的连接

apiserver-kubelet-client.crt和apiserver-kubelet-client.key用来校验apiserver身份的数字证书

serviceaccount私钥和公钥

sa.key和sa.pub是一对数字证书

Etcd相关私钥和数字证书

首先etcd是整个k8s集群的数据中心，只有apiserver才可以访问，所有组件都是通过apiserver存储数据的，为了建立apiserver和etcd的安全通信，kubeadm init会生成apiserver访问etcd的安全证书，即apiserver-etcd-client.crt和apiserver-etcd-client.key，其中apiserver-etcd-client.crt并不是由ca.crt签发的，而是有pki/etcd/ca.crt签发

# 并不是由/etc/kubernetes/pki/ca.crt签发
[root@k8s-master pki]# openssl verify -CAfile ca.crt apiserver-etcd-client.crt 
apiserver-etcd-client.crt: O = system:masters, CN = kube-apiserver-etcd-client
error 20 at 0 depth lookup:unable to get local issuer certificate
# 是由/etc/kubernetes/pki/etcd/ca.crt签发
[root@k8s-master pki]# openssl verify -CAfile etcd/ca.crt apiserver-etcd-client.crt 
apiserver-etcd-client.crt: OK

3.生成控制平面组件kubeconfig文件，这些配置用于组件间的通信鉴权

这些文件生成在/etc/kubernetes目录下

kubelet.conf：被kubelet组件使用，用于访问apiserver
scheduler.conf：被scheduler组件使用，用于访问apiserver
controller-manager.conf：被controller-manager组件使用，用于访问apiserver
admin.conf：包含整个集群的最高权限配置数据

admin.conf中包含cluster、user和context信息

[root@k8s-master kubernetes]# cat admin.conf 
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: 
    server: https://192.168.83.128:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: 
    client-key-data:

一旦配置 $KUBECONFIG 变量，指向/etc/kubernetes/admin.conf，则kubectl就会使用KUBECONFIG变量所配置的信息。contexts用来将users和clusters绑定，从而通过切换context来变换不通身份管理不同集群

4.生成控制平面组件manifest文件

这些文件会被master节点的kubelet读取，并且启动控制平面组件，并维护控制平面组件状态

这些文件存在/etc/kubernetes/manifests目录下

5.下载镜像，等待控制平面组件启动

kubeadm会依赖kubelet下载镜像并且启动static pod，kubelet会一直探测 localhost:6443/healthz（apiserver存活性探针）

查看 kube-apiserver.yaml

...
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: 192.168.83.128
        path: /healthz
        port: 6443
        scheme: HTTPS
      initialDelaySeconds: 15
      timeoutSeconds: 15
...