1.在Master01节点上查看节点是否正常,确定都是Ready

$ kubectl get node
NAME           STATUS   ROLES                  AGE   VERSION
k8s-master01   Ready    control-plane,master   10h   v1.23.17
k8s-master02   Ready    control-plane,master   10h   v1.23.17
k8s-master03   Ready    control-plane,master   10h   v1.23.17
k8s-node01     Ready    <none>                 10h   v1.23.17
k8s-node02     Ready    <none>                 10h   v1.23.17

2.在Master01节点上查看所有Pod是否正常,确定READY都是N/N形式的且STATUS 都为Running

$ kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-6f6595874c-tntnr 1/1 Running 0 87m kube-system calico-node-5mj9g 1/1 Running 1 (79m ago) 87m kube-system calico-node-hhjrv 1/1 Running 2 (79m ago) 87m kube-system calico-node-szjm7 1/1 Running 0 87m kube-system calico-node-xcgwq 1/1 Running 0 87m kube-system calico-node-ztbkj 1/1 Running 1 (79m ago) 87m kube-system calico-typha-6b6cf8cbdf-8qj8z 1/1 Running 0 87m kube-system coredns-65c54cc984-nrhlg 1/1 Running 0 10h kube-system coredns-65c54cc984-xkx7w 1/1 Running 0 10h kube-system etcd-k8s-master01 1/1 Running 1 (108m ago) 10h kube-system etcd-k8s-master02 1/1 Running 1 (108m ago) 10h kube-system etcd-k8s-master03 1/1 Running 1 (107m ago) 10h kube-system kube-apiserver-k8s-master01 1/1 Running 1 (108m ago) 10h kube-system kube-apiserver-k8s-master02 1/1 Running 1 (108m ago) 10h kube-system kube-apiserver-k8s-master03 1/1 Running 2 (107m ago) 10h kube-system kube-controller-manager-k8s-master01 1/1 Running 2 (108m ago) 10h kube-system kube-controller-manager-k8s-master02 1/1 Running 1 (108m ago) 10h kube-system kube-controller-manager-k8s-master03 1/1 Running 1 (107m ago) 10h kube-system kube-proxy-2kz9g 1/1 Running 0 11m kube-system kube-proxy-b54gh 1/1 Running 0 11m kube-system kube-proxy-kclcc 1/1 Running 0 11m kube-system kube-proxy-pv8gc 1/1 Running 0 11m kube-system kube-proxy-xt52m 1/1 Running 0 11m kube-system kube-scheduler-k8s-master01 1/1 Running 2 (108m ago) 10h kube-system kube-scheduler-k8s-master02 1/1 Running 1 (108m ago) 10h kube-system kube-scheduler-k8s-master03 1/1 Running 1 (107m ago) 10h kube-system metrics-server-5cf8885b66-jdjtb 1/1 Running 0 65m kubernetes-dashboard dashboard-metrics-scraper-7fcdff5f4c-jbw5t 1/1 Running 0 52m kubernetes-dashboard kubernetes-dashboard-85f59f8ff7-xm2w6 1/1 Running 0

3.在Master01节点上查看集群网段是否冲突

(1)在Master01节点上查看SVC网段

$ kubectl get svc 
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP   10h

(2)在Master01节点上查看POD网段,主要分为两段,一段是因为使用HostNetwork,所以使用宿主机网段;另一段使用POD网段

$ kubectl get po -A -owide
NAMESPACE              NAME                                         READY   STATUS    RESTARTS       AGE   IP              NODE           NOMINATED NODE   READINESS GATES
kube-system            calico-kube-controllers-6f6595874c-tntnr     1/1     Running   0              92m   172.25.92.67    k8s-master02   <none>           <none>
kube-system            calico-node-5mj9g                            1/1     Running   1 (84m ago)    92m   192.168.1.33    k8s-master03   <none>           <none>
kube-system            calico-node-hhjrv                            1/1     Running   2 (84m ago)    92m   192.168.1.32    k8s-master02   <none>           <none>
kube-system            calico-node-szjm7                            1/1     Running   0              92m   192.168.1.34    k8s-node01     <none>           <none>
kube-system            calico-node-xcgwq                            1/1     Running   0              92m   192.168.1.31    k8s-master01   <none>           <none>
kube-system            calico-node-ztbkj                            1/1     Running   1 (83m ago)    92m   192.168.1.35    k8s-node02     <none>           <none>
kube-system            calico-typha-6b6cf8cbdf-8qj8z                1/1     Running   0              92m   192.168.1.34    k8s-node01     <none>           <none>
kube-system            coredns-65c54cc984-nrhlg                     1/1     Running   0              10h   172.25.92.65    k8s-master02   <none>           <none>
kube-system            coredns-65c54cc984-xkx7w                     1/1     Running   0              10h   172.25.92.66    k8s-master02   <none>           <none>
kube-system            etcd-k8s-master01                            1/1     Running   1 (112m ago)   10h   192.168.1.31    k8s-master01   <none>           <none>
kube-system            etcd-k8s-master02                            1/1     Running   1 (112m ago)   10h   192.168.1.32    k8s-master02   <none>           <none>
kube-system            etcd-k8s-master03                            1/1     Running   1 (112m ago)   10h   192.168.1.33    k8s-master03   <none>           <none>
kube-system            kube-apiserver-k8s-master01                  1/1     Running   1 (112m ago)   10h   192.168.1.31    k8s-master01   <none>           <none>
kube-system            kube-apiserver-k8s-master02                  1/1     Running   1 (112m ago)   10h   192.168.1.32    k8s-master02   <none>           <none>
kube-system            kube-apiserver-k8s-master03                  1/1     Running   2 (112m ago)   10h   192.168.1.33    k8s-master03   <none>           <none>
kube-system            kube-controller-manager-k8s-master01         1/1     Running   2 (112m ago)   10h   192.168.1.31    k8s-master01   <none>           <none>
kube-system            kube-controller-manager-k8s-master02         1/1     Running   1 (112m ago)   10h   192.168.1.32    k8s-master02   <none>           <none>
kube-system            kube-controller-manager-k8s-master03         1/1     Running   1 (112m ago)   10h   192.168.1.33    k8s-master03   <none>           <none>
kube-system            kube-proxy-2kz9g                             1/1     Running   0              16m   192.168.1.32    k8s-master02   <none>           <none>
kube-system            kube-proxy-b54gh                             1/1     Running   0              16m   192.168.1.33    k8s-master03   <none>           <none>
kube-system            kube-proxy-kclcc                             1/1     Running   0              16m   192.168.1.34    k8s-node01     <none>           <none>
kube-system            kube-proxy-pv8gc                             1/1     Running   0              16m   192.168.1.31    k8s-master01   <none>           <none>
kube-system            kube-proxy-xt52m                             1/1     Running   0              16m   192.168.1.35    k8s-node02     <none>           <none>
kube-system            kube-scheduler-k8s-master01                  1/1     Running   2 (112m ago)   10h   192.168.1.31    k8s-master01   <none>           <none>
kube-system            kube-scheduler-k8s-master02                  1/1     Running   1 (112m ago)   10h   192.168.1.32    k8s-master02   <none>           <none>
kube-system            kube-scheduler-k8s-master03                  1/1     Running   1 (112m ago)   10h   192.168.1.33    k8s-master03   <none>           <none>
kube-system            metrics-server-5cf8885b66-jdjtb              1/1     Running   0              70m   172.27.14.193   k8s-node02     <none>           <none>
kubernetes-dashboard   dashboard-metrics-scraper-7fcdff5f4c-jbw5t   1/1     Running   0              57m   172.17.125.1    k8s-node01     <none>           <none>
kubernetes-dashboard   kubernetes-dashboard-85f59f8ff7-xm2w6        1/1     Running   0              57m   172.27.14.194   k8s-node02     <none>           <none>

4.在Master01节点上查看是否正常创建资源

(1)在Master01节点上创建名为cluster-test的deployment

$ kubectl create deploy cluster-test --image=registry.cn-hangzhou.aliyuncs.com/zq-demo/debug-tools -- sleep 3600

(2)在Master01节点上查看deployment创建情况

$ kubectl kubectl get po
NAME                            READY   STATUS    RESTARTS   AGE
cluster-test-79b978867f-429xg   1/1     Running   0          12m

5.在Master01节点上检查Pod 是否能够解析 Service

(1)在Master01节点上解析kubernetes,观察到和上面SVC地址一致

$ kubectl exec -it cluster-test-79b978867f-429xg -- bash
(04:29 cluster-test-79b978867f-429xg:/) nslookup kubernetes
Server:     10.0.0.10
Address:    10.0.0.10#53

Name:   kubernetes.default.svc.cluster.local
Address: 10.0.0.1

(2)在Master01节点上解析kube-dns.kube-system,观察到和上面SVC地址一致

$ kubectl exec -it cluster-test-79b978867f-429xg -- bash
(04:30 cluster-test-79b978867f-429xg:/) nslookup kube-dns.kube-system
Server:     10.0.0.10
Address:    10.0.0.10#53

Name:   kube-dns.kube-system.svc.cluster.local
Address: 10.0.0.10

6.每个节点是否能访问 Kubernetes 的 kubernetes svc 443 和 kube-dns 的 service 53

(1)在每台机器上测试访问 Kubernetes 的 kubernetes svc 443

$ curl https://10.0.0.1:443
curl: (60) Peer's Certificate issuer is not recognized.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

(2)在每台机器上测试访问 Kubernetes 的kube-dns 的 service 53

$ curl 10.0.0.10:53
curl: (52) Empty reply from server

7.Pod 和机器之间是否能正常通讯

(1)在Master01节点上查看pod节点IP

$ kubectl get po -owide
NAME                            READY   STATUS    RESTARTS   AGE   IP             NODE         NOMINATED NODE   READINESS GATES
cluster-test-79b978867f-429xg   1/1     Running   0          25m   172.17.125.2   k8s-node01   <none>           <none>

(2)在Master01节点上ping测试

$ ping -c 2 172.17.125.2
PING 172.17.125.2 (172.17.125.2) 56(84) bytes of data.
64 bytes from 172.17.125.2: icmp_seq=1 ttl=63 time=0.414 ms
64 bytes from 172.17.125.2: icmp_seq=2 ttl=63 time=0.419 ms

--- 172.17.125.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1026ms
rtt min/avg/max/mdev = 0.414/0.416/0.419/0.020 ms

8.检查Pod 和Pod之间是否能正常通讯

(1)在Master01节点上查看default默认命名空间下的Pod

$ kubectl get po -owide
NAME                            READY   STATUS    RESTARTS   AGE   IP             NODE         NOMINATED NODE   READINESS GATES
cluster-test-79b978867f-429xg   1/1     Running   0          38m   172.17.125.2   k8s-node01   <none>           <none>

(2)在Master01节点上kube-system命名空间下的Pod

$ kubectl get po -n kube-system -owide
NAME                                       READY   STATUS    RESTARTS        AGE     IP              NODE           NOMINATED NODE   READINESS GATES
calico-node-5mj9g                          1/1     Running   1 (4h13m ago)   4h21m   192.168.1.33    k8s-master03   <none>           <none>

(3)在Master01节点上进入cluster-test-79b978867f-429xg进行ping测试

$ kubectl exec -it cluster-test-79b978867f-429xg -- bash
(04:57 cluster-test-79b978867f-429xg:/) ping -c 2 192.168.1.33
PING 192.168.1.33 (192.168.1.33) 56(84) bytes of data.
64 bytes from 192.168.1.33: icmp_seq=1 ttl=63 time=0.712 ms
64 bytes from 192.168.1.33: icmp_seq=2 ttl=63 time=0.271 ms

--- 192.168.1.33 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1034ms
rtt min/avg/max/mdev = 0.271/0.491/0.712/0.221 ms