linuxea:处理k8s kubelet.go node "master" not found问题

marksugar
2021-12-21 / 0 评论 / 6,749 阅读 / 正在检测是否收录...
温馨提示:
本文最后更新于2022年02月22日,已超过341天没有更新,若内容或图片失效,请留言反馈。

一个项目的master节点down了之后kubelet起不来,由于是远古时期的项目,早期未作任何监控手段。因此也没有监控,没有文档,非常棘手
经过查看,master节点的kubelet未启动,查看kubelet日志: failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory

failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory

随即在/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf中去掉--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf后重启

master的kubelet已经启动起来,但是还是有问题,status是NotReady

查看master状态也是NotReady,而后就暂停了master的调度SchedulingDisabled

[root@master ~]# kubectl get node
NAME      STATUS                        ROLES    AGE    VERSION
master    NotReady,SchedulingDisabled   master   647d   v1.16.3
worker1   Ready                         <none>   647d   v1.16.3
worker2   Ready                         <none>   647d   v1.16.3

我们留意到worker是ready的,于是我们查看worker节点。发现worker节点调度正常,并且可以进行拉取镜像,并且此时业务也恢复了。
我们使用kubectl describe master查看到master节点还是有问题

20211221-2.png

[root@master ~]# kubectl describe  node master
Name:               master
Roles:              master
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=master
                    kubernetes.io/os=linux
                    node-role.kubernetes.io/master=
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                    node.alpha.kubernetes.io/ttl: 0
                    projectcalico.org/IPv4Address: 192.168.133.6/25
                    projectcalico.org/IPv4IPIPTunnelAddr: 10.100.219.64
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Sat, 14 Mar 2020 11:37:43 +0800
Taints:             node.kubernetes.io/unreachable:NoExecute
                    node-role.kubernetes.io/master:NoSchedule
                    node.kubernetes.io/unreachable:NoSchedule
                    node.kubernetes.io/unschedulable:NoSchedule
Unschedulable:      true
Conditions:
  Type                 Status    LastHeartbeatTime                 LastTransitionTime                Reason              Message
  ----                 ------    -----------------                 ------------------                ------              -------
  NetworkUnavailable   False     Thu, 04 Mar 2021 10:26:49 +0800   Thu, 04 Mar 2021 10:26:49 +0800   CalicoIsUp          Calico is running on this node
  MemoryPressure       Unknown   Tue, 21 Dec 2021 09:17:47 +0800   Tue, 21 Dec 2021 09:18:41 +0800   NodeStatusUnknown   Kubelet stopped posting node status.
  DiskPressure         Unknown   Tue, 21 Dec 2021 09:17:47 +0800   Tue, 21 Dec 2021 09:18:41 +0800   NodeStatusUnknown   Kubelet stopped posting node status.
  PIDPressure          Unknown   Tue, 21 Dec 2021 09:17:47 +0800   Tue, 21 Dec 2021 09:18:41 +0800   NodeStatusUnknown   Kubelet stopped posting node status.
  Ready                Unknown   Tue, 21 Dec 2021 09:17:47 +0800   Tue, 21 Dec 2021 09:18:41 +0800   NodeStatusUnknown   Kubelet stopped posting node status.
Addresses:
  InternalIP:  192.168.133.6
  Hostname:    master
Capacity:
 cpu:                4
 ephemeral-storage:  36805060Ki
 hugepages-2Mi:      0
 memory:             16259676Ki
 pods:               110
Allocatable:
 cpu:                4
 ephemeral-storage:  33919543240
 hugepages-2Mi:      0
 memory:             16157276Ki
 pods:               110
System Info:
 Machine ID:                 abfbcf67c72dca40affb0cd0c8debd6d
 System UUID:                63B3A0C6-AEB7-3841-BA81-2724178E0890
 Boot ID:                    02424d77-4b14-45ea-a632-038dbc70a2c4
 Kernel Version:             3.10.0-957.el7.x86_64
 OS Image:                   CentOS Linux 7 (Core)
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  docker://18.9.7
 Kubelet Version:            v1.16.3
 Kube-Proxy Version:         v1.16.3
PodCIDR:                     10.100.0.0/24
PodCIDRs:                    10.100.0.0/24
Non-terminated Pods:         (6 in total)
  Namespace                  Name                              CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                  ----                              ------------  ----------  ---------------  -------------  ---
  kube-system                calico-node-v2rmv                 250m (6%)     0 (0%)      0 (0%)           0 (0%)         4h15m
  kube-system                etcd-master                       0 (0%)        0 (0%)      0 (0%)           0 (0%)         362d
  kube-system                kube-apiserver-master             250m (6%)     0 (0%)      0 (0%)           0 (0%)         362d
  kube-system                kube-controller-manager-master    200m (5%)     0 (0%)      0 (0%)           0 (0%)         362d
  kube-system                kube-proxy-k7shj                  0 (0%)        0 (0%)      0 (0%)           0 (0%)         4h15m
  kube-system                kube-scheduler-master             100m (2%)     0 (0%)      0 (0%)           0 (0%)         362d
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                800m (20%)  0 (0%)
  memory             0 (0%)      0 (0%)
  ephemeral-storage  0 (0%)      0 (0%)
Events:              <none>

接着查看日志,提示node "master" not found

20211221.png

[root@master ~]# journalctl -u kubelet -f
-- Logs begin at Sun 2021-10-17 15:04:51 CST. --
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.159602   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.259813   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.329948   34678 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.360614   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.460822   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.531167   34678 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: nodes "master" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.560990   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.661174   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.731398   34678 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.761342   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.861818   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.930789   34678 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Dec 21 14:56:18 master kubelet[34678]: E1221 14:56:18.962003   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.062175   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.131629   34678 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.162315   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.262472   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.331431   34678 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.362644   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.462865   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.532574   34678 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: nodes "master" is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.563028   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.663176   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.732705   34678 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.763328   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.863481   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.932104   34678 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:anonymous" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope
Dec 21 14:56:19 master kubelet[34678]: E1221 14:56:19.963637   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:20 master kubelet[34678]: E1221 14:56:20.063823   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:20 master kubelet[34678]: E1221 14:56:20.133218   34678 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User "system:anonymous" cannot list resource "runtimeclasses" in API group "node.k8s.io" at the cluster scope
Dec 21 14:56:20 master kubelet[34678]: E1221 14:56:20.164019   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:20 master kubelet[34678]: E1221 14:56:20.264632   34678 kubelet.go:2267] node "master" not found
Dec 21 14:56:20 master kubelet[34678]: E1221 14:56:20.332795   34678 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
Dec 21 14:56:20 master kubelet[34678]: E1221 14:56:20.364844   34678 kubelet.go:2267] node "master" not found

然而此时master的kubelet 是runing的,但是没有状态还是没有ready,接着我重启了docker程序,仍然没有进展,反倒让proxy和coredns,calico Pending了。
而etcd,controlle,api-server却是正常running的。

由于项目太旧,没有任何文档,半年前做过证书更新,于是查看证书

[root@master ]# kubeadm alpha certs check-expiration
CERTIFICATE                EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
admin.conf                 Dec 22, 2030 06:19 UTC   9y              no
apiserver                  Dec 22, 2030 06:19 UTC   9y              no
apiserver-etcd-client      Dec 22, 2030 06:19 UTC   9y              no
apiserver-kubelet-client   Dec 22, 2030 06:19 UTC   9y              no
controller-manager.conf    Dec 22, 2030 06:19 UTC   9y              no
etcd-healthcheck-client    Dec 22, 2030 06:19 UTC   9y              no
etcd-peer                  Dec 22, 2030 06:19 UTC   9y              no
etcd-server                Dec 22, 2030 06:19 UTC   9y              no
front-proxy-client         Dec 22, 2030 06:19 UTC   9y              no
scheduler.conf             Dec 22, 2030 06:19 UTC   9y              no

发现证书也没有问题
而后我们确认master是否配置正确

[root@master kubernetes]# cat kubelet.conf
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: 
    server: https://master:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: 
    client-key-data: L

在查看hosts

[root@master kubernetes]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.133.6 master

一切都没有问题,

现在怀疑是kubelet.conf的证书权限有问题,于是将node的kubelet拿到master尝试。发现还是没有解决

最后在尝试将master的admin.conf替换成kubelet.conf

[root@master kubernetes]# systemctl restart kubelet

查看日志

[root@master kubernetes]# journalctl -u kubelet -f
-- Logs begin at Sun 2021-10-31 03:30:27 CST. --
Dec 21 15:15:48 master kubelet[39135]: I1221 15:15:48.782529   39135 reconciler.go:301] Volume detached for volume "calico-node-token-sfzfw" (UniqueName: "kubernetes.io/secret/b7039ad3-f568-47b7-a323-737898628333-calico-node-token-sfzfw") on node "master" DevicePath ""
Dec 21 15:15:48 master kubelet[39135]: I1221 15:15:48.782541   39135 reconciler.go:301] Volume detached for volume "coredns-token-6rmpw" (UniqueName: "kubernetes.io/secret/bf57f81b-911a-48b7-aa1e-624193cd7c51-coredns-token-6rmpw") on node "master" DevicePath ""
Dec 21 15:15:48 master kubelet[39135]: I1221 15:15:48.782552   39135 reconciler.go:301] Volume detached for volume "config-volume" (UniqueName: "kubernetes.io/configmap/f4d237c7-2212-4c91-8535-4cad2bb88644-config-volume") on node "master" DevicePath ""
Dec 21 15:15:48 master kubelet[39135]: I1221 15:15:48.782564   39135 reconciler.go:301] Volume detached for volume "coredns-token-6rmpw" (UniqueName: "kubernetes.io/secret/f4d237c7-2212-4c91-8535-4cad2bb88644-coredns-token-6rmpw") on node "master" DevicePath ""
Dec 21 15:15:48 master kubelet[39135]: W1221 15:15:48.993096   39135 kubelet_getters.go:292] Path "/var/lib/kubelet/pods/3114871e-b583-4e02-a8f3-b63a25b2dffa/volumes" does not exist
Dec 21 15:15:49 master kubelet[39135]: W1221 15:15:49.215461   39135 kubelet_getters.go:292] Path "/var/lib/kubelet/pods/02f1318a-3b06-48cf-a96e-83322f66c17b/volumes" does not exist
Dec 21 15:15:49 master kubelet[39135]: W1221 15:15:49.215524   39135 kubelet_getters.go:292] Path "/var/lib/kubelet/pods/b7039ad3-f568-47b7-a323-737898628333/volumes" does not exist
Dec 21 15:15:49 master kubelet[39135]: W1221 15:15:49.215548   39135 kubelet_getters.go:292] Path "/var/lib/kubelet/pods/f4d237c7-2212-4c91-8535-4cad2bb88644/volumes" does not exist
Dec 21 15:15:49 master kubelet[39135]: W1221 15:15:49.215570   39135 kubelet_getters.go:292] Path "/var/lib/kubelet/pods/83515e8e-8ba5-459d-9bc6-3d897a4f18e9/volumes" does not exist
Dec 21 15:15:49 master kubelet[39135]: W1221 15:15:49.215593   39135 kubelet_getters.go:292] Path "/var/lib/kubelet/pods/bf57f81b-911a-48b7-aa1e-624193cd7c51/volumes" does not exist
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384045   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "lib-modules" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-lib-modules") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384101   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "xtables-lock" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-xtables-lock") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384152   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "flexvol-driver-host" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-flexvol-driver-host") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384194   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "cni-net-dir" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-cni-net-dir") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384233   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "host-local-net-dir" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-host-local-net-dir") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384271   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "policysync" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-policysync") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384305   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "calico-node-token-sfzfw" (UniqueName: "kubernetes.io/secret/809554bc-4e5c-4645-b8dd-9963182a8196-calico-node-token-sfzfw") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384332   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "cni-bin-dir" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-cni-bin-dir") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384369   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "var-run-calico" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-var-run-calico") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: I1221 15:15:49.384398   39135 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "var-lib-calico" (UniqueName: "kubernetes.io/host-path/809554bc-4e5c-4645-b8dd-9963182a8196-var-lib-calico") pod "calico-node-k4vx7" (UID: "809554bc-4e5c-4645-b8dd-9963182a8196")
Dec 21 15:15:49 master kubelet[39135]: W1221 15:15:49.986228   39135 kubelet_getters.go:292] Path "/var/lib/kubelet/pods/754ff5ae-8b4f-4f67-9027-68b5f59a8b90/volumes" does not exist
Dec 21 15:15:50 master kubelet[39135]: I1221 15:15:50.387442   39135 reconciler.go:207]

问题暂时得到解决

[root@master kubernetes]# kubectl get pod -A
NAMESPACE     NAME                                       READY   STATUS             RESTARTS   AGE
kube-system   calico-kube-controllers-dc4d48847-mmwdk    1/1     Running            0          4h30m
kube-system   calico-node-9z59b                          1/1     Running            0          21m
kube-system   calico-node-k4vx7                          1/1     Running            0          22m
kube-system   calico-node-xpktb                          1/1     Running            0          21m
kube-system   coredns-dffb59cff-4tnrn                    1/1     Running            0          4h30m
kube-system   coredns-dffb59cff-hjt7v                    1/1     Running            0          4h30m
kube-system   eip-nfs-minio-6c44f59b76-nq8fm             1/1     Running            4          237d
kube-system   etcd-master                                1/1     Running            5          362d
kube-system   kube-apiserver-master                      1/1     Running            3          362d
kube-system   kube-controller-manager-master             1/1     Running            244        362d
kube-system   kube-proxy-7d5qb                           1/1     Running            0          22m
kube-system   kube-proxy-k755k                           1/1     Running            0          21m
kube-system   kube-proxy-pxhb6                           1/1     Running            0          22m
kube-system   kube-scheduler-master                      1/1     Running            241        362d
kube-system   kuboard-7d745b566-wwp8w                    1/1     Running            0          42d
kube-system   metrics-server-7dcd7b44c-dvpp4             1/1     Running            0          87d
[root@master kubernetes]# kubectl get node
NAME      STATUS                     ROLES    AGE    VERSION
master    Ready,SchedulingDisabled   master   647d   v1.16.3
worker1   Ready                      <none>   647d   v1.16.3
worker2   Ready                      <none>   647d   v1.16.3
[root@master kubernetes]# kubectl describe  node master
Name:               master
Roles:              master
...
Unschedulable:      true
Conditions:
  Type                 Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----                 ------  -----------------                 ------------------                ------                       -------
  NetworkUnavailable   False   Tue, 21 Dec 2021 15:15:52 +0800   Tue, 21 Dec 2021 15:15:52 +0800   CalicoIsUp                   Calico is running on this node
  MemoryPressure       False   Tue, 21 Dec 2021 15:46:58 +0800   Tue, 21 Dec 2021 15:15:47 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure         False   Tue, 21 Dec 2021 15:46:58 +0800   Tue, 21 Dec 2021 15:15:47 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure          False   Tue, 21 Dec 2021 15:46:58 +0800   Tue, 21 Dec 2021 15:15:47 +0800   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready                True    Tue, 21 Dec 2021 15:46:58 +0800   Tue, 21 Dec 2021 15:15:47 +0800   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  192.168.133.6
  Hostname:    master
Capacity:
 cpu:                4
 ephemeral-storage:  36805060Ki
 hugepages-2Mi:      0
 memory:             16259676Ki
 pods:               110
Allocatable:
 cpu:                4
 ephemeral-storage:  33919543240
 hugepages-2Mi:      0
 memory:             15211100Ki
 pods:               110
System Info:
 Machine ID:                 abfbcf67c72dca40affb0cd0c8debd6d
 System UUID:                63B3A0C6-AEB7-3841-BA81-2724178E0890
 Boot ID:                    02424d77-4b14-45ea-a632-038dbc70a2c4
 Kernel Version:             3.10.0-957.el7.x86_64
 OS Image:                   CentOS Linux 7 (Core)
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  docker://18.9.7
 Kubelet Version:            v1.16.3
 Kube-Proxy Version:         v1.16.3
PodCIDR:                     10.100.0.0/24
PodCIDRs:                    10.100.0.0/24
Non-terminated Pods:         (6 in total)
  Namespace                  Name                              CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                  ----                              ------------  ----------  ---------------  -------------  ---
  kube-system                calico-node-k4vx7                 250m (6%)     0 (0%)      0 (0%)           0 (0%)         31m
  kube-system                etcd-master                       0 (0%)        0 (0%)      0 (0%)           0 (0%)         362d
  kube-system                kube-apiserver-master             250m (6%)     0 (0%)      0 (0%)           0 (0%)         362d
  kube-system                kube-controller-manager-master    200m (5%)     0 (0%)      0 (0%)           0 (0%)         362d
  kube-system                kube-proxy-pxhb6                  0 (0%)        0 (0%)      0 (0%)           0 (0%)         31m
  kube-system                kube-scheduler-master             100m (2%)     0 (0%)      0 (0%)           0 (0%)         362d
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                800m (20%)  0 (0%)
  memory             0 (0%)      0 (0%)
  ephemeral-storage  0 (0%)      0 (0%)
Events:
  Type    Reason                   Age                From                Message
  ----    ------                   ----               ----                -------
  Normal  Starting                 31m                kubelet, master     Starting kubelet.
  Normal  NodeHasSufficientMemory  31m (x2 over 31m)  kubelet, master     Node master status is now: NodeHasSufficientMemory
  Normal  NodeHasNoDiskPressure    31m (x2 over 31m)  kubelet, master     Node master status is now: NodeHasNoDiskPressure
  Normal  NodeHasSufficientPID     31m (x2 over 31m)  kubelet, master     Node master status is now: NodeHasSufficientPID
  Normal  NodeNotReady             31m                kubelet, master     Node master status is now: NodeNotReady
  Normal  NodeNotSchedulable       31m                kubelet, master     Node master status is now: NodeNotSchedulable
  Normal  NodeAllocatableEnforced  31m                kubelet, master     Updated Node Allocatable limit across pods
  Normal  NodeReady                31m                kubelet, master     Node master status is now: NodeReady
  Normal  Starting                 31m                kube-proxy, master  Starting kube-proxy.

延伸阅读

bootstrap-kubelet.conf: no such file or directory kubelet证书轮换失败

0

评论

博主关闭了当前页面的评论