This repository was archived by the owner on Jun 29, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 46
This repository was archived by the owner on Jun 29, 2022. It is now read-only.
Kubelet/Checkpointer not starting up after master node reboot #1576
Copy link
Copy link
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Description
When I reboot my master node (bare-metal), the kubelet and etcd containers start up. However, nothing happens afterwards. The API server does not start up, checkpointer does not start up.
Impact
Complete loss of management plane. Cannot access cluster.
Environment and steps to reproduce
- Set-up:
Set up a cluster on Proxmox using the bare-metal guide on the Lokomotive docs and iPXE booting it. I'm running 1 master node, and 3 worker nodes. I'm running the stable version of Flatcar, however this issue also happens on the beta build. I'm running the 0.9.0 version of lokoctl. I'm running the following components on the cluster:
- flatcar-linux-update-operator
- openebs-operator
- openebs-storage-class
- metrics-server
- cert-manager
- contour
- web-ui
- httpbin
- Task:
General cluster management, deploying of services, etc... - Action(s):
a. Let the flatcar-linux-update-operator update the master node OS / manually reboot the master node usingsudo reboot - Error:
Kubelet and etcd start up.
Kubelet produces the following error message, somewhere in the logs:
Oct 27 09:11:57 socrates001 docker[424152]: E1027 09:11:57.209231 424127 file.go:187] "Could not process manifest file" err="invalid pod: [spec.volumes[3].projected.sources[0].serviceAccountToken: Forbidden: must not be specified when serviceAccountName is not set]" path="/etc/kubernetes/manifests/kube-system-pod-checkpointer-pr6qb.json"
Expected behavior
I expected checkpointer to properly start, and start up the kube API server as well as all the other required kubernetes components.
Additional information
My cluster is named socrates, cluster domain is cluster.local.
I have 1 master node named socrates001 and 3 worker nodes, named socrates002, socrates003, socrates004.
Log files directly taken from the master node of kubelet.service.
kubelet.log
If you need more information, please ask, I'll be happy to provide it.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working