Skip to main content
DevOps

Return to Question

Added info as per Matt and LArntz request.
Source Link
carrotcakeslayer
carrotcakeslayer
  • 113
  • 6
  • kubelet version --short -> Client Version: v1.17.2
  • kubeadm version -o short -> v1.17.2
  • cat /etc/redhat-release -> CentOS Linux release 7.7.1908 (Core)
  • crictl version -> ["Version: 0.1.0","RuntimeName: cri-o","RuntimeVersion: 1.15.1-2.el7","RuntimeApiVersion: v1alpha1"]
  • crictl info -> {"status": {"conditions": [{"type": "RuntimeReady","status": true,"reason": "","message": ""},{"type": NetworkReady","status": true,"reason": "","message": ""}]}}
  • grep -i cgroup_manager /etc/crio/crio.conf -> cgroup_manager = "systemd"
  • lsmod | grep -i "overlay|br_netfilter" -> ["br_netfilter","overlay"] (both ok).
  • Other "extra args tested" (/etc/sysconfig/kubelet)
  • kubelet version --short -> Client Version: v1.17.2
  • kubeadm version -o short -> v1.17.2
  • grep -i cgroup_manager /etc/crio/crio.conf -> cgroup_manager = "systemd"
  • lsmod | grep -i "overlay|br_netfilter" -> ["br_netfilter","overlay"] (both ok).
  • Other "extra args tested" (/etc/sysconfig/kubelet)
  • kubelet version --short -> Client Version: v1.17.2
  • kubeadm version -o short -> v1.17.2
  • cat /etc/redhat-release -> CentOS Linux release 7.7.1908 (Core)
  • crictl version -> ["Version: 0.1.0","RuntimeName: cri-o","RuntimeVersion: 1.15.1-2.el7","RuntimeApiVersion: v1alpha1"]
  • crictl info -> {"status": {"conditions": [{"type": "RuntimeReady","status": true,"reason": "","message": ""},{"type": NetworkReady","status": true,"reason": "","message": ""}]}}
  • grep -i cgroup_manager /etc/crio/crio.conf -> cgroup_manager = "systemd"
  • lsmod | grep -i "overlay|br_netfilter" -> ["br_netfilter","overlay"] (both ok).
  • Other "extra args tested" (/etc/sysconfig/kubelet)
Added more info to the original question as per "LArntz" and "Weyder Ferreira" suggestions/questions
Source Link
carrotcakeslayer
carrotcakeslayer
  • 113
  • 6
  • kubelet version --short -> Client Version: v1.17.2
  • kubeadm version -o short -> v1.17.2
  • grep -i cgroup_manager /etc/crio/crio.conf -> cgroup_manager = "systemd"
  • lsmod | grep -i "overlay|br_netfilter" -> ["br_netfilter","overlay"] (both ok).
  • Other "extra args tested" (/etc/sysconfig/kubelet)

KUBELET_EXTRA_ARGS=--cgroup-driver=systemd --container-runtime-endpoint=unix:///var/run/crio/crio.sock

  • Full output of some commands: -> "kube admin init --upload-certs" -> https://pastebin.com/isbAynEg -> "while true ; do systemctl status kubelet ; sleep 1 ; done" -> https://pastebin.com/9pgtDMNu -> "cat /etc/sysctl.d/99-kubernetes-cri.conf" -> ["net.bridge.bridge-nf-call-iptables = 1","net.ipv4.ip_forward = 1","net.bridge.bridge-nf-call-ip6tables = 1"] -> "journalctl -xeu kubelet" -> https://pastebin.com/qreJyhsu (this one has some things like "aws credentials" which i dont understand and not sure if are the reason or not for the errors).
  • kubelet version --short -> Client Version: v1.17.2
  • kubeadm version -o short -> v1.17.2
  • grep -i cgroup_manager /etc/crio/crio.conf -> cgroup_manager = "systemd"
  • lsmod | grep -i "overlay|br_netfilter" -> ["br_netfilter","overlay"] (both ok).
  • Other "extra args tested" (/etc/sysconfig/kubelet)

KUBELET_EXTRA_ARGS=--cgroup-driver=systemd --container-runtime-endpoint=unix:///var/run/crio/crio.sock

  • Full output of some commands: -> "kube admin init --upload-certs" -> https://pastebin.com/isbAynEg -> "while true ; do systemctl status kubelet ; sleep 1 ; done" -> https://pastebin.com/9pgtDMNu -> "cat /etc/sysctl.d/99-kubernetes-cri.conf" -> ["net.bridge.bridge-nf-call-iptables = 1","net.ipv4.ip_forward = 1","net.bridge.bridge-nf-call-ip6tables = 1"] -> "journalctl -xeu kubelet" -> https://pastebin.com/qreJyhsu (this one has some things like "aws credentials" which i dont understand and not sure if are the reason or not for the errors).
Source Link
carrotcakeslayer
carrotcakeslayer
  • 113
  • 6

kubeadm init fails when using cri-o

I've been following this doc to deploy kubernetes https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/

After launching the "kubeadm init" command and during its excecution, this is the output:

[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp 127.0.0.1:10248: connect: connection refused. Unfortunately, an error has occurred: timed out waiting for the condition This error is likely caused by: - The kubelet is not running - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled) If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: - 'systemctl status kubelet' - 'journalctl -xeu kubelet' Additionally, a control plane component may have crashed or exited when started by the container runtime. To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker. Here is one example how you may list all Kubernetes containers running in docker: - 'docker ps -a | grep kube | grep -v pause' Once you have found the failing container, you can inspect its logs with: - 'docker logs CONTAINERID' error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster To see the stack trace of this error execute with --v=5 or higher

It runs fine if I install docker and switch the runtime to docker instead of "remote" and "systemd".

I've tried several things, but this is the content of my last try:

cat etc/sysconfig/kubelet

KUBELET_ARGS="--container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --runtime-request-timeout=10m --cluster_domain=topota.madre"

systemctl cat kubelet

# /usr/lib/systemd/system/kubelet.service [Unit] Description=kubelet: The Kubernetes Node Agent Documentation=https://kubernetes.io/docs/

[Service] ExecStart=/usr/bin/kubelet Restart=always StartLimitInterval=0 RestartSec=10

[Install] WantedBy=multi-user.target

# /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf # Note: This dropin only works with kubeadm and kubelet v1.11+ [Service] Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/ Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml" # This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS var EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env # This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user shoul # the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sour EnvironmentFile=-/etc/sysconfig/kubelet ExecStart= ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

cat /var/lib/kubelet/kubeadm-flags.env

is empty right now

Cri-O service ir running ok and Docker is not installed.

Any help please?