A union of curiosity and data science

Knowledgebase and brain dump of a database engineer


Kubernetes Trouble Shooting.

 

Kubelet: View the logs. 

 tail -f /var/log/syslog

journalctl -u kubelet

 

 search helm repo :

 helm search hub -o json matomo|jq

 

Node join command : run this in master to get the join params. 

kubeadm token create --print-join-command
should return something like :  
kubeadm join 192.168.0.172:6443 --token gwllj9.3oj2nsrcx4fgm1yu --discovery-token-ca-cert-hash sha256:1c02343880aa61c6e5debc45a3b82c27e7ee885b6d36186bf31208d3e642efb6 


kubeadm primary commands
kubeadm init , kubeadm reset, kubeadm join

 view configs master

kubectl -n kube-system get cm kubeadm-config -oyaml

 

outputs.... 

[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"e

 

 

 

create a cluster  : 

https://github.com/justmeandopensource/kubernetes/blob/master/docs/install-cluster-ubuntu-20.md

 https://www.youtube.com/watch?v=mMmxMoprxiY

 

 ------------------------------------------------------------------------------------------------------------------

Kubelet goes to a cyclic restart loop when inconsistent container list received from runtimeservice

Find broken container (docker ps -a --filter "label=io.kubernetes.sandbox.id=894f35dca3eda57adef28b69acd0607efdeb34e8814e87e196bc163305576028" <--- id from error message) and remove it using docker rm ID, then restart kubelet.

 The fix was to recreate teh cluster - uber fail. 

 

 ------------------------------------------------------------------------------------------------------------------

Log into kubernetes AWS cluster 

Once you have your .aws credentials and config files setup. 

look for your cluster than then update the config with the following commands. 

aws eks list-clusters

aws eks update-kubeconfig --name <the name of one of the clusters resulting from the previous command>

https://aws.amazon.com/premiumsupport/knowledge-center/eks-cluster-connection/

 If you're not authorized, authorize your arn

aws sts get-caller-identity



 

Add comment