Kubernetes is a powerful container orchestration platform, but real-time errors can
arise due to misconfigurations, resource issues, or runtime challenges. Here's an
overview of common Kubernetes errors and troubleshooting techniques:
---
### 1. **Pod Issues**
#### **Problem: Pod Stuck in `Pending` State**
- **Causes**:
- Insufficient cluster resources (CPU, memory).
- Unsatisfied node affinity or tolerations.
- No matching storage class for PVC.
- **Troubleshooting**:
- Check pod status: `kubectl describe pod <pod-name>`
- Inspect node resources: `kubectl get nodes -o wide` and `kubectl describe node
<node-name>`
- Verify storage claims: `kubectl describe pvc <pvc-name>`
- Adjust resource requests/limits or update scheduling constraints.
#### **Problem: Pod Stuck in `CrashLoopBackOff`**
- **Causes**:
- Application crash or misconfiguration.
- Missing dependencies or incorrect environment variables.
- Resource exhaustion.
- **Troubleshooting**:
- Check pod logs: `kubectl logs <pod-name> --previous`
- Describe pod for events: `kubectl describe pod <pod-name>`
- Verify image and environment variables.
#### **Problem: Pod Stuck in `ContainerCreating`**
- **Causes**:
- Issues pulling the container image.
- Misconfigured volume mounts.
- **Troubleshooting**:
- Check events: `kubectl describe pod <pod-name>`
- Verify image availability: `kubectl get events`
- Check node disk space and permissions.
---
### 2. **Node Issues**
#### **Problem: Node `NotReady`**
- **Causes**:
- Kubelet not running or misconfigured.
- Resource exhaustion on the node.
- Network connectivity issues.
- **Troubleshooting**:
- Check node details: `kubectl describe node <node-name>`
- Verify kubelet logs on the node: `journalctl -u kubelet`
- Ensure sufficient CPU, memory, and disk space.
- Check network connectivity and firewalls.
#### **Problem: Node Drain Fails**
- **Causes**:
- Pod disruption budgets blocking eviction.
- **Troubleshooting**:
- List pods with disruption budgets: `kubectl get poddisruptionbudget`
- Force eviction (if safe): `kubectl drain <node-name> --ignore-daemonsets --
delete-local-data`
---
### 3. **Service and Networking Issues**
#### **Problem: Service Not Accessible**
- **Causes**:
- Incorrect service configuration (type, selectors, ports).
- Network policies blocking traffic.
- External IP or LoadBalancer not provisioned.
- **Troubleshooting**:
- Check service configuration: `kubectl describe service <service-name>`
- Verify endpoints: `kubectl get endpoints`
- Check network policies: `kubectl get networkpolicy`
- Test connectivity using `kubectl exec` or `curl`.
#### **Problem: DNS Resolution Fails**
- **Causes**:
- CoreDNS pod issues.
- Network misconfigurations.
- **Troubleshooting**:
- Check CoreDNS pod: `kubectl get pods -n kube-system -l k8s-app=kube-dns`
- Inspect logs: `kubectl logs <coredns-pod> -n kube-system`
- Verify DNS configuration: `kubectl describe configmap coredns -n kube-system`
---
### 4. **Ingress Issues**
#### **Problem: Ingress Not Routing Traffic**
- **Causes**:
- Misconfigured ingress resource or controller.
- DNS not pointing to ingress.
- **Troubleshooting**:
- Check ingress resource: `kubectl describe ingress <ingress-name>`
- Verify controller pod logs: `kubectl logs <controller-pod>`
- Test endpoints with `curl` and validate DNS configuration.
---
### 5. **Deployment and Scaling Issues**
#### **Problem: Deployment Failing to Scale**
- **Causes**:
- Insufficient resources in the cluster.
- Horizontal Pod Autoscaler (HPA) misconfiguration.
- **Troubleshooting**:
- Verify deployment status: `kubectl describe deployment <deployment-name>`
- Check HPA: `kubectl get hpa` and `kubectl describe hpa <hpa-name>`
- Inspect cluster resource utilization: `kubectl top nodes`
---
### 6. **Storage Issues**
#### **Problem: Persistent Volume Claim (PVC) Not Bound**
- **Causes**:
- No matching Persistent Volume (PV) available.
- Misconfigured StorageClass.
- **Troubleshooting**:
- Check PVC and PV details: `kubectl get pvc` and `kubectl describe pvc <pvc-
name>`
- Verify storage class: `kubectl get storageclass`
#### **Problem: Volume Mount Fails**
- **Causes**:
- Incorrect mount path.
- Node or storage backend issues.
- **Troubleshooting**:
- Check pod and volume details: `kubectl describe pod <pod-name>`
- Verify storage backend and node status.
---
### 7. **Authentication and Authorization Issues**
#### **Problem: Unauthorized Access Errors**
- **Causes**:
- Incorrect RBAC (Role-Based Access Control) configuration.
- **Troubleshooting**:
- Check roles and bindings: `kubectl get
roles,rolebindings,clusterroles,clusterrolebindings`
- Describe the role or binding for details: `kubectl describe <resource> <name>`
- Update RBAC policies to allow appropriate access.
---
### Tools and Best Practices for Troubleshooting:
1. **Monitoring and Logging**:
- Use tools like Prometheus, Grafana, EFK/ELK stack, or Kubernetes-native
solutions for real-time monitoring.
- Leverage `kubectl logs` and `kubectl describe` for debugging.
2. **Networking Debugging**:
- Use tools like `curl`, `telnet`, or `nc` for connectivity tests.
- Tools like `kubectl exec` and `tcpdump` can be helpful for detailed network
traces.
3. **Cluster Inspection**:
- `kubectl get events` provides a snapshot of the cluster's state.
- Tools like `k9s` or `lens` offer enhanced Kubernetes UI for better visibility.
4. **Audit Logs**:
- Enable Kubernetes auditing to track API server events and detect
misconfigurations.
Let me know if you'd like examples or further details on any specific issue!
