Skip to main content

 

Kubernetes POD Troubleshooting Tactics

There’s a joke in the industry:

Debugged failed pods for 8 hours - No luck.

A random restart the next morning - all set!

If you’ve been there, you know the frustration.

But instead of hoping for a miraculous restart, here’s a structured way to troubleshoot Kubernetes pods effectively.

1. Check Logs

kubectl logs <pod_name>

If your pod has multiple containers, specify one:

kubectl logs <pod_name> -c <container_name>

2. Analyze Pod Status

kubectl get pod <pod_name>

Look at the STATUS column.

If it shows CrashLoopBackOff, ImagePullBackOff, or ErrImagePullyou have clear hints on what to check next.

3. Describe Pod

kubectl describe pod <pod_name>

Look for warning events, scheduling failures, and container state details.

4. Verify Pod Configuration

A misconfigured pod can cause all sorts of issues. Review its YAML configuration.

kubectl get pod <pod_name> -o yaml

Check environment variables, resource limits, image versions, and volumes.

5. Check Events

Kubernetes events provide historical context on failures.

kubectl get events --sort-by=.metadata.creationTimestamp

Pay attention to events like FailedScheduling, ImagePullBackOff, or OOMKilled

6. Validate Container Images

Ensure your container images are correct and available:

Check if the image tag exists.

kubectl get pod <pod_name> -o jsonpath='{.spec.containers[*].image}'

Try pulling the image manually.

docker pull <image_name>

7. Restart Pod

Sometimes, instead of deleting the pod, restarting the deployment helps.

kubectl rollout restart deployment/<deployment_name>

8. Review Service Dependencies

Pods may fail if dependent services are unavailable. Check the relevant services.

kubectl get svc

Ensure services are resolving correctly.

nslookup <service_name>

9. Check Network Connectivity

If your pod can’t communicate with another service, test connectivity.

kubectl exec -it <pod_name> -- sh

ping <target_host>

curl <target_url>

10. Inspect Resource Usage

If your pod is OOMKilled or throttled, check resource usage.

kubectl top pod <pod_name>

Compare with defined limits.

Following this structured approach, you save time, avoid frustration, and debug with confidence!

Comments

Popular posts from this blog

Step-by-Step Explanation of Ballooning, Compression & Swapping in VMware

 ðŸ”¹ Step-by-Step Explanation of Ballooning, Compression & Swapping in VMware ⸻ 1️⃣ Memory Ballooning (vmmemctl) Ballooning is the first memory reclamation technique used when ESXi detects memory pressure. ➤ Step-by-Step: How Ballooning Works  1. VMware Tools installs the balloon driver (vmmemctl) inside the guest OS.  2. ESXi detects low free memory on the host.  3. ESXi inflates the balloon in selected VMs.  4. Balloon driver occupies guest memory, making the OS think RAM is full.  5. Guest OS frees idle / unused pages (because it believes memory is needed).  6. ESXi reclaims those freed pages and makes them available to other VMs. Why Ballooning Happens?  • Host free memory is very low.  • ESXi wants the VM to release unused pages before resorting to swapping. Example  • Host memory: 64 GB  • VMs used: 62 GB  • Free: 2 GB → ESXi triggers ballooning  • VM1 (8 GB RAM): Balloon inflates to 2 GB → OS frees 2 GB → ESXi re...

Quick Guide to VCF Automation for VCD Administrators

  Quick Guide to VCF Automation for VCD Administrators VMware Cloud Foundation 9 (VCF 9) has been  released  and with it comes brand new Cloud Management Platform –  VCF Automation (VCFA)  which supercedes both Aria Automation and VMware Cloud Director (VCD). This blog post is intended for those people that know VCD quite well and want to understand how is VCFA similar or different to help them quickly orient in the new direction. It should be emphasized that VCFA is a new solution and not just rebranding of an old one. However it reuses a lot of components from its predecessors. The provider part of VCFA called Tenenat Manager is based on VCD code and the UI and APIs will be familiar to VCD admins, while the tenant part inherist a lot from Aria Automation and especially for VCD end-users will look brand new. Deployment and Architecture VCFA is generaly deployed from VCF Operations Fleet Management (former Aria Suite LCM embeded in VCF Ops. Fleet Management...

ESXi Host Troubleshooting Checklist

  🛠️ ESXi Host Troubleshooting Checklist (With Complete Log Locations) ✅ 1. Host Status & Connectivity Check host state in vCenter (Connected / Not Responding / Disconnected) Review CPU, RAM, and datastore usage Validate HA/DRS recommendations ✅ 2. Hardware Health Monitor hardware sensors (CPU, DIMMs, fans, PSU, RAID) Check RAID controller logs via vendor tools View hardware status in: vCenter → Monitor → Hardware Health DCUI → Hardware Status ✅ 3. Network Validation Verify vSwitches, Port Groups, NIC teaming, VLAN tagging Check vmkernel ports (mgmt, vMotion, iSCSI, vSAN) Test reachability using: Shell vmkping < IP > Show more lines Validate physical NIC status and link speed ✅ 4. Storage & Datastore Checks Confirm datastore accessibility Rescan storage adapters (iSCSI/FC/NFS) Validate multipathing (Round Robin / Fixed / MRU) Check datastore latency with esxtop ✅ **5. Critical Logs & Their Locations (Full List) Here are key ESXi log files that every VMware adm...