Skip to main content

Day 37 - Application Troubleshooting in Kubernetes

 Day 37 - Application Troubleshooting in Kubernetes


Day-37
Today I worked on Application Troubleshooting in Kubernetes — deep-diving into how microservices communicate, how services expose workloads, and how to resolve real-world connectivity issues inside a cluster!

Key Learnings & Fixes:
1️⃣ Deployed the Example Voting App, which consists of 5 microservices — frontend, Redis, .NET worker, PostgreSQL, and results UI.
2️⃣ Verified pod placement using kubectl get po -o wide and traced traffic flow across worker nodes to identify NodePort accessibility issues.
3️⃣ Identified missing service endpoints caused by label mismatches — fixed by aligning pod labels with service selectors.
4️⃣ Debugged Result Service connectivity — the pod listened on port 80 but the service was targeting 8080, so I patched the service using kubectl edit svc result.
5️⃣ Diagnosed vote-submission failures originating from NetworkPolicy restrictions — the policy allowed only app=frontend, while our voting pods used labels like app=vote.
6️⃣ Updated the NetworkPolicy to include the correct labels, restoring connectivity between the voting app and Redis.
7️⃣ Validated everything end-to-end: NodePort access ✔️, service endpoints ✔️, Redis communication ✔️, vote submission ✔️, result page ✔️

Pro Tips Learned Today:
1️⃣ Always validate labels and selectors — most service issues come from mismatches.
2️⃣ If a service shows no endpoints, check pod labels first before diving deeper.
3️⃣ For NodePort issues, confirm node public IP + correct worker placement.
4️⃣ NetworkPolicies can silently block communication — always verify your ingress rules.
5️⃣ Debugging in Kubernetes isn’t guessing — it’s following the network path step-by-step.

ReadMore: https://lnkd.in/dFn4XKHH




Comments

Popular posts from this blog

Step-by-Step Explanation of Ballooning, Compression & Swapping in VMware

 ðŸ”¹ Step-by-Step Explanation of Ballooning, Compression & Swapping in VMware ⸻ 1️⃣ Memory Ballooning (vmmemctl) Ballooning is the first memory reclamation technique used when ESXi detects memory pressure. ➤ Step-by-Step: How Ballooning Works  1. VMware Tools installs the balloon driver (vmmemctl) inside the guest OS.  2. ESXi detects low free memory on the host.  3. ESXi inflates the balloon in selected VMs.  4. Balloon driver occupies guest memory, making the OS think RAM is full.  5. Guest OS frees idle / unused pages (because it believes memory is needed).  6. ESXi reclaims those freed pages and makes them available to other VMs. Why Ballooning Happens?  • Host free memory is very low.  • ESXi wants the VM to release unused pages before resorting to swapping. Example  • Host memory: 64 GB  • VMs used: 62 GB  • Free: 2 GB → ESXi triggers ballooning  • VM1 (8 GB RAM): Balloon inflates to 2 GB → OS frees 2 GB → ESXi re...

ESXi Host Troubleshooting Checklist

  🛠️ ESXi Host Troubleshooting Checklist (With Complete Log Locations) ✅ 1. Host Status & Connectivity Check host state in vCenter (Connected / Not Responding / Disconnected) Review CPU, RAM, and datastore usage Validate HA/DRS recommendations ✅ 2. Hardware Health Monitor hardware sensors (CPU, DIMMs, fans, PSU, RAID) Check RAID controller logs via vendor tools View hardware status in: vCenter → Monitor → Hardware Health DCUI → Hardware Status ✅ 3. Network Validation Verify vSwitches, Port Groups, NIC teaming, VLAN tagging Check vmkernel ports (mgmt, vMotion, iSCSI, vSAN) Test reachability using: Shell vmkping < IP > Show more lines Validate physical NIC status and link speed ✅ 4. Storage & Datastore Checks Confirm datastore accessibility Rescan storage adapters (iSCSI/FC/NFS) Validate multipathing (Round Robin / Fixed / MRU) Check datastore latency with esxtop ✅ **5. Critical Logs & Their Locations (Full List) Here are key ESXi log files that every VMware adm...
  vCenter Troubleshooting Tips, Common Issues & Log Locations 🚀 1. General Troubleshooting Approach A. Check service health first For vCenter Server Appliance (VCSA): Shell https : / / < vcenter - FQDN > : 5480 Show more lines Go to Services → Health to verify: vCenter Server ESXi hosts vSphere Client DNS, NTP, DB connections vpxd, vpxd-svcs, vsphere-ui B. DNS & NTP checks vCenter is highly dependent on correct DNS & time sync. Check: Shell nslookup < vcenter - fqdn > nslookup < esxi - host - fqdn > Show more lines Time drift > 5 minutes causes: Login failures vpxd crashes PSC authentication issues C. Check storage/database health Slow DB/storage = UI slow, tasks hung. For VCSA: Shell df - h du - sh / storage / log / * / var / log Show more lines Free up space if partitions hit >80%. D. Restart critical services safely On VCSA: Shell service - control - - status service - control - - stop - - all service - control - - start - - all ...