Chapter 33: Container Orchestration (Kubernetes)
1. Why Orchestration
- Automated Scheduling: Place containers across cluster based on resource requirements, constraints, and policies
- Self-Healing: Restart failed containers, replace nodes, kill unresponsive containers, reschedule workloads
- Elastic Scaling: Scale applications horizontally (replicas) and vertically (resources) based on demand
Interview: "How does K8s differ from Docker Compose?" — Compose is single-host orchestration; K8s is multi-host with scheduling, self-healing, and declarative state reconciliation.
2. Kubernetes Architecture
graph TB
subgraph Control Plane
API[API Server
kube-apiserver] SCHED[Scheduler
kube-scheduler] CM[Controller Manager
kube-controller-manager] ETCD[(etcd
Key-Value Store)] end subgraph Worker Node 1 KUBELET1[kubelet] PROXY1[kube-proxy] POD1A[Pod] POD1B[Pod] end subgraph Worker Node 2 KUBELET2[kubelet] PROXY2[kube-proxy] POD2A[Pod] POD2B[Pod] end API --> ETCD API --> SCHED API --> CM KUBELET1 --> API KUBELET2 --> API KUBELET1 --> POD1A KUBELET1 --> POD1B KUBELET2 --> POD2A KUBELET2 --> POD2B
kube-apiserver] SCHED[Scheduler
kube-scheduler] CM[Controller Manager
kube-controller-manager] ETCD[(etcd
Key-Value Store)] end subgraph Worker Node 1 KUBELET1[kubelet] PROXY1[kube-proxy] POD1A[Pod] POD1B[Pod] end subgraph Worker Node 2 KUBELET2[kubelet] PROXY2[kube-proxy] POD2A[Pod] POD2B[Pod] end API --> ETCD API --> SCHED API --> CM KUBELET1 --> API KUBELET2 --> API KUBELET1 --> POD1A KUBELET1 --> POD1B KUBELET2 --> POD2A KUBELET2 --> POD2B
| Component | Role |
|---|---|
| API Server | Frontend for control plane; all communication goes through it; validates and processes REST requests |
| Scheduler | Watches for unscheduled Pods; assigns them to nodes based on resources, constraints, affinity |
| Controller Manager | Runs controllers (Deployment, ReplicaSet, StatefulSet); reconciles desired vs actual state |
| etcd | Distributed key-value store; single source of truth for cluster state |
| kubelet | Agent on each node; ensures containers are running; reports node/pod status to API server |
| kube-proxy | Network proxy on each node; maintains network rules for Service routing (iptables/IPVS) |
3. Core Objects
| Object | Description | Use Case |
|---|---|---|
| Pod | Smallest deployable unit; 1+ containers sharing network/storage | Run application container(s); sidecar pattern (logging, proxy) |
| Deployment | Manages ReplicaSets; declarative updates; rolling updates/rollbacks | Stateless applications (web servers, APIs); version management |
| Service | Stable network endpoint for set of Pods; load balancing | Expose applications internally (ClusterIP) or externally (LoadBalancer) |
| ConfigMap | Store non-confidential config data as key-value pairs | Environment variables, config files, command-line arguments |
| Secret | Store sensitive data (passwords, tokens); base64 encoded | Database credentials, API keys, TLS certificates |
| PersistentVolumeClaim (PVC) | Request for storage; binds to PersistentVolume | Databases, file storage; decouple Pod from storage details |
| Ingress | HTTP/HTTPS routing rules; manages external access to Services | Host/path-based routing, TLS termination, virtual hosting |
Interview: "Why use a Deployment instead of managing Pods directly?" — Declarative updates, rolling updates, rollback capability, self-healing via ReplicaSet management.
4. Scheduling
| Mechanism | Purpose | Example |
|---|---|---|
| Node Affinity | Constrain Pods to nodes with specific labels (required/preferred) | Schedule GPU workloads to nodes with gpu=true |
| Pod Affinity | Co-locate Pods (e.g., app + cache on same node) | Place Redis Pod near application Pod for low latency |
| Pod Anti-Affinity | Spread Pods across nodes/zones for high availability | Distribute replicas across availability zones |
| Taints & Tolerations | Repel Pods from nodes unless they tolerate the taint | Dedicate nodes to specific workloads; isolate control plane |
| Resource Requests | Minimum resources guaranteed; used for scheduling decisions | requests: cpu=500m, memory=1Gi |
| Resource Limits | Maximum resources Pod can consume; enforced by kubelet | limits: cpu=1, memory=2Gi (OOMKilled if exceeded) |
5. Autoscaling
| Type | What It Scales | Trigger | Use Case |
|---|---|---|---|
| HPA (Horizontal Pod Autoscaler) |
Pod replicas | CPU/memory utilization or custom metrics | Scale stateless apps based on load (web servers, workers) |
| VPA (Vertical Pod Autoscaler) |
Pod resource requests/limits | Historical resource usage | Right-size Pods; avoid over/under-provisioning |
| Cluster Autoscaler | Node count in cluster | Pending Pods (unschedulable) or underutilized nodes | Add nodes when capacity needed; remove idle nodes (cost) |
| KEDA (Event-Driven) |
Pod replicas (0-N) | External metrics (queue depth, Kafka lag, HTTP requests) | Scale to zero; event-driven workloads (queue consumers) |
Interview: "Can HPA and VPA be used together?" — Not recommended on same metric; HPA scales replicas, VPA changes resources, causing conflicts. Use HPA for horizontal, VPA for batch/non-critical workloads.
6. Networking
| Concept | Description | Details |
|---|---|---|
| CNI (Container Network Interface) | Plugins that configure Pod networking | Calico, Cilium, Flannel, Weave; handles IP allocation, routing |
| ClusterIP | Default Service type; internal cluster IP | Only accessible within cluster; stable DNS name |
| NodePort | Exposes Service on static port on each node | External access via <NodeIP>:<NodePort>; range 30000-32767 |
| LoadBalancer | Provisions external load balancer (cloud provider) | AWS ELB, GCP Load Balancer, Azure LB; public IP assigned |
| Ingress | Layer 7 routing; single entry point for multiple Services | Host/path rules, TLS termination; requires Ingress Controller (nginx, Traefik) |
| NetworkPolicy | Firewall rules for Pods; control ingress/egress traffic | Label-based selectors; default deny, explicit allow; requires CNI support |
Interview: "When to use Ingress vs LoadBalancer Service?" — LoadBalancer for each Service (costly, multiple IPs); Ingress for multiple Services via single LB with routing rules (cost-effective, L7 features).
7. Storage
| Concept | Description |
|---|---|
| PersistentVolume (PV) | Cluster resource representing storage (admin provisioned or dynamic); independent of Pod lifecycle |
| PersistentVolumeClaim (PVC) | User request for storage; binds to PV; abstraction layer between Pod and storage |
| StorageClass | Defines storage types (SSD, HDD, replicated); enables dynamic provisioning (auto-create PV on PVC request) |
| StatefulSet | Manages stateful applications; stable network identity, ordered deployment/scaling, persistent storage per Pod |
StatefulSet vs Deployment: StatefulSets provide stable Pod names (app-0, app-1), persistent PVCs per replica, ordered operations (databases, ZooKeeper). Deployments treat Pods as interchangeable (stateless apps).
8. Deployment Strategies
| Strategy | How It Works | Pros | Cons |
|---|---|---|---|
| Rolling Update | Incrementally replace old Pods with new; configure maxSurge and maxUnavailable |
Zero downtime; gradual rollout; easy rollback | Mixed versions during rollout; slower than recreate |
| Blue-Green | Deploy new version (green) alongside old (blue); switch traffic via Service selector; keep blue for rollback | Instant cutover; full testing pre-switch; easy rollback | 2x resources during deployment; all-or-nothing switch |
| Canary | Route small % of traffic to new version; gradually increase if healthy; monitor metrics | Risk mitigation; real user testing; progressive rollout | Complex routing (Istio, Flagger); requires traffic splitting |
| Recreate | Terminate all old Pods, then create new; downtime accepted | Simple; no mixed versions; useful for non-concurrent versions | Downtime during deployment |
9. Observability
Prometheus: De facto monitoring solution; scrapes metrics from Pods (via annotations or ServiceMonitor); integrates with Grafana for visualization; PromQL for queries; AlertManager for alerts.
| Probe Type | Purpose | Failure Action |
|---|---|---|
| Liveness Probe | Detect deadlocked/hung containers | Restart container (kubelet kills and recreates) |
| Readiness Probe | Determine if Pod ready to serve traffic | Remove Pod from Service endpoints (no traffic routed) |
| Startup Probe | Handle slow-starting containers | Disable liveness/readiness checks until startup succeeds or fails |
Common Probe Methods: httpGet (HTTP 200-399 = success), tcpSocket (TCP connection), exec (command exit code 0).
Interview: "What's the difference between liveness and readiness probes?" — Liveness restarts unhealthy containers; readiness controls traffic routing (Pod may be running but not ready to serve).
10. Security
- RBAC (Role-Based Access Control): Define Roles (permissions on resources) and RoleBindings (assign Roles to users/service accounts); ClusterRole for cluster-wide; namespace-scoped or cluster-scoped; principle of least privilege
- Pod Security Standards: Privileged (unrestricted), Baseline (minimally restrictive, blocks privilege escalation), Restricted (hardened, follows pod hardening best practices); enforced via admission controllers
- Admission Controllers: Intercept API requests before persistence; mutate or validate; PodSecurityPolicy (deprecated), PodSecurityAdmission, ValidatingWebhook, MutatingWebhook; enforce policies (image scanning, resource limits, security contexts)
- Network Policies: Microsegmentation; default deny ingress/egress, explicit allow by label selectors; restrict pod-to-pod communication
- Secrets Management: Encrypt Secrets at rest (etcd encryption); use external providers (AWS Secrets Manager, HashiCorp Vault, Azure Key Vault) via CSI driver; avoid plaintext in manifests
- Service Accounts: Identity for Pods; mount tokens for API access; assign minimal RBAC permissions; disable automounting if not needed
Interview: "How do you secure a K8s cluster?" — RBAC for access control, NetworkPolicies for segmentation, Pod Security Standards for workload restrictions, secrets encryption, admission controllers for policy enforcement, regular security scanning (trivy, kube-bench).
11. Managed Kubernetes
| Provider | Service | Key Features | Integration |
|---|---|---|---|
| AWS | EKS (Elastic Kubernetes Service) | Managed control plane; integrates IAM (IRSA); Fargate for serverless Pods; EKS Distro for on-prem | ALB Ingress, EBS/EFS CSI drivers, CloudWatch, VPC CNI |
| GCP | GKE (Google Kubernetes Engine) | Autopilot mode (fully managed); fast release cycle; Workload Identity for IAM; preemptible nodes | GCE Persistent Disk, Cloud Load Balancing, Cloud Monitoring, Anthos (hybrid) |
| Azure | AKS (Azure Kubernetes Service) | Free control plane; integrates Azure AD; virtual nodes (ACI); Windows node support | Azure Disk/Files, Azure LB, Azure Monitor, Azure Policy, Arc for hybrid |
Selection Criteria: Existing cloud footprint, pricing (EKS charges for control plane, GKE/AKS free control plane), integrations (IAM, monitoring, storage), managed node groups, multi-cluster management (Rancher, Anthos), hybrid/multi-cloud needs.
Interview: "EKS vs self-managed K8s on EC2?" — EKS: managed control plane (HA, patching, upgrades), simpler operations, higher cost. Self-managed: full control, lower cost, operational overhead (etcd backups, upgrades, monitoring).
Summary
Kubernetes is the standard for container orchestration, providing automated scheduling, self-healing, and scaling across clusters. The control plane (API server, scheduler, controller manager, etcd) manages cluster state, while worker nodes (kubelet, kube-proxy) run workloads. Core objects (Pods, Deployments, Services, ConfigMaps, Secrets, PVCs, Ingress) enable declarative application management. Scheduling mechanisms (affinity, taints, resource requests/limits) control Pod placement. Autoscaling (HPA, VPA, Cluster Autoscaler, KEDA) adapts capacity to demand. Networking (CNI, Service types, Ingress, NetworkPolicies) handles connectivity and security. Storage (PV/PVC, StorageClasses, StatefulSets) manages persistent data. Deployment strategies (rolling, blue-green, canary) control release risk. Observability (Prometheus, probes) ensures reliability. Security (RBAC, Pod Security, admission controllers) protects clusters. Managed services (EKS, GKE, AKS) reduce operational burden.