The Kubernetes Scheduler
Explains how a Pod gets scheduled on a node.
The scheduler watches a queue of unscheduled Pods and selects the best node based on resource requests like CPU
and memory.
If a node fails, its Pods are requeued for rescheduling.
Resource Manifests in YAML Format
Kubernetes resources like Deployments and Services are declared in YAML manifest files.
These files represent the desired state of your system.
You can edit YAML files directly and apply them using kubectl apply.
Resources Are Data
Kubernetes stores all resources in its internal database.
Changes to resources (e.g., Deployments) are handled by a reconciliation loop, ensuring the actual state matches the
desired state.
Deployment Manifests
YAML for Deployments includes fields like replicas, selector, template, containers, etc.
Container info such as image, name, and ports is specified inside containers.
Using kubectl apply
Apply changes with kubectl apply -f yourfile.yaml.
Automatically updates resources to match the desired state defined in YAML.
Services
Expose your Pods to the network using a Kubernetes Service.
Services are defined with selectors that match Pods by labels.
Ensures stable DNS/IP for communication, even if Pods are recreated.
Querying the Cluster with kubectl
kubectl get to list resources, kubectl describe to inspect details.
Examples:
kubectl get nodes
kubectl describe pod/demo-pod-name
Use kubectl get all to list all common resources.
Taking Resources to the Next Level
Highlights repetition in YAML (e.g., name, labels).
Introduces Helm as a better way to manage complex YAML setups.
Helm: A Kubernetes Package Manager
Simplifies app deployment using charts.
Helm templates support variables, reducing duplication in YAML.
Popular apps have public Helm charts on Artifact Hub.
Installing Helm
Install via OS-specific instructions.
Install charts with helm install and manage them with helm list, helm upgrade, etc.
Chapter 5: Managing Resources
Understanding Resources
Scheduler needs info on Pod resource requests (minimum needed) and limits (maximum allowed).
Expressed in CPU (millicores) and memory (MiB).
Resource Units
1 Kubernetes CPU = 1 vCPU (AWS, GCP, Azure).
Pods typically use partial CPUs (100m = 0.1 core).
Resource Requests & Limits
If requests arent met, the Pod stays pending.
Exceeding limits can lead to throttling (CPU) or termination (memory).
Quality of Service (QoS)
Guaranteed: Requests = Limits
Burstable: Requests < Limits
BestEffort: No requests/limits
Probes
Liveness: Container is working.
Readiness: App ready to serve traffic.
Startup: App fully started.
PodDisruptionBudgets (PDBs)
Restrict the number of Pods that can be disrupted at once (during upgrades, evictions).
Namespaces
Logical separation of resources within a cluster.
Use kubectl get pods -n namespace.
Cross-Namespace Services
Access via service.namespace.svc.cluster.local.
ResourceQuota and LimitRange
ResourceQuota: Set caps on usage per namespace.
LimitRange: Default resource limits for containers in that namespace.
Optimizing Resource Usage
Use minimal container images.
Dont overprovision cloud storage.
Annotate resources with owners.
Using Preemptible or Reserved Instances
Preemptible: Cheap but can be terminated any time.
Use node affinity to protect critical workloads.
Keeping Workloads Balanced
Scheduler places Pods evenly but doesnt rebalance running Pods.
Cluster may become unbalanced after node failure or update.
Descheduler
Helps redistribute Pods by evicting ones that can be better placed.
Can fix duplication issues and underutilized nodes.
Chapter 6: Operating Clusters
Introduction
Covers real-world operation: scaling, costs, HA, resilience.
Tools like Chaos Monkey introduced.
Capacity Planning
Start small, expand based on load and observation.
Think in terms of how many VMs youd need traditionally.
Smallest Cluster
1 node = dev only.
At least 3 control plane nodes for resilience.
At least 2 worker nodes for HA workloads.
K3s
Lightweight Kubernetes for resource-constrained environments.
Cluster Sizing and Scaling
Kubernetes officially supports:
5,000 nodes
150,000 Pods
300,000 containers
100 Pods per node
Federation
Use federated clusters only if you need global scale or redundancy.
Avoid multiple clusters unless necessaryuse namespaces instead.
Nodes and Instances
Bigger nodes may be cost-effective.
Mix sizes to suit varying workloads.
Use smaller nodes for redundancy.
Cloud Instance Types
Avoid smallest sizes; leave headroom for system Pods.
Large clusters benefit from multiple instance types.
Heterogeneous Nodes
GPUs for ML workloads.
Windows containers need Windows nodes.
Bare-Metal Clusters
Join on-prem physical servers to Kubernetes if you already own them.
Autoscaling
Use Cluster Autoscaler (or GKE/AKS managed scaling).
Use manual scaling first in most setups.
Conformance Checking
Use Sonobuoy to verify Kubernetes behavior.
Ensures compatibility across providers.
CNCF Certifications
Certified Kubernetes (for vendors).
Certified Kubernetes Administrator (for people).
KCSP for service providers.
Audit Logging
Record who did what.
Useful for security and troubleshooting.
Chaos Engineering
Test cluster resilience with:
chaoskube random deletions
kube-monkey opt-in, scheduled chaos
PowerfulSeal scripted or interactive chaos
Best Practices
Always run at least 3 control plane + 2 worker nodes.
Use kubectl drain before removing nodes.
Use PDBs to ensure no downtime.
Run chaos tools on production with care.
