Kubernetes random notes

Kubernetes is designed to abstract infrastructure, so your apps don’t care if they’re running on:

• Bare-metal physical machines
• VMs
• Cloud-managed Kubernetes services (like GKE, EKS, AKS)

Kubernetes thinks in Pods and Nodes, not physical hosts vs virtual machines.

Core Concepts

🖥️ Cluster

• A collection of machines (nodes) managed by Kubernetes.
• Includes the control plane and worker nodes.

🧱 Node

• A physical or virtual machine in the cluster.
• Runs:
  • kubelet
  • container runtime (e.g. containerd)
  • kube-proxy for networking

📦 Pod

• The smallest deployable unit in Kubernetes.
• Wraps one or more containers sharing:
  • Network namespace (same IP)
  • Storage volumes (if defined)

🔁 ReplicaSet

• Maintains a stable set of replica Pods.
• Ensures a specified number of Pods are running at all times.

🚀 Deployment

• Manages ReplicaSets and enables rolling updates, rollback, and scaling.
• Declarative management of Pods.

🔹 Grouping, Configuration & Networking

🗂️ Namespace

• Logical partitions in a cluster.
• Useful for multi-tenancy and organizing resources (e.g. dev, staging, prod).

🔧 ConfigMap

• Stores non-sensitive config data (like app settings, file names, flags).

🔐 Secret

• Stores sensitive data (like passwords, tokens, keys).
• Base64-encoded by default.

📡 Service

• A stable network abstraction to expose Pods.
• Types:
  • ClusterIP (default, internal only)
  • NodePort (exposes on a node’s IP/port)
  • LoadBalancer (cloud providers)
  • Headless (DNS without a ClusterIP)

🌐 Ingress

• Exposes HTTP/HTTPS traffic externally.
• Allows routing based on domain or path.

🔹 Storage & Scheduling

📁 Volume

• Temporary or persistent storage for a Pod.
• Example types: emptyDir, hostPath, persistentVolumeClaim, nfs, cloud storage.

📦 PersistentVolume (PV) / PersistentVolumeClaim (PVC)

• PV: A piece of storage in the cluster.
• PVC: A request for storage by a user.
• Decouples how storage is provisioned and how it’s consumed.

📅 Scheduler

• Decides which node a Pod should run on based on resources, affinity, taints, etc.

🔹Advanced Concepts

🧬 Custom Resource (CR)

• A user-defined API object.
• Lets you create types like ModelTrainingJob, DatabaseCluster, etc.

🧾 CustomResourceDefinition (CRD)

• The schema used to define a new Custom Resource type.
• Once defined, K8s treats your resource like built-in ones.

🧠 Controller

• Watches resources (like Pods, Deployments, or your CRs) and reconciles desired vs actual state.
• Keeps the system in the correct state.
• Runs a control loop: "observe → compare → act".

🧑‍🔧 Operator

• Combines:
  • CRDs (define your custom API)
  • Controllers (logic to manage CRs)
• Encapsulates domain-specific logic for managing complex apps (e.g. PostgreSQL, Kafka).
• Operators act like K8s-native automation scripts.

🔹 Ecosystem & Add-Ons

🧠 How These All Relate (Conceptual Diagram)

[Kubernetes Cluster]
│
├── Namespaces
│   └── Groups resources logically
│
├── Core Resources
│   ├── Pod ← Smallest compute unit
│   ├── Deployment ← Manages ReplicaSets & updates
│   ├── ReplicaSet ← Maintains # of Pods
│   ├── Service ← Exposes Pods
│   ├── Ingress ← Routes traffic to Services
│   └── ConfigMap / Secret ← Inject config/env into Pods
│
├── Storage Resources
│   └── Volume / PersistentVolumeClaim
│
├── Custom Resources (CRs)
│   └── Defined by: CustomResourceDefinition (CRD)
│       └── Watched by: Custom Controller (e.g. Operator)
│
└── Control Plane
    ├── API Server ← Everything talks to it
    ├── Scheduler ← Assigns Pods to nodes
    └── Controller Manager ← Reconciliation loops

🐳 What is a Container Runtime?

• A container runtime is the low-level software responsible for running containers on a host machine.
• A container runtime is what Kubernetes uses to turn your container images into running processes inside Pods. It handles image pulling, process isolation, resource limits, and lifecycle control.
• In a Kubernetes context, it’s what Node’s kubelet uses to create, start, stop, and delete containers.

What a container runtime like containerd does:

🔧 Common Container Runtimes

🧩 How It Fits Into Kubernetes

[ Kubernetes Node ]
├── kubelet
│    └── talks to container runtime via CRI (Container Runtime Interface)
├── containerd / CRI-O
│    └── talks to runc to actually launch the container
└── container
     └── Runs your app inside isolated namespaces with limits and network rules

Kubernetes uses the CRI (Container Runtime Interface) to abstract away which runtime it uses — so you can plug in different runtimes (containerd, CRI-O, etc.).

💻 What does a single physical node contain?

A physical machine may run:

• One or more virtual machines (if using virtualization like KVM, VMware, etc.)
• Or it might run Kubernetes directly (bare-metal setup)

Each Node in Kubernetes is typically a:

VM (most common in cloud setups)
or
Physical host (bare-metal Kubernetes)

On each Node, you will find:

• OS + container runtime (like containerd)
• kubelet (talks to control plane)
• kube-proxy (networking rules)
• Your scheduled Pods, each containing 1 or more containers

📦 Do we need VMs if we have containers?

Not necessarily!

So: VMs and containers solve different problems:

• VMs: Isolate full OS environments
• Containers: Isolate application-level processes, much lighter

Kubernetes runs on VMs or physical nodes—it doesn't care. What it cares about is abstracting compute capacity.

🧊 Can a container belong to different Pods?

🛑 No.

• Each container belongs to exactly one Pod.
• A Pod is the unit of deployment and resource allocation.
• Containers in different Pods are totally independent (even if they run the same image).

🚫 Can one Pod span multiple nodes (machines)?

🛑 No.

• A Pod is always scheduled onto exactly one Node.
• All containers inside that Pod live on the same machine.
• This is by design for tight coupling (e.g., sidecars, shared volumes, local communication)

So if you need to scale across nodes:

• Use multiple Pods, not one big multi-node Pod.
• Let a Service abstract across those Pods.

🧱 6. What’s the real boundary then?

🧠 TL;DR Mental Model

[Physical Hardware]
  └── [Optional: VMs]
        └── [Kubernetes Node]
              └── [Pods]
                    └── [Containers]

✅ Each Pod = runs on one Node only
✅ Each Node = can run many Pods
✅ Containers = belong to only one Pod
✅ Physical vs virtual = blurred on purpose to keep K8s cloud-agnostic

🏢 What is Multi-Tenancy in Kubernetes?

Multi-tenancy = multiple teams, users, or applications sharing the same Kubernetes cluster, but:

• Isolated from each other
• With controlled access

Think of it like:

One office building with many companies → they share the building (cluster) but each has their own space (namespace), keys (permissions), and policies.

🧩 Why use multi-tenancy?

• Avoid running separate clusters for each team/app
• Lower cost, centralized infrastructure
• But still provide logical and secure isolation

🗂️ What are Namespaces?

Namespaces are virtual clusters within a Kubernetes cluster.

Each namespace:

• Contains its own Pods, Deployments, Services, Secrets, ConfigMaps, etc.
• Has its own quotas and policies
• Can be locked down to specific users/teams

Think of it as:

“This is the dev team’s sandbox.”
“This is production. Only ops can touch it.”

🔐 What is RBAC (Role-Based Access Control)?

RBAC = The way Kubernetes manages user permissions.

It defines who can do what in which namespace or cluster.

🔑 Core Components:

📦 Example:

Let’s say you have a dev team and prod team:

Namespaces:
- dev
- prod

Users:
- alice
- bob

Roles:
- dev-reader (can view stuff in dev)
- prod-admin (can manage everything in prod)

RoleBindings:
- dev-reader is bound to alice in dev
- prod-admin is bound to bob in prod

Result:

• ✅ Alice can view Pods in dev
• ❌ Alice cannot do anything in prod
• ✅ Bob can manage everything in prod
• ❌ Bob cannot access dev (unless separately bound)

🔐 Multi-Tenancy = Namespaces + RBAC

Together they create logical and security boundaries for safe multi-tenancy.