Kubernetes Distributions

Teabar supports multiple Kubernetes distributions, from lightweight self-managed options to fully managed cloud services. Choose based on your requirements for security, simplicity, and cloud integration.

Distribution Comparison

Distribution	Type	Provisioning	Use Case
Talos	Self-managed	~4 min	Secure, immutable, production-grade
K3s	Self-managed	~3 min	Lightweight, fast, development
EKS	Managed (AWS)	~10 min	AWS integration, enterprise
AKS	Managed (Azure)	~10 min	Azure integration, enterprise

Tip

**Recommended:** Talos for production security, K3s for development speed.

Talos Linux

Talos is a secure, immutable, minimal Linux distribution designed specifically for Kubernetes. It’s Teabar’s recommended distribution for production environments.

Why Talos?

Feature	Benefit
Immutable	No shell, no SSH - API-only management
Secure	Minimal attack surface, signed images
Declarative	Configuration as YAML, GitOps-friendly
Fast	Boot to Kubernetes in under 2 minutes
Upgrades	Atomic, rollback-capable upgrades

Talos Blueprint

apiVersion: teabar.dev/v1
kind: Blueprint
metadata:
  name: talos-cluster
spec:
  infrastructure:
    provider: hetzner
    location: fsn1

  resources:
    - name: k8s
      type: cluster
      spec:
        distribution: talos
        version: "1.29"
        
        controlPlane:
          count: 1           # 1 for dev, 3 for HA
          size: cx41         # 4 vCPU, 16 GB
          
        workers:
          count: 3
          size: cx31         # 2 vCPU, 8 GB
          
        # Networking
        cni: cilium          # cilium, flannel, or custom
        podCidr: 10.244.0.0/16
        serviceCidr: 10.96.0.0/12
        
        # Addons
        addons:
          - metrics-server
          - local-path-provisioner

Talos Configuration

Customize Talos machine configuration:

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: talos
      version: "1.29"
      
      talosConfig:
        # Control plane patches
        controlPlane:
          patches:
            - op: add
              path: /cluster/apiServer/extraArgs
              value:
                audit-log-path: /var/log/audit.log
                
        # Worker patches  
        worker:
          patches:
            - op: add
              path: /machine/kubelet/extraArgs
              value:
                max-pods: "200"

Accessing Talos Clusters

# Get kubeconfig
teactl access kubeconfig my-env --cluster k8s

# Get talosconfig (for talosctl)
teactl access talosconfig my-env --cluster k8s

# Use talosctl
talosctl --talosconfig ~/.talos/config dashboard

K3s

K3s is a lightweight Kubernetes distribution perfect for development, edge, and resource-constrained environments.

Why K3s?

Feature	Benefit
Lightweight	Single binary, minimal resource usage
Fast	Cluster ready in under 3 minutes
Simple	Built-in ingress, load balancer, storage
Compatible	Full Kubernetes API compatibility

K3s Blueprint

apiVersion: teabar.dev/v1
kind: Blueprint
metadata:
  name: k3s-cluster
spec:
  infrastructure:
    provider: hetzner
    location: fsn1

  resources:
    - name: k8s
      type: cluster
      spec:
        distribution: k3s
        version: "1.29"
        
        controlPlane:
          count: 1
          size: cx21
          
        workers:
          count: 2
          size: cx21
          
        # K3s options
        k3sConfig:
          disableComponents:
            - traefik        # Use your own ingress
          extraArgs:
            - --disable=servicelb

K3s with Built-in Components

K3s includes several components by default:

Component	Default	Purpose
Traefik	Enabled	Ingress controller
ServiceLB	Enabled	Load balancer
Local Path	Enabled	Storage provisioner
CoreDNS	Enabled	DNS
Metrics Server	Disabled	Resource metrics

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: k3s
      version: "1.29"
      
      k3sConfig:
        # Keep built-ins for simple setups
        disableComponents: []
        
        # Or disable and use your own
        disableComponents:
          - traefik
          - servicelb

Amazon EKS

EKS is AWS’s managed Kubernetes service with deep AWS integration.

Why EKS?

Feature	Benefit
Managed	AWS manages control plane
Integration	IAM, ALB, EBS, ECR integration
Compliance	SOC, HIPAA, PCI certifications
Support	AWS enterprise support

EKS Blueprint

apiVersion: teabar.dev/v1
kind: Blueprint
metadata:
  name: eks-cluster
spec:
  infrastructure:
    provider: aws
    region: us-east-1

  resources:
    - name: k8s
      type: cluster
      spec:
        distribution: eks
        version: "1.29"
        
        nodeGroups:
          - name: system
            instanceType: t3.medium
            desiredSize: 2
            minSize: 2
            maxSize: 3
            
          - name: workers
            instanceType: t3.large
            desiredSize: 3
            minSize: 1
            maxSize: 10
            enableAutoScaling: true
            
        # EKS addons
        addons:
          - name: vpc-cni
            version: latest
          - name: coredns
            version: latest
          - name: kube-proxy
            version: latest
          - name: aws-ebs-csi-driver
            version: latest

EKS with IRSA

Enable IAM Roles for Service Accounts:

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: eks
      version: "1.29"
      
      # Enable OIDC provider for IRSA
      oidcProvider:
        enabled: true
        
      nodeGroups:
        - name: workers
          instanceType: t3.large
          desiredSize: 3

  # Create IAM role for a service account
  - name: s3-access
    type: eks-irsa
    spec:
      cluster: k8s
      serviceAccount: my-app
      namespace: default
      policyArns:
        - arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

EKS Node Types

Node Type	Use Case
Managed Node Groups	Default, AWS manages EC2
Self-managed	Custom AMIs, full control
Fargate	Serverless pods

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: eks
      version: "1.29"
      
      # Managed nodes (default)
      nodeGroups:
        - name: managed
          instanceType: t3.large
          desiredSize: 3
          
      # Fargate profile (serverless)
      fargateProfiles:
        - name: serverless
          selectors:
            - namespace: serverless-apps

Azure AKS

AKS is Azure’s managed Kubernetes service with Azure integration.

Why AKS?

Feature	Benefit
Managed	Azure manages control plane (free)
Integration	AAD, ACR, Key Vault, Azure Monitor
Compliance	SOC, HIPAA, PCI certifications
Hybrid	Azure Arc for hybrid/multi-cloud

AKS Blueprint

apiVersion: teabar.dev/v1
kind: Blueprint
metadata:
  name: aks-cluster
spec:
  infrastructure:
    provider: azure
    region: eastus

  resources:
    - name: k8s
      type: cluster
      spec:
        distribution: aks
        version: "1.29"
        
        nodePools:
          - name: system
            vmSize: Standard_D2s_v5
            count: 2
            mode: System
            
          - name: workers
            vmSize: Standard_D4s_v5
            minCount: 1
            maxCount: 10
            enableAutoScaling: true
            mode: User
            
        # Networking
        networkPlugin: azure   # azure or kubenet
        networkPolicy: calico  # azure or calico
        
        # Azure AD integration
        azureAD:
          managed: true
          adminGroupObjectIds:
            - "00000000-0000-0000-0000-000000000000"

AKS with Azure Integrations

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: aks
      version: "1.29"
      
      nodePools:
        - name: workers
          vmSize: Standard_D4s_v5
          count: 3
          
      # Azure Monitor
      monitoring:
        enabled: true
        logAnalyticsWorkspaceId: "/subscriptions/.../workspaces/my-workspace"
        
      # Azure Container Registry
      acr:
        enabled: true
        registryId: "/subscriptions/.../registries/myacr"
        
      # Azure Key Vault Secrets Provider
      keyVault:
        enabled: true
        secretsProvider: true

CNI Options

Cilium (Recommended)

eBPF-based networking with advanced features:

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: talos
      cni: cilium
      
      ciliumConfig:
        hubble:
          enabled: true
          relay:
            enabled: true
          ui:
            enabled: true
        encryption:
          enabled: true
          type: wireguard

Flannel

Simple overlay networking:

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: talos
      cni: flannel
      
      flannelConfig:
        backend: vxlan  # vxlan, host-gw, wireguard

Cloud Provider CNI

Use cloud-native CNI for managed clusters:

# EKS with VPC CNI
resources:
  - name: k8s
    type: cluster
    spec:
      distribution: eks
      cni: aws-vpc-cni  # Uses AWS VPC networking

# AKS with Azure CNI
resources:
  - name: k8s
    type: cluster
    spec:
      distribution: aks
      networkPlugin: azure  # Pods get VNet IPs

Storage Classes

Hetzner (Self-managed)

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: talos
      
      storage:
        default: local-path
        classes:
          - name: local-path
            provisioner: rancher.io/local-path
            default: true
          - name: hcloud-volumes
            provisioner: csi.hetzner.cloud
            parameters:
              type: ssd

AWS EKS

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: eks
      
      addons:
        - name: aws-ebs-csi-driver
          
      storage:
        classes:
          - name: gp3
            provisioner: ebs.csi.aws.com
            parameters:
              type: gp3
              iops: "3000"
            default: true

Azure AKS

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: aks
      
      storage:
        classes:
          - name: managed-premium
            provisioner: disk.csi.azure.com
            parameters:
              skuName: Premium_LRS
            default: true

Cluster Access

Get Kubeconfig

# Download kubeconfig
teactl access kubeconfig my-env --cluster k8s

# Use with kubectl
export KUBECONFIG=~/.teabar/kubeconfig-my-env-k8s
kubectl get nodes

In-cluster Access

For participants accessing the cluster:

spec:
  access:
    kubernetes:
      enabled: true
      # Per-participant namespace
      namespacePerParticipant: true
      # RBAC
      role: edit  # view, edit, admin, cluster-admin

Web Terminal with kubectl

spec:
  access:
    terminal:
      type: shell
      image: bitnami/kubectl:latest
      env:
        - name: KUBECONFIG
          value: /home/user/.kube/config

Cluster Addons

Common addons for workshop environments:

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: talos
      
      addons:
        # Resource monitoring
        - name: metrics-server
          
        # Storage
        - name: local-path-provisioner
          
        # Ingress
        - name: ingress-nginx
          config:
            service:
              type: LoadBalancer
              
        # Certificate management
        - name: cert-manager
          config:
            installCRDs: true
            
        # GitOps
        - name: argocd
          config:
            server:
              ingress:
                enabled: true

High Availability

HA Control Plane

For production environments:

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: talos
      
      controlPlane:
        count: 3           # Odd number for etcd quorum
        size: cx41
        
      # Spread across availability zones (if supported)
      topology:
        zones:
          - fsn1-dc8
          - fsn1-dc14

etcd Configuration

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: talos
      
      etcd:
        snapshotSchedule: "0 */6 * * *"  # Every 6 hours
        snapshotRetention: 5

Upgrades

Automatic Upgrades

resources:
  - name: k8s
    type: cluster
    spec:
      distribution: talos
      version: "1.29"
      
      upgrades:
        automatic: true
        schedule: "0 3 * * 0"  # Sundays at 3 AM
        maxUnavailable: 1

Manual Upgrades

# Check available versions
teactl env upgrade my-env --cluster k8s --list-versions

# Upgrade cluster
teactl env upgrade my-env --cluster k8s --version 1.30

Next Steps

Kubernetes Distributions

Distribution Comparison

Talos Linux

Why Talos?

Talos Blueprint

Talos Configuration

Accessing Talos Clusters

K3s

Why K3s?

K3s Blueprint

K3s with Built-in Components

Amazon EKS

Why EKS?

EKS Blueprint

EKS with IRSA

EKS Node Types

Azure AKS

Why AKS?

AKS Blueprint

AKS with Azure Integrations

CNI Options

Cilium (Recommended)

Flannel

Cloud Provider CNI

Storage Classes

Hetzner (Self-managed)

AWS EKS

Azure AKS

Cluster Access

Get Kubeconfig

In-cluster Access

Web Terminal with kubectl

Cluster Addons

High Availability

HA Control Plane

etcd Configuration

Upgrades

Automatic Upgrades

Manual Upgrades

Next Steps

Networking Guide

Blueprint Resources

Hetzner Provider

Participant Access