Bootstrapping New Cluster

Bootstrapping a new Cluster is Easy.

We just need a few things.

Name	Description
K3S Version	Version of K3s to install
VM or Host Machine	Server to install K3s Node on
IP Address	IP address of the Node (10.10.60.67)
DNS Names	DNS address of the Node (name.clstr.c0.spakl)

Cluster Config

It is Much easier to create a Config File over using flags

k3s_config.yml

cluster-init: true
write-kubeconfig-mode: 0644

tls-san:
  - "127.0.0.1" # Keep
  ## Set your IPS and Hostnames ##
  - "10.10.60.67"
  - "10.10.60.68"
  - "10.10.60.69"
  - "pltfrm.c0.clstr.spakl"
  - "n0.pltfrm.c0.clstr.spakl"
  - "n1.pltfrm.c0.clstr.spakl"
  - "n2.pltfrm.c0.clstr.spakl"

disable:
  - "servicelb"
  - "traefik"

kubelet-arg:
- "containerd=/run/k3s/containerd/containerd.sock"
- "node-status-update-frequency=60s"

kube-apiserver-arg:
- "default-not-ready-toleration-seconds=30"
- "default-unreachable-toleration-seconds=30"
## OAuth ##
- "oidc-issuer-url=https://auth.spakl.io"
- "oidc-client-id=29967478786@kubernetes"
- "oidc-username-claim=email"
- "oidc-groups-claim=groups"

kube-controller-arg:
- "node-monitor-period=60s"
- "node-monitor-grace-period=60s"

## Metrics ##
etcd-expose-metrics: true

kube-controller-manager-arg:
- "bind-address=0.0.0.0"

kube-scheduler-arg:
- "bind-address=0.0.0.0"

kube-proxy-arg:
- "metrics-bind-address=0.0.0.0"

TLS San

It is really important to fill this out. If you don't, when you try to use kubectl to talk to your cluster it will get a certificate error. Keep 127.0.0.1. That is needed.

You can replace my examples with your own. I like to add...

• IP Addresses of nodes
• Dns Address of nodes
• KubeVIP Dns Name i will use for HA

Disbale

We are going to deploy Traefik and MetalLB so we will disable the built in services.

Now the Config File is Ready to Go

Install Script

Create a bootstrap.sh script that will run and deploy the cluster

bootstrap.sh

#!/usr/bin/env bash
K3S_VERSION=v1.30.6+k3s1

## Increase Limits for FS (for grafana alloy and log collecting)
sudo sysctl fs.inotify.max_user_instances=1280
sudo sysctl fs.inotify.max_user_watches=655360

## Move config file into correct spot
sudo mkdir -p /etc/rancher/k3s
sudo cp ./k3s_config.yml /etc/rancher/k3s/config.yaml


## Download and install cluster at Version
curl -sfL https://get.k3s.io \
        | INSTALL_K3S_VERSION=${K3S_VERSION} sh -s -

## For Convenience
mkdir -p $HOME/k3s
sudo cp /var/lib/rancher/k3s/server/node-token $HOME/k3s/node-token
sudo cp /etc/rancher/k3s/k3s.yaml $HOME/k3s/k3s.yml
sudo chmod 644 $HOME/k3s/node-token

## Done
echo "${HOSTNAME} is ready!"
sudo kubectl get nodes

Deploy

However you see fit, copy these two files into the server.

The location they exist at doesnt matter.

I will put them at ~/ or $HOME

$HOME/
├── bootstrap.sh
└── k3s_config.yml

Run Script

Give Script Executable Permission

chmod +x ./bootstrap.sh

Run script

./bootstrap.sh

When it is done it will spit out your node-token (for joining other nodes) and kubeconfig in ~/k3s

$HOME/
├── k3s
│   ├── k3s.yml
│   └── node-token
├── bootstrap.sh
└── k3s_config.yml

Copy the k3s.yml to your $HOME/.kube/config and update the ip from

127.0.0.1 to the dns name you picked or cluster ip

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ==
    ## All are valid if they exist in the tls-san section of the cluster config
    server: https://n0.pltfrm.c0.clstr.spakl:6443
    # server: https://10.10.60.67:6443
    # server: https://pltfrm.c0.clstr.spakl:6443

...OMITTED...