Host-level WireGuard for Oracle node (reachable kubelet)¶

This gives the Oracle node an IP the cluster can reach (e.g. 10.99.0.2) so metrics-server and kubectl logs/exec work.

Tunnel: 10.99.0.0/24 — server 10.99.0.1, Oracle 10.99.0.2
WG port: 51820 (open this on the homelab firewall if needed)

You can either use Part 1 (Linux server on yoda) or Part 1b (UCG/UDM as server).

Part 1b: Using UCG Max (or UDM) as the WireGuard server¶

If your UniFi Cloud Gateway Max (or UDM/UDM Pro) has WireGuard server built in, use it instead of running WG on a node. The UCG is usually your LAN gateway (e.g. 192.168.4.1), so cluster nodes can reach the tunnel without extra routes.

On the UCG (UniFi UI)¶

Settings → VPN → WireGuard (or Network → VPN → WireGuard depending on UniFi OS).
Add WireGuard Server (or create a new WG instance):
Name: e.g. oracle-node
Network: create or use a subnet that doesn’t clash with your LAN, e.g. 10.99.0.0/24.
Port: e.g. 51820 (ensure this port is open on WAN or port‑forwarded).
Note the server’s public key and the VPN gateway IP (e.g. 10.99.0.1). The UI may show a sample client config.
Add a peer for the Oracle node:
Name: oracle
Allowed IPs: 10.99.0.2/32 (one IP for this peer).

You need the peer’s public key. Generate it on Oracle (see below) or on any Linux box:

wg genkey | tee /tmp/oracle_private.key | wg pubkey > /tmp/oracle_public.key
cat /tmp/oracle_public.key   # paste this into UCG as the peer public key
cat /tmp/oracle_private.key  # use on Oracle in wg0.conf as PrivateKey

Paste the public key into the UCG peer and save. The UI may show a peer VPN IP (e.g. 10.99.0.2) or you assign it via Allowed IPs.

If the UCG only lets you pick from a pool (e.g. 10.99.0.2, 10.99.0.3…), use the one it assigns (e.g. 10.99.0.2) for Oracle’s K3S_NODE_IP below.

Routing¶

If the UCG is the default gateway for your cluster (e.g. 192.168.4.1), it will route 10.99.0.0/24 over the WireGuard interface. No extra routes on k3s nodes are needed.
If your cluster nodes use a different gateway, add on each node:
ip route add 10.99.0.0/24 via 192.168.4.1 (replace with your UCG LAN IP).

Oracle side (same as Part 2, with UCG values)¶

Endpoint: UCG’s public IP (or DynDNS) and the port you set (e.g. 51820).
Peer PublicKey: the WireGuard server’s public key from the UCG UI.
PrivateKey: the Oracle peer’s private key (from the wg genkey above, or generate on Oracle).
Address: the IP the UCG assigns to this peer (e.g. 10.99.0.2/24).

Then set K3S_NODE_IP=10.99.0.2 (or the IP the UCG gave this peer) and restart k3s-agent as in Part 2.

Part 1: Homelab (one control-plane node, e.g. yoda `192.168.4.100`)¶

SSH to the node that will be the WireGuard server (e.g. yoda).

# 1. Install WireGuard (Debian/Ubuntu)
sudo apt-get update && sudo apt-get install -y wireguard

# 2. Generate server keys
sudo mkdir -p /etc/wireguard/oracle-node
cd /etc/wireguard/oracle-node
sudo wg genkey | sudo tee server_private.key | sudo wg pubkey | sudo tee server_public.key
sudo chmod 600 server_private.key

# 3. Generate Oracle (client) key pair — run this once, you'll need client_public.key on Oracle
sudo wg genkey | sudo tee client_private.key | sudo wg pubkey | sudo tee client_public.key
sudo chmod 600 client_private.key

# 4. Create WG interface config for the Oracle tunnel
sudo tee /etc/wireguard/wg0-oracle.conf << 'WGEOF'
[Interface]
Address = 10.99.0.1/24
ListenPort = 51820
PrivateKey = SERVER_PRIVATE_KEY
# Allow forwarding so cluster can reach Oracle
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT

[Peer]
# Oracle node
PublicKey = CLIENT_PUBLIC_KEY
AllowedIPs = 10.99.0.2/32
WGEOF

# 5. Insert keys into config (replace placeholders)
sudo sed -i "s/SERVER_PRIVATE_KEY/$(sudo cat /etc/wireguard/oracle-node/server_private.key)/" /etc/wireguard/wg0-oracle.conf
sudo sed -i "s/CLIENT_PUBLIC_KEY/$(sudo cat /etc/wireguard/oracle-node/client_public.key)/" /etc/wireguard/wg0-oracle.conf

# 6. Start WireGuard
sudo wg-quick up wg0-oracle

# 7. Enable on boot
sudo systemctl enable wg-quick@wg0-oracle

# 8. Show client private key (copy for Oracle)
echo "--- Use this on Oracle as PrivateKey ---"
sudo cat /etc/wireguard/oracle-node/client_private.key
echo "--- Server public key (Oracle needs this as Peer PublicKey) ---"
sudo cat /etc/wireguard/oracle-node/server_public.key

Add route on other cluster nodes so they can reach 10.99.0.2 via yoda. On each other node (plumbus, jet, rex, spike — not Oracle):

# Run on plumbus, jet, rex, spike (replace 192.168.4.100 with yoda's IP if different)
sudo ip route add 10.99.0.0/24 via 192.168.4.100 2>/dev/null || true

To make the route persistent (Debian, one option):

echo 'up ip route add 10.99.0.0/24 via 192.168.4.100' | sudo tee /etc/network/if-up.d/route-wg-oracle
sudo chmod +x /etc/network/if-up.d/route-wg-oracle

Firewall: ensure UDP 51820 is open on the host that runs the WG server (yoda), and that your router/firewall forwards 51820 → 192.168.4.100 if you're behind NAT. Your external IP (e.g. 88.207.216.67) is what Oracle will connect to.

Part 2: Oracle node¶

SSH to Oracle. You need from Part 1: client private key, server public key, and homelab public IP (e.g. 88.207.216.67) and port 51820.

# 1. Install WireGuard (Oracle Linux / RHEL)
sudo dnf install -y wireguard-tools

# 2. Create WireGuard config
sudo mkdir -p /etc/wireguard
sudo tee /etc/wireguard/wg0.conf << 'WGEOF'
[Interface]
Address = 10.99.0.2/24
PrivateKey = PASTE_CLIENT_PRIVATE_KEY_FROM_HOMELAB

[Peer]
PublicKey = PASTE_SERVER_PUBLIC_KEY_FROM_HOMELAB
Endpoint = HOMELAB_PUBLIC_IP:51820
AllowedIPs = 10.99.0.0/24
PersistentKeepalive = 25
WGEOF

# 3. Replace placeholders (run each line, paste the real values)
# sudo nano /etc/wireguard/wg0.conf
#   - PASTE_CLIENT_PRIVATE_KEY_FROM_HOMELAB  → output of: sudo cat /etc/wireguard/oracle-node/client_private.key (on homelab)
#   - PASTE_SERVER_PUBLIC_KEY_FROM_HOMELAB  → output of: sudo cat /etc/wireguard/oracle-node/server_public.key (on homelab)
#   - HOMELAB_PUBLIC_IP  → e.g. 88.207.216.67

# 4. Bring up WireGuard
sudo wg-quick up wg0

# 5. Enable on boot (so it's up before k3s-agent)
sudo systemctl enable wg-quick@wg0

# 6. Verify you have 10.99.0.2
ip -4 addr show wg0

Point k3s-agent at the new IP:

# 7. Create env file for k3s-agent (use the WireGuard IP)
echo 'K3S_NODE_IP=10.99.0.2' | sudo tee /etc/systemd/system/k3s-agent.service.env

# 8. Restart k3s-agent so it re-registers with 10.99.0.2
sudo systemctl daemon-reload
sudo systemctl restart k3s-agent

Ensure WireGuard starts before k3s-agent:

# 9. Add dependency so k3s-agent starts after wg0
sudo mkdir -p /etc/systemd/system/k3s-agent.service.d
sudo tee /etc/systemd/system/k3s-agent.service.d/after-wireguard.conf << 'EOF'
[Unit]
After=network-online.target wg-quick@wg0.service
Wants=wg-quick@wg0.service
EOF

sudo systemctl daemon-reload

Part 3: Verify¶

On homelab (from a node that has the route, or from yoda):

# Ping Oracle's tunnel IP
ping -c 2 10.99.0.2

# Optional: test kubelet from a pod (after node re-registers)
kubectl get nodes -o wide
# Oracle should show InternalIP 10.99.0.2

From your laptop (with kubeconfig):

kubectl get nodes -o wide
kubectl top nodes
kubectl logs -n kube-system deployment/metrics-server --tail=20

Oracle should appear with InternalIP 10.99.0.2 and metrics-server should stop timing out on that node.

One-liner reference¶

Option A – UCG Max / UDM as server: In UniFi UI add a WireGuard server (e.g. 10.99.0.0/24, port 51820), add one peer for Oracle (AllowedIPs 10.99.0.2/32) with a generated key pair. On Oracle: wg0 with that IP, server public key, endpoint = UCG public IP:51820, then K3S_NODE_IP=10.99.0.2 and restart k3s-agent. No routes on nodes if UCG is your gateway.

Option B – Homelab (yoda): install WG, create keys, wg0-oracle with server 10.99.0.1, peer 10.99.0.2, start and enable. Export client private key and server public key for Oracle.

Other nodes (only if not using UCG as gateway): sudo ip route add 10.99.0.0/24 via 192.168.4.100 (or via UCG 192.168.4.1).

Oracle: install WG, wg0 with Address = 10.99.0.2/24, Peer = server public key, Endpoint = UCG or homelab public IP:51820, then K3S_NODE_IP=10.99.0.2 and restart k3s-agent.