Home-Lab k3s Monitoring in 1.5 GiB of RAM: Prometheus, Grafana, Loki, Flux-ready

A complete monitoring stack for a small home-lab k3s cluster. It fits in about 1.5 GiB of RAM, emails you when something breaks, and deploys with one Git push. Made for people who run k3s on a few mini PCs or a NAS VM, host real sites on it, and have an hour a week for ops, which they would honestly rather spend on something else. Why I made it On my first morning with the new cluster, the control plane had restarted ten times in the night. I stopped things one by one to find out why, and pausing the monitoring was one of the biggest wins. Then I started Prometheus again, and k3s crashed again within minutes. The thing meant to watch my cluster was helping to knock it over. The stock kube-prometheus-stack is built for a data centre. On three mini PCs it still collects hundreds of thousands of series, re-checks its rules every few seconds and eats the RAM your apps wanted. And when a site goes down, it says nothing, so your first alert is a friend asking "is your site down?". Friends are very good at that message. So the stack went on a diet, from 204 rules to 46, and this recipe is what came out of it. What you get The full stack, Flux-ready. Prometheus, Alertmanager, Grafana, Loki with fluent-bit, and blackbox probes for your sites. Small on purpose. About 1.5 GiB of RAM and 40 millicores of CPU on a 3-node cluster. Every trim is explained in the README, so you can undo any of it if you disagree with me. A Golden Signals dashboard. Latency, traffic, errors and saturation in 19 panels, plus a small generator for cloudflared, Garage and nginx panels. Seven alerts that matter, by email. Site down, target down, node not ready, crash looping, disk or volume filling up, CPU steal. The email is built to be read at 3 a.m.: one loud colour per severity, one card per alert, and a button to Grafana. Optional: high availability on three nodes. Two Prometheus, two clustered Alertmanagers and three Lokis with the logs in S3, plus two Grafanas on Postgres if you want them. Lose a node and you still get the email about it, and you get it once. A runbook to deploy, verify, update and tear down, with every setting in one table. Made for AI coding agents It ships with an AGENTS.md and a CLAUDE.md written as an operating procedure: what to ask you, what to check first, the exact steps, how to verify, and where to stop for your approval (Git pushes, cluster changes). Your agent does the typing, and you do the approving. How it was tested I deployed it on an empty VM, followed the README as written, and wrote down what happened: ✅ Ready 2 to 3 minutes after the push ✅ 15 targets up, 7 of 7 alert rules healthy, 3 of 3 probes succeeding ✅ Dashboard present at first start, logs queryable ✅ Test alert delivered by email ✅ Same result on x86_64 and on arm64 ✅ HA on three nodes, with one stopped mid-run: the alert was emailed, one alert through two Prometheus and two Alertmanagers arrived once, and no log line was lost (x86_64 and arm64) Is it for you? Yes, if you run k3s on one to a few small nodes and want to hear that something broke before your friends tell you. No, if you want high availability on one or two nodes. By default everything runs once, on one node's disk, so if that node dies, your monitoring dies with it and can't even send you the email. With three nodes and an S3 bucket, the included HA option keeps alerts, metrics and logs going for about 0.7 GiB more. Retention is 3 days of metrics and 7 days of logs by default. Each is one setting, and longer retention mostly needs disk: about 65 MiB a day for metrics and 10 MiB a day for logs on a 3-node home cluster. The README shows how to measure yours. What you need k3s with the local-path storage class Flux v2.10 or newer, and kubectl An SMTP account for the email alerts For HA: three nodes and an S3 bucket (the S3 + static sites recipe works) Runs on x86_64 and arm64 (64-bit). On arm64 you set one documented value for the operator image. Not tested on a Raspberry Pi itself. Pinned versions This recipe: version 1.1.0 k3s v1.37.2 and Flux v2.10.0 kube-prometheus-stack 93.1.1: Prometheus 3.16.0, Grafana 13.3.0, operator v0.95.1 (v0.94.2 on arm64) Loki 3.7.9, fluent-bit 5.2.0, blackbox-exporter v0.29.1