rttf

rttf · recipe pack · version 1.1.0

Home-lab k3s monitoring in 1.5 GiB of RAM

A complete monitoring stack for a small k3s cluster. It emails you when a site goes down, fits beside your apps instead of eating their RAM, and deploys with one Git push.

  • Deployed from zero
  • x86_64 and arm64
  • Ready for AI coding agents
Grafana Golden Signals dashboard: sites up, tunnel error share, request rate, CPU steal, and latency per site
~1.5 GiBRAM for the whole stack
7alerts that matter, by email
2–3 minfrom Git push to ready
15/15targets up in the test

The problem

The stock chart is built for a data centre. Your cluster is three mini PCs.

Install kube-prometheus-stack with its defaults and you get hundreds of thousands of series and rules evaluated every few seconds. And when a site goes down, it says nothing. Your first alert is a friend asking "is your site down?"

The default install

  • Eats the RAM your apps need
  • Dashboards you never open
  • No alert goes anywhere
  • Rebuilt by hand after every reinstall

This recipe

  • About 1.5 GiB of RAM, every trim explained
  • One Golden Signals dashboard that answers "is it OK?"
  • Seven alerts, delivered to your inbox
  • Lives in Git; Flux puts it back

What you get

Everything for "is my cluster OK?", in one folder

Flux manifests, a small dashboard generator and a runbook. Copy the folder into your Flux repository, fill in one settings file, push.

The full stack, Flux-ready

Prometheus, Alertmanager, Grafana, Loki with fluent-bit, and blackbox probes for your sites.

Small by design

About 1.5 GiB of RAM and 40 millicores of CPU on a 3-node cluster. The README explains every trim, so you can undo any of it.

Golden Signals dashboard

Latency, traffic, errors and saturation in 19 panels, plus a generator for cloudflared, Garage and nginx panels.

Seven alerts, by email

Site down, target down, node not ready, crash looping, disk or volume filling up, CPU steal.

A runbook

Deploy, verify, update and tear down, with every setting in one table.

Longer retention, measured

3 days of metrics and 7 of logs by default; each is one setting. The README shows the disk it costs and how to measure yours.

Made for AI coding agents

Your agent does the typing. You do the approving.

The recipe ships with an AGENTS.md and a CLAUDE.md written as an operating procedure: what to ask you, what to check first, the exact steps, how to verify, and where to stop for your approval.

Illustration of the steps in AGENTS.md, not a recorded session.

Tested, not promised

Deployed on an empty VM, following the README as written

On x86_64 and again on arm64. Every check below passed, and each run started from VM creation.

  • Ready 2 to 3 minutes after the push
  • 15 targets up, 7 of 7 alert rules healthy
  • 3 of 3 probes succeeding
  • Dashboard present at first start, logs queryable
  • Test alert delivered by email
  • Same result on x86_64 and on arm64
  • HA on three nodes, one stopped mid-run: alert emailed once, no log line lost
Two alert emails side by side: a red FIRING NodeNotReady card and the green RESOLVED one twelve minutes later, each with the summary, labels as boxed chips and an Open Grafana button
The alert email: one loud colour per severity, and the all-clear when it's over.
File tree of the 1.1.0 download: START-HERE.md and recipes/observability with AGENTS.md, CHANGELOG.md, CLAUDE.md, README.md, VERSIONS.md, a flux folder with the manifests, the ha overlay and the grafana-postgres component, and a dashboard generator
Exactly what is in the zip.

Is it for you?

Built for small clusters, on purpose

Yes, if you run k3s on one to a few small nodes, host real sites on it, and want to know when something breaks.

No, if you want high availability on one or two nodes: by default everything runs once, on one node's disk, and if that node dies your monitoring dies with it. With three nodes and an S3 bucket, the included HA option keeps alerting, metrics and logs going when a node is lost.

What you need

  • k3s with the local-path storage class
  • Flux v2.10 or newer, and kubectl
  • An SMTP account for the email alerts
  • x86_64 or arm64 (64-bit)

Pinned versions

  • k3s v1.37.2 · Flux v2.10.0
  • kube-prometheus-stack 93.1.1
  • Prometheus 3.16.0 · Grafana 13.3.0
  • operator v0.95.1 (v0.94.2 on arm64)
  • Loki 3.7.9 · fluent-bit 5.2.0 · blackbox v0.29.1

Questions

Before you buy

Does it run on a Raspberry Pi?

It runs on 64-bit ARM: it was tested from zero on an arm64 VM, where you set one documented value for the operator image. It was not tested on a Pi itself, and 32-bit ARM is not supported.

Can I keep more than 3 days of metrics?

Yes. Retention is one setting for metrics and one for logs. Longer retention mainly needs disk: about 65 MiB a day for metrics and 10 MiB a day for logs on a 3-node home cluster. The README shows how to measure yours and how to cap the size.

Is it highly available?

Not by default: the 1.5 GiB setup runs everything once. Since version 1.1.0 the download includes an HA option for three nodes: two Prometheus, two clustered Alertmanagers and three Lokis with the logs in S3 (the S3 recipe works), plus two Grafanas on Postgres if you want them. Tested from zero with a node stopped mid-run: the alert about it was emailed, one alert through two Prometheus and two Alertmanagers arrived once, and no log line was lost. It costs about 0.7 GiB more on a 3-node home cluster.

Do I need an AI agent to use it?

No. The README is a complete runbook for doing it by hand. The AGENTS.md is there if you would rather have your coding agent follow the same steps, asking you before each change.

What do I actually download?

A zip with Flux manifests, a settings example, a dashboard generator in Python, a README runbook, a VERSIONS.md listing what was tested, and the agent files. Nothing in it needs access to anyone else's infrastructure.

Know when something breaks. Keep your RAM for your apps.

One folder, one settings file, one Git push. Tested from zero on x86_64 and arm64.

Get the recipe €24