mirror of
https://github.com/Katakate/k7.git
synced 2026-11-18 18:04:20 +00:00
Observability, usage metering, tenants and quotas, and roles with OIDC login. A Helm chart installs K7 on a cluster you already run, API keys carry a role, and k7-agent no longer needs host privileges.
1.3 KiB
1.3 KiB
K7UsageSpoolOverflow
A k7-agent dropped usage events in the last 10 minutes
(k7_usage_events_dropped_total). Its spool at
/var/lib/k7/usage-spool reached metering.spool.maxBytes, so the
oldest segment was deleted before every sink had read it. Usage for
those windows is lost downstream, and the agent's /health reports
usage.status: degraded for 10 minutes.
First three commands
kubectl -n kube-system logs -l app=k7-agent --since=30m --prefix | grep -E '"k7.metering|sink ' | tail -20
k7 nodes storage
kubectl -n kube-system exec ds/k7-agent -- python -c "import json,urllib.request as u;print(json.dumps(json.load(u.urlopen('http://127.0.0.1:8000/health'))['usage'],indent=1))"
Reading them
- A sink with
stoppedset, orlast_errorrepeating: that sink is not draining the spool. See K7UsageSinkStopped, or fix the endpoint; the spool drains on its own once the sink accepts events again. pending_bytesnearmax_bytesfor every sink, with no errors: the sinks keep up too slowly for the event volume. Raise--metering-spool-size(metering.spool.maxBytes), raise the sink'sbatchSize, or scale the sink.- The lost windows are not re-created. Downstream totals for the dropped
period are low by the dropped events' seconds;
last_dropin the spool status bounds when it happened.