LLM / MCP layer

Goal: let an LLM (Claude via MCP, or Grafana’s in-app assistant) query metrics, dashboards, and logs for alert-noise review, threshold tuning proposals, and overnight anomaly summaries.

Grafana LLM plugin

grafana-llm-app is preinstalled by the compose file (GF_INSTALL_PLUGINS). Configure it under Administration → Plugins → LLM: choose Anthropic (API key) or a local Ollama endpoint. v0.10+ supports non-OpenAI backends. This powers in-Grafana features (panel explanations, incident summaries).

Grafana MCP server (external agent access)

1. Create a service account: Administration → Service accounts → role Viewer (read-only is enough for querying) → add token. Record the token reference in inventory.md.
2. Run mcp-grafana where your MCP client lives, e.g. for Claude Code:

claude mcp add grafana -e GRAFANA_URL=https://grafana.<domain> \
  -e GRAFANA_API_KEY=<service-account-token> \
  -- docker run --rm -i -e GRAFANA_URL -e GRAFANA_API_KEY mcp/grafana -t stdio

The agent can then list dashboards, run PromQL/LogQL, and inspect alerts — which covers Prometheus and Loki too.

Netdata MCP

Netdata’s agent ships a built-in MCP server (free/OSS, on by default in v2.x). Point an MCP client at the Netdata container’s MCP endpoint (see Netdata → Integrations → MCP in its UI for the exact URL/port of your version). Netdata’s edge ML anomaly scoring (anomaly_detection_* charts) is on by default — this is the stack’s anomaly-detection layer, queryable by the LLM.

Uptime Kuma MCP

The community ecosystem is still immature (low-star repos). The most visible option for Kuma 2.x is DavidFuchs/mcp-uptime-kuma (monitor list, maintenance windows, silencing). Treat it as experimental; revisit before relying on it.

Suggested agent workflows

• Alert-noise review: weekly, have the agent pull 7 days of Alertmanager history + firing counts per rule, and propose threshold changes to stack/prometheus/alerts/baseline.yml as a diff.
• Overnight anomaly summary: morning query of Netdata anomaly rates
  • Loki error spikes + Graylog stream volumes, correlated on the shared host/site/role labels.
• Capacity check: monthly predict_linear sweep over disk/memory trends.

Security notes: give agents read-only tokens; the MCP servers run inside your LAN behind Caddy; record every token + scope in inventory.md.