What to bring in
The signals that matter most when agents are investigating or writing RCAs:- Alert state (Grafana Alerting API) to surface firing alerts as insight candidates.
- Dashboards (
/api/dashboards/*) for the panels a team uses during an incident. - Datasource proxy queries (
/api/datasources/proxy/...) to execute PromQL, LogQL, or SQL through Grafana’s datasource layer. - Annotations to read deploy markers and runbook entries.
Which path to pick
Start with REST
Grafana ships a stable HTTP API. Point the custom REST integration at it with an OpenAPI wrapper.
Move to MCP when you need more
Wrap datasource queries behind named tools (for example
prom_via_grafana(service, window)).Quick recipe: REST path
Create a Grafana service account token
In Grafana, create a service account with a read role and generate a token.
Register the endpoint
In the RubixKube console, go to Integrations → Custom REST and register the Grafana API.
Enable the calls you care about
Start with alerts list, dashboard read, and datasource proxy queries for the specific datasources you want to reach through Grafana.
Set auth
Add the service account token to the workspace secret vault. Map it to the
Authorization: Bearer <token> header.Example tools worth having
grafana_firing_alerts(service)— returns alerts currently firing that tag or match the service.grafana_dashboard(id)— returns the panels and queries from a dashboard, useful for agents trying to reproduce what a human would look at.grafana_datasource_query(datasource, query, window)— runs a query through Grafana’s datasource proxy (PromQL, LogQL, SQL, etc).
Related guides
Custom REST Integrations
The full REST flow with auth options and approval rules.
Custom MCP Servers
When you want richer tool schemas and multi-call workflows.
Skills
How to compose Grafana queries into custom workflows.