From c76c429e12123f67393535bd2ce99d7748e97400 Mon Sep 17 00:00:00 2001 From: Ondrej Patocka <80347071+ondrejpatocka@users.noreply.github.com> Date: Wed, 8 Apr 2026 15:43:17 +0200 Subject: [PATCH 1/2] Add AGENTS.md maintenance protocol and dashboard/QAN component guides --- .../instructions/dashboards.instructions.md | 58 +++++++ .github/instructions/qan-app.instructions.md | 149 ++++++++++++++++++ AGENTS.md | 23 ++- 3 files changed, 228 insertions(+), 2 deletions(-) create mode 100644 .github/instructions/dashboards.instructions.md create mode 100644 .github/instructions/qan-app.instructions.md diff --git a/.github/instructions/dashboards.instructions.md b/.github/instructions/dashboards.instructions.md new file mode 100644 index 0000000000..9a39e3a1b9 --- /dev/null +++ b/.github/instructions/dashboards.instructions.md @@ -0,0 +1,58 @@ +--- +applyTo: dashboards/dashboards/**,dashboards/misc/** +--- +# Grafana Dashboards Development Guidelines + +> **Parent guide**: [AGENTS.md](../../AGENTS.md) — product overview, architecture, domain model, global conventions +> **Related**: [qan-app.instructions.md](qan-app.instructions.md) (Grafana plugin that bundles these dashboards) · [managed.instructions.md](managed.instructions.md) (server backend providing metrics data) + +The `dashboards/dashboards/` directory contains Grafana dashboard JSON definitions organized by database and domain area. These are the canonical source for all PMM monitoring dashboards. The `dashboards/misc/` directory provides Python helper scripts for importing, exporting, and converting dashboard JSON files. + +## Architecture + +Dashboard JSON files are standard Grafana dashboard exports. They are loaded into Grafana through two mechanisms: + +1. **Plugin bundling** — `pmm-app/src/plugin.json` declares each dashboard in its `includes` array. The build copies them into the plugin `dist/` directory, and Grafana provisions them when the pmm-app plugin is loaded. +2. **Grafana provisioning** — the PMM Server Ansible role configures Grafana to load dashboards from the plugin's `dist/dashboards/` path. + +``` +dashboards/dashboards/*.json + → pmm-app build (copied to dist/dashboards/) + → Grafana provisioning on PMM Server + → Grafana UI (visualization) +``` + +## Dashboard Categories + +| Directory | Domain | +|-----------|--------| +| `MySQL/` | MySQL, PXC/Galera, Aurora, ProxySQL, HAProxy | +| `MongoDB/` | MongoDB, WiredTiger, MMAPv1, InMemory, PBM, ReplSet | +| `PostgreSQL/` | PostgreSQL, Patroni | +| `OS/` | Node, CPU, memory, disk, network, NUMA, processes | +| `Valkey/` | Valkey/Redis clients, cluster, memory, replication, slowlog | +| `Insight/` | Home Dashboard, Advanced Data Exploration, VictoriaMetrics, Exporters | +| `Experimental/` | Environment, DB Cluster, PMM Health, PostgreSQL Vacuum | +| `Query Analytics/` | QAN panel wrapper (`pmm-qan.json`) | +| `Kubernetes (experimental)/` | Kubernetes operator monitoring | + +## Patterns and Conventions + +### Do +- Design dashboards in the Grafana UI, then export the JSON +- Use `misc/cleanup-dash.py` to normalize exported JSON before committing +- Follow the existing directory structure when adding dashboards for a new domain +- Keep one dashboard per JSON file, named to match the dashboard title +- Register new dashboards in `pmm-app/src/plugin.json` under the `includes` array + +### Don't +- Don't edit dashboard JSON by hand unless making targeted fixes — use the Grafana UI for design work +- Don't duplicate dashboard JSON inside `pmm-app/src/` — the canonical source is `dashboards/dashboards/` +- Don't commit Grafana-generated volatile fields (e.g., `version`, `iteration`) — use `cleanup-dash.py` to strip them + +## Key Files to Reference + +- `dashboards/dashboards/` — all dashboard JSON definitions, organized by domain +- `dashboards/pmm-app/src/plugin.json` — plugin manifest that registers dashboards in Grafana +- `dashboards/README.md` — featured dashboards list +- `dashboards/CONTRIBUTING.md` — contribution workflow and local dev setup diff --git a/.github/instructions/qan-app.instructions.md b/.github/instructions/qan-app.instructions.md new file mode 100644 index 0000000000..34d569c41c --- /dev/null +++ b/.github/instructions/qan-app.instructions.md @@ -0,0 +1,149 @@ +--- +applyTo: dashboards/pmm-app/** +--- +# QAN App (pmm-app) Development Guidelines + +> **Parent guide**: [AGENTS.md](../../AGENTS.md) — product overview, architecture, domain model, global conventions +> **Related**: [dashboards.instructions.md](dashboards.instructions.md) (dashboard JSON definitions bundled by this plugin) · [ui.instructions.md](ui.instructions.md) (main PMM frontend) · [api.instructions.md](api.instructions.md) (API definitions consumed by QAN) · [qan-api2.instructions.md](qan-api2.instructions.md) (QAN backend) + +The `dashboards/pmm-app/` directory contains a **Grafana application plugin** (`type: app`, `id: pmm-app`) that bundles PMM dashboard JSON definitions and provides the custom **Query Analytics (QAN) panel** (`pmm-qan-app-panel`). It is built with TypeScript and React on top of Grafana's plugin SDK. + +## Architecture + +### Plugin Structure + +The pmm-app plugin consists of two sub-plugins registered in their respective `plugin.json` manifests: + +1. **App plugin** (`pmm-app/src/plugin.json`) — declares the application, registers PMM dashboard JSON includes, and exposes the QAN panel. +2. **Panel plugin** (`pmm-app/src/pmm-qan/plugin.json`) — declares the `pmm-qan-app-panel` panel type used by `Query Analytics/pmm-qan.json`. + +``` +pmm-app/src/module.ts → AppPlugin() (minimal app shell) +pmm-app/src/pmm-qan/module.ts → PanelPlugin(QueryAnalyticsPanel) + +plugin.json includes[]: + - dashboards from dashboards/dashboards/**/*.json + - panel: pmm-qan-app-panel + +Build (webpack) → dist/ + → deployed to Grafana plugins directory on PMM Server +``` + +### Key Technology Choices + +| Technology | Role | +|------------|------| +| **TypeScript** | Type-safe development | +| **React 18** | UI framework | +| **Webpack** | Build tooling (Grafana plugin scaffolding) | +| **Yarn 1.x** | Package manager (`packageManager: yarn@1.22.21`) | +| **SCSS / LESS** | Styling | +| **@grafana/data, @grafana/ui, @grafana/runtime** | Grafana plugin SDK (`>=11.x.x`) | +| **Ant Design** | Additional UI components (QAN panel) | +| **axios** | HTTP client for QAN API calls | +| **react-table** | Table rendering in QAN Overview | +| **d3** | Data visualization | +| **Jest 29** | Unit testing (`@swc/jest`, `jest-environment-jsdom`) | + +## QAN Panel + +The Query Analytics panel lives in `pmm-app/src/pmm-qan/` and is registered as a `PanelPlugin` wrapping the `QueryAnalytics` React component. + +### Key Sub-Components + +| Component | Path | Purpose | +|-----------|------|---------| +| **QueryAnalytics** | `pmm-qan/panel/QueryAnalytics.tsx` | Root panel component | +| **Overview** | `pmm-qan/panel/components/Overview/` | Main query table with sortable metrics columns | +| **Details** | `pmm-qan/panel/components/Details/` | Query detail view: Explain, Metrics, Metadata, Table | +| **Filters** | `pmm-qan/panel/components/Filters/` | Filter sidebar (dimension, value filtering) | +| **BarChart** | `pmm-qan/panel/components/BarChart/` | Time-distribution bar chart | +| **ManageColumns** | `pmm-qan/panel/components/ManageColumns/` | Column visibility picker | + +### Shared Code + +`pmm-app/src/shared/` contains reusable code across the QAN panel: +- `components/` — common UI elements (Table, Modal, Charts, Icons, Form controls) +- `components/helpers/` — humanization, formatting, validators +- `components/hooks/` — shared React hooks (e.g., window size) +- `global-styles/themes/` — dark/light theme SCSS variables + +## Patterns and Conventions + +### Do +- Co-locate test files next to components (`*.test.tsx`) +- Use `@testing-library/react` for component tests +- Use `@grafana/data` and `@grafana/ui` APIs for Grafana integration +- Use the existing provider pattern in `pmm-qan/panel/provider/` for QAN state +- Follow the Grafana plugin SDK conventions for panel lifecycle + +### Don't +- Don't modify files under `.config/` — they are scaffolded by `@grafana/create-plugin` and carry "do not edit" warnings +- Don't introduce new state management libraries — use React state/context as in existing QAN code +- Don't duplicate dashboard JSON inside `pmm-app/src/` — the canonical source is `dashboards/dashboards/` +- Don't bypass the Grafana plugin SDK APIs for data queries or runtime services + +## Testing + +- **Framework**: Jest 29 with `@swc/jest` transform, `jest-environment-jsdom` +- **Libraries**: `@testing-library/react`, `@testing-library/jest-dom`, `@testing-library/user-event`, `jest-canvas-mock`, `mockdate` +- **Config**: `pmm-app/jest.config.js` extends `pmm-app/.config/jest.config.js`; sets `TZ=GMT` +- **Pattern**: ~35 co-located `*.test.tsx` / `*.test.ts` files under `pmm-app/src/` +- **Run**: `make test` (from `dashboards/`) or `yarn test:ci` (from `pmm-app/`) +- **Linting**: `yarn lint` runs ESLint on `src/**/*.{ts,tsx}`; `yarn typecheck` runs `tsc --noEmit` + +## Development Workflow + +```bash +# Prerequisites: Node >= 18, Yarn 1.x +cd dashboards/pmm-app + +# Install dependencies +yarn install --frozen-lockfile + +# Start webpack in watch mode (development) +yarn dev + +# Production build +yarn build + +# Run tests (CI mode) +yarn test:ci + +# Lint and typecheck +yarn lint && yarn typecheck +``` + +### Docker Development + +`pmm-app/docker-compose.yaml` provides a local Grafana environment that mounts `./dist` into the PMM Server plugin directory: + +```bash +cd dashboards/pmm-app +docker-compose up -d +yarn dev +``` + +### Makefile Targets (from `dashboards/`) + +| Target | Purpose | +|--------|---------| +| `make install` | `yarn install --frozen-lockfile` in `pmm-app/` | +| `make build` | `yarn build` in `pmm-app/` | +| `make release` | `install` + `build` | +| `make test` | `release` + `yarn test:ci` | +| `make clean` | Remove `pmm-app/dist/` | +| `make install-plugins` | Download ClickHouse and Polystat Grafana plugins | + +## Key Files to Reference + +- `dashboards/Makefile` — build, test, and release targets +- `dashboards/pmm-app/package.json` — dependencies, scripts, engine requirements +- `dashboards/pmm-app/src/plugin.json` — app plugin manifest (dashboard includes, panel registration) +- `dashboards/pmm-app/src/pmm-qan/plugin.json` — QAN panel plugin manifest +- `dashboards/pmm-app/src/module.ts` — app plugin entry point +- `dashboards/pmm-app/src/pmm-qan/module.ts` — QAN panel entry point +- `dashboards/pmm-app/src/pmm-qan/panel/QueryAnalytics.tsx` — root QAN panel component +- `dashboards/pmm-app/jest.config.js` — test configuration +- `dashboards/pmm-app/docker-compose.yaml` — local development environment +- `dashboards/CONTRIBUTING.md` — contribution workflow and local dev setup diff --git a/AGENTS.md b/AGENTS.md index f1629fcdd4..5d305bc69f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,7 +7,23 @@ - .github/copilot-instructions.md (GitHub Copilot auto-discovery) Component-specific guides: - .github/instructions/.instructions.md (GitHub Copilot applyTo scoping) - Last reviewed: 2026-03 --> + Last reviewed: 2026-04 --> + +## Maintaining This Document + +This file is read by every AI agent at session start. **You are responsible for keeping it accurate.** After completing work, check whether any of these apply: + +- Added, removed, or renamed a top-level directory or component +- Added or removed a component guide in `.github/instructions/` +- Changed the tech stack (new dependency in `go.mod`, new tool, removed technology) +- Changed build targets in `Makefile` / `Makefile.include` +- Changed global conventions (code style, error handling, testing patterns) +- Changed architecture or data-flow (new pipeline, changed communication protocol) +- Changed the development environment (`docker-compose.yml`, `.devcontainer/`) + +If any apply, update the relevant sections of this file and bump the `Last reviewed` date in the HTML comment at the top. Also update the matching component guide in `.github/instructions/` if one exists for the affected area. + +Do **not** update this file for routine code changes (bug fixes, minor feature implementation) that don't alter the repo's structure or conventions. ## How This Documentation Is Organized @@ -26,6 +42,8 @@ Each PMM component has a dedicated guide with architecture, directory structure, | **qan-api2** (query analytics) | [qan-api2.instructions.md](.github/instructions/qan-api2.instructions.md) | `qan-api2/**` | | **vmproxy** (VictoriaMetrics proxy) | [vmproxy.instructions.md](.github/instructions/vmproxy.instructions.md) | `vmproxy/**` | | **UI** (React frontend) | [ui.instructions.md](.github/instructions/ui.instructions.md) | `ui/**` | +| **Dashboards** (Grafana dashboard definitions) | [dashboards.instructions.md](.github/instructions/dashboards.instructions.md) | `dashboards/dashboards/**` | +| **QAN App** (Grafana plugin & QAN panel) | [qan-app.instructions.md](.github/instructions/qan-app.instructions.md) | `dashboards/pmm-app/**` | | **API Tests** (integration tests) | [api-tests.instructions.md](.github/instructions/api-tests.instructions.md) | `api-tests/**` | | **Build & Packaging** | [build.instructions.md](.github/instructions/build.instructions.md) | `build/**` | @@ -102,6 +120,8 @@ Relationships: | `/qan-api2` | qan-api2 | Query Analytics API: ClickHouse ingestion and analytics | [qan-api2.instructions.md](.github/instructions/qan-api2.instructions.md) | | `/vmproxy` | vmproxy | VictoriaMetrics reverse proxy with LBAC filtering | [vmproxy.instructions.md](.github/instructions/vmproxy.instructions.md) | | `/ui` | UI | React/TypeScript PMM frontend (Vite, MUI, TanStack Query) | [ui.instructions.md](.github/instructions/ui.instructions.md) | +| `/dashboards/dashboards` | Grafana Dashboards | Grafana dashboard JSON definitions for MySQL, MongoDB, PostgreSQL, OS, and more | [dashboards.instructions.md](.github/instructions/dashboards.instructions.md) | +| `/dashboards/pmm-app` | QAN App | Grafana application plugin bundling dashboards and the Query Analytics panel | [qan-app.instructions.md](.github/instructions/qan-app.instructions.md) | | `/api-tests` | API Tests | Integration tests against live PMM Server | [api-tests.instructions.md](.github/instructions/api-tests.instructions.md) | | `/build` | Build & Packaging | Docker, RPM/DEB, Packer, Ansible | [build.instructions.md](.github/instructions/build.instructions.md) | @@ -120,7 +140,6 @@ Relationships: | Repository | Purpose | |------------|---------| -| [percona/grafana-dashboards](https://github.com/percona/grafana-dashboards) | PMM Grafana dashboards for database monitoring | | [percona/grafana](https://github.com/percona/grafana) | Percona's Grafana fork with PMM customizations | | [percona/node_exporter](https://github.com/percona/node_exporter) | Machine-level metrics exporter | | [percona/mysqld_exporter](https://github.com/percona/mysqld_exporter) | MySQL server metrics exporter | From c20571bbb09ad291b4c4b109212a8824049b61ec Mon Sep 17 00:00:00 2001 From: "ondrej.patocka" <80347071+ondrejpatocka@users.noreply.github.com> Date: Thu, 9 Apr 2026 10:17:14 +0200 Subject: [PATCH 2/2] Update AGENTS.md - remove the commented notes to shorten the file --- AGENTS.md | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 5d305bc69f..f3d8119e4c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,14 +1,5 @@ # PMM Development Guide for AI Agents - - ## Maintaining This Document This file is read by every AI agent at session start. **You are responsible for keeping it accurate.** After completing work, check whether any of these apply: @@ -21,7 +12,7 @@ This file is read by every AI agent at session start. **You are responsible for - Changed architecture or data-flow (new pipeline, changed communication protocol) - Changed the development environment (`docker-compose.yml`, `.devcontainer/`) -If any apply, update the relevant sections of this file and bump the `Last reviewed` date in the HTML comment at the top. Also update the matching component guide in `.github/instructions/` if one exists for the affected area. +If any apply, update the relevant sections of this file. Also update the matching component guide in `.github/instructions/` if one exists for the affected area. Do **not** update this file for routine code changes (bug fixes, minor feature implementation) that don't alter the repo's structure or conventions.