Document pre-compare transforms for tenzir-test

src/content/docs/guides/testing/write-tests.mdx

@@ -142,6 +142,36 @@ quiet; the final result line includes `attempts=3/3` (or the actual number on a
 success). Use this as a guardrail while you investigate the underlying flake and
 keep the budget small to avoid masking issues.
 
+### Handle non-deterministic output
+
+Some tests produce output in an unpredictable order, for example when querying
+hash-based aggregations or parallel operations. Instead of retrying until the
+order happens to match, use `pre-compare` transforms to normalize both sides
+before comparison:
+
+```yaml
+---
+pre-compare: sort
+---
+```
+
+The `sort` transform sorts output lines lexicographically before comparing them
+against the baseline. This lets you capture the raw output with `--update` while
+still verifying correctness regardless of order.
+
+```sh
+#!/usr/bin/env bash
+# pre-compare: sort
+
+# Output order depends on filesystem enumeration
+ls /tmp/*.log 2>/dev/null || echo "no logs"
+```
+
+Transforms only affect comparison. The baseline file stores the original
+untransformed output, keeping it human-readable. See
+[pre-compare transforms](/reference/test-framework/#pre-compare-transforms) in
+the reference documentation for the full list of available transforms.
+
 ### Run multiple projects together
 
 Large organisations often split tests across several repositories but still

src/content/docs/reference/test-framework/index.mdx

@@ -443,6 +443,7 @@ Common frontmatter keys:
 | `inputs`       | string          | project   | Override `TENZIR_INPUTS` for this directory or test.             |
 | `retry`        | integer         | `1`       | Total attempt budget for flaky tests (see below).                |
 | `package-dirs` | list of strings | inherit   | Directory-only; extra packages merged with CLI `--package-dirs`. |
+| `pre-compare`  | string or list  | `[]`      | Transform names to apply before comparison (e.g., `sort`).       |
 
 `test.yaml` files accept the same keys and apply recursively to child
 directories. A relative `inputs:` value resolves against the file that defines
@@ -458,6 +459,67 @@ line includes `attempts=N/M` whenever the budget exceeds one. Keep the value
 small and treat it as a temporary guardrail while you fix the underlying
 flakiness.
 
+### Pre-compare transforms
+
+Use `pre-compare` transforms to normalize output before comparison with
+baselines. This helps test scenarios that produce non-deterministic output,
+such as unordered result sets from hash-based aggregations or parallel
+operations.
+
+Configure transforms in `test.yaml` or per-test frontmatter:
+
+```yaml
+# tests/aggregation/test.yaml
+pre-compare: sort
+```
+
+Or as a list for multiple transforms:
+
+```yaml
+pre-compare: [sort]
+```
+
+In shell scripts, use a comment:
+
+```sh
+#!/usr/bin/env bash
+# pre-compare: sort
+
+echo "zebra"
+echo "alpha"
+echo "bravo"
+```
+
+Available transforms:
+
+| Transform | Description                                                                                                                                              |
+| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `sort`    | Sort output lines lexicographically. Handles unordered results from parallel processing, hash-based aggregations, or other non-deterministic operations. |
+
+Transforms apply in order to both the actual test output and the baseline before
+comparison. The original output and baseline files remain unchanged on disk.
+
+**Comparison workflow:**
+
+1. The harness runs the test and captures raw output.
+2. When updating baselines (`--update`), the raw output writes to the `.txt`
+   file without any transformation.
+3. During comparison, both the baseline and actual output pass through the
+   configured transforms before the harness checks for equality.
+4. Diffs display the _transformed_ content when a mismatch occurs, making it
+   easier to identify the actual difference.
+
+This design keeps baselines human-readable while still handling non-determinism.
+You can inspect the `.txt` file to see what the test actually produced, even
+when the comparison uses transformed data.
+
+Common use cases:
+
+- Hash-based aggregations that produce results in arbitrary order
+- Parallel operations where output order depends on execution scheduling
+- Set operations where mathematical correctness does not require order
+- Tests that enumerate filesystem entries or process IDs
+
 ### Tenzir configuration files
 
 - The harness inspects the directory that owns each test. If it finds
@@ -618,6 +680,10 @@ uvx tenzir-test --update
 `--purge` removes stale artifacts (diffs, temporary files). Keep generated
 `.txt` files under version control so future runs can diff against them.
 
+For tests with non-deterministic output, use
+[pre-compare transforms](#pre-compare-transforms) to normalize output before
+comparison while keeping original baselines unchanged.
+
 ## Troubleshooting
 
 - **Missing binaries** – The harness auto-detects binaries on `PATH` and falls