New executor documentation

[mavam]

Summary

Feature branch for the new executor documentation.

Individual sub-PRs will target this branch so they can be reviewed and merged
incrementally. Once all pieces are in place, this branch merges into main.

(Continuation of #192, moved to a topic branch after reverting the premature merge to main.)

Test plan

• Sub-PRs pass CI individually
• Final merge into main passes build

[github-actions]

📦 Preview  ·  View →  ·  🟢 Live

_{Verified for 3bc10df  ·  Auto-updates on push}

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds MySQL integration docs and diagram; introduces new operators (from_stdin, from_mysql, read_tql, accept_tcp, to_tcp, serve_tcp, from_tcp); updates many TCP-related docs to new accept/to/serve operator forms; adjusts several operator docs (batch, measure, parallel) and sidebar to include MySQL.

Changes

Cohort / File(s)	Summary
MySQL integration src/content/docs/integrations/mysql.mdx, src/content/docs/integrations/mysql.excalidraw	New MySQL integration page with examples (from_mysql), deployment notes, and accompanying Excalidraw diagram asset.
New operator docs src/content/docs/reference/operators/from_stdin.mdx, src/content/docs/reference/operators/from_mysql.mdx, src/content/docs/reference/operators/read_tql.mdx, src/content/docs/reference/operators/accept_tcp.mdx, src/content/docs/reference/operators/to_tcp.mdx, src/content/docs/reference/operators/serve_tcp.mdx, src/content/docs/reference/operators/from_tcp.mdx	Adds reference pages for new operators: syntax, examples, TLS options, behaviors (streaming, reconnection, routing, piping bytes), and See Also links.
TCP-related docs & examples src/content/docs/integrations/index.mdx, src/content/docs/integrations/tcp.mdx, src/content/docs/guides/collecting/get-data-from-the-network.mdx, src/content/docs/integrations/syslog.mdx, src/content/docs/integrations/microsoft/windows-event-logs.mdx, src/content/docs/guides/node-setup/configure-tls.mdx, src/content/docs/explanations/configuration.mdx, src/content/docs/guides/routing/*.md, src/content/docs/reference/operators/*read_*.md	Replaces many load_tcp/save_tcp usages and prose with new accept_tcp/to_tcp/serve_tcp/from_tcp forms; consolidates TLS guidance into tls option object; updates examples to nested block form where applicable.
Operator adjustments src/content/docs/reference/operators/batch.md, src/content/docs/reference/operators/measure.md, src/content/docs/reference/operators/parallel.mdx, src/content/docs/reference/operators/read_tql.mdx, src/content/docs/reference/operators/read_tql.mdx	Batch: clarified per-schema buffers and explicit default timeout; Measure: removed real_time option from signature; Parallel: added optional jobs and route_by semantics, examples, and reordering notes; added read_tql doc.
Cross-references & see-also updates src/content/docs/reference/operators.mdx, src/content/docs/reference/operators/load_tcp.mdx, src/content/docs/reference/operators/save_tcp.mdx, src/content/docs/reference/operators/write_tql.mdx, src/content/docs/reference/operators/*	Inserted new operator entries (from_stdin, from_mysql, read_tql, from_tcp, to_tcp, accept_tcp, serve_tcp) into operator lists and See Also sections; updated deprecation notes and removed/reshuffled older TCP references.
Sidebar / Index src/sidebar.ts, src/content/docs/integrations/index.mdx	Added integrations/mysql to the sidebar Data Tools list; reformatted integration index prose and added AI-assisted package generation reference.

Sequence Diagram(s)

mermaid
sequenceDiagram
participant Client as Client
participant Tenzir as Tenzir (from_mysql)
participant MySQL as MySQL DB

Client->>Tenzir: Submit pipeline using from_mysql (table/sql/stream)
Tenzir->>MySQL: Connect using URI/host+creds (TLS optional)
MySQL-->>Tenzir: Return rows / stream rows
Tenzir->>Tenzir: Map MySQL types -> internal types, track watermark (tracking_column)
Tenzir-->>Client: Emit events (initial results / live streamed rows)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 I nibbled docs by lantern light,
Added MySQL and operators bright,
TCP now listens, serves, and sends,
Pipelines hop to new rabbit friends,
Diagrams, examples—hoppy night!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'New executor documentation' is vague and generic, using non-descriptive terms that don't clearly convey the specific changes in this PR.	Consider a more specific title that reflects the main documentation additions, such as 'Add MySQL and TCP operator documentation' or 'Document new MySQL integration and TCP operators'.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description check	✅ Passed	The PR description clearly explains that this is a feature branch for new executor documentation, noting it's a continuation of previous work and will be merged incrementally via sub-PRs.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch topic/new-executor

📝 Coding Plan

• Generate coding plan for human review comments

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/content/docs/reference/operators/measure.md (1)

39-90: 🧹 Nitpick | 🔵 Trivial

Consider adding an example demonstrating the cumulative parameter.

The examples effectively show basic usage of measure, but none demonstrate the cumulative parameter—the only configurable option for this operator. Adding an example with cumulative=true would help users understand the difference between per-batch statistics and running totals.

For instance, you could add an example showing how measure cumulative=true produces running totals for events or bytes, making it clearer when to use this option versus post-processing with summarize.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/content/docs/reference/operators/measure.md` around lines 39 - 90, Add a
new example demonstrating the cumulative parameter for the measure operator:
show a short TQL snippet that loads a file (e.g., load_file "eve.json" /
read_suricata) and runs measure with cumulative=true (measure cumulative=true)
and include the expected output illustrating running totals (increasing events
or bytes across batches) so readers can compare it to the existing
non-cumulative examples; reference the measure operator and the cumulative=true
flag and mention summarizing with summarize (e.g., summarize schema,
events=sum(events)) as a contrast.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/content/docs/reference/operators.mdx`:
- Around line 362-373: Add the new operator entry to the operators index
frontmatter and the rendered card grid by inserting an operators list item with
name: 'from_tcp', description: 'Reads events from a TCP socket.', example:
'from_tcp "0.0.0.0:8090" { read_json }' and path: 'reference/operators/from_tcp'
into the frontmatter operators: array, and add a corresponding ReferenceCard
block for from_tcp in the grid (match the existing card structure used for
from_stdin/from_kafka/from_mysql) so the new reference/operators/from_tcp page
appears in the overview and card grid.

In `@src/content/docs/reference/operators/from_mysql.mdx`:
- Around line 24-29: The doc incorrectly states that show is mutually exclusive
with table; clarify that the three modes are: Table mode (use table to read
rows), SQL mode (use sql to run custom queries), and Show mode (use show to list
objects) but note that show is mutually exclusive with sql while some show
values (e.g., show="columns") require a table parameter; update the top summary
and the other occurrences referencing the modes (mentions of table/sql/show and
the example using show="columns") to explicitly state "show is exclusive with
sql, but show='columns' accepts/needs table='<name>'" and adjust the wording at
the other sections (lines referenced) accordingly so examples and constraints
are consistent with the parameter behavior.
- Around line 30-32: Update the sentence about MySQL prepared statements to
clarify they protect bound values, not identifiers: change the line mentioning
"prevent SQL injection through user-provided table and column names" to say that
internal metadata queries (e.g., resolving tracking columns for live mode)
validate or allowlist user-provided table and column names before executing
prepared statements, and that prepared statements are used to safely bind
values; for example: "Internal metadata queries (such as resolving tracking
columns for live mode) validate user-provided table and column names against a
safe list and then use MySQL prepared statements to safely bind values and
prevent SQL injection."

In `@src/content/docs/reference/operators/from_stdin.mdx`:
- Around line 30-40: Update the shell example so it actually runs the TQL
pipeline shown below: pipe the JSON into the tenzir CLI and invoke the TQL query
that uses from_stdin { read_json } (i.e. pass the TQL snippet to tenzir so the
input is parsed by read_json). Ensure the echo ... | tenzir invocation includes
the TQL query for from_stdin/read_json so the command is copy-paste runnable.

---

Outside diff comments:
In `@src/content/docs/reference/operators/measure.md`:
- Around line 39-90: Add a new example demonstrating the cumulative parameter
for the measure operator: show a short TQL snippet that loads a file (e.g.,
load_file "eve.json" / read_suricata) and runs measure with cumulative=true
(measure cumulative=true) and include the expected output illustrating running
totals (increasing events or bytes across batches) so readers can compare it to
the existing non-cumulative examples; reference the measure operator and the
cumulative=true flag and mention summarizing with summarize (e.g., summarize
schema, events=sum(events)) as a contrast.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: d95507d1-9d7b-4695-89c3-ccfd6f408c66

📥 Commits

Reviewing files that changed from the base of the PR and between 5299540 and 2621f79.

📒 Files selected for processing (16)

• src/content/docs/integrations/index.mdx
• src/content/docs/integrations/mysql.excalidraw
• src/content/docs/integrations/mysql.mdx
• src/content/docs/integrations/tcp.mdx
• src/content/docs/reference/operators.mdx
• src/content/docs/reference/operators/batch.md
• src/content/docs/reference/operators/from_mysql.mdx
• src/content/docs/reference/operators/from_stdin.mdx
• src/content/docs/reference/operators/from_tcp.mdx
• src/content/docs/reference/operators/load_tcp.mdx
• src/content/docs/reference/operators/measure.md
• src/content/docs/reference/operators/parallel.mdx
• src/content/docs/reference/operators/read_tql.mdx
• src/content/docs/reference/operators/save_tcp.mdx
• src/content/docs/reference/operators/write_tql.mdx
• src/sidebar.ts

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/content/docs/integrations/tcp.mdx`:
- Around line 39-45: In the "SSL/TLS" section remove the duplicated fragment
"like `certfile` and `keyfile`." so the sentence reads only once; locate the
repeated text under the SSL/TLS heading and delete the extra occurrence, then
verify spacing and punctuation around the remaining sentence to ensure the
paragraph reads cleanly.

In `@src/content/docs/reference/operators/save_tcp.mdx`:
- Around line 73-75: Update the "See Also" block to remove the deprecated
reference to load_tcp and replace it with the new migration targets—use to_tcp
and serve_tcp alongside from_tcp and keep the Integration tcp marker; locate the
See Also entries (symbols: from_tcp, load_tcp, Integration tcp) in save_tcp.mdx
and change load_tcp to to_tcp and/or serve_tcp to match the deprecation guidance
and the migration pattern (also consider updating the example referenced by the
example at Line 58 in a separate follow-up for consistency).

In `@src/content/docs/reference/operators/to_tcp.mdx`:
- Around line 29-32: Add a consistent TLS section header before the TLSOptions
partial: insert a "### tls = record (optional)" header immediately above where
TLSOptions is rendered (the <TLSOptions/> partial import/render in to_tcp.mdx)
so it matches the style used in accept_tcp.mdx and from_tcp.mdx; ensure the
header text exactly matches the other files for consistency.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: ebfdf061-7af7-4bff-a708-3be8c6917466

📥 Commits

Reviewing files that changed from the base of the PR and between 2621f79 and 6956c8c.

📒 Files selected for processing (21)

• src/content/docs/explanations/configuration.mdx
• src/content/docs/guides/collecting/get-data-from-the-network.mdx
• src/content/docs/guides/node-setup/configure-tls.mdx
• src/content/docs/guides/routing/load-balance-pipelines.mdx
• src/content/docs/guides/routing/send-to-destinations.mdx
• src/content/docs/integrations/microsoft/windows-event-logs.mdx
• src/content/docs/integrations/syslog.mdx
• src/content/docs/integrations/tcp.mdx
• src/content/docs/reference/functions/secret.mdx
• src/content/docs/reference/operators/accept_tcp.mdx
• src/content/docs/reference/operators/from.mdx
• src/content/docs/reference/operators/from_tcp.mdx
• src/content/docs/reference/operators/load_balance.mdx
• src/content/docs/reference/operators/load_tcp.mdx
• src/content/docs/reference/operators/read_delimited_regex.mdx
• src/content/docs/reference/operators/read_gelf.mdx
• src/content/docs/reference/operators/read_lines.mdx
• src/content/docs/reference/operators/read_syslog.mdx
• src/content/docs/reference/operators/save_tcp.mdx
• src/content/docs/reference/operators/serve_tcp.mdx
• src/content/docs/reference/operators/to_tcp.mdx