diff --git a/src/content/docs/guides/collecting/index.mdx b/src/content/docs/guides/collecting/index.mdx index c2b683232..1762f945f 100644 --- a/src/content/docs/guides/collecting/index.mdx +++ b/src/content/docs/guides/collecting/index.mdx @@ -71,6 +71,17 @@ from_kafka "security-events", offset="end" See the [message broker guide](/guides/collecting/read-from-message-brokers) for broker-specific configurations. +### Data stores + +Query external data stores such as MySQL and ClickHouse: + +```tql +from_clickhouse sql="SELECT * FROM events WHERE severity >= 3", tls=false +``` + +See the [data store guide](/guides/collecting/read-from-data-stores) for table +reads, SQL pushdown, metadata inspection, and live MySQL polling. + ### Network data Receive data over TCP or UDP sockets, or capture packets from network @@ -98,5 +109,6 @@ connections. - collecting/read-and-watch-files - collecting/fetch-via-http-and-apis - collecting/read-from-message-brokers +- collecting/read-from-data-stores - collecting/get-data-from-the-network - routing/send-to-destinations diff --git a/src/content/docs/guides/collecting/read-from-data-stores.mdx b/src/content/docs/guides/collecting/read-from-data-stores.mdx new file mode 100644 index 000000000..5277b587b --- /dev/null +++ b/src/content/docs/guides/collecting/read-from-data-stores.mdx @@ -0,0 +1,149 @@ +--- +title: Read from data stores +--- + +This guide shows you how to read from external data stores with TQL. You'll learn how to read full tables, push filters into SQL, inspect metadata, and stream new rows from MySQL. + +Today, this guide focuses on from_mysql and from_clickhouse. As Tenzir adds more data store integrations, the same patterns will apply. + +:::note +`from_clickhouse` currently requires the new pipeline executor. Run pipelines that use it with `tenzir --neo`. +::: + +## Read a full table + +Use `table=...` when you want to fetch all rows from a table. + +Read a ClickHouse table: + +```tql +from_clickhouse table="security.events", + host="clickhouse.example.com", + password=secret("CLICKHOUSE_PASSWORD"), + tls=false +``` + +Or use a ClickHouse URI: + +```tql +from_clickhouse uri="clickhouse://default:secret@clickhouse.example.com:9000/security", + table="events", + tls=false +``` + +Read a MySQL table: + +```tql +from_mysql table="users", + host="mysql.example.com", + user="tenzir", + password=secret("MYSQL_PASSWORD"), + database="identity" +``` + +Use this mode when you want Tenzir to treat the table as the source of truth and +apply filtering later in the pipeline. + +## Push filters and projections into SQL + +Use `sql=...` when the data store should do the filtering, sorting, or column +selection before Tenzir receives the rows. + +Filter in ClickHouse: + +```tql +from_clickhouse sql="SELECT time, host, severity, message FROM events WHERE severity >= 3 ORDER BY time DESC", + host="clickhouse.example.com", + password=secret("CLICKHOUSE_PASSWORD"), + tls=false +``` + +Filter in MySQL: + +```tql +from_mysql sql="SELECT id, user, last_login FROM users WHERE active = 1 ORDER BY last_login DESC", + host="mysql.example.com", + user="tenzir", + password=secret("MYSQL_PASSWORD"), + database="identity" +``` + +This pattern reduces network traffic and lets you use the source system's query +planner and indexes. + +## Inspect metadata + +Both operators can return metadata instead of table rows. + +List ClickHouse tables in a database: + +```tql +from_clickhouse sql="SHOW TABLES FROM security", + host="clickhouse.example.com", + password=secret("CLICKHOUSE_PASSWORD"), + tls=false +``` + +Show the columns of a MySQL table: + +```tql +from_mysql table="users", + show="columns", + host="mysql.example.com", + user="tenzir", + password=secret("MYSQL_PASSWORD"), + database="identity" +``` + +Use metadata queries when you want to discover available tables, inspect a +schema, or validate assumptions before you run a larger pipeline. In +ClickHouse, prefer regular SQL such as `SHOW`, `DESCRIBE`, or queries against +system catalogs. + +## Poll for new rows from MySQL + +MySQL supports a live polling mode for tables with a monotonically increasing +integer tracking column. + +```tql +from_mysql table="audit_log", + live=true, + tracking_column="id", + host="mysql.example.com", + user="tenzir", + password=secret("MYSQL_PASSWORD"), + database="security" +where severity == "high" +``` + +Use this mode when you want to turn a database table into a continuously polled +event source. + +`from_clickhouse` does not currently provide a comparable live mode. It runs a +query, emits the result, and then finishes. + +## Shape rows after reading + +Both operators produce structured events, so you can transform the result right +away. + +```tql +from_clickhouse sql="SELECT host, severity, message FROM events", + host="clickhouse.example.com", + password=secret("CLICKHOUSE_PASSWORD"), + tls=false +where severity >= 3 +message = message.upper() +``` + +This lets you treat database rows like any other event stream in Tenzir. + +## See also + +- collecting/read-from-message-brokers +- transformation/filter-and-select-data +- routing/send-to-destinations +- from_clickhouse +- from_mysql +- clickhouse +- mysql diff --git a/src/content/docs/guides/routing/send-to-destinations.mdx b/src/content/docs/guides/routing/send-to-destinations.mdx index 5e0deb9f4..9fb57ff90 100644 --- a/src/content/docs/guides/routing/send-to-destinations.mdx +++ b/src/content/docs/guides/routing/send-to-destinations.mdx @@ -3,13 +3,14 @@ title: Send to destinations --- This guide shows you how to send data to various destinations using TQL output -operators. You'll learn about destination operators, file output patterns, and -expression-based serialization. +operators. You'll learn about message destinations, data stores, file output +patterns, and expression-based serialization. ## Destination operators -TQL provides `to_*` operators for sending events to various destinations. These -operators accept expressions for flexible serialization. +TQL provides `to_*` operators for sending events to various destinations. +Message-oriented operators accept expressions for flexible serialization, while +data store operators write structured rows directly. ### Message brokers @@ -32,6 +33,34 @@ to_kafka "events", message=this.print_json() The `message` parameter accepts any expression that evaluates to a string or blob. +### Data stores + +Send events to data stores like clickhouse and +snowflake. + +Send structured events to ClickHouse: + +```tql +subscribe "security-events" +to_clickhouse table="alerts", primary=time, mode="create_append", tls=false +``` + +Write batches to Snowflake with bulk ingestion: + +```tql +export +to_snowflake \ + account_identifier="org-account", + user_name="tenzir_user", + password=secret("SNOWFLAKE_PASSWORD"), + database="SECURITY", + schema="PUBLIC", + table="EVENTS" +``` + +These operators preserve event structure instead of requiring a `message` +expression. + ### Analytics platforms Send data to platforms like splunk, @@ -57,8 +86,8 @@ to_opensearch "https://opensearch.example.com:9200", ### Cloud services -Route events to cloud destinations like amazon/sqs -and google/cloud-pubsub. +Route events to cloud destinations like [Amazon SQS](/integrations/amazon/sqs) +and [Google Cloud Pub/Sub](/integrations/google/cloud-pubsub). Send to SQS: @@ -71,7 +100,9 @@ Send to Pub/Sub: ```tql subscribe "events" -to_gcp_pubsub "projects/my-project/topics/events" +to_google_cloud_pubsub project_id="my-project", + topic_id="events", + message=this.print_json() ``` ## File output @@ -105,12 +136,15 @@ save_file "s3://bucket/logs/events.jsonl" Send NDJSON over tcp: ```tql -to_tcp "collector.example.com:5044" { write_json } +to_tcp "collector.example.com:5044" { + write_json +} ``` -## Expression-based serialization +## Expression-based serialization for message destinations -Destination operators use expressions for flexible message formatting: +Message-oriented destination operators use expressions for flexible message +formatting: ### Serialize the entire event @@ -150,9 +184,12 @@ to_kafka f"events.{event_type}", message=this.print_json() ## See also +- collecting/read-from-data-stores - routing/load-balance-pipelines - routing/split-and-merge-streams +- to_clickhouse - to_kafka -- to_splunk - to_opensearch +- to_snowflake +- to_splunk - fork diff --git a/src/content/docs/integrations/clickhouse.excalidraw b/src/content/docs/integrations/clickhouse.excalidraw new file mode 100644 index 000000000..61c2f2dd2 --- /dev/null +++ b/src/content/docs/integrations/clickhouse.excalidraw @@ -0,0 +1,2787 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://app.excalidraw.com", + "elements": [ + { + "id": "8MN5I4hT3wVen_QUHAUnu", + "type": "rectangle", + "x": 1023.6306664679144, + "y": 476.4109026851011, + "width": 151.42450136036612, + "height": 143.23101048605469, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "Zy", + "roundness": { + "type": 3 + }, + "seed": 239993862, + "version": 521, + "versionNonce": 33394306, + "isDeleted": false, + "boundElements": [ + { + "type": "text", + "id": "Q90BHXvcB9LwBRiLiSjA9" + } + ], + "updated": 1776405904244, + "link": null, + "locked": false + }, + { + "id": "Q90BHXvcB9LwBRiLiSjA9", + "type": "text", + "x": 1061.767916385158, + "y": 481.4109026851011, + "width": 75.1500015258789, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "Zz", + "roundness": null, + "seed": 1597739846, + "version": 502, + "versionNonce": 1309437506, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false, + "text": "Database", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "center", + "verticalAlign": "top", + "containerId": "8MN5I4hT3wVen_QUHAUnu", + "originalText": "Database", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "type": "rectangle", + "version": 511, + "versionNonce": 925925210, + "index": "a0", + "isDeleted": true, + "id": "pxsLHRu0DqDRMIfSBq1IP", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "dotted", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 312.6865893307831, + "y": 326, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 438.62682133843384, + "height": 260, + "seed": 612159942, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false + }, + { + "id": "Y7J38UXbiwPtdxyo9fYNr", + "type": "rectangle", + "x": 392.6865893307831, + "y": 406, + "width": 280, + "height": 100, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a1", + "roundness": null, + "seed": 1774889222, + "version": 25, + "versionNonce": 1906129734, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false + }, + { + "id": "lXsZ5Iq09sJ6hKvV8PUW9", + "type": "text", + "x": 497.211590856662, + "y": 411, + "width": 70.94999694824219, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "hachure", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a2", + "roundness": null, + "seed": 1306236998, + "version": 25, + "versionNonce": 1851491354, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "text": "Pipeline", + "fontSize": 20, + "fontFamily": 5, + "textAlign": "center", + "verticalAlign": "top", + "containerId": "Y7J38UXbiwPtdxyo9fYNr", + "originalText": "Pipeline", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "type": "rectangle", + "version": 489, + "versionNonce": 1512170118, + "index": "a3", + "isDeleted": true, + "id": "MS113qib66n_byZ-dYsHP", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "dotted", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 852.6865893307831, + "y": 326, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 438.62682133843384, + "height": 260, + "seed": 948739974, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false + }, + { + "id": "7tWVFQ1hznFjHbRX77A40", + "type": "rectangle", + "x": 932.6865893307831, + "y": 386, + "width": 300, + "height": 180, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a4", + "roundness": { + "type": 3 + }, + "seed": 236633798, + "version": 108, + "versionNonce": 994570458, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false + }, + { + "id": "2J7vPgwu8j0T0zofCOk6w", + "type": "text", + "x": 1045.1115885678437, + "y": 391, + "width": 75.1500015258789, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a5", + "roundness": null, + "seed": 589493766, + "version": 90, + "versionNonce": 1913627078, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "text": "Database", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "center", + "verticalAlign": "top", + "containerId": "7tWVFQ1hznFjHbRX77A40", + "originalText": "Database", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "e_FOcnN9fF4M0BdfxaDrN", + "type": "rectangle", + "x": 972.6865893307831, + "y": 426, + "width": 220, + "height": 100, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a6", + "roundness": { + "type": 3 + }, + "seed": 815940934, + "version": 127, + "versionNonce": 831047066, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false + }, + { + "id": "rrcWQeq44H5YTrv0GY8Bx", + "type": "text", + "x": 1050.5949221554902, + "y": 431, + "width": 64.18333435058594, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a7", + "roundness": null, + "seed": 1207282822, + "version": 114, + "versionNonce": 478378246, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "text": "Schemas", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "center", + "verticalAlign": "top", + "containerId": "e_FOcnN9fF4M0BdfxaDrN", + "originalText": "Schemas", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "type": "arrow", + "version": 1532, + "versionNonce": 2052129370, + "isDeleted": true, + "id": "ijxomphViqLxL8CKt_0IF", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 692.6865893307831, + "y": 466, + "strokeColor": "#e03131", + "backgroundColor": "#ffec99", + "width": 220, + "height": 0, + "seed": 1188703174, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": { + "elementId": "7tWVFQ1hznFjHbRX77A40", + "mode": "orbit", + "fixedPoint": [ + 0.4444444444444442, + 0.4444444444444444 + ] + }, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 220, + 0 + ] + ], + "index": "a8" + }, + { + "id": "yN4z2aruFERH55r0brpmx", + "type": "rectangle", + "x": 772.6865893307831, + "y": 446, + "width": 60, + "height": 40, + "angle": 0, + "strokeColor": "#e03131", + "backgroundColor": "#f8f9fa", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "a9", + "roundness": { + "type": 3 + }, + "seed": 1265419014, + "version": 102, + "versionNonce": 1358587974, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false + }, + { + "id": "YP3kZRs98Jib89suc6x0T", + "type": "text", + "x": 779.9115897122529, + "y": 456, + "width": 45.54999923706055, + "height": 20, + "angle": 0, + "strokeColor": "#e03131", + "backgroundColor": "#ffec99", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aA", + "roundness": null, + "seed": 673870406, + "version": 102, + "versionNonce": 544450330, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "text": "ADBC", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "yN4z2aruFERH55r0brpmx", + "originalText": "ADBC", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "LBHcAyLWAiRS1VWAnYrWL", + "type": "rectangle", + "x": 993.1573163962655, + "y": 457.10806858517185, + "width": 79.72278435602915, + "height": 56.34336263774654, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aB", + "roundness": null, + "seed": 1475361158, + "version": 658, + "versionNonce": 256995206, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false + }, + { + "id": "uhjj5mdyx378bMwZOmacA", + "type": "line", + "x": 993.5181039582695, + "y": 481.73451980525397, + "width": 78.67259902626967, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aC", + "roundness": { + "type": 2 + }, + "seed": 1912281286, + "version": 235, + "versionNonce": 1550037978, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 78.67259902626967, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "wxTOD61F5OPXXBIDtpGxt", + "type": "line", + "x": 1009.2526237635235, + "y": 481.73451980525397, + "width": 0.12437256409457782, + "height": 31.7169114176644, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aD", + "roundness": { + "type": 2 + }, + "seed": 501747718, + "version": 232, + "versionNonce": 1405456070, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.12437256409457782, + 31.7169114176644 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "AcUsH6RUS9OLVvg18AAKQ", + "type": "line", + "x": 1025.2422604774129, + "y": 482.05089551798244, + "width": 0.17604688976762406, + "height": 31.400535704935898, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aE", + "roundness": { + "type": 2 + }, + "seed": 129244998, + "version": 240, + "versionNonce": 1614031002, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.17604688976762406, + 31.400535704935898 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "lSYkjHCVf2YIeF5v7QZjU", + "type": "line", + "x": 1041.4312823638481, + "y": 482.0360932873964, + "width": 0.42710638798645123, + "height": 31.415337935521947, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aF", + "roundness": { + "type": 2 + }, + "seed": 1404847750, + "version": 240, + "versionNonce": 183748102, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.42710638798645123, + 31.415337935521947 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "osejFYryyYUlSgz6iQVXc", + "type": "line", + "x": 1056.8729787460302, + "y": 481.95251889913084, + "width": 0.06915961804813092, + "height": 31.498912323787575, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aG", + "roundness": { + "type": 2 + }, + "seed": 314410438, + "version": 240, + "versionNonce": 597165402, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.06915961804813092, + 31.498912323787575 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "QJe234d_RzbV02CQo2Uzr", + "type": "text", + "x": 1012.4241827594619, + "y": 458.7444843651541, + "width": 42.39994812011719, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aH", + "roundness": null, + "seed": 903142662, + "version": 121, + "versionNonce": 98112838, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "text": "Table", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Table", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "X6VRn41RLIDfBeVhxV29d", + "type": "rectangle", + "x": 1092.7584223962656, + "y": 456.8270985851718, + "width": 79.72278435602915, + "height": 56.34336263774654, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aI", + "roundness": null, + "seed": 41897030, + "version": 706, + "versionNonce": 803359258, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false + }, + { + "id": "5o6eLAhe7QDtwfTp0CkhY", + "type": "line", + "x": 1093.1192099582695, + "y": 481.45354980525394, + "width": 78.67259902626967, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aJ", + "roundness": { + "type": 2 + }, + "seed": 110744454, + "version": 283, + "versionNonce": 1842510982, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 78.67259902626967, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "gvZuZ0NqSRado6wU51goL", + "type": "line", + "x": 1108.8537297635235, + "y": 481.45354980525394, + "width": 0.12437256409457782, + "height": 31.7169114176644, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aK", + "roundness": { + "type": 2 + }, + "seed": 207712966, + "version": 280, + "versionNonce": 1812330202, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.12437256409457782, + 31.7169114176644 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "yVzJb6tPtqkBULz3LqRQa", + "type": "line", + "x": 1124.843366477413, + "y": 481.7699255179824, + "width": 0.17604688976762406, + "height": 31.400535704935898, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aL", + "roundness": { + "type": 2 + }, + "seed": 242428422, + "version": 288, + "versionNonce": 949812166, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.17604688976762406, + 31.400535704935898 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "9rbT0wMUwUK6Ga-xkfsT1", + "type": "line", + "x": 1141.0323883638482, + "y": 481.7551232873964, + "width": 0.42710638798645123, + "height": 31.415337935521947, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aM", + "roundness": { + "type": 2 + }, + "seed": 734787910, + "version": 288, + "versionNonce": 1754663834, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.42710638798645123, + 31.415337935521947 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "jOEeTGeIkgds_XcW51U0P", + "type": "line", + "x": 1156.4740847460303, + "y": 481.6715488991308, + "width": 0.06915961804813092, + "height": 31.498912323787575, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aN", + "roundness": { + "type": 2 + }, + "seed": 466149510, + "version": 288, + "versionNonce": 475370246, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.06915961804813092, + 31.498912323787575 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "xEI-VKH-Qj0Aweqeg26JH", + "type": "text", + "x": 1112.025288759462, + "y": 458.4635143651542, + "width": 42.39994812011719, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aO", + "roundness": null, + "seed": 572972998, + "version": 169, + "versionNonce": 734365786, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "text": "Table", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Table", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "type": "line", + "version": 1524, + "versionNonce": 58278470, + "isDeleted": true, + "id": "I-akP1h_MfsBs3WnlaqpY", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 498.58828246518095, + "y": 446.8875532984406, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 70.35754617365805, + "height": 35.178773086829025, + "seed": 50525958, + "groupIds": [ + "Zm4VUaypghdAlHCLFMc64" + ], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": null, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": null, + "points": [ + [ + 0, + 0 + ], + [ + 17.589386543414523, + 17.58938654341451 + ], + [ + 3.552713678800501e-15, + 35.178773086829025 + ], + [ + 52.768159630243545, + 35.17877308682902 + ], + [ + 70.35754617365805, + 17.589386543414538 + ], + [ + 52.76815963024356, + 0 + ], + [ + 0, + 0 + ] + ], + "index": "aP", + "polygon": false + }, + { + "type": "line", + "version": 1676, + "versionNonce": 2103930138, + "isDeleted": true, + "id": "hTHCXGLox1-6KFItcVf6X", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 574.0022304422087, + "y": 447.09035574946415, + "strokeColor": "#1971c2", + "backgroundColor": "#ced4da", + "width": 70.3575461736581, + "height": 35.28193106911539, + "seed": 1994008134, + "groupIds": [ + "Zm4VUaypghdAlHCLFMc64" + ], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": null, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": null, + "points": [ + [ + 0, + 0 + ], + [ + 16.054749690096525, + 17.36392457284035 + ], + [ + -1.5346368533179948, + 34.953311116254866 + ], + [ + 68.8229093203401, + 34.95331111625485 + ], + [ + 68.58605202297224, + -0.3286199528605209 + ], + [ + 0, + 0 + ] + ], + "index": "aQ", + "polygon": false + }, + { + "type": "line", + "version": 1710, + "versionNonce": 15010182, + "isDeleted": true, + "id": "StHUX5ckaob2vDhCP1NVO", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 423.06775260815704, + "y": 446.976207295103, + "strokeColor": "#000000", + "backgroundColor": "#ced4da", + "width": 70.35754617365805, + "height": 35.178773086829025, + "seed": 512322950, + "groupIds": [ + "Zm4VUaypghdAlHCLFMc64" + ], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": null, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": null, + "points": [ + [ + 0, + 0 + ], + [ + 3.552713678800501e-15, + 35.178773086829025 + ], + [ + 52.768159630243545, + 35.17877308682902 + ], + [ + 70.35754617365805, + 17.589386543414538 + ], + [ + 52.76815963024356, + 0 + ], + [ + 0, + 0 + ] + ], + "index": "aR", + "polygon": false + }, + { + "id": "Y8Ntms1glDCjSG_0XGCL5", + "type": "image", + "x": 1008.3122995580559, + "y": 337.03124975000003, + "width": 148.7485795454546, + "height": 35.570312500000014, + "angle": 0, + "strokeColor": "transparent", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aS", + "roundness": null, + "seed": 2001587398, + "version": 268, + "versionNonce": 649625798, + "isDeleted": true, + "boundElements": [], + "updated": 1741097843325, + "link": null, + "locked": false, + "status": "error", + "fileId": "7bce60b8c15c30d4b3b4e7fa5fc02163346bfd973ce9a5d018fa64fdb8d35a4cd875428988936279276f795843fab791", + "scale": [ + 1, + 1 + ], + "crop": null + }, + { + "id": "ZSdQbLPBJQ-3vi-SDkUnG", + "type": "image", + "x": 423.75250910769563, + "y": 337.03125, + "width": 217.8681609999999, + "height": 35.57031199999998, + "angle": 0, + "strokeColor": "transparent", + "backgroundColor": "#ffc9c9", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aT", + "roundness": null, + "seed": 1321485318, + "version": 237, + "versionNonce": 854505946, + "isDeleted": true, + "boundElements": [], + "updated": 1741097881189, + "link": null, + "locked": false, + "status": "saved", + "fileId": "e82434d4ac1cd6730a4ed376c59edff2e9c0022ad4ee9b38c806c3d7b63a8e86555f9d9c604e3478ba272e9154108d4a", + "scale": [ + 1, + 1 + ], + "crop": null + }, + { + "type": "rectangle", + "version": 482, + "versionNonce": 990543198, + "index": "aU", + "isDeleted": false, + "id": "MdBLs5DrRJv7pPr1ECBdP", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "dotted", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 860, + "y": 400.0000004254188, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 460.00000000000006, + "height": 259.99999957458124, + "seed": 1012386970, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1776407010951, + "link": null, + "locked": false + }, + { + "type": "arrow", + "version": 1436, + "versionNonce": 484510594, + "isDeleted": false, + "id": "g_xmr_rAUoQx04wSxq2yQ", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 760, + "y": 540.3267832689439, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "width": 239.4116874766886, + "height": 0.14914230391627825, + "seed": 61286746, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1776405922494, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": null, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 239.4116874766886, + -0.14914230391627825 + ] + ], + "index": "aV" + }, + { + "type": "line", + "version": 1404, + "versionNonce": 427313986, + "isDeleted": false, + "id": "K1eRli9Ss4OCth8Rxisx0", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 580, + "y": 501.7969713603986, + "strokeColor": "#000000", + "backgroundColor": "#e9ecef", + "width": 160, + "height": 80, + "seed": 174911002, + "groupIds": [ + "nS55pT0iEmYN13lrATL_r" + ], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1776405922494, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": null, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": null, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 39.67197250080198 + ], + [ + -1.4210854715202004e-14, + 80 + ], + [ + 160, + 80 + ], + [ + 160, + 0 + ], + [ + 0, + 0 + ] + ], + "index": "aW", + "polygon": false + }, + { + "type": "image", + "version": 3811, + "versionNonce": 1217499906, + "index": "aX", + "isDeleted": false, + "id": "Ob5zHrhyWqtZkihkGCreM", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 646.1123898142422, + "y": 515.6199661898443, + "strokeColor": "transparent", + "backgroundColor": "#ffc9c9", + "width": 61.54214099463984, + "height": 50.681763172056336, + "seed": 1704729306, + "groupIds": [ + "fCtmhUGKWexr3OXqoPqQN", + "nS55pT0iEmYN13lrATL_r" + ], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1776405922494, + "link": null, + "locked": false, + "status": "saved", + "fileId": "d1ee05fd41e4fad60a7372901fb1a4d206070a057f9b3a48768235e2c5775230a90d16ba6804ee1fbd79befd1ec8cbbf", + "scale": [ + 1, + 1 + ], + "crop": null + }, + { + "id": "wiIzNpyZMDMVPVOdx3eVn", + "type": "rectangle", + "x": 1024.4167779398936, + "y": 485.22529758183066, + "width": 120, + "height": 80, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#f8f9fa", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "ED3zbSqQFl4xgPIcdlXCR" + ], + "frameId": null, + "index": "aY", + "roundness": { + "type": 3 + }, + "seed": 331853722, + "version": 266, + "versionNonce": 64235034, + "isDeleted": true, + "boundElements": [], + "updated": 1741098217025, + "link": null, + "locked": false + }, + { + "id": "TXbEZ48uEoiWDzIqPH1M_", + "type": "text", + "x": 1061.9834454966563, + "y": 490.22529758183066, + "width": 44.86666488647461, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#f8f9fa", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "ED3zbSqQFl4xgPIcdlXCR" + ], + "frameId": null, + "index": "aZ", + "roundness": null, + "seed": 1227009114, + "version": 271, + "versionNonce": 1870422150, + "isDeleted": true, + "boundElements": [], + "updated": 1741098217025, + "link": null, + "locked": false, + "text": "Index", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "center", + "verticalAlign": "top", + "containerId": "wiIzNpyZMDMVPVOdx3eVn", + "originalText": "Index", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "C4Vg7NQrQ7CRH2oimHPhh", + "type": "text", + "x": 1063.897760976568, + "y": 517.8553174757881, + "width": 44, + "height": 25, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#f8f9fa", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "ED3zbSqQFl4xgPIcdlXCR" + ], + "frameId": null, + "index": "aa", + "roundness": null, + "seed": 998248730, + "version": 385, + "versionNonce": 492589786, + "isDeleted": true, + "boundElements": [], + "updated": 1741098217025, + "link": null, + "locked": false, + "text": "main", + "fontSize": 20, + "fontFamily": 8, + "textAlign": "center", + "verticalAlign": "top", + "containerId": null, + "originalText": "main", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "U43N5fGAQhuW1feXA8vj9", + "type": "image", + "x": 935.1033669398937, + "y": 395.5924855818307, + "width": 320, + "height": 61.935483870967744, + "angle": 0, + "strokeColor": "transparent", + "backgroundColor": "#ffec99", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ab", + "roundness": null, + "seed": 1850924506, + "version": 172, + "versionNonce": 732740038, + "isDeleted": true, + "boundElements": [], + "updated": 1741097953019, + "link": null, + "locked": false, + "status": "saved", + "fileId": "427536ece8193bbdf039f2e7d8e9c52ddca850d9c1f5ab9cb392be289df6a930d6b9508e72a47654204b34fa73df0b84", + "scale": [ + 1, + 1 + ], + "crop": null + }, + { + "id": "f0yXEXn5F8SJ5p_TyNUwb", + "type": "arrow", + "x": 1025.2986369398936, + "y": 525.2252975818308, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ac", + "roundness": null, + "seed": 2125337242, + "version": 285, + "versionNonce": 862609350, + "isDeleted": true, + "boundElements": [], + "updated": 1741098217025, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "elbowed": false + }, + { + "id": "7SvpfQ9JKp7TWxk_iPRuj", + "type": "ellipse", + "x": 963.4167779398937, + "y": 515.2252975818307, + "width": 20, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ad", + "roundness": { + "type": 2 + }, + "seed": 1866772314, + "version": 219, + "versionNonce": 1455482778, + "isDeleted": true, + "boundElements": [], + "updated": 1741098217025, + "link": null, + "locked": false + }, + { + "id": "tYpVzQAJgHPTVmA_sLOkS", + "type": "text", + "x": 880.2823889115372, + "y": 551.1903207025754, + "width": 108.73332977294922, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ae", + "roundness": null, + "seed": 1953084442, + "version": 246, + "versionNonce": 809152834, + "isDeleted": false, + "boundElements": [], + "updated": 1776407002754, + "link": null, + "locked": false, + "text": "clickhouse-cpp", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "clickhouse-cpp", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "KN1ayyvhKKjnePaEKeaIF", + "type": "image", + "x": 820.9327548851652, + "y": 144.86044771263136, + "width": 648, + "height": 198, + "angle": 0, + "strokeColor": "transparent", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "af", + "roundness": null, + "seed": 1748074566, + "version": 7, + "versionNonce": 611016410, + "isDeleted": true, + "boundElements": [], + "updated": 1741097970404, + "link": null, + "locked": false, + "status": "saved", + "fileId": "cf677ae2acf110ab1ada213681646441e0c13120f42e07e34dd682f16c145e211079120b235b660943d40ff50c8c0009", + "scale": [ + 1, + 1 + ], + "crop": null + }, + { + "id": "xYtTaORtPfTaBfyVaE9Jh", + "type": "image", + "x": 735.7119039438548, + "y": 105.09331477743947, + "width": 648, + "height": 198, + "angle": 0, + "strokeColor": "transparent", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ag", + "roundness": null, + "seed": 538466886, + "version": 7, + "versionNonce": 301543046, + "isDeleted": true, + "boundElements": [], + "updated": 1741097998524, + "link": null, + "locked": false, + "status": "saved", + "fileId": "cf677ae2acf110ab1ada213681646441e0c13120f42e07e34dd682f16c145e211079120b235b660943d40ff50c8c0009", + "scale": [ + 1, + 1 + ], + "crop": null + }, + { + "id": "NsxTI00fJ7Tk4JFzEOGL_", + "type": "image", + "x": 988.6590468390034, + "y": 400, + "width": 221.3677416168746, + "height": 67.5359211712499, + "angle": 0, + "strokeColor": "transparent", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ah", + "roundness": null, + "seed": 135755354, + "version": 510, + "versionNonce": 1302936962, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false, + "status": "saved", + "fileId": "74235aaa80d5377e7af618a88eaf3165fae35889019cba2a4a62571f5ce6a7f9ad2989f927cd6093f58b899f2e11d669", + "scale": [ + 1, + 1 + ], + "crop": null + }, + { + "id": "vOB755ry3Yy5dgsquEXEw", + "type": "rectangle", + "x": 1008.2342083148578, + "y": 717.6959755193367, + "width": 220, + "height": 100, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ak", + "roundness": { + "type": 3 + }, + "seed": 842349190, + "version": 157, + "versionNonce": 1028906566, + "isDeleted": true, + "boundElements": [], + "updated": 1741098226665, + "link": null, + "locked": false + }, + { + "id": "CC4blV5bRJpXB76sprPmw", + "type": "text", + "x": 1086.1425411395649, + "y": 722.6959755193367, + "width": 64.18333435058594, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "al", + "roundness": null, + "seed": 502790598, + "version": 143, + "versionNonce": 1494520090, + "isDeleted": true, + "boundElements": [], + "updated": 1741098226665, + "link": null, + "locked": false, + "text": "Schemas", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "center", + "verticalAlign": "top", + "containerId": "vOB755ry3Yy5dgsquEXEw", + "originalText": "Schemas", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "NHNLon77B1UpJWm0TtITQ", + "type": "text", + "x": 1062.9663276537221, + "y": 759.4757254233818, + "width": 11.199999809265137, + "height": 35, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 4, + "strokeStyle": "solid", + "roughness": 0, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "amV", + "roundness": null, + "seed": 1291795610, + "version": 3, + "versionNonce": 1291828998, + "isDeleted": true, + "boundElements": [], + "updated": 1741098244079, + "link": null, + "locked": false, + "text": "", + "fontSize": 28, + "fontFamily": 5, + "textAlign": "center", + "verticalAlign": "middle", + "containerId": "grimbjvvQaXcX0fZ1QGkH", + "originalText": "", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "grimbjvvQaXcX0fZ1QGkH", + "type": "rectangle", + "x": 1059.430991911885, + "y": 522.742581081342, + "width": 79.72278435602915, + "height": 56.34336263774654, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "3KLhhHW84bohbevIB1Hzc" + ], + "frameId": null, + "index": "amd", + "roundness": null, + "seed": 1503711494, + "version": 764, + "versionNonce": 644496706, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false + }, + { + "id": "jDCSnbgRZUSNDrfjEd8Fc", + "type": "line", + "x": 1059.7917794738892, + "y": 547.369032301424, + "width": 78.67259902626967, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "3KLhhHW84bohbevIB1Hzc" + ], + "frameId": null, + "index": "aml", + "roundness": { + "type": 2 + }, + "seed": 816627782, + "version": 335, + "versionNonce": 108348674, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 78.67259902626967, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "x1F877POVQOiJKca5RDRq", + "type": "line", + "x": 1075.5262992791431, + "y": 547.369032301424, + "width": 0.12437256409457782, + "height": 31.7169114176644, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "3KLhhHW84bohbevIB1Hzc" + ], + "frameId": null, + "index": "amt", + "roundness": { + "type": 2 + }, + "seed": 1996015494, + "version": 332, + "versionNonce": 122246338, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.12437256409457782, + 31.7169114176644 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "okS3KPOvVBZwY3ofsj0Uc", + "type": "line", + "x": 1091.5159359930326, + "y": 547.6854080141526, + "width": 0.17604688976762406, + "height": 31.400535704935898, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "3KLhhHW84bohbevIB1Hzc" + ], + "frameId": null, + "index": "an", + "roundness": { + "type": 2 + }, + "seed": 2093082310, + "version": 343, + "versionNonce": 981822594, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.17604688976762406, + 31.400535704935898 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "ZJrOj4tP-Rs6DlBgjxBwu", + "type": "line", + "x": 1107.7049578794677, + "y": 547.6706057835664, + "width": 0.42710638798645123, + "height": 31.415337935521947, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "3KLhhHW84bohbevIB1Hzc" + ], + "frameId": null, + "index": "anV", + "roundness": { + "type": 2 + }, + "seed": 44375558, + "version": 339, + "versionNonce": 635997250, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.42710638798645123, + 31.415337935521947 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "vYTHyZXj8YpcxC4gKL6cW", + "type": "line", + "x": 1123.1466542616495, + "y": 547.5870313953009, + "width": 0.06915961804813092, + "height": 31.498912323787575, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "3KLhhHW84bohbevIB1Hzc" + ], + "frameId": null, + "index": "ao", + "roundness": { + "type": 2 + }, + "seed": 765422918, + "version": 339, + "versionNonce": 1062319106, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.06915961804813092, + 31.498912323787575 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "AL2_d8YDszRMsO8BuMuM5", + "type": "text", + "x": 1078.6978582750817, + "y": 524.3789968613241, + "width": 42.39994812011719, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [ + "3KLhhHW84bohbevIB1Hzc" + ], + "frameId": null, + "index": "ap", + "roundness": null, + "seed": 2064503942, + "version": 224, + "versionNonce": 1757406146, + "isDeleted": false, + "boundElements": [], + "updated": 1776405904244, + "link": null, + "locked": false, + "text": "Table", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Table", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "id": "EjkVOqbp5FvWVeBdAETVV", + "type": "rectangle", + "x": 1128.3060413803403, + "y": 748.5230741045085, + "width": 79.72278435602915, + "height": 56.34336263774654, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "at", + "roundness": null, + "seed": 883774406, + "version": 701, + "versionNonce": 1153008838, + "isDeleted": true, + "boundElements": [], + "updated": 1741098228386, + "link": null, + "locked": false + }, + { + "id": "E-WgXxnOCbXfeYW0mfWG3", + "type": "line", + "x": 1128.6668289423444, + "y": 773.1495253245906, + "width": 78.67259902626967, + "height": 0, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "au", + "roundness": { + "type": 2 + }, + "seed": 1687874310, + "version": 278, + "versionNonce": 140182534, + "isDeleted": true, + "boundElements": [], + "updated": 1741098236529, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 78.67259902626967, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "xjWYRtXFibG7uLFMuFR8a", + "type": "line", + "x": 1144.4013487475984, + "y": 773.1495253245906, + "width": 0.12437256409457782, + "height": 31.7169114176644, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "av", + "roundness": { + "type": 2 + }, + "seed": 1230215750, + "version": 275, + "versionNonce": 30123866, + "isDeleted": true, + "boundElements": [], + "updated": 1741098236529, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.12437256409457782, + 31.7169114176644 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "4iUma5gpDHV3Ss8YN1ZnC", + "type": "line", + "x": 1160.3909854614876, + "y": 773.4659010373191, + "width": 0.17604688976762406, + "height": 31.400535704935898, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "aw", + "roundness": { + "type": 2 + }, + "seed": 1236558214, + "version": 283, + "versionNonce": 262705990, + "isDeleted": true, + "boundElements": [], + "updated": 1741098236529, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.17604688976762406, + 31.400535704935898 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "YTALf8QivrY8Y3ukdnM48", + "type": "line", + "x": 1176.580007347923, + "y": 773.4510988067331, + "width": 0.42710638798645123, + "height": 31.415337935521947, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ax", + "roundness": { + "type": 2 + }, + "seed": 1761231046, + "version": 283, + "versionNonce": 189901850, + "isDeleted": true, + "boundElements": [], + "updated": 1741098236529, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -0.42710638798645123, + 31.415337935521947 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "uw8IKhs5_yw6le7eBDmog", + "type": "line", + "x": 1192.021703730105, + "y": 773.3675244184675, + "width": 0.06915961804813092, + "height": 31.498912323787575, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#e9ecef", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "ay", + "roundness": { + "type": 2 + }, + "seed": 985806854, + "version": 283, + "versionNonce": 1291991686, + "isDeleted": true, + "boundElements": [], + "updated": 1741098236529, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.06915961804813092, + 31.498912323787575 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": null, + "polygon": false + }, + { + "id": "WfZ3V-cKbT8O4P8aYo7dF", + "type": "text", + "x": 1147.5729077435367, + "y": 750.1594898844909, + "width": 42.39994812011719, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "az", + "roundness": null, + "seed": 1460225862, + "version": 164, + "versionNonce": 1328099162, + "isDeleted": true, + "boundElements": [], + "updated": 1741098229156, + "link": null, + "locked": false, + "text": "Table", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Table", + "autoResize": true, + "lineHeight": 1.25 + }, + { + "type": "arrow", + "version": 565, + "versionNonce": 320025794, + "isDeleted": true, + "id": "pBotRGLBg9X5mnuVmfKnB", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 1240, + "y": 299.590408778781, + "strokeColor": "#1e1e1e", + "backgroundColor": "transparent", + "width": 205.2903283354567, + "height": 0, + "seed": 487225986, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1776405941366, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": null, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 205.2903283354567, + 0 + ] + ], + "index": "b00", + "moveMidPointsWithElement": false + }, + { + "type": "line", + "version": 1216, + "versionNonce": 1354649154, + "isDeleted": false, + "id": "F0orhRaHAqTuALLSnO9Xy", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 1461.4896611218805, + "y": 501.1228355476312, + "strokeColor": "#000000", + "backgroundColor": "#e9ecef", + "width": 160, + "height": 80, + "seed": 1352391234, + "groupIds": [ + "ue6SNJ8D0rwUulbQkKEMs" + ], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1776405935461, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": null, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": null, + "points": [ + [ + 0, + 0 + ], + [ + -1.4896611218806584, + 80 + ], + [ + 118.51033887811936, + 79.99999999999999 + ], + [ + 158.51033887811934, + 40.000000000000014 + ], + [ + 118.51033887811934, + -7.105427357601002e-15 + ], + [ + 0, + 0 + ] + ], + "index": "b01", + "polygon": false + }, + { + "type": "image", + "version": 3823, + "versionNonce": 1310668290, + "index": "b02", + "isDeleted": false, + "id": "u7uGFZKvYTJg3ZaveldwI", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 1500.215355777485, + "y": 514.0164108401761, + "strokeColor": "transparent", + "backgroundColor": "#ffc9c9", + "width": 61.54214099463984, + "height": 50.681763172056336, + "seed": 309019138, + "groupIds": [ + "DdDt_26QqNoMYFN5KrQht", + "ue6SNJ8D0rwUulbQkKEMs" + ], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1776405935461, + "link": null, + "locked": false, + "status": "saved", + "fileId": "d1ee05fd41e4fad60a7372901fb1a4d206070a057f9b3a48768235e2c5775230a90d16ba6804ee1fbd79befd1ec8cbbf", + "scale": [ + 1, + 1 + ], + "crop": null + }, + { + "type": "arrow", + "version": 1463, + "versionNonce": 1552216990, + "isDeleted": false, + "id": "tKM0pRpx3mTuxNwnoBONG", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "angle": 0, + "x": 1200, + "y": 540.291388253688, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "width": 240, + "height": 0.2913882536879555, + "seed": 1924097822, + "groupIds": [], + "frameId": null, + "roundness": null, + "boundElements": [], + "updated": 1776405936994, + "link": null, + "locked": false, + "startBinding": null, + "endBinding": null, + "lastCommittedPoint": null, + "startArrowhead": null, + "endArrowhead": "triangle", + "points": [ + [ + 0, + 0 + ], + [ + 240, + -0.2913882536879555 + ] + ], + "index": "b03" + }, + { + "id": "pxMAkKbSTRDYrKzd8E1cn", + "type": "text", + "x": 1200, + "y": 551.190321, + "width": 108.73332977294922, + "height": 20, + "angle": 0, + "strokeColor": "#1e1e1e", + "backgroundColor": "#ffec99", + "fillStyle": "hachure", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "index": "b04", + "roundness": null, + "seed": 1604514014, + "version": 385, + "versionNonce": 436460290, + "isDeleted": false, + "boundElements": [], + "updated": 1776407000712, + "link": null, + "locked": false, + "text": "clickhouse-cpp", + "fontSize": 16, + "fontFamily": 5, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "clickhouse-cpp", + "autoResize": true, + "lineHeight": 1.25 + } + ], + "appState": { + "gridSize": 20, + "gridStep": 5, + "gridModeEnabled": true, + "viewBackgroundColor": "#ffffff", + "lockedMultiSelections": {} + }, + "files": { + "d1ee05fd41e4fad60a7372901fb1a4d206070a057f9b3a48768235e2c5775230a90d16ba6804ee1fbd79befd1ec8cbbf": { + "mimeType": "image/svg+xml", + "id": "d1ee05fd41e4fad60a7372901fb1a4d206070a057f9b3a48768235e2c5775230a90d16ba6804ee1fbd79befd1ec8cbbf", + "dataURL": "data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI4NSIgaGVpZ2h0PSI3MCIgdmlld0JveD0iMCAwIDg1IDcwIiBmaWxsPSJub25lIj4KPHBhdGggZD0iTTI5LjQ1IDQ0LjQyTDQyLjc3IDEyLjkzQzQyLjgyNjkgMTIuODA1OCA0Mi44NTc4IDEyLjY3MTMgNDIuODYwNyAxMi41MzQ4QzQyLjg2MzYgMTIuMzk4MiA0Mi44Mzg2IDEyLjI2MjUgNDIuNzg3IDEyLjEzNkM0Mi43MzU1IDEyLjAwOTUgNDIuNjU4NiAxMS44OTQ5IDQyLjU2MTEgMTEuNzk5M0M0Mi40NjM1IDExLjcwMzcgNDIuMzQ3NSAxMS42MjkgNDIuMjIgMTEuNThDNDIuMDkyOCAxMS41Mjg2IDQxLjk1NzEgMTEuNTAxNSA0MS44MiAxMS41SDkuMjc5OTlDOS4wNzYzMiAxMS40OTU4IDguODc2MjEgMTEuNTUzOCA4LjcwNjQzIDExLjY2NjRDOC41MzY2NiAxMS43NzkgOC40MDUzMiAxMS45NDA3IDguMzI5OTkgMTIuMTNMMy4xMDk5OSAyNC42M0MzLjA1IDI0Ljc1NTggMy4wMTY5NSAyNC44OTI3IDMuMDEyOTggMjUuMDMyQzMuMDA5MDEgMjUuMTcxMyAzLjAzNDE5IDI1LjMwOTkgMy4wODY5MiAyNS40Mzg5QzMuMTM5NjUgMjUuNTY3OSAzLjIxODc1IDI1LjY4NDQgMy4zMTkxNiAyNS43ODExQzMuNDE5NTcgMjUuODc3NyAzLjUzOTA3IDI1Ljk1MjMgMy42Njk5OSAyNkMzLjc5MzMxIDI2LjA1MjUgMy45MjU5NCAyNi4wNzk3IDQuMDU5OTkgMjYuMDhIMTkuODJDMjAuMDg1MiAyNi4wOCAyMC4zMzk2IDI2LjE4NTQgMjAuNTI3MSAyNi4zNzI5QzIwLjcxNDYgMjYuNTYwNCAyMC44MiAyNi44MTQ4IDIwLjgyIDI3LjA4QzIwLjgxODUgMjcuMjE3MiAyMC43OTE0IDI3LjM1MjggMjAuNzQgMjcuNDhMNy41OTk5OSA1OS4wN0M3LjU0MzMyIDU5LjE5NDggNy41MTI5MiA1OS4zMjk5IDcuNTEwNjUgNTkuNDY2OUM3LjUwODM5IDU5LjYwNCA3LjUzNDMyIDU5Ljc0IDcuNTg2ODMgNTkuODY2NkM3LjYzOTM1IDU5Ljk5MzIgNy43MTczMiA2MC4xMDc3IDcuODE1OTIgNjAuMjAyOUM3LjkxNDUyIDYwLjI5OCA4LjAzMTYzIDYwLjM3MiA4LjE1OTk5IDYwLjQyQzguMjgzMzEgNjAuNDcyNSA4LjQxNTk0IDYwLjQ5OTcgOC41NDk5OSA2MC41SDQzQzQzLjIxMzIgNjAuNTE0MSA0My40MjUzIDYwLjQ1OTYgNDMuNjA1MiA2MC4zNDQ1QzQzLjc4NTIgNjAuMjI5MyA0My45MjM1IDYwLjA1OTUgNDQgNTkuODZMNDkuMjIgNDcuMjhDNDkuMjc2NyA0Ny4xNTUyIDQ5LjMwNzEgNDcuMDIwMSA0OS4zMDkzIDQ2Ljg4MzFDNDkuMzExNiA0Ni43NDYgNDkuMjg1NyA0Ni42MSA0OS4yMzMxIDQ2LjQ4MzRDNDkuMTgwNiA0Ni4zNTY4IDQ5LjEwMjcgNDYuMjQyNCA0OS4wMDQxIDQ2LjE0NzJDNDguOTA1NSA0Ni4wNTIgNDguNzg4MyA0NS45NzggNDguNjYgNDUuOTNDNDguNTM2NyA0NS44Nzc1IDQ4LjQwNCA0NS44NTAzIDQ4LjI3IDQ1Ljg1SDMwLjRDMzAuMTM0OCA0NS44NSAyOS44ODA0IDQ1Ljc0NDYgMjkuNjkyOSA0NS41NTcxQzI5LjUwNTMgNDUuMzY5NiAyOS40IDQ1LjExNTIgMjkuNCA0NC44NUMyOS4zODggNDQuNzA0OCAyOS40MDUgNDQuNTU4NiAyOS40NSA0NC40MloiIGZpbGw9IiMxMjEyMTIiLz4KPHBhdGggZD0iTTM3LjE3IDQxLjU4SDUwLjg4QzUxLjA4MzcgNDEuNTg0MyA1MS4yODM4IDQxLjUyNjIgNTEuNDUzNSA0MS40MTM2QzUxLjYyMzMgNDEuMzAxIDUxLjc1NDcgNDEuMTM5MyA1MS44MyA0MC45NUw1Ny44MyAyNi42OUM1Ny45MDUzIDI2LjUwMDcgNTguMDM2NyAyNi4zMzkgNTguMjA2NCAyNi4yMjY0QzU4LjM3NjIgMjYuMTEzOCA1OC41NzYzIDI2LjA1NTcgNTguNzggMjYuMDZINzYuNDFDNzYuNjIyIDI2LjA3NDkgNzYuODMzMiAyNi4wMjE4IDc3LjAxMyAyNS45MDg1Qzc3LjE5MjggMjUuNzk1MyA3Ny4zMzE5IDI1LjYyNzYgNzcuNDEgMjUuNDNMODIuNjMgMTIuOTNDODIuNjg1MiAxMi44MDA3IDgyLjcxMjYgMTIuNjYxMyA4Mi43MTAzIDEyLjUyMDdDODIuNzA3OSAxMi4zODAxIDgyLjY3NiAxMi4yNDE2IDgyLjYxNjYgMTIuMTE0M0M4Mi41NTcxIDExLjk4NjkgODIuNDcxNSAxMS44NzM0IDgyLjM2NTIgMTEuNzgxNEM4Mi4yNTkgMTEuNjg5MyA4Mi4xMzQ1IDExLjYyMDcgODIgMTEuNThDODEuODc2NyAxMS41Mjc1IDgxLjc0NCAxMS41MDAzIDgxLjYxIDExLjVINDguODJDNDguNjA4IDExLjQ4NTEgNDguMzk2OCAxMS41MzgyIDQ4LjIxNyAxMS42NTE1QzQ4LjAzNzEgMTEuNzY0NyA0Ny44OTgxIDExLjkzMjQgNDcuODIgMTIuMTNMMzYuMTcgNDAuMTNDMzYuMTEzMSA0MC4yNTQyIDM2LjA4MjIgNDAuMzg4NyAzNi4wNzkzIDQwLjUyNTJDMzYuMDc2MyA0MC42NjE4IDM2LjEwMTQgNDAuNzk3NSAzNi4xNTI5IDQwLjkyNEMzNi4yMDQ1IDQxLjA1MDUgMzYuMjgxNCA0MS4xNjUxIDM2LjM3ODkgNDEuMjYwN0MzNi40NzY0IDQxLjM1NjMgMzYuNTkyNSA0MS40MzEgMzYuNzIgNDEuNDhDMzYuODYxMiA0MS41NDQ3IDM3LjAxNDYgNDEuNTc4OCAzNy4xNyA0MS41OFoiIGZpbGw9IiMxMjEyMTIiLz4KPC9zdmc+", + "created": 1684733732697 + }, + "74235aaa80d5377e7af618a88eaf3165fae35889019cba2a4a62571f5ce6a7f9ad2989f927cd6093f58b899f2e11d669": { + "mimeType": "image/svg+xml", + "id": "74235aaa80d5377e7af618a88eaf3165fae35889019cba2a4a62571f5ce6a7f9ad2989f927cd6093f58b899f2e11d669", + "dataURL": "data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIGlkPSJMYXllcl8xIiB2ZXJzaW9uPSIxLjEiIHZpZXdCb3g9IjAgMCA2NDkuMyAxOTguMyIgd2lkdGg9IjY0OS4zIiBoZWlnaHQ9IjE5OC4zIj4KICA8IS0tIEdlbmVyYXRvcjogQWRvYmUgSWxsdXN0cmF0b3IgMjkuMS4wLCBTVkcgRXhwb3J0IFBsdWctSW4gLiBTVkcgVmVyc2lvbjogMi4xLjAgQnVpbGQgMTQyKSAgLS0+CiAgPGRlZnM+CiAgICA8c3R5bGU+CiAgICAgIC5zdDAgewogICAgICAgIGZpbGw6ICMxZTFlMWU7CiAgICAgIH0KICAgIDwvc3R5bGU+CiAgPC9kZWZzPgogIDxnPgogICAgPHBhdGggY2xhc3M9InN0MCIgZD0iTTMyLjIsNDguOGMwLS43LjUtMS4yLDEuMi0xLjJoOWMuNywwLDEuMi41LDEuMiwxLjJ2MTAwLjdjMCwuNy0uNSwxLjItMS4yLDEuMmgtOWMtLjcsMC0xLjItLjUtMS4yLTEuMlY0OC44WiIvPgogICAgPHBhdGggY2xhc3M9InN0MCIgZD0iTTU1LjEsNDguOGMwLS43LjUtMS4yLDEuMi0xLjJoOWMuNywwLDEuMi41LDEuMiwxLjJ2MTAwLjdjMCwuNy0uNSwxLjItMS4yLDEuMmgtOWMtLjcsMC0xLjItLjUtMS4yLTEuMiwwLDAsMC0xMDAuNywwLTEwMC43WiIvPgogICAgPHBhdGggY2xhc3M9InN0MCIgZD0iTTc4LDQ4LjhjMC0uNy41LTEuMiwxLjItMS4yaDljLjcsMCwxLjIuNSwxLjIsMS4ydjEwMC43YzAsLjctLjUsMS4yLTEuMiwxLjJoLTljLS43LDAtMS4yLS41LTEuMi0xLjJWNDguOFoiLz4KICAgIDxwYXRoIGNsYXNzPSJzdDAiIGQ9Ik0xMDAuOSw0OC44YzAtLjcuNS0xLjIsMS4yLTEuMmg5Yy43LDAsMS4yLjUsMS4yLDEuMnYxMDAuN2MwLC43LS41LDEuMi0xLjIsMS4yaC05Yy0uNywwLTEuMi0uNS0xLjItMS4yVjQ4LjhaIi8+CiAgICA8cGF0aCBjbGFzcz0ic3QwIiBkPSJNMTIzLjgsODguOWMwLS43LjUtMS4yLDEuMi0xLjJoOWMuNywwLDEuMi41LDEuMiwxLjJ2MjAuNWMwLC43LS41LDEuMi0xLjIsMS4yaC05Yy0uNywwLTEuMi0uNS0xLjItMS4ydi0yMC41WiIvPgogIDwvZz4KICA8cGF0aCBjbGFzcz0ic3QwIiBkPSJNMTk1LjIsNzUuN2MtMy4yLDAtNi4xLjYtOC43LDEuNy0yLjYsMS4xLTQuOCwyLjctNi42LDQuOS0xLjgsMi4yLTMuMSw0LjktNC4xLDgtLjksMy4xLTEuNCw2LjYtMS40LDEwLjVzLjgsOS42LDIuMywxMy40YzEuNSwzLjcsMy44LDYuNiw2LjksOC42LDMuMSwyLDYuOSwzLDExLjUsM3M1LjUtLjIsOC4xLS43LDUuMy0xLjMsOC0yLjF2OC41Yy0yLjYsMS01LjIsMS43LTcuOSwyLjEtMi43LjUtNS44LjctOS4zLjctNi43LDAtMTIuMi0xLjQtMTYuNy00LjEtNC40LTIuNy03LjctNi43LTkuOS0xMS43LTIuMi01LjEtMy4zLTExLTMuMy0xNy44cy43LTkuNSwyLTEzLjZjMS40LTQuMiwzLjQtNy43LDYtMTAuN3M1LjktNS4zLDkuNy02LjljMy45LTEuNiw4LjQtMi41LDEzLjUtMi41czYuNS40LDkuNywxLjEsNiwxLjYsOC42LDIuOWwtMy43LDguMmMtMi4xLTEtNC41LTEuOC03LTIuNi0yLjMtLjUtNC45LS45LTcuNy0uOVoiLz4KICA8cGF0aCBjbGFzcz0ic3QwIiBkPSJNMjMzLjksMTMzLjVoLTkuN1Y2My45aDkuN3Y2OS42WiIvPgogIDxwYXRoIGNsYXNzPSJzdDAiIGQ9Ik0yNTUuMiw4NC4xdjQ5LjRoLTkuN3YtNDkuNGg5LjdaTTI1MC41LDY1LjRjMS41LDAsMi43LjQsMy44LDEuM3MxLjYsMi4zLDEuNiw0LjMtLjUsMy40LTEuNiw0LjMtMi40LDEuMy0zLjgsMS4zLTIuOS0uNC00LTEuM2MtMS0uOS0xLjYtMi40LTEuNi00LjNzLjUtMy41LDEuNi00LjNjMS0uOSwyLjQtMS4zLDQtMS4zWiIvPgogIDxwYXRoIGNsYXNzPSJzdDAiIGQ9Ik0yODguNSwxMzQuNGMtNC41LDAtOC41LS45LTExLjktMi43cy02LjEtNC41LTgtOC4yYy0xLjktMy44LTIuOC04LjYtMi44LTE0LjRzMS0xMS4xLDMtMTQuOGMyLjEtMy44LDQuOS02LjYsOC4zLTguMywzLjUtMS44LDcuNi0yLjcsMTIuMS0yLjdzNS4zLjMsNy43LjljMi40LjUsNC41LDEuMiw2LDEuOWwtMi45LDcuOWMtMS43LS43LTMuNS0xLjMtNS41LTEuNy0xLjktLjUtMy43LS43LTUuNC0uNy0zLjEsMC01LjYuNy03LjYsMnMtMy40LDMuMi00LjQsNS45Yy0uOSwyLjYtMS40LDUuOC0xLjQsOS43cy41LDYuOSwxLjUsOS40YzEsMi42LDIuNCw0LjUsNC4zLDUuOSwyLDEuMyw0LjQsMS45LDcuMiwxLjlzNS4yLS4zLDcuMi0uOWMyLjEtLjYsNC0xLjQsNS45LTIuNHY4LjRjLTEuOCwxLTMuNywxLjgtNS44LDIuMy0xLjkuNC00LjUuNi03LjUuNloiLz4KICA8cGF0aCBjbGFzcz0ic3QwIiBkPSJNMzIxLjIsMTA1LjdjLjctLjcsMjEuMS0yMS42LDIxLjEtMjEuNmgxMS4zbC0xOS45LDIxLjIsMjEuMiwyOC4yaC0xMS41bC0xNi4zLTIyLjEtNS43LDQuOHYxNy4zaC05LjZWNjMuOWg5LjZsLS4yLDQxLjhaIi8+CiAgPHBhdGggY2xhc3M9InN0MCIgZD0iTTQxNS4xLDEzMy41aC05Ljl2LTI5LjloLTMxLjJ2MjkuOWgtOS45di02NS40aDkuOXYyNy4xaDMxLjJ2LTI3LjFoOS45djY1LjRaIi8+CiAgPHBhdGggY2xhc3M9InN0MCIgZD0iTTQ3MC40LDEwOC44YzAsNC4xLS42LDcuNy0xLjYsMTAuOS0xLjEsMy4yLTIuNyw1LjktNC43LDguMXMtNC41LDMuOS03LjMsNWMtMi45LDEuMS02LjEsMS42LTkuNywxLjZzLTYuNC0uNS05LjItMS42Yy0yLjgtMS4yLTUuMy0yLjgtNy4zLTVzLTMuNi00LjktNC44LTguMWMtMS4yLTMuMi0xLjctNi44LTEuNy0xMXMuOS0xMCwyLjctMTMuOGMxLjktMy44LDQuNi02LjcsOC4xLTguNiwzLjUtMiw3LjctMywxMi41LTNzOC41LDEsMTEuOSwzYzMuNSwyLDYuMiw0LjgsOC4yLDguNiwxLjksMy43LDIuOSw4LjQsMi45LDEzLjlaTTQzNCwxMDguOGMwLDMuNy41LDYuOCwxLjQsOS41LjksMi42LDIuNCw0LjYsNC4zLDYsMiwxLjQsNC41LDIuMSw3LjUsMi4xczUuNi0uNyw3LjUtMi4xYzItMS40LDMuNC0zLjQsNC4zLTYsLjktMi43LDEuNC01LjksMS40LTkuNXMtLjUtNy0xLjUtOS41Yy0uOS0yLjYtMi40LTQuNS00LjMtNS45cy00LjQtMi4xLTcuNS0yLjFjLTQuNiwwLTcuOSwxLjUtMTAsNC42LTIuMSwzLTMuMSw3LjMtMy4xLDEyLjlaIi8+CiAgPHBhdGggY2xhc3M9InN0MCIgZD0iTTUyMi4xLDg0LjF2NDkuNGgtNy44bC0xLjMtNi41aC0uNWMtMSwxLjctMi40LDMuMS00LDQuMi0xLjYsMS4xLTMuNSwxLjktNS40LDIuNC0yLC41LTQsLjgtNiwuOC0zLjgsMC03LjEtLjYtOS44LTEuOC0yLjYtMS4zLTQuNi0zLjItNi01LjktMS4zLTIuNy0yLTYuMS0yLTEwLjN2LTMyLjNoOS44djMwLjljMCwzLjguOCw2LjcsMi40LDguNnM0LjIsMi44LDcuNiwyLjgsNi4xLS43LDguMS0yLDMuMy0zLjMsNC4xLTUuOWMuOS0yLjYsMS4zLTUuOCwxLjMtOS40di0yNWg5LjVaIi8+CiAgPHBhdGggY2xhc3M9InN0MCIgZD0iTTU2Ni41LDExOS42YzAsMy4yLS44LDYtMi40LDguMi0xLjYsMi4yLTMuOSwzLjktNi45LDUtMywxLjEtNi42LDEuNi0xMC44LDEuNnMtNi41LS4zLTktLjhjLTIuNC0uNS00LjctMS4yLTYuOC0yLjJ2LTguNWMyLjIsMSw0LjcsMiw3LjYsMi44LDIuOS44LDUuNywxLjIsOC40LDEuMnM2LjMtLjYsNy45LTEuN2MxLjYtMS4yLDIuNS0yLjcsMi41LTQuN3MtLjMtMi4xLS45LTIuOWMtLjYtLjktMS44LTEuOC0zLjUtMi43LTEuNy0xLTQuMS0yLjEtNy4zLTMuMy0zLjEtMS4zLTUuOC0yLjUtOC0zLjhzLTMuOS0yLjgtNS00LjVjLTEuMi0xLjgtMS43LTQtMS43LTYuOCwwLTQuMywxLjctNy41LDUuMS05LjgsMy41LTIuMyw4LjEtMy41LDEzLjctMy41czUuOC4zLDguNC45YzIuNy41LDUuMywxLjQsNy44LDIuNmwtMy4yLDcuNGMtMS40LS43LTIuOS0xLjItNC40LTEuNi0xLjUtLjUtMy0uOS00LjUtMS4yLTEuNS0uMy0zLS41LTQuNi0uNS0yLjksMC01LjIuNS02LjcsMS40cy0yLjMsMi4yLTIuMywzLjguMywyLjIsMSwzYy43LjksMiwxLjcsMy44LDIuNnM0LjIsMS45LDcuMiwzLjFjMy4xLDEuMiw1LjcsMi40LDcuOCwzLjYsMi4yLDEuMiwzLjgsMi43LDQuOSw0LjUsMS4zLDEuOCwxLjksNC4xLDEuOSw2LjhaIi8+CiAgPHBhdGggY2xhc3M9InN0MCIgZD0iTTU5Ni4yLDgzLjJjNC4zLDAsOC4xLjksMTEuMiwyLjcsMy4xLDEuOCw1LjUsNC4zLDcuMSw3LjcsMS43LDMuNCwyLjYsNy4zLDIuNiwxMS45djUuM2gtMzMuMmMwLDUuMSwxLjQsOC45LDMuOSwxMS42LDIuNiwyLjcsNi4xLDQsMTAuNyw0czYtLjMsOC40LS44YzIuNS0uNiw1LjEtMS41LDcuNy0yLjd2OC4xYy0yLjQsMS4xLTQuOSwxLjktNy41LDIuNS0yLjUuNS01LjUuOC05LC44cy05LS45LTEyLjYtMi44Yy0zLjctMS45LTYuNS00LjctOC42LTguNC0yLTMuOC0zLTguNC0zLTEzLjlzLjktMTAuMywyLjctMTQuMWMxLjktMy44LDQuNS02LjgsNy44LTguOCwzLjQtMiw3LjQtMy4xLDExLjgtMy4xWk01OTYuMiw5MC43Yy0zLjUsMC02LjMsMS4xLTguNCwzLjQtMi4xLDIuMy0zLjMsNS41LTMuNyw5LjZoMjMuM2MwLTIuNi0uNC00LjgtMS4yLTYuOHMtMi0zLjUtMy43LTQuNmMtMS42LTEtMy43LTEuNi02LjMtMS42WiIvPgo8L3N2Zz4=", + "created": 1741098070064, + "version": 2 + } + } +} \ No newline at end of file diff --git a/src/content/docs/integrations/clickhouse.mdx b/src/content/docs/integrations/clickhouse.mdx index 191d80b28..fce0f6c44 100644 --- a/src/content/docs/integrations/clickhouse.mdx +++ b/src/content/docs/integrations/clickhouse.mdx @@ -5,15 +5,16 @@ title: ClickHouse [ClickHouse](https://clickhouse.com/clickhouse) is an open-source analytical database. It lets you run real-time analytics with SQL queries. -![ClickHouse integration](clickhouse.svg) +![ClickHouse integration](clickhouse.excalidraw) ## How Tenzir Connects to ClickHouse Tenzir connects to ClickHouse over the network using the native ClickHouse TCP protocol using the official [clickhouse-cpp](https://github.com/ClickHouse/clickhouse-cpp) library. Tenzir -communicates with ClickHouse via the host and port you specify in the -to_clickhouse operator. This means: +communicates with ClickHouse via the connection details you specify in the +from_clickhouse and to_clickhouse operators, either through a +ClickHouse URI or through explicit host and port settings. This means: - **Network**: Tenzir and ClickHouse can run on the same machine (using `localhost`) or on different machines in the same network. You just need to @@ -84,11 +85,40 @@ These examples assume that the ClickHouse server is running on the same host as Tenzir and that it allows non-TLS connections (hence using `tls=false` in the pipelines). -You can find out more about how to configure TLS on the -to_clickhouse documentation and the -[Clickhouse SSL-TLS configuration +You can find out more about how to configure TLS in the +from_clickhouse and to_clickhouse references and the +[ClickHouse SSL/TLS configuration guide](https://clickhouse.com/docs/guides/sre/configuring-ssl) +### Read data from ClickHouse + +Tenzir can also query an existing ClickHouse table and return the result as +structured events: + +```tql +from_clickhouse table="my_table", tls=false +``` + +You can also connect through a URI: + +```tql +from_clickhouse uri="clickhouse://default:secret@localhost:9000/default", + table="my_table", + tls=false +``` + +You can also run custom SQL queries: + +```tql +from_clickhouse sql="SELECT * FROM my_table WHERE severity >= 3", tls=false +``` + +To inspect column metadata, query the table schema instead: + +```tql +from_clickhouse sql="DESCRIBE TABLE my_table", tls=false +``` + ### 1. Easy Mode: Automatic table creation Tenzir can automatically create tables in ClickHouse based on the incoming data @@ -100,6 +130,21 @@ ocsf::cast encode_variants=true, null_fill=true to_clickhouse table=f"ocsf.{class_name.replace(" ","_")}", primary=time, tls=false ``` +A URI works here as well: + +```tql +from "ocsf_network_activity.json" +ocsf::cast encode_variants=true, null_fill=true +to_clickhouse uri="clickhouse://default:secret@localhost:9000/default", + table="ocsf_events", + primary=time, + tls=false +``` + +When the URI includes a database and `table` is unqualified, Tenzir writes to +that database. In `mode="create"` and `mode="create_append"`, Tenzir also +creates the selected database if needed. + When creating a table, the to_clickhouse operator uses the first event to determine the schema. You must take care that there are no untyped nulls in this event, as the operator cannot transmit diff --git a/src/content/docs/integrations/clickhouse.svg b/src/content/docs/integrations/clickhouse.svg deleted file mode 100644 index e7c971d5e..000000000 --- a/src/content/docs/integrations/clickhouse.svg +++ /dev/null @@ -1,2 +0,0 @@ -Databaseclickhouse-cppTable \ No newline at end of file diff --git a/src/content/docs/integrations/mysql.mdx b/src/content/docs/integrations/mysql.mdx index 2199bb694..5c6d65e80 100644 --- a/src/content/docs/integrations/mysql.mdx +++ b/src/content/docs/integrations/mysql.mdx @@ -47,4 +47,5 @@ from_mysql table="events", live=true, host="localhost", database="mydb" ## See Also +- collecting/read-from-data-stores - from_mysql diff --git a/src/content/docs/reference/operators.mdx b/src/content/docs/reference/operators.mdx index 6f821338f..7c4351a39 100644 --- a/src/content/docs/reference/operators.mdx +++ b/src/content/docs/reference/operators.mdx @@ -347,6 +347,10 @@ operators: description: 'Reads one or multiple files from Azure Blob Storage.' example: 'from_azure_blob_storage "abfs://container/data/**.json"' path: 'reference/operators/from_azure_blob_storage' + - name: 'from_clickhouse' + description: 'Reads rows or metadata from ClickHouse.' + example: 'from_clickhouse table="my_table"' + path: 'reference/operators/from_clickhouse' - name: 'from_file' description: 'Reads one or multiple files from a filesystem.' example: 'from_file "s3://data/**.json"' @@ -2187,6 +2191,14 @@ from_azure_blob_storage "abfs://container/data/**.json" + + +```tql +from_clickhouse table="my_table" +``` + + + ```tql diff --git a/src/content/docs/reference/operators/from_clickhouse.mdx b/src/content/docs/reference/operators/from_clickhouse.mdx new file mode 100644 index 000000000..b30cd6043 --- /dev/null +++ b/src/content/docs/reference/operators/from_clickhouse.mdx @@ -0,0 +1,161 @@ +--- +title: from_clickhouse +category: Inputs/Events +example: 'from_clickhouse table="my_table"' +--- + +import TLSOptions from '@partials/operators/TLSOptions.mdx'; + +This reference documents the `from_clickhouse` operator. You'll learn how to read table rows, run SQL queries, and inspect ClickHouse metadata through SQL. + +```tql +from_clickhouse [table=string, sql=string, + uri=string | (host=string, port=int, user=string, password=string), + tls=bool|record] +``` + +## Description + +Use exactly one of `table` or `sql`. + +Use either `uri` or the explicit connection arguments `host`, `port`, `user`, +and `password`. + +Currently, `from_clickhouse` is available only with the new pipeline executor. +Run it with `tenzir --neo` until it is available in the legacy executor. + +### `uri = string (optional)` + +A ClickHouse connection URI in the format: + +```text +clickhouse://[user[:password]@]host[:port][/database] +``` + +When present, the URI supplies the connection endpoint and optionally the +current database. + +Use `tls` separately to control TLS. + +### `table = string` + +The table to read from. + +You can qualify the table as `.`. If you omit the database, +ClickHouse uses the current database selected by the URI or server defaults. + +Use this mode when you want to read a whole table and preserve named tuple +fields from the table schema. + +### `sql = string` + +A custom SQL query to execute. + +Use this mode when you want ClickHouse to filter, project, sort, or cast data +before Tenzir reads it. + +For metadata queries such as `SHOW TABLES`, `DESCRIBE TABLE`, or queries +against `system.tables` and `system.columns`, use `sql`. + +### `host = string (optional)` + +The hostname for the ClickHouse server. + +Defaults to `"localhost"`. + +Mutually exclusive with `uri`. + +### `port = int (optional)` + +The port for the ClickHouse server. + +Defaults to `9000` without TLS and `9440` with TLS. + +Mutually exclusive with `uri`. + +### `user = string (optional)` + +The user to use for authentication. + +Defaults to `"default"`. + +Mutually exclusive with `uri`. + +### `password = string (optional)` + +The password for the given user. + +Defaults to `""`. + +Mutually exclusive with `uri`. + + + +## Types + +Tenzir maps ClickHouse types to Tenzir types as follows: + +| ClickHouse | Tenzir | Comment | +| :------------------------------------------------ | :-------- | :--------------------------------------------- | +| `Bool` | `bool` | | +| `Int8`, `Int16`, `Int32`, `Int64` | `int64` | | +| `UInt8`, `UInt16`, `UInt32`, `UInt64` | `uint64` | | +| `Float32`, `Float64` | `double` | | +| `String`, `FixedString(N)` | `string` | | +| `UUID` | `string` | Emitted as canonical UUID text. | +| `Enum8`, `Enum16` | `string` | Emitted as the enum label. | +| `Decimal`, `Decimal32`, `Decimal64`, `Decimal128` | `string` | Emitted as decimal text to preserve precision. | +| `Date`, `Date32`, `DateTime`, `DateTime64` | `time` | | +| `IPv4`, `IPv6` | `ip` | | +| `Tuple(...)` | `record` | | +| `Array(T)` | `list` | | +| `Array(UInt8)` | `blob` | | +| `Nullable(T)` | `T` | Null values stay null. | + +`Map(...)` is not currently supported. Cast unsupported columns in `sql` or +omit them from the query result. + +Some ClickHouse types are not perfect inverses of to_clickhouse. For +example, to_clickhouse writes `bool` as `UInt8`, so reading the same +table back yields `uint64` unless the ClickHouse column type is `Bool`. + +## Examples + +### Read all rows from a table + +```tql +from_clickhouse table="events", tls=false +``` + +### Use a connection URI + +```tql +from_clickhouse uri="clickhouse://default:secret@clickhouse.example.com:9000/security", + table="events", + tls=false +``` + +### Run a filtered SQL query + +```tql +from_clickhouse sql="SELECT * FROM events WHERE severity >= 3 ORDER BY time DESC", + tls=false +``` + +### List tables in the current database + +```tql +from_clickhouse sql="SHOW TABLES", tls=false +``` + +### Show the columns for a table + +```tql +from_clickhouse sql="DESCRIBE TABLE events", tls=false +``` + +## See Also + +- to_clickhouse +- collecting/read-from-data-stores +- clickhouse diff --git a/src/content/docs/reference/operators/from_mysql.mdx b/src/content/docs/reference/operators/from_mysql.mdx index cdfece6f3..86bfe6066 100644 --- a/src/content/docs/reference/operators/from_mysql.mdx +++ b/src/content/docs/reference/operators/from_mysql.mdx @@ -238,5 +238,6 @@ from_mysql table="events", live=true, tracking_column="event_id", ## See Also +- collecting/read-from-data-stores - to_clickhouse - mysql diff --git a/src/content/docs/reference/operators/to_clickhouse.mdx b/src/content/docs/reference/operators/to_clickhouse.mdx index a650c7fa4..1573f72fb 100644 --- a/src/content/docs/reference/operators/to_clickhouse.mdx +++ b/src/content/docs/reference/operators/to_clickhouse.mdx @@ -7,9 +7,9 @@ example: 'to_clickhouse table="my_table"' Sends events to a ClickHouse table. ```tql -to_clickhouse table=string, [host=string, port=int, user=string, password=string, - mode=string, primary=field, database=string, - tls=record] +to_clickhouse table=string, + [uri=string | (host=string, port=int, user=string, password=string)], + [mode=string, primary=field, tls=bool|record] ``` ## Description @@ -21,8 +21,25 @@ The name of the table you want to write to. This can be a dynamic expression, allowing you to automatically write to different tables based on the data. -The `.
` notation can be used to also specify a table. If no ``, -is provided, `"default"` will be used. +The `.
` notation can be used to also specify a table. If no `` +is provided, Tenzir writes to the database selected by the URI, if any, or the server default. + +### `uri = string (optional)` + +A ClickHouse connection URI in the format: + +```text +clickhouse://[user[:password]@]host[:port][/database] +``` + +When present, the URI supplies the connection endpoint and optionally the +current database. + +If the URI includes `/database` and `table` is unqualified, Tenzir writes to +that database. In `mode="create"` and `mode="create_append"`, Tenzir also +creates the selected database if it does not exist yet. + +Use `tls` separately to control TLS. ### `host = string (optional)` @@ -30,24 +47,32 @@ The hostname for the ClickHouse server. Defaults to `"localhost"`. +Mutually exclusive with `uri`. + ### `port = int (optional)` The port for the ClickHouse server. Defaults to `9000` without TLS and `9440` with TLS. +Mutually exclusive with `uri`. + ### `user = string (optional)` The user to use for authentication. Defaults to `"default"`. +Mutually exclusive with `uri`. + ### `password = string (optional)` The password for the given user. Defaults to `""`. +Mutually exclusive with `uri`. + ### `mode = string (optional)` - `"create"` Create a table and database. Fails the table already exists. @@ -153,6 +178,18 @@ from "my_file.csv" to_clickhouse table="my_table", tls=false ``` +### Use a connection URI + +```tql +from "my_file.csv" +to_clickhouse uri="clickhouse://default:secret@clickhouse.example.com:9000/security", + table="alerts", + primary=time, + tls=false +``` + +This writes to `security.alerts`. + ### Send OCSF data to ClickHouse When sending OCSF data to ClickHouse, it is important to ensure that a consistent @@ -188,5 +225,7 @@ This creates the following table: ## See Also +- from_clickhouse - ocsf::cast +- routing/send-to-destinations - clickhouse diff --git a/src/content/docs/reference/operators/to_snowflake.mdx b/src/content/docs/reference/operators/to_snowflake.mdx index b3f5f9878..9622aaedf 100644 --- a/src/content/docs/reference/operators/to_snowflake.mdx +++ b/src/content/docs/reference/operators/to_snowflake.mdx @@ -96,4 +96,5 @@ to_snowflake \ ## See Also +- routing/send-to-destinations - snowflake diff --git a/src/sidebar.ts b/src/sidebar.ts index 5bbaefa7e..de0c2bb04 100644 --- a/src/sidebar.ts +++ b/src/sidebar.ts @@ -89,6 +89,7 @@ export const guides = [ "guides/collecting/read-and-watch-files", "guides/collecting/fetch-via-http-and-apis", "guides/collecting/read-from-message-brokers", + "guides/collecting/read-from-data-stores", "guides/collecting/get-data-from-the-network", ], },