Document to_file, to_s3, to_azure_blob_storage, to_google_cloud_storage#280
Draft
raxyte wants to merge 3 commits intotopic/new-executorfrom
Draft
Document to_file, to_s3, to_azure_blob_storage, to_google_cloud_storage#280raxyte wants to merge 3 commits intotopic/new-executorfrom
raxyte wants to merge 3 commits intotopic/new-executorfrom
Conversation
Adds reference pages for the new filesystem sink operators implemented on top of `ToArrowFsOperator` (tenzir/tenzir#6053), along with a shared `ToArrowFsCommonParams` partial covering `max_size`, `timeout`, `partition_by`, and the required byte-producing subpipeline. Also registers the new pages in `reference/operators.mdx`.
github-actions
Bot
added
reference
The new-style `from_file`, `from_s3`, `from_gcs`, and `from_azure_blob_storage` no longer accept `path_field`. Attaching the source path to events is now done in the subpipeline via the `$file.path` variable exposed by the operator, which also preserves the existing `mtime` field and lets users perform arbitrary per-event transformations.
Brings the operator name in line with the new `to_google_cloud_storage` sibling and the renamed plugin file on disk, and uses the full name consistently with `from_azure_blob_storage`. Updates the reference page, the operators index, integration and testing guides. Historical changelog entries continue to refer to `from_gcs` as the original release notes.
github-actions
Bot
added
guide
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion docs for tenzir/tenzir#6053.
Adds reference pages for the new filesystem sink operators built on top of
ToArrowFsOperator, and introduces a sharedToArrowFsCommonParamspartial coveringmax_size,timeout,partition_by, and the required byte-producing subpipeline.