-
Notifications
You must be signed in to change notification settings - Fork 15
feat: M-LOG-STRUCTURED guide #26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
343f28a
feat: M-LOG-STRUCTURED guide
martintmk cda2ae7
shorten the guide
martintmk 8121c4a
cleanup
martintmk a81f8e8
Delete TELEMETRY.md
martintmk 77068c1
cleanup
martintmk 32f850e
Update M-LOG-STRUCTURED.md
martintmk 0bd2cea
Update M-LOG-STRUCTURED.md
martintmk aa4a3d6
Update M-LOG-STRUCTURED.md
martintmk 421945b
Update M-LOG-STRUCTURED.md
martintmk 7f5d1b7
clarification
martintmk f076d83
Update M-LOG-STRUCTURED.md
martintmk 0b829da
Cleanup, align with existing guidelines, link.
ralfbiedert e78f39d
Disable annoying lint.
ralfbiedert File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| <!-- Copyright (c) Microsoft Corporation. Licensed under the MIT license. --> | ||
|
|
||
| ## Use Structured Logging with Message Templates (M-LOG-STRUCTURED) { #M-LOG-STRUCTURED } | ||
|
|
||
| <why>To minimize the cost of logging and to improve filtering capabilities.</why> | ||
| <version>0.1</version> | ||
|
|
||
| Logging should use structured events with named properties and message templates following | ||
| the [message templates](https://messagetemplates.org/) specification. | ||
|
|
||
| > **Note:** Examples use the [`tracing`](https://docs.rs/tracing/) crate's `event!` macro, | ||
| but these principles apply to any logging API that supports structured logging (e.g., `log`, | ||
| `slog`, custom telemetry systems). | ||
|
|
||
| ### Avoid String Formatting | ||
|
|
||
| String formatting allocates memory at runtime, even if logs are filtered out. Message templates defer | ||
| formatting until viewing time. | ||
|
|
||
| ```rust,ignore | ||
| // DON'T: String formatting causes allocations | ||
| tracing::info!("file opened: {}", path); | ||
| tracing::info!(format!("file opened: {}", path)); | ||
|
|
||
| // DO: Use message templates with named properties | ||
| event!( | ||
| name: "file.open.success", | ||
| Level::INFO, | ||
| file.path = path.display(), | ||
| "file opened: {{file.path}}", | ||
| ); | ||
| ``` | ||
|
|
||
| > **Note**: Use `{{property}}` syntax in message templates. Double braces preserve the literal text | ||
| > while escaping Rust's format syntax. String formatting is deferred until logs are viewed. | ||
| > | ||
| > This pattern may trigger Clippy's [`literal_string_with_formatting_args`](https://rust-lang.github.io/rust-clippy/stable/index.html#literal_string_with_formatting_args) | ||
| > warning so consider suppressing it for logging. | ||
|
|
||
| ### Name Your Events | ||
|
|
||
| Use hierarchical dot-notation: `<component>.<operation>.<state>` | ||
|
|
||
| ```rust,ignore | ||
| // DON'T: Unnamed events | ||
| event!( | ||
| Level::INFO, | ||
| file.path = file_path, | ||
| "file {{file.path}} processed succesfully", | ||
| ); | ||
|
|
||
| // DO: Named events | ||
| event!( | ||
| name: "file.processing.success", // event identifier | ||
| Level::INFO, | ||
| file.path = file_path, | ||
| "file {{file.path}} processed succesfully", | ||
| ); | ||
| ``` | ||
|
|
||
| Named events enable grouping and filtering across log entries. | ||
|
|
||
| ### Follow OpenTelemetry Semantic Conventions | ||
|
|
||
| Use [OTel semantic conventions](https://opentelemetry.io/docs/specs/semconv/) for common attributes if needed. | ||
| This enables standardization and interoperability. | ||
|
|
||
| ```rust,ignore | ||
| event!( | ||
| name: "file.write.success", | ||
| Level::INFO, | ||
| file.path = path.display(), // Standard OTel name | ||
| file.size = bytes_written, // Standard OTel name | ||
| file.directory = dir_path, // Standard OTel name | ||
| file.extension = extension, // Standard OTel name | ||
| file.operation = "write", // Custom name | ||
| "{{file.operation}} {{file.size}} bytes to {{file.path}} in {{file.directory}} extension={{file.extension}}", | ||
| ); | ||
| ``` | ||
|
|
||
| Common conventions: | ||
|
|
||
| - HTTP: `http.request.method`, `http.response.status_code`, `url.scheme`, `url.path`, `server.address` | ||
| - File: `file.path`, `file.directory`, `file.name`, `file.extension`, `file.size` | ||
| - Database: `db.system.name`, `db.namespace`, `db.operation.name`, `db.query.text` | ||
| - Errors: `error.type`, `error.message`, `exception.type`, `exception.stacktrace` | ||
|
|
||
| ### Redact Sensitive Data | ||
|
|
||
| Do not log plain sensitive data as this might lead to privacy and security incidents. | ||
|
|
||
| ```rust,ignore | ||
| // DON'T: Log potentially sensitive data | ||
| event!( | ||
| name: "file.operation.started", | ||
| Level::INFO, | ||
| user.email = user.email, // Sensitive data | ||
| file.name = "license.txt", | ||
| "reading file {{file.name}} for user {{user.email}}", | ||
| ); | ||
|
|
||
| // DO: Redact sensitive parts | ||
| event!( | ||
| name: "file.operation.started", | ||
| Level::INFO, | ||
| user.email.redacted = redact_email(user.email), | ||
| file.name = "license.txt", | ||
| "reading file {{file.name}} for user {{user.email.redacted}}", | ||
| ); | ||
| ``` | ||
|
|
||
| Sensitive data include user email, file paths revealing user identity, filenames containing secrets or tokens, | ||
| file contents with PII, temporary file paths with session IDs and more. | ||
|
|
||
| Consider using the [`data_privacy`](https://crates.io/crates/data_privacy) crate for consistent redaction. | ||
|
|
||
| ### Further Reading | ||
|
|
||
| - [Message Templates Specification](https://messagetemplates.org/) | ||
| - [OpenTelemetry Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/) | ||
| - [OWASP Logging Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.