[clickhouse] Skip emitting empty storage_health payloads#23553
Merged
sangeetashivaji merged 3 commits intomasterfrom May 1, 2026
Merged
Conversation
Skip the database_monitoring_query_activity emission when every parts-and-merges collection (parts, merges, mutations, replication queue, detached parts, thresholds) is empty. This is the case for fresh ClickHouse Cloud instances with no user data and restricted system tables, and for transient total-collection-failure scenarios. Empty storage_health payloads previously wedged dbm-events-processor partitions on the dd-go side; that processor is being fixed independently in DataDog/dd-go#234748. This change also avoids emitting one useless Kafka message per collection cycle for any genuinely-idle/empty instance. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…mpty Pins the wiring between _collect_and_emit and _emit_events for the case where every collection query fails or returns no rows, which is the scenario that fires when all queries hit exceptions (auth/network) or on a fresh, restricted ClickHouse Cloud instance. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Validation ReportAll 20 validations passed. Show details
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files🚀 New features to boost your workflow:
|
Contributor
🎉 All green!❄️ No new flaky tests detected 🎯 Code Coverage (details) 🔗 Commit SHA: b26b6cc | Docs | Datadog PR Page | Give us feedback! |
sangeetashivaji
commented
Apr 30, 2026
| mock.patch('datadog_checks.clickhouse.parts_and_merges.datadog_agent') as agent_mock, | ||
| ): | ||
| agent_mock.get_version.return_value = '7.64.0' | ||
| job._collect_and_emit() |
Contributor
Author
There was a problem hiding this comment.
make every collector empty
eric-weaver
approved these changes
May 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Skip the
storage_healthevent emission inClickhousePartsAndMerges._emit_eventswhen every collection (parts, merges, mutations, replication queue, detached parts, thresholds) is empty. Previously the integration emitted a payload every collection cycle regardless of whether the queries returned anything, including when every collector caught an exception and returned[].Why this matters
The integration was sending one
storage_healthevent per collection cycle (default 60s) for everymonitored ClickHouse instance, even when there was nothing to report. Two scenarios where this is
undesirable:
data, but the integration was still emitting an empty payload every minute.
ClickHouse restart, restricted system tables on managed services), all eight collectors return
[].The integration was emitting a payload claiming "everything is empty," which is misleading — we don't
actually know the state of the database in that case.
The new guard makes the agent emit only when at least one collection has rows. For any production
instance with actual data, this is a no-op.
Behavior in real environments
The guard short-circuits only when every collection is empty:
partsandthresholdspopulated → emits as beforepartspopulated → emits as beforeparts/thresholdspopulated → emits as beforethresholdspopulated → emits thresholds-only payloadFor any production customer with actual data, this is a no-op.
Test plan
test_emit_events_shape— all collections populated, payload contains every sectiontest_emit_events_uses_query_activity_channel_not_metadata— emits via the query-activitychannel rather than the metadata channel (updated to use non-empty input)
test_emit_events_skips_when_all_collections_empty— (new) every collection empty, noemission
test_collect_and_emit_skips_when_all_collectors_empty— (new) end-to-end: every_collect_*returns empty,
_collect_and_emitproduces no emissiontest_collect_and_emit_runs_with_partial_failures— pre-existing, still passes: when only onecollector returns rows, emission still happens
🤖 Generated with Claude Code