Skip to content

[sqlserver] Fix Azure SQL schema query selection#23533

Merged
lu-zhengda merged 1 commit intomasterfrom
codex/fix-azure-sql-db-schema-version-detection
May 1, 2026
Merged

[sqlserver] Fix Azure SQL schema query selection#23533
lu-zhengda merged 1 commit intomasterfrom
codex/fix-azure-sql-db-schema-version-detection

Conversation

@lu-zhengda
Copy link
Copy Markdown
Contributor

@lu-zhengda lu-zhengda commented Apr 29, 2026

What changed

This updates SQL Server schema collection so Azure SQL Database and Azure SQL Managed Instance no longer use boxed SQL Server ProductMajorVersion alone to choose the schema query path. The collector records each database's sys.databases.compatibility_level and uses it as the database-scoped guard for the modern schema query.

The emitted database schema payload now includes compatibility_level; the backend ignores unknown fields, and the metadata tests normalize the value because it varies by SQL Server version.

Query selection logic

The modern schema query depends on two different SQL Server capabilities:

  • STRING_AGG is an engine feature. For boxed SQL Server, the collector still requires SQL Server 2017 or later because SQL Server 2016 can run a database at compatibility level 130 but still does not support STRING_AGG.
  • JSON output is database-scoped, so the collector also checks each database's compatibility_level and uses the legacy query when it is below 130 or missing.
  • Azure SQL Database and Azure SQL Managed Instance skip the boxed engine-version gate because their ProductMajorVersion is not comparable to boxed SQL Server releases. They can report version 12 while still supporting STRING_AGG.

Why

Azure SQL Database and Azure SQL Managed Instance can report version strings such as Microsoft SQL Azure (RTM) - 12.0.2000.8, where major version 12 is not SQL Server 2014. Treating that value as a boxed SQL Server release incorrectly sends modern Azure SQL deployments through the legacy schema path.

Connection reuse for the legacy table-detail path is split into #23544. Managed Instance AO secondary lag handling is split into #23558.

https://datadoghq.atlassian.net/browse/SDBM-2571

@lu-zhengda lu-zhengda force-pushed the codex/fix-azure-sql-db-schema-version-detection branch 2 times, most recently from 1d36032 to e06368d Compare April 29, 2026 18:50
@lu-zhengda lu-zhengda changed the title [codex] Fix Azure SQL Database schema detection [sqlserver] Fix Azure SQL Database schema detection Apr 29, 2026
@lu-zhengda lu-zhengda force-pushed the codex/fix-azure-sql-db-schema-version-detection branch 2 times, most recently from 29dad99 to 56330c9 Compare April 29, 2026 19:07
@lu-zhengda lu-zhengda changed the title [sqlserver] Fix Azure SQL Database schema detection [sqlserver] Fix Azure SQL schema collection Apr 29, 2026
@lu-zhengda lu-zhengda force-pushed the codex/fix-azure-sql-db-schema-version-detection branch 2 times, most recently from dcab759 to ea3a14c Compare April 29, 2026 19:33
@lu-zhengda lu-zhengda changed the title [sqlserver] Fix Azure SQL schema collection [sqlserver] Fix Azure SQL schema query selection and pre-2017 connection reuse Apr 29, 2026
@lu-zhengda lu-zhengda force-pushed the codex/fix-azure-sql-db-schema-version-detection branch from ea3a14c to cbebc43 Compare April 29, 2026 19:41
@lu-zhengda lu-zhengda changed the title [sqlserver] Fix Azure SQL schema query selection and pre-2017 connection reuse [sqlserver] Fix Azure SQL schema query selection and legacy connection reuse Apr 29, 2026
@datadog-official
Copy link
Copy Markdown
Contributor

datadog-official Bot commented Apr 29, 2026

Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 90.14% (+2.99%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 76ac311 | Docs | Datadog PR Page | Give us feedback!

Comment thread sqlserver/tests/test_unit.py Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.44%. Comparing base (3fb0c5c) to head (76ac311).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lu-zhengda lu-zhengda changed the title [sqlserver] Fix Azure SQL schema query selection and legacy connection reuse [sqlserver] Fix Azure SQL schema query selection Apr 30, 2026
def _record_database_compatibility_levels(self, databases: list[DatabaseInfo]) -> None:
self._database_compatibility_levels = {}
for database in databases:
self._database_compatibility_levels[database["name"]] = int(database["compatibility_level"])
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for this int to raise an error if the value isn't numeric as expected?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compatibility_level is a required tinyint type so it would't be an issue


def _should_use_legacy_schema_query(self, database_name: str) -> bool:
engine_edition = self._check.static_info_cache.get(STATIC_INFO_ENGINE_EDITION)
if is_azure_database(engine_edition):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to branch on azure here? Is compatibility_level not supported on other SQL Server installations?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trying to reduce the blast radius but it could also be a good idea to completely rely on compatibility_level for schema. the field is available in all MS documented sqlserver versions

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point that compatibility_level is available broadly. I updated this to use it for all engine editions as the database-scoped guard, but kept the Azure branch to skip only the boxed ProductMajorVersion check. Compatibility level alone is not sufficient on boxed SQL Server because SQL Server 2016 can have compatibility level 130 while still lacking STRING_AGG; Azure SQL DB/MI report a non-boxed major version, so they need the engine-edition exception.


major_version = int(self._check.static_info_cache.get(STATIC_INFO_MAJOR_VERSION) or 0)
if major_version == 0:
self._check.log.debug("major_version is not available yet, defaulting to 2016 or earlier")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should maintain the previous logic and not collect if we don't have the version yet.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this to preserve the legacy fallback for the missing-information cases. Non-Azure still defaults to the legacy schema query when major_version is unavailable, and any engine edition defaults to legacy if the per-database compatibility_level was not recorded. Azure SQL DB/MI only bypass the boxed major-version check because their reported ProductMajorVersion is not comparable.

@lu-zhengda lu-zhengda force-pushed the codex/fix-azure-sql-db-schema-version-detection branch 3 times, most recently from 815c6cc to 0c12d4d Compare April 30, 2026 23:13
@lu-zhengda lu-zhengda force-pushed the codex/fix-azure-sql-db-schema-version-detection branch from 0c12d4d to 9727169 Compare April 30, 2026 23:14
@lu-zhengda lu-zhengda force-pushed the codex/fix-azure-sql-db-schema-version-detection branch 4 times, most recently from 41e026c to b9e74a6 Compare May 1, 2026 14:15
@lu-zhengda lu-zhengda requested a review from sethsamuel May 1, 2026 15:46
@lu-zhengda lu-zhengda force-pushed the codex/fix-azure-sql-db-schema-version-detection branch from b9e74a6 to 76ac311 Compare May 1, 2026 17:55
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 1, 2026

Validation Report

All 20 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and Codecov settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@lu-zhengda lu-zhengda added this pull request to the merge queue May 1, 2026
Merged via the queue into master with commit 475d23d May 1, 2026
76 checks passed
@lu-zhengda lu-zhengda deleted the codex/fix-azure-sql-db-schema-version-detection branch May 1, 2026 20:33
@dd-octo-sts dd-octo-sts Bot added this to the 7.79.0 milestone May 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants