Skip to content

ENH: add xarray-native open_datatree with engine= parameter#335

Draft
aladinor wants to merge 8 commits intoopenradar:mainfrom
aladinor:feat/open-datatree-engine
Draft

ENH: add xarray-native open_datatree with engine= parameter#335
aladinor wants to merge 8 commits intoopenradar:mainfrom
aladinor:feat/open-datatree-engine

Conversation

@aladinor
Copy link
Copy Markdown
Member

@aladinor aladinor commented Feb 20, 2026

Summary

Closes #329

Note: This is a draft. Full implementation depends on #333 (station coords + optional_groups) and #332 (NEXRAD chunk support) being merged first. After those land, this branch will be rebased to incorporate:

  • optional_groups parameter flow-through in open_groups_as_dict / open_datatree
  • Station coordinates as root coordinates (propagates automatically via _assign_root)
  • NEXRAD chunk list/tuple input support in open_groups_as_dict

Adds open_datatree() and open_groups_as_dict() methods with supports_groups = True to three prototype backends (ODIM, CfRadial1, NEXRAD Level2), enabling:

import xradar as xd
import xarray as xr

# New unified xradar API
dtree = xd.open_datatree("file.h5", engine="odim")

# xarray native API
dtree = xr.open_datatree("file.h5", engine="odim")

# Legacy (emits FutureWarning)
dtree = xd.io.open_odim_datatree("file.h5")

Changes

  • xradar/io/backends/common.py: Add shared _build_groups_dict() helper and _deprecation_warning() utility
  • xradar/io/backends/odim.py: Add supports_groups=True, open_groups_as_dict(), open_datatree(); refactor open_odim_datatree() to delegate with deprecation warning
  • xradar/io/backends/cfradial1.py: Same pattern (unique single-file slicing logic)
  • xradar/io/backends/nexrad_level2.py: Same pattern (store-based with sweep resolution)
  • xradar/io/__init__.py: Add engine registry and unified open_datatree(file, engine=) function
  • xradar/__init__.py: Export open_datatree at package level
  • requirements.txt: Exclude broken netCDF4 1.7.3/1.7.4
  • tests/io/test_backend_datatree.py: 24 new tests (integration, deprecation, equivalence)
  • examples/notebooks/Open-Datatree-Engine.ipynb: Demo notebook

Depends on

Test plan

@review-notebook-app
Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@aladinor aladinor force-pushed the feat/open-datatree-engine branch from 3542304 to 173ae84 Compare February 26, 2026 23:18
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 27, 2026

Codecov Report

❌ Patch coverage is 98.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.57%. Comparing base (e59d9e6) to head (a9c9519).

Files with missing lines Patch % Lines
xradar/io/backends/nexrad_level2.py 94.73% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #335      +/-   ##
==========================================
+ Coverage   93.52%   93.57%   +0.04%     
==========================================
  Files          27       27              
  Lines        5717     5755      +38     
==========================================
+ Hits         5347     5385      +38     
  Misses        370      370              
Flag Coverage Δ
notebooktests ?
unittests 93.57% <98.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

aladinor and others added 7 commits March 30, 2026 18:06
Implement open_datatree/open_groups_as_dict on three prototype backends
(ODIM, CfRadial1, NEXRAD Level2) with supports_groups=True, enabling
xr.open_datatree(file, engine="odim") and a unified xd.open_datatree()
entry point. Deprecate standalone open_*_datatree functions with
FutureWarning.

Closes openradar#329

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Resolve rebase conflicts with openradar#333 (station coords + optional_groups):
- Thread optional_groups parameter through open_groups_as_dict and
  _build_groups_dict for all three backends (ODIM, CfRadial1, NEXRAD)
- Update test assertions to match optional_groups=False default
- Fix read-only array bug in util._ipol_time (use .values.copy())
Rebase fixes:
- Resolve conflicts in common.py, odim.py, cfradial1.py, nexrad_level2.py
- Keep _prepare_backend_ds from PR openradar#345 alongside _build_groups_dict
- Remove unused _attach_sweep_groups imports

Bug fixes:
- Fix site_coords → site_as_coords parameter name in ODIM and NEXRAD
- Fix _assign_root tuple unpacking in NEXRAD open_groups_as_dict
- Add incomplete_sweep drop/pad/chunk logic to NEXRAD open_groups_as_dict
- Fix site_coords undefined in open_cfradial1_datatree deprecation wrapper
- Remove duplicate _deprecation_warning call in open_odim_datatree
- Guard against empty sweep list (sweep=[]) with ValueError
- Remove unused exp_sweeps variable

New tests (30 total, up from 24):
- test_site_coords_false: verifies kwarg flows through correctly
- test_xr_open_datatree_cfradial1: was missing
- test_nexrad_groups_dict_optional_groups: was missing
- test_nexrad_empty_sweep_list_raises: edge case
- TestEngineRegistry: verifies registry completeness
- Deprecation warning count changed from >= 1 to == 1
Phase 2: Convert 7 standard-pattern backends (GAMIC, IRIS, Furuno,
Rainbow, DataMet, HPL, Metek) to support open_groups_as_dict() and
open_datatree() with supports_groups=True. All 10 backends now
registered in _ENGINE_REGISTRY.

Code improvements:
- Extract _resolve_sweeps helper to common.py, replacing duplicated
  sweep normalization in all backends
- Fix stacklevel=3 → 4 in _deprecation_warning
- Fix HPL sweep=[N] incorrect +1 offset
- Fix open_hpl_datatree optional=None → True
- Fix GAMIC/IRIS deprecated wrappers missing site_as_coords remap
- Strip station vars from sweeps in _build_groups_dict

Tests (74 total):
- All 9 engines parametrized (basic open, int/string sweep, kwargs,
  empty list guard)
- CfRadial1 tested separately (needs engine="h5netcdf")
- xr.open_datatree tested for 5 engines
- supports_groups verified for all 10 engines
- Engine registry exact match assertion
- Deprecation FutureWarning tested for all 10 deprecated functions
UF backend:
- Add supports_groups=True, open_groups_as_dict(), open_datatree()
- Use _resolve_sweeps for sweep normalization
- Drop _STATION_VARS from sweeps in groups_dict
- Deprecate open_uf_datatree() with FutureWarning
- Register "uf" in _ENGINE_REGISTRY (11/11 complete)

Bug fixes:
- Fix NEXRAD deprecated wrapper: site_coords=site_as_coords instead
  of hardcoded False
- Fix NEXRAD/UF: drop _STATION_VARS from sweep datasets in
  open_groups_as_dict (matching _build_groups_dict behavior)

Tests (87 total):
- Add xr.open_datatree tests for all 11 engines
- UF added to all parametrized test fixtures
- Engine registry now asserts all 11 engines
@aladinor aladinor force-pushed the feat/open-datatree-engine branch from a9c9519 to 6986717 Compare March 31, 2026 02:51
@aladinor aladinor marked this pull request as ready for review March 31, 2026 02:55
@aladinor aladinor marked this pull request as draft March 31, 2026 03:04
@aladinor aladinor mentioned this pull request Apr 1, 2026
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Implement xarray-native open_datatree with engine parameter

1 participant