Add resourceAllocation field to trace record by pditommaso · Pull Request #6973 · nextflow-io/nextflow

pditommaso · 2026-03-25T09:32:15Z

Summary

Add resourceAllocation transient field to TraceRecord exposing scheduler-allocated resources (cpuShares, memoryMiB, accelerators, time)
Source the value from the last TaskAttempt.resources, falling back to TaskState.resourceAllocation when no attempts exist
Forward the field through TowerClient to the platform
Bump sched-client to 0.46.0-SNAPSHOT (renames TaskState.resourceRequirement → resourceAllocation)

Test plan

Unit tests for getResourceAllocation() in SeqeraTaskHandlerTest (null state, single attempt, multiple attempts, fallback scenarios)
Unit test for resourceAllocation propagation in TowerClientTest.makeTasksReq()
Existing trace metadata test updated to verify resourceAllocation included in trace record

🤖 Generated with Claude Code

Expose scheduler-allocated resources (cpuShares, memoryMiB, accelerators, time) in the trace record. The value is taken from the last TaskAttempt's resources, falling back to the TaskState's resourceAllocation if no attempts exist. Also bump sched-client to 0.46.0-SNAPSHOT which renames TaskState.resourceRequirement to resourceAllocation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

netlify · 2026-03-25T09:32:21Z

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	`f5f840c`
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs-staging/deploys/69d53c449fb06c000847b643

This reverts commit 00f35b3. The accelerator and accelerator_type fields in the trace record are superseded by the resourceAllocation field which carries the actual scheduler-allocated resources including accelerator info. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

pditommaso · 2026-03-25T14:16:50Z

@bentsherman have a look at this. I think we can remove accelerator for "classic" trace and only keep as platform trace via resourceAllocation

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

bentsherman · 2026-03-30T15:33:51Z

modules/nextflow/src/main/groovy/nextflow/trace/TraceRecord.groovy

    transient private ContainerMeta containerMeta
    transient private Integer numSpotInterruptions
    transient private String logStreamId
+    transient private Map<String,Object> resourceAllocation


not blocking, but the naming is a bit awkward. how about resourcesAllocated ?

Eventually, it's prefer to keep aligned to sched naming for now

modules/nextflow/src/main/groovy/nextflow/trace/TraceRecord.groovy

plugins/nf-seqera/src/test/io/seqera/executor/SeqeraTaskHandlerTest.groovy

* Bump sched-client to 0.47.0 and update prediction model support - Upgrade sched-client from 0.41.0-SNAPSHOT to 0.47.0 - Add qr/v2 prediction model to supported values description - Remove client-side prediction model validation (moved to backend) - Fix getResourceRequirement() -> getResourceAllocation() API change Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> * Fix test to use renamed resourceAllocation API Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> --------- Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…i skip] * Docs: document Wave support for module resources directory Update Wave and module docs to explain that the module `resources/` directory is automatically included in Wave-provisioned containers, removing the need for ADD/COPY Dockerfile commands. Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> * Update docs/module.md [ci skip] Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> * Update docs/wave.md [ci skip] Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> --------- Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Chris Hakkaart <chris.hakkaart@seqera.io>

--------- Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Signed-off-by: jorgee <jorge.ejarque@seqera.io>

Co-authored-by: Ben Sherman <bentshermann@gmail.com>

… fast] * Use npr-client API instead of custom ModuleRegistryClient Replace the custom ModuleRegistryClient with the npr-client library, delegating HTTP registry interactions to the shared client. This removes ~500 lines of duplicated HTTP/retry/auth logic. - Delete ModuleRegistryClient.groovy - Update call sites to use new npr-client method names (getModule, getModuleRelease, searchModules, downloadModuleRelease, publishModuleRelease) - Update all tests to match new API Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> * Bump npr-api and npr-client to 0.22.0 [ci fast] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> --------- Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Keep the existing trace fields for requested accelerators alongside the new resourceAllocation field for allocated resources. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

Set accelerator request count and type in the trace record from the process accelerator directive. Add corresponding test coverage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

pditommaso requested a review from bentsherman March 30, 2026 14:55

pditommaso marked this pull request as ready for review March 30, 2026 14:55

Merge branch 'master' into add-resource-allocation-trace-field

63a908b

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

bentsherman reviewed Mar 30, 2026

View reviewed changes

modules/nextflow/src/main/groovy/nextflow/trace/TraceRecord.groovy Show resolved Hide resolved

bentsherman reviewed Mar 30, 2026

View reviewed changes

plugins/nf-seqera/src/test/io/seqera/executor/SeqeraTaskHandlerTest.groovy Show resolved Hide resolved

pditommaso and others added 17 commits April 7, 2026 19:16

Bump org.apache.groovy from 4.0.30 to 4.0.31 (#6985) [ci fast]

601c3f0

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Remove stale reference to soon-to-be EOL'd AWS Linux 2 (#6970) [ci skip]

d5ce609

Use inline metadata from trace create response (#6976)

c50e75d

--------- Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com> Signed-off-by: jorgee <jorge.ejarque@seqera.io>

Replace Auth0 with Platform OIDC PKCE for auth login (#6953)

8c52655

Document workflow output lineage types (#6972)

9ea212d

Co-authored-by: Ben Sherman <bentshermann@gmail.com>

docs: Add output labels use cases and details (#6986)

a8687cb

Co-authored-by: Ben Sherman <bentshermann@gmail.com>

docs: Improve process directive docs (#6990)

a0b4319

Co-authored-by: Ben Sherman <bentshermann@gmail.com>

Typed workflows (#6881)

ae10318

Add module create subcommand (#6992)

3249b51

Co-authored-by: Ben Sherman <bentshermann@gmail.com>

Add module validate subcommand (#6993)

4cd2ea2

Co-authored-by: Ben Sherman <bentshermann@gmail.com>

Add module spec command (#6859)

ab50174

Co-authored-by: Ben Sherman <bentshermann@gmail.com>

docs: Add migration guide reference to legacy operators page (#7010)

93924a8

pditommaso requested review from a team as code owners April 7, 2026 17:17

pditommaso requested a review from bentsherman April 7, 2026 17:17

Merge branch 'master' into add-resource-allocation-trace-field

f5f840c

pditommaso merged commit a274293 into master Apr 7, 2026
25 of 26 checks passed

pditommaso deleted the add-resource-allocation-trace-field branch April 7, 2026 19:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add resourceAllocation field to trace record#6973

Add resourceAllocation field to trace record#6973
pditommaso merged 21 commits intomasterfrom
add-resource-allocation-trace-field

pditommaso commented Mar 25, 2026

Uh oh!

netlify bot commented Mar 25, 2026 •

edited

Loading

Uh oh!

pditommaso commented Mar 25, 2026

Uh oh!

bentsherman Mar 30, 2026

Uh oh!

pditommaso Apr 7, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

pditommaso commented Mar 25, 2026

Summary

Test plan

Uh oh!

netlify bot commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for nextflow-docs-staging canceled.

Uh oh!

pditommaso commented Mar 25, 2026

Uh oh!

bentsherman Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

pditommaso Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

netlify bot commented Mar 25, 2026 •

edited

Loading