datacrumbs-utils

datacrumbs-utils contains the probe builder, client library, user wrappers, tests, and developer tooling that complement the main datacrumbs runtime.

datacrumbs and datacrumbs-utils now have distinct responsibilities:

• datacrumbs
  • builds the runtime daemon and BPF programs
  • owns install-time system configuration generation
  • installs the systemd unit and scheduler service scripts
• datacrumbs-utils
  • builds libdatacrumbs_client.so
  • builds datacrumbs_probe_configurator
  • installs user-facing wrappers such as datacrumbs_wrap, datacrumbs_track, datacrumbs_untrack, datacrumbs_run, datacrumbs_stop, datacrumbs_salloc, datacrumbs_sbatch, and datacrumbs_service_wrapper
  • owns tools, tests, formatting scripts, and Docker helper scripts

Current flow

1. Build and install datacrumbs. datacrumbs bootstraps datacrumbs-utils into the same install prefix as part of its build.
2. datacrumbs generates the install-time system configuration and probe secret if they are missing.
3. Run datacrumbs_probe_configurator from datacrumbs-utils against a host YAML to generate a signed probes file.
4. Start datacrumbs directly or through datacrumbs@<run-id>.service.
5. Launch applications with datacrumbs_wrap or preload/track them with datacrumbs_track.

Important installed files

• bin/datacrumbs_probe_configurator
• bin/datacrumbs_wrap
• bin/datacrumbs_track
• bin/datacrumbs_untrack
• bin/datacrumbs_run
• bin/datacrumbs_stop
• bin/datacrumbs_salloc
• bin/datacrumbs_sbatch
• bin/datacrumbs_service_wrapper
• lib/libdatacrumbs_client.so or lib64/libdatacrumbs_client.so
• etc/datacrumbs/configs/<host>.yaml
• etc/datacrumbs/configs/project.env.local

Probe generation

Generate a signed probes file from an installed host config:

<install-prefix>/bin/datacrumbs_probe_configurator \
  <install-prefix>/etc/datacrumbs/configs/<host>.yaml \
  /path/to/probes.json.gz

The probe builder reads the install-time system configuration and secret generated by datacrumbs. It validates the selected runtime functions against DATACRUMBS_MAX_RUNTIME_FUNCTIONS and fails early if the selection is too large.

Wrapper commands

datacrumbs_wrap is the simplest ad hoc entry point:

datacrumbs_wrap dd if=/dev/zero of=/tmp/out.bin bs=1M count=1 status=none

datacrumbs_track and datacrumbs_untrack patch ELF executables to add or remove libdatacrumbs_client.so.

datacrumbs_run and datacrumbs_stop are multi-node helpers. They expand a node list, connect over SSH, and start or stop datacrumbs@<run-id>.service on each node.

datacrumbs_run --node-list "node[1-4]" --probe-file /path/to/probes.json.gz --run-id myrun
datacrumbs_stop --node-list "node[1-4]" --run-id myrun

datacrumbs_salloc and datacrumbs_sbatch inject DataCrumbs metadata into SLURM job comments:

datacrumbs_salloc --datacrumbs-enable --datacrumbs-probe-file /path/to/probes.json.gz -N 2
datacrumbs_sbatch --datacrumbs-enable --datacrumbs-probe-file /path/to/probes.json.gz job.sh

datacrumbs_service_wrapper exists mainly for testing the scheduler service flow outside a real prolog or epilog:

datacrumbs_service_wrapper start <job-id> <user> <probe-file>
datacrumbs_service_wrapper stop <job-id> <user> <probe-file>

Build and test

cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure

The utility Docker and formatting workflows now live in this repository as well.