Skip to content

Downgrade gRPC errors to warnings in remote execution logging#3511

Open
sean- wants to merge 1 commit intothought-machine:masterfrom
sean-:grpc-log-downgrade-errors
Open

Downgrade gRPC errors to warnings in remote execution logging#3511
sean- wants to merge 1 commit intothought-machine:masterfrom
sean-:grpc-log-downgrade-errors

Conversation

@sean-
Copy link
Contributor

@sean- sean- commented Mar 26, 2026

Remote-side errors - like CAS cache misses (NotFound) - are logged as ERROR by the gRPC interceptor, but the client recovers gracefully by falling back to local execution. Downgrade these to WARNING to avoid noisy error output during normal builds.

13:35:53.445  NOTICE: Build running for 4m20s, 3601 / 3835 tasks done (234 left), 4 workers busy, parsing 0 BUILD files
13:36:03.446  NOTICE: Build running for 4m30s, 3626 / 3835 tasks done (209 left), 10 workers busy, parsing 0 BUILD files
13:36:13.446  NOTICE: Build running for 4m40s, 3647 / 3835 tasks done (188 left), 7 workers busy, parsing 0 BUILD files
13:36:23.446  NOTICE: Build running for 4m50s, 3673 / 3835 tasks done (162 left), 3 workers busy, parsing 0 BUILD files
13:36:28.895   ERROR: Error handling /build.bazel.remote.execution.v2.ContentAddressableStorage/GetTree: 1 error occurred:
        * rpc error: code = NotFound desc = Blob d8199952771ae2bf004da92854cd172dfa9d125a10f8d0b5ff9ccb816308d1b7 not found


13:36:33.446  NOTICE: Build running for 5m0s, 3702 / 3835 tasks done (133 left), 6 workers busy, parsing 0 BUILD files
13:36:43.445  NOTICE: Build running for 5m10s, 3728 / 3835 tasks done (107 left), 10 workers busy, parsing 0 BUILD files
13:36:53.446  NOTICE: Build running for 5m20s, 3762 / 3835 tasks done (73 left), 8 workers busy, parsing 0 BUILD files
13:37:03.445  NOTICE: Build running for 5m30s, 3787 / 3835 tasks done (48 left), 3 workers busy, parsing 0 BUILD files

Remote-side errors like CAS cache misses (NotFound) are logged as ERROR
by the gRPC interceptor, but the client recovers gracefully by falling
back to local execution. Downgrade these to WARNING to avoid noisy
error output during normal builds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant