Skip to content

fix: shader compilation error on Firefox with Nvidia driver #10069

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lsdch wants to merge 2 commits into
base: master
Choose a base branch
from
Open

fix: shader compilation error on Firefox with Nvidia driver #10069

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1502cdb
fix(gpu-aggregator): use sentinel value for empty bins in shaders to …
lsdch Mar 6, 2026
bfcd415
Merge branch 'master' into fix-nan-firefox-nvidia
chrisgervang Mar 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions ...es/aggregation-layers/src/common/aggregator/gpu-aggregator/webgl-aggregation-transform.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,9 @@ flat out vec2 values;
flat out vec3 values;
#endif

const float NAN = intBitsToFloat(-1);
// Sentinel value to indicate an empty bin instead of NaN,
// which can cause issues with compilation in some drivers
const float EMPTY_BIN = -3e38;
Copy link
Copy Markdown

@cursor cursor bot Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sentinel value passes Number.isFinite check, corrupting scale computations

Medium Severity

The sentinel value EMPTY_BIN = -3e38 is a finite number, so Number.isFinite(-3e38) returns true. The scale utility functions in scale-utils.ts rely on Number.isFinite() to filter out empty bin values — applyScaleQuantile uses .filter(Number.isFinite) and applyScaleOrdinal checks Number.isFinite(x). Previously, NaN was correctly excluded by these filters. Now, -3e38 values from empty bins pass through and contaminate percentile threshold calculations and ordinal category mappings, skewing results when quantile or ordinal scale types are used.

Additional Locations (1)

Fix in Cursor Fix in Web


void main() {
int row = gl_VertexID / SAMPLER_WIDTH;
Expand All @@ -158,7 +160,7 @@ void main() {
aggregatorTransform.isMean
);
if (weights.a == 0.0) {
value3 = vec3(NAN);
value3 = vec3(EMPTY_BIN);
}

#if NUM_DIMS == 1
Expand Down Expand Up @@ -201,6 +203,9 @@ flat in vec3 values;

out vec4 fragColor;

// Same sentinel as vertex shader
const float EMPTY_BIN = -3e38;

void main() {
vec3 value3;
#if NUM_CHANNELS == 3
Expand All @@ -210,7 +215,8 @@ void main() {
#else
value3.x = values;
#endif
if (isnan(value3.x)) discard;
// Skip empty bins, with epsilon to account for potential floating point precision issues
if (value3.x <= EMPTY_BIN + 1e-5) discard;
Copy link
Copy Markdown

@cursor cursor bot Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epsilon 1e-5 is meaningless at magnitude 3e38

Low Severity

The comparison value3.x <= EMPTY_BIN + 1e-5 adds an epsilon of 1e-5 to -3e38, but at that magnitude, float32 precision is on the order of ~1e31. The addition (-3e38) + 1e-5 evaluates to exactly -3e38 in floating-point arithmetic, so the epsilon provides zero additional tolerance despite the comment claiming it accounts for "floating point precision issues." The check still works for exact sentinel matches, but the claimed safety margin is illusory.

Fix in Cursor Fix in Web

// This shader renders into a 2x1 texture with blending=max
// The left pixel yields the max value of each channel
// The right pixel yields the min value of each channel
Expand Down