Skip to content

feat(api): optimize api rate limiting and fix token/semaphore leaks#11

Open
zeusoo001 wants to merge 1 commit intobase-linefrom
fix/api-rate-limiter-semaphore-leak
Open

feat(api): optimize api rate limiting and fix token/semaphore leaks#11
zeusoo001 wants to merge 1 commit intobase-linefrom
fix/api-rate-limiter-semaphore-leak

Conversation

@zeusoo001
Copy link
Copy Markdown
Owner

Core changes:

  • GlobalRateLimiter: check IP quota before global QPS so per-IP rejections do not consume a global token; reorder to ip-first
  • RateLimiterServlet.service: check per-endpoint limiter before global to avoid wasting global IP/QPS quota on requests that will be rejected anyway; track perEndpointAcquired separately so IPreemptibleRateLimiter permits are always released in finally
  • RateLimiterInterceptor.interceptCall: same ordering fix; add explicit permit release in the early-return path (global rejects after per-endpoint acquired) and in the catch block when next.startCall() throws, preventing a permanent semaphore leak that would eventually block all gRPC requests
  • GlobalPreemptibleStrategy: remove unused timeout constant and @slf4j annotation

Tests:

  • GlobalRateLimiterTest: verify ip-first ordering conserves global tokens; cover no-ip fallback and independent per-ip limits
  • RateLimiterServletTest: verify per-endpoint rejection skips global; permit released on global rejection and on normal completion
  • RateLimiterInterceptorTest: verify per-endpoint rejection skips global; permit released on global rejection, on startCall exception, and via listener onComplete/onCancel
  • AdaptorTest: minor cleanup

What does this PR do?

Why are these changes required?

This PR has been tested by:

  • Unit Tests
  • Manual Testing

Follow up

Extra details

Core changes:
- GlobalRateLimiter: check IP quota before global QPS so per-IP
  rejections do not consume a global token; reorder to ip-first
- RateLimiterServlet.service: check per-endpoint limiter before
  global to avoid wasting global IP/QPS quota on requests that
  will be rejected anyway; track perEndpointAcquired separately
  so IPreemptibleRateLimiter permits are always released in finally
- RateLimiterInterceptor.interceptCall: same ordering fix; add
  explicit permit release in the early-return path (global rejects
  after per-endpoint acquired) and in the catch block when
  next.startCall() throws, preventing a permanent semaphore leak
  that would eventually block all gRPC requests
- GlobalPreemptibleStrategy: remove unused timeout constant and
  @slf4j annotation

Tests:
- GlobalRateLimiterTest: verify ip-first ordering conserves global
  tokens; cover no-ip fallback and independent per-ip limits
- RateLimiterServletTest: verify per-endpoint rejection skips global;
  permit released on global rejection and on normal completion
- RateLimiterInterceptorTest: verify per-endpoint rejection skips
  global; permit released on global rejection, on startCall exception,
  and via listener onComplete/onCancel
- AdaptorTest: minor cleanup

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant