Skip to content

支持New优雅关闭,避免重启时流式回复被打断 #4258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
feitianbubu wants to merge 1 commit into
base: main
Choose a base branch
from
+27 −3
Open

支持New优雅关闭,避免重启时流式回复被打断 #4258

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 27 additions & 3 deletions main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,17 @@ package main

import (
"bytes"
"context"
"embed"
"errors"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"time"

"github.com/QuantumNous/new-api/common"
Expand Down Expand Up @@ -192,10 +196,30 @@ func main() {
// Log startup success message
common.LogStartupSuccess(startTime, port)

err = server.Run(":" + port)
if err != nil {
common.FatalLog("failed to start HTTP server: " + err.Error())
srv := &http.Server{
Addr: ":" + port,
Handler: server,
}

go func() {
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
common.FatalLog("failed to start HTTP server: " + err.Error())
}
Comment on lines 196 to +207
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Bind the listener before logging startup success.

Line 197 logs a successful startup before Line 205 has actually bound the port. If the bind fails, operators still get a false-positive “started” log and the real failure only appears from a background goroutine.

Suggested shape 
- common.LogStartupSuccess(startTime, port)
-
- srv := &http.Server{
- 	Addr:    ":" + port,
- 	Handler: server,
- }
-
- go func() {
- 	if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
- 		common.FatalLog("failed to start HTTP server: " + err.Error())
- 	}
- }()
+ ln, err := net.Listen("tcp", ":"+port)
+ if err != nil {
+ 	common.FatalLog("failed to start HTTP server: " + err.Error())
+ }
+ common.LogStartupSuccess(startTime, port)
+
+ srv := &http.Server{
+ 	Handler: server,
+ }
+
+ go func() {
+ 	if err := srv.Serve(ln); err != nil && !errors.Is(err, http.ErrServerClosed) {
+ 		common.FatalLog("failed to start HTTP server: " + err.Error())
+ 	}
+ }()

Also add the net import.

</details>

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against the current code and only fix it if needed.

In @main.go around lines 196 - 207, Move the LogStartupSuccess call to after you
successfully bind the socket: perform a net.Listen("tcp", ":"+port) first (add
the net import), if listen returns without error then call
common.LogStartupSuccess(startTime, port), then start the server with
srv.Serve(listener) in the goroutine (or wrap listener in the existing srv).
Update the goroutine to use srv.Serve(listener) and still treat
http.ErrServerClosed as non-fatal; ensure the listener is closed on shutdown so
resources are cleaned up.


</details>

<!-- fingerprinting:phantom:medusa:grasshopper:322129d1-9e03-4e22-80a1-b6f30e657d48 -->

<!-- This is an auto-generated comment by CodeRabbit -->

}()

quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
sig := <-quit
common.SysLog(fmt.Sprintf("received signal: %v, shutting down...", sig))

// SSE streams may run for minutes; give them time to finish before forced exit
shutdownTimeout := time.Duration(common.GetEnvOrDefault("SHUTDOWN_TIMEOUT_SECONDS", 120)) * time.Second
ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
defer cancel()
if err := srv.Shutdown(ctx); err != nil {
common.SysError(fmt.Sprintf("server forced to shutdown: %v", err))
}
common.SysLog("server exited")
Comment on lines +215 to +222
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Wait for relay/background workers before returning from main.

srv.Shutdown(ctx) only waits for the http.Server. The relay path also uses common.relayGoPool (common/gopool.go:1-23), and this file starts several long-lived goroutines above. Line 222 returns from main immediately afterward, so anything still running outside the HTTP server is still terminated abruptly on process exit. That leaves a gap in the graceful-shutdown guarantee for streams offloaded to those workers.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@main.go` around lines 215 - 222, The HTTP server shutdown only stops srv; you
must also wait for the relay/background goroutine pool (common.relayGoPool from
common/gopool.go) to drain before exiting main. After calling srv.Shutdown(ctx)
and before returning, call the relay pool's shutdown/wait method (e.g.,
common.relayGoPool.Shutdown(ctx) or common.relayGoPool.Wait()) or implement a
drain loop that checks relayGoPool.ActiveWorkers()/len(relayGoPool.workers) and
blocks until zero (respecting the same ctx timeout) so long-lived relay tasks
complete gracefully.

}

func InjectUmamiAnalytics() {
Expand Down