feat(reverse_proxy): lb_retry_match now contains status field

[seroperson]

Closes #6267

lb_retry_match now contains status field for retries based on response.

I had to do some weird parsing to be able to put it into lb_retry_match block, but this definition in lb_retry_match looks so good that I could not resist. I can do any changes if necessary.

Some examples of how this PR now works:

reverse_proxy upstream1:8080 upstream2:8080 {
  lb_retries 3
}

• That's default behavior, nothing is changed here
• GET requests retry on any error
• Non-GET requests are never retried
• Dial errors always retry

---

reverse_proxy upstream1:8080 upstream2:8080 {
  lb_retries 3
  lb_retry_match {
    status 502 503
  }
}

• 502/503 responses trigger retry for any request method
• Default error retry behavior unchanged

---

reverse_proxy upstream1:8080 upstream2:8080 {
  lb_retries 3
  lb_retry_match {
    status 5xx
  }
}

• Any 5xx response triggers retry
• Default error behavior unchanged

---

reverse_proxy upstream1:8080 upstream2:8080 {
  lb_retries 3
  lb_retry_match {
    method POST
    status 503
  }
}

• 503 POST -> retried
• 503 PUT -> not retried
• 503 GET -> not retried
• POST error, GET error -> retried

---

reverse_proxy upstream1:8080 upstream2:8080 {
  lb_retries 3
  lb_retry_match {
    method PUT
  }
}

• PUT requests retry on any error (in addition to GET)
• No new behavior - pre-existing lb_retry_match functionality

---

reverse_proxy upstream1:8080 upstream2:8080 {
  lb_retries 3
  lb_retry_match {
    method POST
    status 503
  }
  lb_retry_match {
    status 502
  }
}

• 503 POST -> retried
• 503 GET -> not retried
• 502 for any method -> retried
• POST error -> retried
• PUT error -> not retried
• GET error -> retried

Initial PR version

Added a new field, lb_retry_on which is responsible for retrying based on response.

Available conditions:

• Status codes: 502, 503, 5xx-like
• empty_response - socket closed before receiving a response
• tls_handshake_error - TLS negotiation failed
• response_timeout - upstream was too slow to respond
• other - any other unclassified error

In Caddyfile it looks like:

reverse_proxy backend1:8080 backend2:8080 {
    lb_retries 3
    lb_retry_on 503 502 response_timeout
}

If lb_retry_on is unset, the old default behavior is preserved if retry-related options are set:

1. Dial errors (connection refused, DNS failure, etc) - always retried
2. All other errors (EOF, timeout, TLS errors, etc):
   2.1 GET requests - always retried
   2.2 Non-GET requests - retried only if lb_retry_match is configured and the request matches it
3. Status codes - never trigger retries

Now lb_retry_on allows you to mostly override the default behavior (except of dial errors, they're always retried).

Assistance Disclosure

Co-Authored with Claude Opus 4.6, mostly for tests and consulting about the codebase.

[CLAassistant]

All committers have signed the CLA.

[seroperson]

I've revisited this PR to make it simpler. Shortly, the only thing left is retrying based on status, which I moved to the lb_retry_match block.

I've removed retrying based on conditions as it messes with default behavior too much and it makes everything too complicated and unclear. I still can implement them in this PR, but feels like you should point me on how they must be defined and handled.

Updated the PR's description, previous one is now hidden under "Initial version" spoiler.

[mohammed90]

Asking to think out loud, shouldn't this be MatcherSet?

[seroperson]

Not right exactly this place, as it's a JSON-serializable struct. It becomes MatcherSet later in runtime.