Skip to content

Add docs for replay fallbacks and timeouts #2368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bglw merged 1 commit into from
Mar 20, 2026
Merged

Add docs for replay fallbacks and timeouts #2368

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions networking/dynamic-request-routing.html.markerb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@
|`app` | The name of another app to route to |
|`state` | Optional string included in `fly-replay-src` header on replay |
|`elsewhere` | If `true`, excludes responding Machine from next load-balance |
|`timeout` | Duration to attempt the replay before giving up (e.g. `10s`, `800ms`) |
|`fallback` | If the replay fails, route back to the original Machine. `force_self` or `prefer_self` (see [Replay Timeout and Fallback](#replay-timeout-and-fallback)) |

### Example Usage

Expand Down Expand Up @@ -70,6 +72,11 @@
fly-replay: region="sjc,any";app=target-app
```

Route to another app with a timeout and fallback to the original Machine:
```
fly-replay: app=my-worker;timeout=10s;fallback=force_self
```

<div class="note icon">
**Note**: A comma-separated list of regions must be quoted.
</div>
Expand All @@ -87,10 +94,27 @@
| `us`, `usa` | United States |
| `any` | Earth |

### Replay Timeout and Fallback

You can set a `timeout` and `fallback` on a replay to handle cases where the replay target is unreachable.

**`timeout`** sets how long the proxy tries to reach the replay target. The actual duration may slightly exceed this value. Accepts duration strings like `10s`, `500ms`. Without `fallback`, a timeout makes the replay error faster instead of waiting for the default error timeout.

**`fallback`** tells the proxy to route the request back to the Machine that issued the replay if the replay fails due to timeout, exhausted retries, or no available candidate:

- `force_self`: Route back to the exact Machine that issued the replay. Returns a proxy error if that Machine is no longer available.
- `prefer_self`: Try the original Machine first, but fall back to any Machine in the original app if it is unavailable.

When a fallback triggers, the original Machine receives the request again with a [`fly-replay-failed`](#the-fly-replay-failed-header) request header containing details about the failed replay attempt. Since this is still the original request, your app can respond with a useful error instead of the client receiving a generic proxy error.

<div class="note icon">
**Note**: Fallback requests cannot themselves issue `fly-replay` responses.
</div>

## Replay JSON Format

Your app can set the response content-type to `application/vnd.fly.replay+json` and include replay instructions in the response body.
For a complex replay the JSON body is easier to compose than the header format, and also supports more functionality.

Check warning on line 117 in networking/dynamic-request-routing.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
[Fly.WordList] Consider using 'capability' or 'feature' instead of 'functionality'.

Raw Output:
{"message": "[Fly.WordList] Consider using 'capability' or 'feature' instead of 'functionality'.", "location": {"path": "networking/dynamic-request-routing.html.markerb", "range": {"start": {"line": 117, "column": 104}}}, "severity": "INFO"}

### JSON Structure

Expand All @@ -104,6 +128,8 @@
|`app` | The name of another app to route to |
|`state` | Optional string included in `fly-replay-src` header on replay |
|`elsewhere` | If `true`, excludes responding Machine from next load-balance |
|`timeout` | Duration to attempt the replay before giving up (e.g. `"10s"`, `"800ms"`) |
|`fallback` | If the replay fails, route back to the original Machine. `"force_self"` or `"prefer_self"` (see [Replay Timeout and Fallback](#replay-timeout-and-fallback)) |
|`transform.path` | Rewrite the path and query parameters of the request |
|`transform.delete_headers` | Delete headers from the request, hiding them from the replay target |
|`transform.set_headers` | Set new headers on the request, overwriting headers of the same name |
Expand All @@ -130,6 +156,16 @@
}
```

Route to another app with a timeout and fallback:

```json
{
"app": "my-worker",
"timeout": "10s",
"fallback": "force_self"
}
```

## Replay Caching

Replay caching allows Fly Proxy to remember and reuse replay decisions, reducing both load on your application and the latency of replayed requests. There are two types of replay caching:
Expand All @@ -138,7 +174,7 @@
- **Session-based**: Cache replays for specific cookie or header values

<div class="note icon">
**Note**: Replay caching is an optimization, not a guarantee. Your app should _not_ depend on this mechanism to function.

Check warning on line 177 in networking/dynamic-request-routing.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
[Fly.Spelling] Is '_not_' a typo?

Raw Output:
{"message": "[Fly.Spelling] Is '_not_' a typo?", "location": {"path": "networking/dynamic-request-routing.html.markerb", "range": {"start": {"line": 177, "column": 80}}}, "severity": "INFO"}
The app issuing `fly-replay` still serves as the ultimate source of truth, and we may decide to consult that app at any moment even if a replay cache has previously been set.
To ensure reliable operation, the app issuing `fly-replay` should still have multiple instances deployed in multiple regions.
</div>
Expand Down Expand Up @@ -196,8 +232,8 @@
```

In this example:
- Replays for requests to `/` (and its subpaths) are cached for 5 minutes based on the `session_id` cookie value

Check warning on line 235 in networking/dynamic-request-routing.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
[Fly.Spelling] Is 'subpaths' a typo?

Raw Output:
{"message": "[Fly.Spelling] Is 'subpaths' a typo?", "location": {"path": "networking/dynamic-request-routing.html.markerb", "range": {"start": {"line": 235, "column": 40}}}, "severity": "INFO"}
- Replays for requests to `/api` (and its subpaths) are cached for 10 minutes based on the `Authorization` header value

Check warning on line 236 in networking/dynamic-request-routing.html.markerb

View workflow job for this annotation

GitHub Actions / Vale linter 

[vale] reported by reviewdog 🐶
[Fly.Spelling] Is 'subpaths' a typo?

Raw Output:
{"message": "[Fly.Spelling] Is 'subpaths' a typo?", "location": {"path": "networking/dynamic-request-routing.html.markerb", "range": {"start": {"line": 236, "column": 43}}}, "severity": "INFO"}
- When multiple rules match, the longest matching path takes precedence

If your app accepts requests for multiple hostnames, you can narrow the configuration by specifying a hostname in `path_prefix`:
Expand Down Expand Up @@ -286,6 +322,7 @@
- Transformations for the headers or path cannot be defined.
- The TTL needs to be a minimum of 10 seconds
- Only one step of lookup is performed in the cache; as such, if the target app issues another `fly-replay-cache`, the caching behavior in this case is undefined
- The `timeout` and `fallback` fields cannot be set in the `fly-replay` intended to be cached
- The `fly-replay-src` header (described below) will _not_ be set for requests replayed through the cache

### The fly-replay-src Header
Expand All @@ -309,6 +346,26 @@

In these cases, the request will be delivered to a different Machine that matches the remaining fields in your replay. Along with the other Fly.io-specific headers, a `fly-preferred-instance-unavailable` header will be set containing the ID of the instance that could not be reached.

### The fly-replay-failed Header

When a replay [fallback](#replay-timeout-and-fallback) triggers, Fly Proxy delivers the request back to the original Machine with a `fly-replay-failed` request header. This header contains semicolon-separated metadata about the failed replay attempt:

|Field |Description |
|---|---|
|`instance` | ID of Machine the replay was targeting |
|`app` | App the replay was targeting |
|`region` | Region the replay was targeting |
|`replay_source` | ID of the Machine that originally issued the replay |
|`reason` | Why the replay failed: `timeout`, `retries_exhausted`, or `no_candidate` |
|`elapsed_ms` | Time in milliseconds spent attempting the replay |

Example header value:
```
fly-replay-failed: instance=00bb33ff;app=target-app;region=iad;replay_source=11aa44ee;reason=timeout;elapsed_ms=10000
```

Your app can use this header to detect that a fallback occurred and respond accordingly, for example by serving a helpful error to the client.

### Web Socket Considerations

It is worth noting that an application returning `fly-replay` headers should not negotiate a web socket upgrade itself. Some frameworks automatically handle this process. Instead, the application or instance receiving the requests should handle the upgrade.
Expand Down
Loading