mirror of
https://github.com/diegosouzapw/OmniRoute.git
synced 2026-11-16 17:09:09 +00:00
docs(codex): document session affinity and stream idle for long tasks (#8709)
* docs(codex): document session affinity and stream idle for long tasks Operators running multi-hour Codex sessions need both knobs spelled out: sessionAffinityTtlMs (default off) and STREAM_IDLE_TIMEOUT_MS (10 min), with a concrete recipe and an explicit keep-defaults decision (#7287). * fix(ci): keep #7287 docs-only so base-red gates stay skipped Drop the unit content-guard that classified the PR as code (triggering i18n/unit/eslint/dast on a red release tip). Sync agent-skills so check:agent-skills-sync passes (config-codex-cli blank line + stale cli-backup-sync catalog drift).
This commit is contained in:
committed by
GitHub
parent
7db3ba615b
commit
85128984f9
1
changelog.d/maintenance/7287-codex-long-running-tasks.md
Normal file
1
changelog.d/maintenance/7287-codex-long-running-tasks.md
Normal file
@@ -0,0 +1 @@
|
||||
- **docs(codex):** document session affinity (`sessionAffinityTtlMs`) and stream idle timeout (`STREAM_IDLE_TIMEOUT_MS`) for multi-hour Codex CLI tasks, with a concrete recommended config, diagnosability string, and an explicit "defaults stay opt-in" decision — mirrored in `skills/config-codex-cli` (#7287)
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "Codex CLI — Configuration with OmniRoute"
|
||||
version: 3.8.40
|
||||
lastUpdated: 2026-06-28
|
||||
version: 3.8.49
|
||||
lastUpdated: 2026-07-26
|
||||
---
|
||||
|
||||
# Codex CLI — Configuration with OmniRoute
|
||||
@@ -389,6 +389,110 @@ Inside an interactive session:
|
||||
|
||||
---
|
||||
|
||||
## Long-running tasks
|
||||
|
||||
Two OmniRoute defaults can silently sabotage multi-hour Codex CLI sessions. Neither is a Codex CLI setting — both live on the OmniRoute side. Users migrating a config from upstream proxies that pin accounts and disable idle cutoffs often hit both and conclude OmniRoute “cannot sustain a long session.”
|
||||
|
||||
| Symptom | Likely cause | Knob |
|
||||
| -------------------------------------------------------------------------------- | ------------------------------------------------------------------ | ------------------------ |
|
||||
| Session keeps switching accounts / prompt-cache continuity is lost between turns | Session affinity TTL is `0` (disabled) | `sessionAffinityTtlMs` |
|
||||
| Connection dies mid-reasoning with no client-facing prompt | Stream idle watchdog fired after 10 minutes with no upstream chunk | `STREAM_IDLE_TIMEOUT_MS` |
|
||||
|
||||
Related discussions: [#7126](https://github.com/diegosouzapw/OmniRoute/discussions/7126) (long task drops), [#5718](https://github.com/diegosouzapw/OmniRoute/discussions/5718) (why affinity defaults off). Tracking: [#7287](https://github.com/diegosouzapw/OmniRoute/issues/7287).
|
||||
|
||||
### 1. Session affinity — pin one conversation to one account
|
||||
|
||||
**Default:** `sessionAffinityTtlMs = 0` (disabled).
|
||||
|
||||
**Where to set it**
|
||||
|
||||
- Dashboard → **Settings → Routing** → **Session affinity** → **Affinity TTL (seconds)** (`ComboDefaultsTab`)
|
||||
- Or PATCH settings with `sessionAffinityTtlMs` in **milliseconds** (Zod range `0`–`86_400_000`, i.e. up to 24 hours)
|
||||
|
||||
> Renamed in #7274 from the Codex-only `codexSessionAffinityTtlMs`. The legacy key is still accepted as a read-only alias; new configs should use `sessionAffinityTtlMs`. Affinity now applies to **any** provider once the TTL is above `0`, not only Codex — see [`docs/architecture/RESILIENCE_GUIDE.md`](../architecture/RESILIENCE_GUIDE.md) → Session affinity.
|
||||
|
||||
**What breaks when it stays at 0**
|
||||
|
||||
Every turn of a multi-turn Codex conversation is routed independently by the active combo strategy and can land on a **different account per turn**. That breaks upstream session / prompt-cache continuity. OmniRoute only consults Codex session headers (`x-codex-session-id` / `x-session-id` / `x-omniroute-session`) and body fields such as `prompt_cache_key` / `session_id` when the TTL is greater than `0` (`extractSessionAffinityKey` in `src/sse/services/auth.ts`).
|
||||
|
||||
**Recommended for a multi-hour single task**
|
||||
|
||||
Set the TTL **above the expected wall-clock length of the task** (UI max is **86400 seconds** = 24 hours):
|
||||
|
||||
| Expected task length | Affinity TTL (UI, seconds) | `sessionAffinityTtlMs` |
|
||||
| -------------------- | -------------------------- | ---------------------- |
|
||||
| A few hours | `14400` (4h) | `14400000` |
|
||||
| Overnight / ~12h | `43200` (12h) | `43200000` |
|
||||
| Full day | `86400` (24h, maximum) | `86400000` |
|
||||
|
||||
Opt-in is deliberate: disabling affinity favors load-balancing across accounts; enabling it favors continuity for one long agent session. This guide does **not** change the default — operators running long Codex tasks must opt in.
|
||||
|
||||
### 2. Stream idle timeout — do not kill quiet reasoning turns
|
||||
|
||||
**Default:** `STREAM_IDLE_TIMEOUT_MS = 600000` (10 minutes). It inherits from `REQUEST_TIMEOUT_MS` when unset; the shared baseline is also 600000. See [`docs/guides/SETUP_GUIDE.md`](SETUP_GUIDE.md) → Timeouts.
|
||||
|
||||
**What breaks at the default**
|
||||
|
||||
A Codex reasoning / tool turn that stays silent for more than 10 minutes with **no real upstream chunk** is force-closed by the SSE idle watchdog (`open-sse/utils/stream.ts`). The client often sees a bare connection drop — matching “stopped automatically without any notification.”
|
||||
|
||||
Critical detail: OmniRoute’s synthetic SSE **heartbeat does not reset** the idle clock. Only a real upstream body chunk updates `lastChunkTime`. A quiet model that is still “thinking” looks identical to a stalled upstream from the watchdog’s point of view.
|
||||
|
||||
Related Undici body inactivity: `FETCH_BODY_TIMEOUT_MS` (also defaults to the same 10-minute baseline; `0` disables it). For streaming, `FETCH_TIMEOUT_MS` only covers connection setup / first headers — once the stream is active, stalls are governed by `STREAM_IDLE_TIMEOUT_MS` and `FETCH_BODY_TIMEOUT_MS`.
|
||||
|
||||
**Recommended for a multi-hour single task**
|
||||
|
||||
In the OmniRoute process environment (`.env` / compose / systemd):
|
||||
|
||||
```bash
|
||||
# Disable stream idle + body inactivity cutoffs for long reasoning turns
|
||||
STREAM_IDLE_TIMEOUT_MS=0
|
||||
FETCH_BODY_TIMEOUT_MS=0
|
||||
```
|
||||
|
||||
Or raise them above the longest quiet gap you expect (values are milliseconds):
|
||||
|
||||
```bash
|
||||
# Example: allow up to 2 hours of silence between upstream chunks
|
||||
STREAM_IDLE_TIMEOUT_MS=7200000
|
||||
FETCH_BODY_TIMEOUT_MS=7200000
|
||||
```
|
||||
|
||||
Restart OmniRoute after changing these env vars.
|
||||
|
||||
### Concrete recipe — multi-hour Codex task
|
||||
|
||||
1. **Pin the account:** Dashboard → Settings → Routing → Session affinity → Affinity TTL = `43200` (12h) or `86400` (24h max).
|
||||
2. **Raise / disable idle cutoffs** in OmniRoute’s environment:
|
||||
|
||||
```bash
|
||||
STREAM_IDLE_TIMEOUT_MS=0
|
||||
FETCH_BODY_TIMEOUT_MS=0
|
||||
```
|
||||
|
||||
3. Keep the usual Codex `config.toml` (`wire_api = "responses"`, correct `base_url`, `OMNIROUTE_API_KEY`) — no Codex-side affinity/idle knobs exist for these two behaviors.
|
||||
4. Restart OmniRoute, then start the long Codex task.
|
||||
|
||||
### Defaults decision (#7287)
|
||||
|
||||
| Knob | Ship default | Change in this guide? |
|
||||
| ------------------------ | ----------------- | ----------------------------------------------------------------------------------------- |
|
||||
| `sessionAffinityTtlMs` | `0` (off) | **No** — remains opt-in (load-balancing vs continuity; see Discussion #5718) |
|
||||
| `STREAM_IDLE_TIMEOUT_MS` | `600000` (10 min) | **No** — remains 10 minutes for general traffic; long Codex operators raise or disable it |
|
||||
|
||||
Flipping either default globally would change behavior for every client of an instance, not only Codex. Document the knobs; leave the defaults alone until an explicit operator decision says otherwise.
|
||||
|
||||
### Diagnosing idle cuts
|
||||
|
||||
When the idle watchdog fires, OmniRoute logs a line shaped like:
|
||||
|
||||
```text
|
||||
[STREAM] Idle timeout: no data from codex for 600000ms (model: cx/gpt-5.5)
|
||||
```
|
||||
|
||||
Grep for `Idle timeout: no data from` (or the code `stream_idle_timeout` / error name `StreamIdleTimeoutError`). The provider segment is whatever OmniRoute used for that request (`codex`, another provider id, or `provider` if unknown) — it is not always the literal string `codex`.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**`Error: wire_api = "chat" is no longer supported`**
|
||||
@@ -411,3 +515,6 @@ Lower `model_auto_compact_token_limit` to 80–85% of the window. Never set abov
|
||||
|
||||
**Profile not loading (`-p <name>` silently ignored)**
|
||||
Confirm the file exists at `~/.codex/<name>.config.toml` (no `profile-` prefix). Run `ls ~/.codex/*.config.toml`.
|
||||
|
||||
**Long Codex task drops mid-run / switches accounts between turns**
|
||||
See [Long-running tasks](#long-running-tasks). Enable session affinity (TTL above task length) and raise or disable `STREAM_IDLE_TIMEOUT_MS` / `FETCH_BODY_TIMEOUT_MS`. Grep OmniRoute logs for `Idle timeout: no data from`.
|
||||
|
||||
@@ -18,3 +18,39 @@ omniroute --version
|
||||
## Subcommands
|
||||
|
||||
_No CLI subcommands mapped for this family yet._
|
||||
|
||||
<!-- skill:custom-start -->
|
||||
|
||||
## Long-running Codex tasks (#7287)
|
||||
|
||||
Two OmniRoute defaults silently break multi-hour Codex sessions. Document them whenever configuring Codex for overnight / multi-hour work. Full guide: `docs/guides/CODEX-CLI-CONFIGURATION.md` → **Long-running tasks**.
|
||||
|
||||
### Session affinity (default off)
|
||||
|
||||
- Setting: `sessionAffinityTtlMs` (ms; UI shows **Affinity TTL (seconds)** under Dashboard → Settings → Routing → Session affinity). Legacy alias: `codexSessionAffinityTtlMs`.
|
||||
- Default `0` = disabled. Each turn can land on a different account and break prompt-cache / session continuity.
|
||||
- Codex session keys (`x-codex-session-id`, `x-session-id`, `x-omniroute-session`, body `prompt_cache_key` / `session_id`) are only used for pinning when TTL > 0.
|
||||
- Max: `86400` seconds / `86400000` ms (24h). Set TTL **above** the expected task length (e.g. `43200` s for ~12h).
|
||||
|
||||
### Stream idle timeout (default 10 minutes)
|
||||
|
||||
- Env: `STREAM_IDLE_TIMEOUT_MS` (default `600000`). Also consider `FETCH_BODY_TIMEOUT_MS` (same baseline; `0` disables).
|
||||
- Synthetic SSE heartbeats do **not** reset the idle clock — only real upstream chunks do.
|
||||
- A quiet reasoning turn past the idle window is force-closed (`stream_idle_timeout` / `StreamIdleTimeoutError`). Grep logs for `Idle timeout: no data from`.
|
||||
|
||||
### Recommended multi-hour recipe
|
||||
|
||||
1. Dashboard → Settings → Routing → Session affinity → Affinity TTL = `43200` (12h) or `86400` (24h max).
|
||||
2. OmniRoute environment:
|
||||
|
||||
```bash
|
||||
STREAM_IDLE_TIMEOUT_MS=0
|
||||
FETCH_BODY_TIMEOUT_MS=0
|
||||
```
|
||||
|
||||
3. Restart OmniRoute. Leave Codex `config.toml` as usual (`wire_api = "responses"`, correct `base_url`).
|
||||
|
||||
### Defaults decision
|
||||
|
||||
Do **not** flip ship defaults in code for this skill: `sessionAffinityTtlMs` stays `0` and `STREAM_IDLE_TIMEOUT_MS` stays `600000`. Long-running operators must opt in. See Discussion #5718 and issue #7287.
|
||||
<!-- skill:custom-end -->
|
||||
|
||||
Reference in New Issue
Block a user