Skip to content

fix(sdk): recover mid-stream provider 5xx/429 like severed connections - #1162

Closed
nordicnode wants to merge 9045 commits into
CodebuffAI:mainfrom
nordicnode:oss/recover-provider-5xx-midstream-1155
Closed

fix(sdk): recover mid-stream provider 5xx/429 like severed connections#1162
nordicnode wants to merge 9045 commits into
CodebuffAI:mainfrom
nordicnode:oss/recover-provider-5xx-midstream-1155

Conversation

@nordicnode

Copy link
Copy Markdown

Summary

  • a provider-reported 500/429 arriving mid-stream — the openai-compatible shim enqueues it as an error part with finishReason='error' — no longer ends the entire run with an error
  • extend classifyThrownStreamRecovery (already consulted for both the thrown-iterator and error-chunk shapes) so retryable APICallErrors (429, any 5xx) take the same capped recovery path as a severed body: note injected into the conversation, retry step forced, capped at MAX_CONSECUTIVE_STREAM_RECOVERIES
  • the recovery message names the HTTP status instead of claiming the connection dropped
  • client-error statuses (400/401/402/403) are deterministic — retrying cannot help — so they stay fatal and still propagate to the run's error handling, including the existing 402 rethrow
  • add regression tests: 500 and 429 recover with a status-naming message, a wrapped 503 behind a RetryError cause chain recovers, client-error statuses and post-cancel 5xx stay fatal

This addresses the run-killing part of #1155: the recoverable class was "the connection failed to speak" (isTransientNetworkError, a socket-level allowlist) and the fatal class was "the provider reported a failure", which is backwards for flaky endpoints, where both are the same transient event.

Validation

  • bun test sdk/src/__tests__/stream-interruption.test.ts (19 pass, 5 new)
  • bun test sdk (493 pass; the 100 failures are network/env-dependent and pre-exist on main — same count with this change stashed)
  • bunx tsc --noEmit -p sdk/tsconfig.json clean

Refs #1155

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants