Skip to content

Think schedules _chatRecoveryContinue for pre-first-chunk stalls, causing recovery to skip silently #1941

Description

@arthurfiorette

(another ai generated issue but it properly explains the problem :) )


When a Think chat stream stalls before producing its first assistant chunk, the recovery system schedules _chatRecoveryContinue.

At this point there is no assistant message to continue from. The latest conversation leaf is still the user's message.

The scheduled callback eventually calls continueLastTurn(), which intentionally returns status: "skipped" when the latest leaf is not an assistant message. As a result, the original user turn is never retried and the recovery finishes silently.

This was observed with:

  • @cloudflare/think@0.12.1
  • Chat recovery enabled
  • A stream stall before the first assistant part
  • The current @cloudflare/think@0.13.0 source appears to contain the same behavior

Reproduction

  1. Create a Think chat agent with chat recovery enabled.
  2. Configure a relatively short chatStreamStallTimeoutMs.
  3. Use a model/provider that accepts the request but does not produce a first stream chunk before the timeout.
  4. Send a user message.
  5. Allow the watchdog to detect the stall.
  6. Observe the scheduled _chatRecoveryContinue callback.

Actual behavior

The live stream stall follows this path:

ChatStreamStalledError
  -> _routeStallToBoundedRecovery()
  -> schedule _chatRecoveryContinue
  -> close the live stream with an empty done response
  -> _chatRecoveryContinue()
  -> continueLastTurn()
  -> latest leaf is the user message
  -> status: "skipped"

_routeStallToBoundedRecovery() always creates the incident with:

recoveryKind: "continue"

and always schedules:

callback: "_chatRecoveryContinue"

even when targetAssistantId is absent because no assistant content was produced.

continueLastTurn() then returns early:

if (!lastLeaf || lastLeaf.role !== "assistant") {
  return { requestId: "", status: "skipped" }
}

The incident is updated to skipped, but the original user turn is not retried.

Lifecycle hook gap

This path also does not invoke any terminal application lifecycle hook:

  • onChatRecovery() is invoked by fiber recovery, but not when this live-stall callback is scheduled.
  • onChatResponse() is not called because no continuation turn starts.
  • onChatError() is not called because the stall was routed into recovery.
  • onExhausted() is not called because skipped is not treated as exhaustion.

This prevents applications from reliably cleaning up turn-scoped resources such as telemetry spans, metrics, timers, or custom state.

The client receives a clean done: true response with an empty body, so the visible result may simply be a silent turn with no assistant response or terminal recovery message.

Expected behavior

A stall before the first assistant chunk should retry the last user turn rather than trying to continue a nonexistent assistant message.

One possible strategy:

const hasPartialAssistant = Boolean(input.targetAssistantId)

const recoveryKind = hasPartialAssistant ? "continue" : "retry"
const callback = hasPartialAssistant
  ? "_chatRecoveryContinue"
  : "_chatRecoveryRetry"

The retry callback could use the latest user message ID already resolved by _routeStallToBoundedRecovery().

Additionally, every scheduled recovery should eventually produce an application-visible terminal lifecycle event, including skipped recoveries.

Possible API options:

  • Invoke onChatRecovery() when live-stall recovery is scheduled.
  • Add onChatRecoverySkipped().
  • Invoke onChatResponse() with status: "skipped".
  • Route an unrecoverable skip through onExhausted() and the configured terminal message.

Impact

  • The user's original message is never retried.
  • The chat can end silently without an assistant response.
  • The configured terminal recovery message is not delivered.
  • Turn-scoped application resources can remain open until another turn starts.
  • Telemetry may show an unfinished turn despite the recovery callback having completed.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions