Skip to content

agent hosts: late result after terminal settlement is acknowledged as already applied and drops incurred usage #13238

Description

@Nakagawa-master

Problem

After #12582 merged, applyHostRunResult() still treats any terminal run carrying the same attempt + hostId + leaseId as if the incoming Host result had already been applied:

if (
  (run.status === 'completed' ||
    run.status === 'failed' ||
    run.status === 'cancelled') &&
  run.attempts === input.attempt &&
  run.lease?.attempt === input.attempt &&
  run.lease.hostId === input.hostId &&
  run.lease.leaseId === input.leaseId
) {
  return {
    ok: true,
    value: { thread: current, alreadyApplied: true },
  };
}

That conflates two different states:

same lease / attempt identity
!=
the same Host result was already accepted

logical terminal settlement
!=
no physical work happened after settlement

Reproduced behavior

The maintainer acceptance run for #12582 already captured the case as N2:

  1. a managed Host run is executing;
  2. the Host is suspended past the recovery grace;
  3. the coordinator terminalizes the run as failed at about lease expiry + 121s;
  4. the Host resumes and physically completes the turn;
  5. the gateway records 1,050 tokens incurred by that attempt;
  6. the Host posts completed with the original host/lease/attempt;
  7. the answer is correctly not applied, but the coordinator replies as alreadyApplied;
  8. the run's usageByRound remains empty, so the incurred usage is not charged.

Acceptance report: #12582 (comment)

A follow-up contract sketch was also posted before merge:
#12582 (comment)

The merged main implementation still contains the same early terminal branch.

Why this matters

State safety is currently good: the stale answer does not revive or overwrite the terminal run.

The remaining problem is idempotency truth + accounting truth:

  • alreadyApplied is false for a result that was never accepted;
  • known physical usage can disappear from the budget ledger;
  • the Host gets no signal that its result was stale rather than previously committed.

This can matter for per-thread token limits, operator cost visibility, and debugging late Host behavior.

Suggested contract

Separate accepted-result idempotency from late physical-effect accounting.

A durable version could be:

  1. Persist a small accepted-result receipt when a Host result is actually committed, bound to the exact attempt/lease and, if needed, a result identity/digest.
  2. Return alreadyApplied: true only when the incoming result matches that receipt.
  3. If the run is already terminal for another reason (recovery timeout, cancellation, Host removal) and there is no matching receipt:
    • do not apply the stale answer;
    • do not revive the run;
    • do not emit another terminal/parent event;
    • return an explicit stale/terminalized outcome rather than alreadyApplied;
    • apply an explicit, tested policy for verifiable usage physically incurred by the old attempt.
  4. Keep old-attempt accounting separate from any reclaimed attempt N+1.

Regressions worth pinning

A. accepted completed result -> exact retry
   state unchanged
   usage unchanged
   alreadyApplied = true

B. recovery sweep terminalizes run -> late completed result from old lease
   stale answer not applied
   no second terminal event
   response != alreadyApplied
   incurred usage follows the explicit accounting policy

C. operator cancellation -> late completed result
   cancellation remains authoritative
   no reply / revival / rebooking
   usage treatment is explicit

D. run reclaimed as attempt N+1 -> attempt N late result
   N+1 state and usage remain independent
   old-attempt usage is not attributed to N+1

I would not use usageByRound presence itself as the idempotency receipt: zero/partial usage is possible, and an explicit receipt makes restart/retry semantics auditable.

Scope / source relation

This issue is scoped to the N2 behavior already reproduced on #12582 and now present on main.

A related structural distinction is that historical lineage does not by itself preserve present write authority after a transition: an old attempt can still be historically related to the run without retaining authority to mutate current state. That framing is documented here as a reusable source, but it is not required to adopt any particular fix:
https://github.com/Nakagawa-master/nakagawa-theory-archive/blob/main/derivatives/307/README.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    category/coreCore engine and logicpriority/P2Medium - Moderately impactful, noticeable problemroadmap/multi-agentRoadmap: Multi-agent collaborationscope/token-managementToken handling and limitstype/bugSomething isn't working as expected

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions