Skip to content
Prev Previous commit
test(runtime-broker): Pin local confirmation retry (#12839)
  • Loading branch information
jinye.djy jinye.djy
jinye.djy authored and jinye.djy committed Sep 28, 2026
commit 2d95f1aed8076f711755cbc7ede1d219da48ea7f
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,36 @@ void aLostAttestationNeverYieldsAReadyLease(int lostAttestation)
.getBindingId());
}

@Test
void aTransientConfirmFailureKeepsAnOwnedWorkerReady()
throws Exception {
broker.warm(HARNESS).requireOk();
RuntimeBindingRecord original = rig.activeBinding();
proxy.schedule("attest", FaultProxy.Action.RESET);

BrokerProcess.Reply failed = broker.warm(HARNESS);
assertFalse(failed.ok());
assertEquals(503, failed.status());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-38: The gate added to pin the transient-confirm retry asserts the numeric status and retryable flag but not the error code

503 with retryable=true is what at least three semantically different codes return on this path, including runtime_broker_runtime_lost — the permanent-loss answer that, per the settled PR discussion, never clears without stop proof. Changing the confirm-failure code from runtime_provision_failed to runtime_broker_runtime_lost keeps every assertion in aTransientConfirmFailureKeepsAnOwnedWorkerReady green (not ok, 503, retryable, binding still READY with the same bindingId, third warm READY, cross-workspace PROVISIONING) while telling a client to give up on a healthy, still-owned worker. The file's own convention is to pin the code (:117 and :146 both assert managed_runtime_unavailable); this is the only reply assertion in the file that checks a bare numeric status. Since this gate is the one the PR added to close the round-4 mutation gap, its own discrimination matters.

Witness:

traced rather than run in the reporting round (no JDK 21 or Maven there) and then confirmed by reading the three decisive sites: LocalProcessRuntimeProvisioner.java:451-455 returns `new RuntimeBrokerException(503, "runtime_provision_failed", message, true, cause)`; RuntimeBrokerService:1490 with :2442-2450 makes runtime_broker_runtime_lost also 503 retryable=true; and IDENTITY_FAILURES (:50-52) does not contain it, so the mutation leaves `invalidate = !canRetryFailedConfirm(live.lease())` unchanged and the binding READY. The same verifier ran this gate on the intact tree (`Tests run: 1, Failures: 0`) while ruling a sibling finding, so the harness can execute it.

Suggested fix: Add one line after assertTrue(failed.retryable()), following the file's convention: assertEquals("runtime_provision_failed", failed.code());

One premise this fix must not break: The actual code is fixed by LocalProcessRuntimeProvisioner.java:453 — confirm -> attestOwned -> attest's catch branches (:370, :372) route through failed(...), so the asserted literal must be runtime_provision_failed, not managed_runtime_unavailable.

Fix acceptance — That assertion is itself the pin. Falsification: changing the code in LocalProcessRuntimeProvisioner.failed(...) to runtime_broker_runtime_lost reddens the gate with the assertion and leaves it green without. Please prove it by mutation: apply the fix, then revert it and confirm that test goes red.

中文说明

[Suggestion] R1-38:在同一条 ensureBinding READY 分支上,503 + retryable=true 至少对应三种语义完全不同的 code,其中包括 runtime_broker_runtime_lost——按本 PR 已 settled 的讨论,那是没有停写证明就永不清除的丢失语义。把 confirm 失败对外抛出的 code 从 runtime_provision_failed 改成 runtime_broker_runtime_lost,aTransientConfirmFailureKeepsAnOwnedWorkerReady 的每一条断言都仍然成立(not ok、503、retryable、绑定仍 READY 且 bindingId 不变、第三次 warm READY、跨 workspace PROVISIONING),而客户端会被告知放弃一个进程仍存活、绑定仍 READY 的健康 worker。同文件的惯例恰恰是钉 code 的(:117 与 :146 都断言 managed_runtime_unavailable),这是全文件唯一一处只断言裸数字 status 的应答断言。由于这道门禁正是本 PR 为补上第四轮变异缺口而新增的,它自己的判别力尤其重要。

修复建议:在 assertTrue(failed.retryable()); 之后按本文件惯例补一行:assertEquals("runtime_provision_failed", failed.code());

修复不得违反的既有事实:实际 code 由 LocalProcessRuntimeProvisioner.java:453 决定——confirm -> attestOwned -> attest 的 catch 分支(:370、:372)走的正是 failed(...),所以断言的字面量必须是 runtime_provision_failed,不是 managed_runtime_unavailable。

修复验收:上方 “Fix acceptance” 一句点名的测试即验收标准(测试名与标识符在两种语言中逐字保留);请用变异验证——先应用修复,再回退它,确认该测试变红。

证据见上方 Witness 代码块:那是程序输出,按规则逐字保留、不翻译。

— kimi-k3 via Qwen Code /review (v0.24.7)

assertTrue(failed.retryable());
assertEquals(RuntimeBindingRecord.State.READY,
rig.activeBinding().getState());
assertEquals(original.getBindingId(), rig.activeBinding()
.getBindingId());

assertEquals("READY", broker.warm(HARNESS).object()
.getString("state"));
assertEquals(original.getBindingId(), rig.activeBinding()
.getBindingId());
RuntimeScope other = new RuntimeScope(rig.scope.getTenantId(),
"workspace-b", rig.scope.getWorkspaceGeneration(),
rig.scope.getCanonicalCwd(), rig.scope.getCapabilityDigest(),
rig.scope.getIsolationClass());
RuntimeProvisionRequest candidate = new RuntimeProvisionRequest(
other, null, LocalProcessRuntimeProvisioner.KIND);
assertEquals(RuntimeBindingRecord.State.PROVISIONING,
rig.bindings.findOrCreate(candidate).getState());
}

private void acquire() {
assertEquals("READY", broker.warm(HARNESS).object()
.getString("state"), rig.logs());
Expand Down
Loading