Repository navigation
fix(agent-sre): measure benchmark duration with a monotonic clock - #4206
Conversation
Signed-off-by: Naveen Chatlapalli <[email protected]>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
|
🟡 Contributor Check: MEDIUM
Automated check by AGT Contributor Check. |
|
Verified in code and locally: a wall-clock step on main produced a negative latency and a PASSED result; Before I approve, two process items that apply to all nine PRs in this batch: please use the repository template with its Checklist, Attribution, AI Assistance and IP sections, and tick the attestations in your own words. The AI note says Codex prepared and submitted the changes at your request; CONTRIBUTING asks that no PR be submitted by an agent without a human reviewing the specific changes, so one sentence confirming you reviewed this diff before it was opened settles it. Once that and the gated checks are green I will approve and merge. |
|
I reviewed the specific changes in this diff before the PR was opened. I have updated the description to use the repository template sections, completed the attestations with the validation scope and limitations, and clarified the AI-assistance note. |
MohammadHaroonAbuomar
left a comment
There was a problem hiding this comment.
Approving at 8323314. Benchmark duration now uses time.monotonic; wall-clock jump reproduces the negative duration on main and not here; tests pass; ruff clean. Description uses the repository template with the attestations completed, and the author confirmed reviewing the diff before opening the PR. All 12 pull_request workflow runs on this head completed successfully. Commits are signed off.
MohammadHaroonAbuomar
left a comment
There was a problem hiding this comment.
Approving at 8323314. Benchmark duration now uses time.monotonic; wall-clock jump reproduces the negative duration on main and not here; tests pass; ruff clean. Description uses the repository template with the attestations completed, and the author confirmed reviewing the diff before opening the PR. All 12 pull_request workflow runs on this head completed successfully. Commits are signed off.
Description
BenchmarkRunner measures elapsed duration using the wall clock. The existing
test_latency_trackedfails on unchanged main under Windows because fast calls report zero duration; wall-clock adjustments can also distort timeout classification.Use
time.perf_counter()for duration measurements in both success and exception paths. Report timestamps continue using the wall clock.Validation
Two deterministic cases cover a frozen wall clock, a successful call exceeding its timeout, and an exception. Both duration cases fail before the fix. Benchmark tests: 25 passed. Full package suite: 1,497 passed, 3 skipped.
All regressions were reproduced on origin/main
406ddd4dbefore the production change. Whitespace checks pass. Ruff passes on the changed files. Changed-line spelling passes with the existing API identifierDeduplicatorallowed in the temporary check input.Scope: the existing Agent-SRE source tree, which remains exercised by repository CI. Combined validation of these six independent SRE fixes: 1,505 passed, 3 skipped on Windows/Python 3.12.
Type of Change
Package(s) Affected
Checklist
Attribution & Prior Art
Prior art / related projects: Uses the existing Agent-SRE implementation and standard library operations. No code was copied or adapted from an external project.
AI Assistance
I reviewed the specific changes in this diff before this PR was opened.
Codex assisted with implementation, regression tests, and local validation under my direction.
IP, Patents, and Licensing
Related Issues
Newly reproduced defect; the reproduction and regression coverage are described above.