Skip to content

feat(runner): Gemini Code Assist as a second outside reviewer - #95

Closed
TurtIeSocks wants to merge 3 commits into
mainfrom
worktree-kelpie-92
Closed

TurtIeSocks wants to merge 3 commits into
mainfrom
worktree-kelpie-92

Conversation

@TurtIeSocks

Copy link
Copy Markdown
Contributor
  • [gemini] settings: enabled (off when the table is absent) and max_rounds (default 3, 1 to 4)
  • Green CI runs a Gemini round before CodeRabbit's. Kelpie posts /gemini review, reads the review of the head that answers it, and the judge rules on its inline comments by their severity badge
  • Held findings go to the worker, refuted ones are resolved. A review the judge holds nothing of satisfies the round
  • The cap counts every Gemini review on the pull request. At the cap the item goes on to CodeRabbit without a ruling, after the worker fixes the last round's held findings
  • The dog holds a Gemini window per account: 100 summons in a rolling day. A refusal reopens it a day later, and a missing review goes the way CodeRabbit's does
  • Reviews Gemini posts on its own are never read
  • CodeRabbit's round is now runner/outside.rs, shared by both. Old coderabbit-* rulings in a state file still load
  • Design log: why Gemini goes first, what satisfies a round, the cap, and Gemini's review shape under Facts

Gemini's shape comes from public repos (google-gemini/gemini-cli#29499, sednalabs/codex#626, jmmaloney4/codex-proxy#6), recorded in crates/kelpie/fixtures/. The /gemini review summons on shep-pm/shep #614, #617, #637 and #638 had no answer yet when this was opened.

Tests, per the ticket:

  • Summon posted once and recorded as kelpie's: a_summon_posts_the_comment_once_and_is_recorded_as_kelpies, plus a failed post and a restart that reads the comment back
  • Only refuted findings satisfy the round: a_review_with_only_refuted_findings_satisfies_the_round
  • A confirmed finding goes to the worker: a_confirmed_finding_goes_to_the_worker_and_its_fix_is_summoned_again
  • Lease shared across two projects: two_projects_share_one_gemini_lease
  • Rounds off never post: with_gemini_rounds_off_no_summon_is_ever_posted

fmt, clippy, 620 tests, rustdoc and cargo +1.88 check pass. tests/gpu_lease.rs failed once under full-suite load and passed 3 of 3 alone. runner/outside/coderabbit_tests.rs was already 958 lines, and a helper keeps it at 993 after the new reviewer fields.

Resolves #92

🤖 Generated with Claude Code

TurtIeSocks and others added 3 commits September 28, 2026 23:55
CodeRabbit's free allowance is one review an hour, which paces shep to
about one pull request an hour. Gemini Code Assist gives an installation
at least 100 reviews a day, can be asked for one with a `/gemini review`
comment, and runs a different model family. This adds Gemini rounds
beside CodeRabbit's, per #92.

One round shape for both reviewers. `runner/coderabbit.rs` becomes
`runner/outside.rs`, keyed by a new `Outside` reviewer (Gemini,
CodeRabbit). Marking ready, taking the lease, summoning, awaiting the
answer, judging threads, the fix turn and its push check are shared. What
differs is small and matched on the reviewer:

- The summon. CodeRabbit keeps its `review please` label. Gemini's is a
  `/gemini review` comment. The round saves the summon as unposted
  before the post and marks it posted after, so a failed post is tried
  on the next pass and a restart reads the comment back from the forge
  rather than posting a second one.
- What is read back. `crate::gemini` reads Gemini's activity: a review
  of the head posted since kelpie's summon answers it, and only that
  review's threads (tied by `pullRequestReview.databaseId`) are judged.
  A review Gemini posts on its own is never read, and a thread held in
  an earlier round is not judged again. Gemini always posts a review
  object, clean or not, so "clean" needs no walkthrough trick. Its quota
  refusal is an issue comment asking to wait up to 24 hours, so the
  window reopens a day later.
- The cap. The maintainer changed #92 while this was built: a pull
  request gets at most `[gemini] max_rounds` Gemini reviews (default 3,
  1 to 4), counting every Gemini review on it, asked for or not. At the
  cap the work item goes on through CI to CodeRabbit without a ruling.
  The last round's held findings still go to the worker first, so the
  later gate reviews the fix. CodeRabbit keeps its divisor formula and
  its cap ruling.

Order: green CI starts Gemini's round before CodeRabbit's, so the scarce
CodeRabbit review sees code Gemini's findings already cleaned up. A
worker turn clears both reviewers' satisfaction. Gemini's round also
marks a draft ready first, since whether Gemini reviews a draft on
request is not documented.

The lease. The dog's review window now has terms per kind: CodeRabbit's
hour with a quota of one until a footer says otherwise, Gemini's rolling
day of 100 (Google's quotas page: "at least 100 pull request reviews per
day"). The book file keeps no span, so a restored window takes its
kind's. An unreadable book counts a summon in every window.

Settings: `[gemini]` with `enabled` and `max_rounds`, off when the table
is absent so existing project files load unchanged.

State and log wire changes. The CodeRabbit rulings, fix and resume
become `outside-cap`, `outside-silent`, `fix-not-pushed` with an
`outside` fix, and `outside-fix`, each carrying `reviewer`. The old
`coderabbit-*` names are serde aliases and a missing reviewer reads as
CodeRabbit, so a state file written before this still loads; tests pin
both. The phase gains `gemini`, the work item a `gemini` tally, and the
`summoned` stage a `posted` flag written only when false. Step reports
`summoned` and `summon-refused` carry the reviewer, and the CodeRabbit
round's `code-rabbit-reviewed|judged|satisfied` lines become
`outside-reviewed|judged|satisfied` with the reviewer, plus a new
`gemini-capped`.

The gh adapter's thread query, resolve and `--jq` line parsing move to
`adapters/gh/outside.rs`, shared by both. The thread query now also asks
for each first comment's review id.

Gemini's shape was taken from public repos, since no Gemini review had
reached shep when this was built: google-gemini/gemini-cli#29499,
sednalabs/codex#626 and jmmaloney4/codex-proxy#6, recorded through the
adapter's own calls into `crates/kelpie/fixtures/`. The design log's
Facts section records what they show.

Tests: the reader against the recordings; the lease book sharing one
Gemini window across two projects; and the runner through its stand-ins
for a summon posted once and recorded as kelpie's (including a failed
post and a restart), a review with only refuted findings satisfying the
round, a held finding going to the worker and its fix summoned again,
reviews already on the pull request counting toward the cap, the cap
moving on to CodeRabbit without a ruling, a refusal, Gemini running
before CodeRabbit, and rounds off never posting a comment.

`runner/outside/coderabbit_tests.rs` grew to 1031 lines from the new
`reviewer` fields, and a helper for the repeated summon check brings it
back to 993.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
serde's skip_serializing_if took Clone::clone to skip a true flag, which
reads as a copy rather than a check. A named function says what it is.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@TurtIeSocks

Copy link
Copy Markdown
Contributor Author

Closing: the free version of Gemini Code Assist on GitHub was shut down on 2026-07-17, and the enterprise version needs Google Cloud billing. The generic second-reviewer gate here can come back for whichever service replaces it.

@TurtIeSocks
TurtIeSocks deleted the worktree-kelpie-92 branch September 29, 2026 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini Code Assist as a second outside reviewer

1 participant