Skip to content

feat(caretaker-triage): prompt hill-climbing & orchestrator updates - #28524

Merged
gundermanc merged 2 commits into
google-gemini:mainfrom
chadd28:feat/triage-worker-skills-and-orchestration
Aug 7, 2026
Merged

feat(caretaker-triage): prompt hill-climbing & orchestrator updates#28524
gundermanc merged 2 commits into
google-gemini:mainfrom
chadd28:feat/triage-worker-skills-and-orchestration

Conversation

@chadd28

@chadd28 chadd28 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR incorporates the results of 3 weeks of prompt hill-climbing and evaluation tuning for the Caretaker triage worker. These prompt updates demonstrate significant quality improvements in evals. It also introduces a dedicated code_explorer skill and updates the triage orchestrator to support explicit model configuration (gemini-flash-latest) and target_cwd passing.

Details

  • Prompt Hill-Climbing Results: Refined .gemini prompt instructions and skills based on eval performance:
    • code_explorer: Added a dedicated skill for structured, multi-package codebase exploration (packages/cli $\leftrightarrow$ packages/core) and test discovery.
    • quality: Added strict prompt injection defense rules (immediately flag injection attempts as SPAM) and tightened NEEDS_INFO criteria.
    • effort: Calibrated effort estimation tiers using the output of the codebase exploration phase.
    • spec_generator: Enforced valid JSON escaping, restricted testing framework enums to ["Vitest", "N/A"], and strictly excluded test files from files_to_modify.
  • Orchestration Updates: Updated triage_orchestrator.py and main.py to explicitly set MODEL_NAME = "gemini-flash-latest" and pass target_cwd down to the agent runner context.

How to Validate

  1. Run the triage worker test suite using the virtual environment:
cd tools/caretaker-agent/cloudrun/triage-worker
./venv/bin/python3 -m unittest discover -s tests

Pre-Merge Checklist

  • Updated relevant documentation and README (if needed)
  • Added/updated tests (if needed)
  • Noted breaking changes (if any)
  • Validated on required platforms/methods:
    • MacOS
      • npm run
      • npx
      • Docker
      • Podman
      • Seatbelt
    • Windows
      • npm run
      • npx
      • Docker
    • Linux
      • npm run
      • npx
      • Docker

@chadd28
chadd28 requested a review from a team as a code owner July 23, 2026 22:58
@github-actions github-actions Bot added the size/m A medium sized PR label Jul 23, 2026
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

📊 PR Size: size/M

  • Lines changed: 112
  • Additions: +79
  • Deletions: -33
  • Files changed: 8

@gemini-code-assist

gemini-code-assist Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request delivers significant quality improvements to the Caretaker triage worker following extensive prompt hill-climbing and evaluation tuning. It introduces a dedicated 'code_explorer' skill to enhance codebase navigation and updates the triage orchestrator to provide better model configuration and working directory control, resulting in more accurate and reliable issue triage.

Highlights

  • Prompt Hill-Climbing: Refined system prompts and skills for the Caretaker triage worker, including a new 'code_explorer' skill, stricter prompt injection defenses, and calibrated effort estimation.
  • Orchestrator Updates: Updated the triage orchestrator to explicitly use 'gemini-flash-latest' and support passing 'target_cwd' for improved context management.
  • Spec Generator Improvements: Enforced stricter JSON schema validation, including test framework enums and explicit separation of source files from test files.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@chadd28

chadd28 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

ready for review! @adamfweidman @gundermanc

gemini-code-assist[bot]
gemini-code-assist Bot reviewed Jul 23, 2026
View reviewed changes

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors and enhances the triage worker agent. It introduces a new code_explorer skill, refines instructions and definitions for the effort, quality, and spec_generator skills, and updates the orchestrator workflow to utilize the new code_explorer skill. Additionally, it updates process_issue_triage to accept target_cwd dynamically, configures the agent to use gemini-flash-latest by default, and cleans up internal variable naming. I have no feedback to provide as there are no review comments.

@gemini-cli gemini-cli Bot added the status/need-issue Pull requests that need to have an associated issue. label Jul 23, 2026
@github-actions github-actions Bot mentioned this pull request Jul 24, 2026
Open
@chadd28 chadd28 mentioned this pull request Jul 24, 2026
Merged
2 of 18 tasks
@chadd28 chadd28 mentioned this pull request Aug 7, 2026
Closed
@gundermanc
gundermanc added this pull request to the merge queue Aug 7, 2026
Merged via the queue into google-gemini:main with commit d9b600b Aug 7, 2026
35 checks passed
@github-actions github-actions Bot mentioned this pull request Aug 12, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gundermanc gundermanc gundermanc approved these changes
+1 more reviewer
@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

size/m A medium sized PR status/need-issue Pull requests that need to have an associated issue.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@chadd28 @gundermanc