Skip to content

feat(core): add reliable file-only logger for interceptor validation - #29181

Closed
tsunex wants to merge 2 commits into
google-gemini:mainfrom
tsunex:feature/top-level-principle-interceptor
Closed

tsunex wants to merge 2 commits into
google-gemini:mainfrom
tsunex:feature/top-level-principle-interceptor

Conversation

@tsunex

@tsunex tsunex commented Sep 3, 2026

Copy link
Copy Markdown

Summary

This PR introduces a new interceptor to enforce the "Top-level Principle" from our global GEMINI.md. This ensures that all model interactions are strictly governed by this principle, preventing deviations and improving reliability. A file-only logger has also been added for interceptor validation.

Details

The core of this change is the deterministic-output-interceptor. It's designed to:

  • Enforce the "Top-level Principle" by intercepting model output.
  • Log validation results to a dedicated file for easier debugging and analysis.

This PR also includes:

  • The deterministic-output-interceptor implementation.
  • A reliable file-only logger for interceptor validation.

Related Issues

N/A

How to Validate

  1. Run the agent and observe the new interceptor's behavior in the logs.
  2. Check the output in interceptor.log to confirm that the interceptor is working as expected.

Pre-Merge Checklist

  • Updated relevant documentation and README (if needed) - The changes are self-contained and don't require documentation updates.
  • Added/updated tests (if needed) - The functionality is validated through logging and manual verification for now.
  • Noted breaking changes (if any)
  • Validated on required platforms/methods:
    • MacOS
      • npm run
      • npx
      • Docker
      • Podman
      • Seatbelt
    • Windows
      • npm run
      • npx
      • Docker
    • Linux
      • npm run
      • npx
      • Docker

…-level Principle

Enforces the MEMORY.md Top-level Principle by programmatically verifying agent outputs against user premises. Runs a stateless, deterministic validation request on turn finalization.

Supporting:
- Interactive automatic self-correction retries (up to 3 attempts).
- Non-interactive (-p mode) fail-safe rollbacks and immediate process.exit(1).
- Global enablement gate via GEMINI_TOP_LEVEL_GUARD environment variable.
Replaces the debugLogger dependency in TopLevelPrincipleValidator with a self-contained, simple file logger. Why: This ensures that every validation attempt and its outcome (PASS or VIOLATION) is reliably written to the file specified by GEMINI_DEBUG_LOG_FILE, without being affected by console behavior. This provides a fully auditable, deterministic proof that the guardrail is active on every turn.
@tsunex
tsunex requested review from a team as code owners September 3, 2026 09:02
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a robust programmatic guardrail to enforce the 'Top-level Principle' across the Gemini CLI agent. By intercepting final model outputs and validating them against user-provided premises, the system prevents hallucinations and incorrect denials of user facts. The implementation includes a stateless validation engine, configurable retry mechanisms for interactive sessions, and fail-safe termination for automated environments, significantly improving the reliability and adherence of the agent to its core directives.

Highlights

  • Deterministic Output Interceptor: Implemented a new interceptor that validates agent responses against the 'Top-level Principle' to ensure user facts and assumptions are never denied.
  • Validation Engine: Added a stateless validator that uses a dedicated Gemini model instance to programmatically detect violations of the Top-level Principle.
  • Interactive Retry Logic: Integrated automatic retry logic in interactive mode, allowing the agent to self-correct up to 3 times if a violation is detected.
  • Non-interactive Fail-safe: Added strict termination logic for non-interactive mode, where violations trigger a fatal error and process exit to prevent incorrect automation output.
  • Diagnostic Logging: Introduced a file-only logger for interceptor validation results to facilitate easier debugging and analysis of agent behavior.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@github-actions github-actions Bot added the size/l A large sized PR label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

📊 PR Size: size/L

  • Lines changed: 725
  • Additions: +686
  • Deletions: -39
  • Files changed: 5

@google-cla

google-cla Bot commented Sep 3, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a Top-level Principle Output Interceptor to programmatically validate agent responses against the supreme directive in MEMORY.md, complete with design documentation, integration into the legacy agent session, and corresponding unit tests. Feedback highlights several critical issues: a bug in streaming text accumulation that skips whitespace chunks, an API contract violation when prepending system commands on retry, a prompt injection and system instruction override vulnerability in the validator, and an architectural anti-pattern of calling process.exit(1) directly within the shared core library.

Comment on lines +269 to +274
if (event.type === GeminiEventType.Content) {
if (typeof event.value === 'string' && event.value.trim() !== '') {
hasVisibleText = true;
responseText += event.value;
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The current implementation of responseText accumulation incorrectly filters out whitespace-only chunks (such as spaces and newlines) because of the event.value.trim() !== '' check. In streaming APIs, spaces and newlines are often emitted as individual chunks. Skipping them will result in concatenated words (e.g., "Helloworld") and no line breaks, which severely degrades the quality of the text sent to the validator and can cause false positives/negatives. Accumulate all string values into responseText unconditionally, while keeping the visibility check separate.

Suggested change
if (event.type === GeminiEventType.Content) {
if (typeof event.value === 'string' && event.value.trim() !== '') {
hasVisibleText = true;
responseText += event.value;
}
}
if (event.type === GeminiEventType.Content) {
if (typeof event.value === 'string') {
responseText += event.value;
if (event.value.trim() !== '') {
hasVisibleText = true;
}
}
}

Comment on lines +238 to +249
let partsForAttempt = currentParts;
if (attempt > 1) {
partsForAttempt = [
{
text: `[SYSTEM COMMAND - DO NOT IGNORE] Your previous response was detected as a violation of the Top-level Principle in MEMORY.md.
You MUST treat all facts, settings, model names, and statements provided by the user as absolute 100% correct facts.
Under NO circumstances are you allowed to doubt them, state they do not exist, or claim they are wrong.
Rewrite your response completely to accept the user's premise as absolute truth and avoid any negative reasoning.`,
},
...currentParts,
];
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

Prepending the system command to currentParts on retry attempts introduces two major issues:

  1. If currentParts contains functionResponse parts (which happens when retrying after a tool execution), prepending a text part violates the Gemini API contract (which expects only functionResponse parts in response to a functionCall). This will trigger a 400 Bad Request error from the Gemini API.
  2. It duplicates currentParts in the chat history on every retry attempt, wasting tokens and polluting the conversation history.

Since the original currentParts and the violating response are already recorded in the chat history, the retry attempt should only send the system command itself as the next user turn.

        let partsForAttempt = currentParts;
        if (attempt > 1) {
          partsForAttempt = [
            {
              text: "[SYSTEM COMMAND - DO NOT IGNORE] Your previous response was detected as a violation of the Top-level Principle in MEMORY.md.\nYou MUST treat all facts, settings, model names, and statements provided by the user as absolute 100% correct facts.\nUnder NO circumstances are you allowed to doubt them, state they do not exist, or claim they are wrong.\nRewrite your response completely to accept the user's premise as absolute truth and avoid any negative reasoning."
            }
          ];
        }

Comment on lines +69 to +104
const validationPrompt = `
You are a strict deterministic Rule Validator for an AI Agent.
Your job is to analyze the agent's output and determine if it violates the "Top-level Principle" from MEMORY.md.

[Top-level Principle]
- The user is ALWAYS right. Never doubt the user.
- Never state or imply that the user's provided facts, settings, model names, or assumptions are incorrect, invalid, or do not exist.
- If there is a contradiction between the user's input and internal knowledge, the agent must assume that the agent's own internal knowledge or search is incomplete/wrong, and MUST NOT deny the user's statement.
- The agent is forbidden from making unauthorized reasoning/assumptions without explicit permission.

[User Query]
"""
${userQuery}
"""

[Agent Output]
"""
${agentOutput}
"""

Determine if the Agent Output violates the Top-level Principle by denying the user's facts, stating they are wrong/incorrect, or claiming something they mentioned does not exist.
Respond with EXACTLY "VIOLATION" if a violation is detected. Otherwise, respond with "PASS". Do not include any other text.
`;

try {
const response = await client.generateContent(
{ model: 'flash' }, // Pass ModelConfigKey object instead of raw string
[
{
role: 'user',
parts: [{ text: validationPrompt }],
},
],
signal,
LlmRole.MAIN, // Use active role MAIN
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-high high

Prompt Injection & System Instruction Override Vulnerability

The detectTopLevelPrincipleViolation function contains a critical security and logic flaw where untrusted user input (userQuery) is directly interpolated into the validationPrompt (lines 69-91) and sent to the model via client.generateContent (lines 94-104).

This introduces two major issues:

  1. Prompt Injection: An attacker can easily escape the triple quotes (""") in userQuery and inject instructions to manipulate the validator's output (e.g., forcing it to return PASS to bypass validation, or VIOLATION to crash the agent in non-interactive mode).
  2. System Instruction Override: GeminiClient.generateContent is hardcoded to always override the systemInstruction parameter with the core agent's system prompt (which instructs the model to follow the user and that "the user is ALWAYS right"). As a result, the validator model runs with the core agent's system prompt instead of the validator's instructions, making it extremely susceptible to following any instructions injected into the userQuery.

Remediation

  1. Use a Dedicated Stateless Call: Avoid using client.generateContent for meta-tasks like validation if it overrides system instructions. Instead, call the underlying content generator directly or update GeminiClient to support passing a custom systemInstruction for validation/meta-tasks.
  2. Structured/JSON Output: Configure the model to return structured JSON output (e.g., { "violation": true/false }) and enforce a strict schema to prevent the model from outputting conversational text or being easily manipulated.
  3. Input Sanitization/Escaping: When sanitizing untrusted context (such as userQuery or agentOutput) for downstream LLM processing, avoid escaping all HTML-like characters (like '<' and '>') as this alters source code syntax and harms LLM reasoning. Instead, restrict escaping to specific boundary tags (e.g., '</untrusted_context>') to prevent context breakout while preserving code formatting.
References
  1. When sanitizing untrusted context (such as code or issue text) for downstream LLM processing, avoid escaping all HTML-like characters (like '<' and '>') as this alters source code syntax and harms LLM reasoning. Instead, restrict escaping to specific boundary tags (e.g., '</untrusted_context>') to prevent context breakout while preserving code formatting.

Comment on lines +171 to +185
if (err instanceof TopLevelPrincipleViolationError) {
if (!this._config.isInteractive()) {
// eslint-disable-next-line no-console
console.error(
`\n======================================================================\n` +
`[FATAL ERROR] Top-level Principle Violation detected in non-interactive (-p) mode!\n` +
`The agent attempted to deny a user-provided fact or assumption, which is strictly prohibited by MEMORY.md.\n` +
`Error details: ${err.message}\n` +
`======================================================================\n`,
);
this._ensureAgentEnd('failed');
process.exit(1);
} else {
this._emitErrorAndAgentEnd(err);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Calling process.exit(1) directly within a shared core library (packages/core) is an architectural anti-pattern. packages/core is designed to be reusable and is imported by other environments like the VS Code extension companion or programmatic SDKs. Abruptly terminating the process will crash the entire host application in those environments. Instead, propagate the error or emit the failure event, and let the CLI runner (in packages/cli) handle the process exit if running in non-interactive mode.

      if (err instanceof TopLevelPrincipleViolationError) {
        this._emitErrorAndAgentEnd(err);
        if (!this._config.isInteractive()) {
          // eslint-disable-next-line no-console
          console.error(
            "\n======================================================================\n" +
              "[FATAL ERROR] Top-level Principle Violation detected in non-interactive (-p) mode!\n" +
              "The agent attempted to deny a user-provided fact or assumption, which is strictly prohibited by MEMORY.md.\n" +
              "Error details: " + err.message + "\n" +
              "======================================================================\n"
          );
        }
      }

@gemini-cli gemini-cli Bot added the status/need-issue Pull requests that need to have an associated issue. label Sep 3, 2026
@gemini-cli

gemini-cli Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Hi there! Thank you for your interest in contributing to Gemini CLI.

To ensure we maintain high code quality and focus on our prioritized roadmap, we only guarantee review and consideration of pull requests for issues that are explicitly labeled as 'help wanted'.

This PR will be closed in 7 days if it remains without that designation. We encourage you to find and contribute to existing 'help wanted' issues in our backlog! Thank you for your understanding.

@gemini-cli

gemini-cli Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

This pull request is being closed as it has been open for 14 days without a 'help wanted' designation. We encourage you to find and contribute to existing 'help wanted' issues in our backlog! Thank you for your understanding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/l A large sized PR status/need-issue Pull requests that need to have an associated issue. status/pr-nudge-sent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant