fix(core): preserve line terminators in truncateString by diegogodinezr · Pull Request #29673 · google-gemini/gemini-cli

diegogodinezr · 2026-10-07T22:52:16Z

Summary

Fixes truncateString in @google/gemini-cli-core so that line terminators (\n, \r, \r\n, \u2028, \u2029) and complex Unicode grapheme clusters are preserved in the truncated output and counted toward the maxLength budget.

Details

Previously, graphemeRegex in truncateString (/(?:[\uD800-\uDBFF][\uDC00-\uDFFF]|.)\p{M}*/gu) omitted the s (dotAll) flag and used a manual regular expression that skipped line terminators and could split multi-code-point grapheme clusters such as \r\n (CRLF) or ZWJ emoji sequences.

This change replaces the manual regex with Intl.Segmenter (granularity: 'grapheme') so that line terminators, CRLF sequences, and complex Unicode grapheme clusters are treated as atomic segments and counted toward maxLength, and adds unit test coverage for multi-line strings and ZWJ emoji clusters.

Related Issues

Fixes #29562

How to Validate

Run the textUtils unit tests in @google/gemini-cli-core:

npm test -w @google/gemini-cli-core -- src/utils/textUtils.test.ts

Verify that truncateString preserves line terminators and counts grapheme clusters toward maxLength:
- truncateString('aa\nbb\ncc', 4, '') returns 'aa\nb'
- truncateString('a\nb\nc', 3, '') returns 'a\nb'
- truncateString('x\r\ny', 2, '') returns 'x'
- truncateString('x\r\ny', 3, '') returns 'x\r\n'

Pre-Merge Checklist

Add the dotAll (s) flag to graphemeRegex in truncateString so that '.' matches line terminators (\n, \r, \u2028, \u2029), preserving multi-line formatting and counting line terminators toward the maxLength budget. Fixes google-gemini#29562

gemini-code-assist · 2026-10-07T22:52:25Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where line terminators were being stripped and ignored during string truncation. By enabling the dotAll flag in the regex used for grapheme parsing, the function now correctly treats line breaks as characters, ensuring they are preserved in the output and accounted for within the specified length budget.

Highlights

Regex Update: Updated the grapheme cluster regular expression in truncateString to include the 's' (dotAll) flag, ensuring line terminators are correctly matched and counted.
Test Coverage: Added comprehensive unit tests to verify that multi-line strings and various line terminators are preserved during string truncation.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

github-actions · 2026-10-07T22:52:31Z

📊 PR Size: size/S

Lines changed: 33
Additions: +22
Deletions: -11
Files changed: 2

diegogodinezr · 2026-10-07T22:53:11Z

/gemini review

gemini-code-assist

Code Review

This pull request updates the truncateString utility in textUtils.ts by adding the s (dotAll) flag to the grapheme regular expression, ensuring that line terminators are preserved and counted toward the maximum length budget. It also adds corresponding unit tests in textUtils.test.ts. The review feedback recommends treating \r\n (CRLF) as a single, indivisible grapheme cluster to prevent splitting that could corrupt terminal rendering in CLI environments, and suggests updating the test assertions to reflect this behavior.

gemini-code-assist · 2026-10-07T22:53:28Z

+    expect(truncateString('x\r\ny', 2, '')).toBe('x\r');
+    expect(truncateString('x\r\ny', 3, '')).toBe('x\r\n');


Update the test assertions to reflect that \r\n (CRLF) is treated as a single indivisible grapheme cluster, preventing a dangling \r from being returned when truncating.

Suggested change

expect(truncateString('x\r\ny', 2, '')).toBe('x\r');

expect(truncateString('x\r\ny', 3, '')).toBe('x\r\n');

expect(truncateString('x\r\ny', 2, '')).toBe('x');

expect(truncateString('x\r\ny', 3, '')).toBe('x\r\n');

References

When truncating strings that may contain multi-byte Unicode characters, use methods that operate on grapheme clusters instead of UTF-16 code units to prevent character splitting.

gemini-code-assist

Code Review

This pull request updates the truncateString utility by adding the s (dotAll) flag to its grapheme regular expression, ensuring line terminators are preserved and counted toward the length budget, and adds corresponding unit tests. However, the review feedback highlights that this custom regex approach still fails to handle complex Unicode grapheme clusters correctly, such as splitting CRLF (\r\n) into separate characters (which is also codified in the new tests). Since the project targets Node.js >= 20, it is recommended to replace the custom regex with the built-in Intl.Segmenter API for robust Unicode support.

… handling

diegogodinezr · 2026-10-07T23:09:37Z

/gemini review

gemini-code-assist

Code Review

This pull request refactors the truncateString utility in packages/core/src/utils/textUtils.ts to use the native Intl.Segmenter API with grapheme granularity instead of a custom regular expression. This improves the handling of complex grapheme clusters, such as zero-width joiner (ZWJ) emojis and line terminators. Additionally, corresponding unit tests have been added to packages/core/src/utils/textUtils.test.ts to verify these behaviors. There are no review comments, and I have no feedback to provide.

diegogodinezr requested a review from a team as a code owner October 7, 2026 22:52

github-actions Bot added the size/s A small PR label Oct 7, 2026

Merge branch 'main' into fix/textutils-truncate-line-terminators

41c7f9c

gemini-code-assist Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread packages/core/src/utils/textUtils.ts Outdated

Comment thread packages/core/src/utils/textUtils.test.ts Outdated

fix(core): use Intl.Segmenter in truncateString for grapheme and CRLF…

31455f7

… handling

gemini-code-assist Bot reviewed Oct 7, 2026

View reviewed changes

gemini-cli Bot added the area/core Issues related to User Interface, OS Support, Core Functionality label Oct 7, 2026

Navigation Menu

Repository navigation

fix(core): preserve line terminators in truncateString - #29673

fix(core): preserve line terminators in truncateString#29673
diegogodinezr wants to merge 3 commits into
google-gemini:mainfrom
diegogodinezr:fix/textutils-truncate-line-terminators

diegogodinezr commented Oct 7, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot commented Oct 7, 2026

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

diegogodinezr commented Oct 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

gemini-code-assist Bot Oct 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

diegogodinezr commented Oct 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		expect(truncateString('x\r\ny', 2, '')).toBe('x\r');
		expect(truncateString('x\r\ny', 3, '')).toBe('x\r\n');

Conversation

diegogodinezr commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Details

Related Issues

How to Validate

Pre-Merge Checklist

Uh oh!

gemini-code-assist Bot commented Oct 7, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

diegogodinezr commented Oct 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist Bot Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

diegogodinezr commented Oct 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

diegogodinezr commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading