Implement 'Tactful Extraction' logic for token-frugal surgical reads #19561

Motivation

Current context baseline is approximately 36.6k tokens per turn. Large file reads often 'firehose' the context, leading to bloat (previously observed at +15k tokens/turn).

The goal of 'Tactful Extraction' is to establish a surgical code-discovery hierarchy:

1. grep_search: Primary scout for locating symbols.
2. shell/sed: scalpel for semantic block extraction (preferred for specific line ranges).
3. read_file: surgical range-reading (migrated to 1-based start/end lines).

Telemetry indicates that making ~2.1% more targeted tool calls results in a ~6.9% reduction in total input tokens.

Changes

• read_file migration: Switched from 0-based 'offset'/'limit' to 1-based 'start_line'/'end_line' for better model alignment.
• Surgical Guidance: Updated tool descriptions and system prompts to mandate 'token-frugal' behavior.
• Tool Hierarchy: Emphasized the use of 'grep_search' and 'sed' via 'run_shell_command' for efficient context management.

Status

This work is temporarily sidelined in favor of other priorities but remains a key architectural goal for efficiency.

Linked Draft PR: #18924