Skip to content

image: preserve metadata when resizing prompt images - #27266

Merged
fjord-oai merged 1 commit into
mainfrom
fjord/responses_strict_mode_v2---4ixyx7vs5bptfr
Jun 11, 2026
Merged

fjord-oai merged 1 commit into
mainfrom
fjord/responses_strict_mode_v2---4ixyx7vs5bptfr

Conversation

@fjord-oai

@fjord-oai fjord-oai commented Jun 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Preserve ICC profiles and EXIF metadata when resizing and re-encoding prompt images.
  • Retain EXIF orientation metadata without rotating or otherwise modifying the pixel data locally.
  • Support metadata preservation for PNG, JPEG, and WebP outputs.
  • Continue returning the original bytes when an image does not require re-encoding.

This intentionally preserves the metadata most important for rendering prompt images faithfully. Other format-specific metadata is not copied.

Motivation

Client-side resizing previously discarded image metadata during re-encoding. This could lose color-profile information and EXIF orientation needed by downstream image consumers.

git stack

@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 10, 2026 02:42
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 5dcded4 to 6c8d994 Compare June 10, 2026 02:42
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 10, 2026 02:42
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 10, 2026 03:45
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 6c8d994 to 9b4dd45 Compare June 10, 2026 03:45
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 10, 2026 03:45
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 10, 2026 03:57
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 9b4dd45 to 1b7f442 Compare June 10, 2026 03:57
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 10, 2026 03:57
@fjord-oai
fjord-oai marked this pull request as ready for review June 10, 2026 04:13

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b7f442138

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/utils/image/src/lib.rs
Comment thread codex-rs/utils/image/src/lib.rs
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 10, 2026 19:37
@fjord-oai
fjord-oai requested a review from a team as a code owner June 10, 2026 19:37
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 1b7f442 to 8fa93de Compare June 10, 2026 19:37
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 10, 2026 19:37
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 10, 2026 19:54
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 8fa93de to 392d948 Compare June 10, 2026 19:54
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 10, 2026 19:54
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 10, 2026 20:19
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 392d948 to 016461f Compare June 10, 2026 20:19
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 10, 2026 20:19
fjord-oai added a commit that referenced this pull request Jun 10, 2026
## Summary

Add shared image-processing primitives needed for centralized image
preparation in a follow-up PR.

- Add `load_data_url_for_prompt` for decoding and preparing base64 image
data URLs.
- Add configurable maximum-dimension and 32px patch-budget resizing.
- Enforce a 1 GiB sanity limit on both encoded and decoded data-URL
representations.
- Preserve original PNG, JPEG, and WebP bytes when resizing is
unnecessary.
- Preserve the existing GIF-to-PNG behavior.
- Move image utility tests into the existing sidecar test module.

## Behavior

This PR is intended to be runtime behavior-preserving.

Existing production callers continue using
`PromptImageMode::ResizeToFit` and `PromptImageMode::Original` with
their existing semantics. The new data-URL entrypoint and configurable
resize mode have no production callers in this PR; they are used by the
next PR in the stack.

This PR does not change user-input handling, `view_image`, history
insertion, request construction, HTTP image URL forwarding, or
app-server behavior.


#### [git stack](https://github.com/magus/git-stack-cli)
- 👉 `1` #27245
- ⏳ `2` #27247
- ⏳ `3` #27246
- ⏳ `4` #27266
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 10, 2026 22:49
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 016461f to 70c2840 Compare June 10, 2026 22:49
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 10, 2026 22:49
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 11, 2026 00:37
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 70c2840 to a410ab3 Compare June 11, 2026 00:37
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 11, 2026 00:37
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 11, 2026 01:16
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from a410ab3 to 854a32c Compare June 11, 2026 01:16
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 11, 2026 01:16
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 11, 2026 01:26
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 854a32c to af333bf Compare June 11, 2026 01:26
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 11, 2026 01:26
fjord-oai added a commit that referenced this pull request Jun 11, 2026
## Summary

Adds complete client-side image preparation behind the default-off
`resize_all_images` feature flag.

When enabled, local image producers defer decoding and resizing. Images
are prepared centrally before insertion into conversation history,
covering user input, `view_image`, and structured tool-output images.

## Behavior

- Processes base64 `data:` images in messages and function/custom tool
outputs.
- Leaves non-data URLs, including HTTP(S) URLs, unchanged.
- Applies image-detail budgets:
  - `high` and omitted: 2048px maximum dimension and 2.5K 32px patches.
  - `original`: 6000px maximum dimension and 10K 32px patches.
  - `auto`: uses the same 2048px / 2.5K-patch budget as high.
  - `low`: unsupported and replaced with an actionable placeholder.
- Preserves original image bytes when no resize or format conversion is
needed.
- Enforces the shared 1 GiB encoded and decoded data-URL sanity limits.
- Replaces only an image that fails preparation, preserving sibling
content and tool-output metadata.
- Uses bounded placeholders distinguishing generic processing failures,
oversized images, and unsupported `low` detail.
- Prepares resumed and forked history before installing it as live
history without modifying persisted rollouts.

## Flag-Off Behavior

When `resize_all_images` is disabled:

- Existing local user-input and `view_image` processing remains
unchanged.
- Existing decoding and error behavior remains unchanged.
- Arbitrary tool-output images are not processed.
- HTTP(S) image URLs continue to be forwarded unchanged.


#### [git stack](https://github.com/magus/git-stack-cli)
- ✅ `1` #27245
- 👉 `2` #27247
- ⏳ `3` #27246
- ⏳ `4` #27266
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 11, 2026 02:21
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from af333bf to ccf7872 Compare June 11, 2026 02:21
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 11, 2026 02:21
@fjord-oai
fjord-oai changed the base branch from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 11, 2026 03:29
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from ccf7872 to 0c4882b Compare June 11, 2026 03:29
@fjord-oai
fjord-oai changed the base branch from main to fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- June 11, 2026 03:29
fjord-oai added a commit that referenced this pull request Jun 11, 2026
## Summary

- Strip image `detail` fields from every Responses Lite request.
- Apply stripping to message images and function/custom tool-output
images.
- Transform only the formatted request copy without mutating stored
history.
- Preserve image URLs byte-for-byte, including HTTP(S) URLs, without
downloading, validating, or resizing them.
- Preserve all image `detail` fields for non-Responses-Lite models.

## Motivation

Responses Lite does not support image `detail` tags, so Codex must omit
them whenever `model_info.use_responses_lite` is enabled. This transport
requirement is independent of the `resize_all_images` feature.

Stored history retains the original detail values. This keeps
request-specific formatting isolated from conversation state and
preserves the information for local image preparation and
non-Responses-Lite requests.


#### [git stack](https://github.com/magus/git-stack-cli)
- ✅ `1` #27245
- ✅ `2` #27247
- 👉 `3` #27246
- ⏳ `4` #27266
Base automatically changed from fjord/responses_strict_mode_v2---4ixxk-hc0zkmy- to main June 11, 2026 03:43
git-stack-id: fjord/responses_strict_mode_v2---4ixyx7vs5bptfr
git-stack-title: image: preserve metadata when resizing prompt images
@fjord-oai
fjord-oai force-pushed the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch from 0c4882b to 42aa9fe Compare June 11, 2026 03:43
@fjord-oai
fjord-oai merged commit 5981251 into main Jun 11, 2026
46 checks passed
@fjord-oai
fjord-oai deleted the fjord/responses_strict_mode_v2---4ixyx7vs5bptfr branch June 11, 2026 04:17
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants