Skip to content

[FIXED] Last msg ID is restored only if it is in duplicate window - #8564

Merged
neilalexander merged 1 commit into
mainfrom
daniele/nats-expected-last-msg-id
Sep 4, 2026
Merged

neilalexander merged 1 commit into
mainfrom
daniele/nats-expected-last-msg-id

Conversation

@sciascid

@sciascid sciascid commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

On restart, the last message id was restored only if the last message was still within the duplicate window, and would otherwise be forgotten. A valid message using Nats-Expected-Last-Msg-Id could later be rejected.

@sciascid
sciascid requested a review from a team as a code owner September 4, 2026 09:16

@MauriceVanVeen MauriceVanVeen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 668b02e516

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread server/stream.go
Comment on lines +1608 to +1611
if state.LastSeq > 0 {
if sm, err := mset.store.LoadMsg(state.LastSeq, &smv); err == nil {
mset.lmsgId = getMsgId(sm.hdr)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist the last ID independently of the stored message

When the last assigned sequence has no stored message—such as an InterestPolicy stream publishing with no consumers, where SkipMsgNoInterest advances the sequence while the publish path still sets mset.lmsgId—this LoadMsg fails and recovery leaves lmsgId empty. Consequently, an expected-last-ID publish that succeeds before shutdown is rejected after restart; expiration, purge, or deletion of the latest message produces the same problem. The last seen message ID needs durable state that does not depend on the corresponding message remaining loadable.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good point, but that would require a much larger change. Probably not worth it, given that the consequences are not that high, worst case we reject a conditional message that was otherwise valid.

@sciascid

sciascid commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

@claude review

@synadia-claude-reviewer

synadia-claude-reviewer Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Claude finished @sciascid's task in 2m 15s —— View job


Code Review

  • Gather context
  • Read changed files and diff
  • Analyze fix logic
  • Review test coverage
  • Post review findings

Summary

This PR fixes a real bug: after server restart, mset.lmsgId (used to validate Nats-Expected-Last-Msg-Id headers) was only restored if the last published message was still within the deduplicate window. If the server was restarted after the window expired, the field would be left empty, causing subsequent conditional publishes to be incorrectly rejected.


Fix Analysis (server/stream.go)

Old code — lmsgId was set inside the loop body, but only when seq == state.LastSeq:

if seq == state.LastSeq {
    mset.lmsgId = msgId  // only reached if last msg is in the dedupe window
}

New code — lmsgId is set unconditionally from LoadMsg(state.LastSeq) before any window logic runs:

if state.LastSeq > 0 {
    if sm, err := mset.store.LoadMsg(state.LastSeq, &smv); err == nil {
        mset.lmsgId = getMsgId(sm.hdr)
    }
}

This is correct and minimal. The change in variable declaration placement (moving smv/state to the top) is a necessary consequence and does not affect behavior.

Additional implicit fix: Previously, when Duplicates == 0 (dedup window disabled), the function returned early before ever setting lmsgId. Now lmsgId is restored even for streams with dedup disabled, which is the right behavior since Nats-Expected-Last-Msg-Id is independent of the dedup window.

Edge case acknowledged in review: If the last sequence has no stored message — e.g. SkipMsgNoInterest, or the message was deleted/purged/expired — LoadMsg will return an error and lmsgId will remain empty. The author correctly identified this as a separate, larger problem requiring durable state for the last-seen msg ID, and this PR's scope is appropriate.


Test Analysis (server/jetstream_test.go)

TestJetStreamRestoreLastMsgIDPastDedupeWindow directly exercises the bug:

  1. Creates a stream with a 100ms dedupe window
  2. Publishes a message with a known msgId ("A")
  3. Waits for the message to exit the window via checkFor + GetSeqFromTime
  4. Shuts down and restarts the server with the same store directory
  5. Sends a publish with Nats-Expected-Last-Msg-Id: A and asserts it succeeds at sequence 2

The test is well-structured and the polling approach to confirm window expiry is robust. Only FileStorage is tested (correct, since memory storage doesn't survive restarts).

Minor observation: The test doesn't cover the case where Duplicates is 0 (the implicit additional fix noted above), but that's a low-risk gap given the simplicity of that path.


Verdict

The fix is correct, focused, and well-tested. The known limitation around non-loadable last messages is an accepted trade-off with reasonable consequences (a false rejection at worst). LGTM.

@neilalexander
neilalexander merged commit 95791cb into main Sep 4, 2026
72 of 73 checks passed
@neilalexander
neilalexander deleted the daniele/nats-expected-last-msg-id branch September 4, 2026 09:52
neilalexander added a commit that referenced this pull request Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants