Skip to content

Exact PostgreSQL promoted matches can duplicate #1153

Description

@bcdonadio

Observed behavior: Importing exact existing promoted text into a PostgreSQL project inserts a duplicate even after the import correctly searches the project-owned records. The real pinned PostgreSQL18 regression starts with normal public /promote, then imports identical content Orchard pruning architecture decision for perennial fruit trees. The search result uses negative PostgreSQL relevance (approximately -0.1), and deduplication rejects the exact match against the shared -15 threshold.

Expected behavior: Exact content identity deduplicates regardless of whether lexical search reports positive fallback rank or negative full-text relevance. Preserve the existing threshold contract for nonidentical fuzzy matches.

Root cause: src/promotion/dedup.ts isDuplicateCandidate checks exact equality only for candidate.rank >= 0. Negative PostgreSQL ts_rank_cd scores go directly to the SQLite/BM25-style threshold comparison, so exact content can fail. Existing rank-zero fixture tests do not cover the actual PostgreSQL score path. This shared behavior predates the #618 provenance correction.

How to reproduce: In the isolated repository PostgreSQL18 operational harness, bind a project whose local path hash differs from its remote UUID, use the normal public /promote route to create the text above, then call importKnowledge with identical text after using owner-project-scoped search. Read active promoted_memories rows: two instead of one. A direct deterministic unit reproduction is an identical-content candidate with rank -0.1 and dedupBm25Threshold: 15.

Environment:

Discovered during #618 / PR #1143 normal-promotion provenance validation, starting at bcd215163f1a4077bf93400c3ed7fff83596f8a6. This is a separate pre-existing shared deduplication Bug outside the fixed Epic #224 inventory. It is not parented into that campaign. The acceptance regression is retained under test/postgresql/portable-knowledge.integration.ts in the unpublished correction candidate. Ownership/scope of the minimal exact-match correction remains pending.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    Medium

    Effort

    None yet

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions