MongoDB Vector Search now supports vector search against nested embeddings and arrays of embeddings.

XMLWordPrintableJSON

    • Type: Epic
    • Resolution: Unresolved
    • Priority: Critical - P2
    • None
    • Component/s: AI/ML
    • None
    • Vector Search over Nested Embeddings and Arrays of Embeddings
    • Database Experience
    • Needed
    • New features need to be documented--see downstream changes for the new features.
    • Hide

      Builder-only feature (no wire-protocol change). Two areas:

      • Vector index builder: accept a vector field path that points into a nested document or array of embeddings (dotted/embedded path), not just a top-level field.
      • $vectorSearch builder: allow nested/array path; add parent-document filtering that emits parentFilter (instead of filter) and, for arrays of embeddings, a way to set the score-combination mode.
      • ⚠️ Compat gotcha: filter works on server 8.0 + 8.3; parentFilter is 8.3+ only. Emit parentFilter only when the caller explicitly opts into parent filtering — don't rewrite an existing filter, or you break 8.0 (regressed then fixed in C#: PR #1975).
      • ⚠️ Server limitation: mongot (≤0.65.1 / Atlas Local) returns 0 results when filter is combined with a nestedRoot index, so the nested filter codepath can't be validated end-to-end yet — see CSHARP-5840.

      Commits for syncing spec/prose tests (or language POC)

      • No shared spec/prose tests exist for this feature — builder-only, covered by driver-specific tests.
      • C# POC: mongo-csharp-driver PR #1900 (initial nested/array support) + PR #1975 (filter/parentFilter 8.0-vs-8.3 fix). C# tickets: CSHARP-5762 (Dev Complete), CSHARP-5985 (Closed). API added: VectorSearchOptions.NestedFilter + EmbeddedScoreMode; nested paths in CreateVectorSearchIndexModel.

      Context for referenced/linked tickets

      • Splits: JAVA-5987 (Backlog — not started), PHPLIB-1733 (Done), C# (done). Initiative DRIVERS-3316; cloud parent CLOUDP-260588.
      • Blocked validation: CSHARP-5840 (mongot filter + nestedRoot).
      • Sibling builder features for precedent: DRIVERS-3309 (lexical prefilters), DRIVERS-3300 (storedSource).
      Show
      Builder-only feature (no wire-protocol change). Two areas: Vector index builder: accept a vector field path that points into a nested document or array of embeddings (dotted/embedded path), not just a top-level field. $vectorSearch builder: allow nested/array path; add parent-document filtering that emits parentFilter (instead of filter) and, for arrays of embeddings, a way to set the score-combination mode. ⚠️ Compat gotcha: filter works on server 8.0 + 8.3; parentFilter is 8.3+ only. Emit parentFilter only when the caller explicitly opts into parent filtering — don't rewrite an existing filter, or you break 8.0 (regressed then fixed in C#: PR #1975). ⚠️ Server limitation: mongot (≤0.65.1 / Atlas Local) returns 0 results when filter is combined with a nestedRoot index, so the nested filter codepath can't be validated end-to-end yet — see CSHARP-5840 . Commits for syncing spec/prose tests (or language POC) No shared spec/prose tests exist for this feature — builder-only, covered by driver-specific tests. C# POC: mongo-csharp-driver PR #1900 (initial nested/array support) + PR #1975 (filter/parentFilter 8.0-vs-8.3 fix). C# tickets: CSHARP-5762 (Dev Complete), CSHARP-5985 (Closed). API added: VectorSearchOptions.NestedFilter + EmbeddedScoreMode; nested paths in CreateVectorSearchIndexModel. Context for referenced/linked tickets Splits: JAVA-5987 (Backlog — not started), PHPLIB-1733 (Done), C# (done). Initiative DRIVERS-3316; cloud parent CLOUDP-260588. Blocked validation: CSHARP-5840 (mongot filter + nestedRoot). Sibling builder features for precedent: DRIVERS-3309 (lexical prefilters), DRIVERS-3300 (storedSource).
    • In Progress
    • 0
    • 0
    • 0
    • 100
    • None
    • None
    • None
    • Builder Changes Needed
    • $i18n.getText("admin.common.words.hide")
      Key Status/Resolution FixVersion
      CSHARP-5762 Done 3.9.0
      JAVA-5987 Backlog
      PHPLIB-1733 Fixed 2.4.0
      $i18n.getText("admin.common.words.show")
      #scriptField, #scriptField *{ border: 1px solid black; } #scriptField{ border-collapse: collapse; } #scriptField td { text-align: center; /* Center-align text in table cells */ } #scriptField td.key { text-align: left; /* Left-align text in the Key column */ } #scriptField a { text-decoration: none; /* Remove underlines from links */ border: none; /* Remove border from links */ } /* Add green background color to cells with FixVersion */ #scriptField td.hasFixVersion { background-color: #00FF00; /* Green color code */ } #scriptField td.willNotDo { background-color: #FF0000; /* Red color code */ } /* Center-align the first row headers */ #scriptField th { text-align: center; } Key Status/Resolution FixVersion CSHARP-5762 Done 3.9.0 JAVA-5987 Backlog PHPLIB-1733 Fixed 2.4.0
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      What is the problem or use case, what are we trying to achieve?

      We are releasing support for performing $vectorSearch queries against nested embeddings and arrays of embeddings (detail) on 1/19 and need to add driver support for both index time and query time changes.

      Motivation

      Customers will be able to more flexibly model their data for various vector search use cases, and easily hydrate their existing collections with vectors without significantly changing their data model and query pattern.

      Who is the affected end user?

      Who are the stakeholders?

      $vectorSearch (and eventually $search.vectorSearch customers) looking to query nested data models.

      How does this affect the end user?

      Are they blocked? Are they annoyed? Are they confused?

      They have to unwind nested documents to root level, which is annoying and requires parent metadata duplication.

      How likely is it that this problem or use case will occur?

      Main path? Edge case?

      No known edge cases

      If the problem does occur, what are the consequences and how severe are they?

      Minor annoyance at a log message? Performance concern? Outage/unavailability? Failover can't complete?

      Is this issue urgent?

      Does this ticket have a required timeline? What is it?

      Yes, mongot is shipping with this feature anticipated 12/15 (always liable to change) for phase 1 support defined here

      Is this ticket required by a downstream team?

      Needed by e.g. Atlas, Shell, Compass?

      Yes

      Is this ticket only for tests?

      Is this ticket have any functional impact, or is it just test improvements?

      No

      Cast of Characters

      Engineering Lead: chunbin.lin@mongodb.com
      Document Author:
      POCers:
      Product Owner: henry.weller@mongodb.com
      Program Manager:
      Stakeholders:

      Channels & Docs

      Slack Channel: #vector-search-nested-embeddings (ask henry to be added)

      Scope Document

      Technical Design Document

      [Parent Epic|CLOUDP-260588]

            Assignee:
            Unassigned
            Reporter:
            Henry Weller
            None
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              None
              None
              None
              None