[SERVER-83756] [$vectorSearch, $project, $match] not passing documents when match depends on meta field from project Created: 30/Nov/23  Updated: 02/Feb/24

Status: In Code Review
Project: Core Server
Component/s: None
Affects Version/s: 6.0.11, 6.0.12
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Evan Nixon Assignee: Reilly McBride
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Query Integration
Operating System: ALL
Steps To Reproduce:
  1. Create a search index named vector-index with the provided index definition over a new collection.
  2. Insert the provided sample document to the indexed collection.
  3. Run the aggregation pipeline provided.
Sprint: QI 2023-12-25, QI 2024-01-08, QI 2024-01-22, QI 2024-02-05, QI 2024-02-19
Participants:

 Description   

This issue was noticed in https://jira.mongodb.org/browse/HELP-52233.

The following pipeline does not match documents when expected in 6.0.11 and 6.0.12. The pipeline does not error, and unexpectedly does not return any documents.

[
  {
    "$vectorSearch": {
      "index": "vector-index",
      "path": "vector",
      "queryVector": [
        1.1,
        2.1
      ],
      "numCandidates": 150,
      "limit": 10
    }
  },
  {
    "$project": {
      "vector": 0,
      "score": {
        "$meta": "vectorSearchScore"
      }
    }
  },
  {
    "$match": {
      "score": {
        "$gt": 0.19
      }
    }
  }
] 

When running the same pipeline with just the first [$vectorSearch, $project] stages, documents return that do have score values greater than 0.19.

Interestingly, it seems like this issue only occurs when the $project stage excludes a field. If the above pipeline removes the vector: 0 part of the $project stage, the pipeline returns the expected set of documents.

Sample Document

{
  "_id": {
    "$oid": "573a1390f29313caabcd42e8"
  },
  "fullplot": "Among the earliest existing films in American cinema - notable as the first film that presented a narrative story to tell - it depicts a group of cowboy outlaws who hold up a train and rob the passengers. They are then pursued by a Sheriff's posse. Several scenes have color included - all hand tinted.",
  "title": "The Great Train Robbery",
  "vector": [
    -0.11765024065971375,
    -0.21213145554065704,
    -0.2577494978904724,
    0.022029373794794083,
    0.18414953351020813,
    0.36586806178092957,
    0.11862823367118835,
    0.08321523666381836,
    0.29645663499832153,
    0.0022856774739921093,
    0.2824995219707489,
    0.29961642622947693,
    0.04239010438323021,
    0.4067825973033905,
    -0.1598043292760849,
    0.2148534059524536,
    0.1679493486881256,
    0.16969089210033417,
    0.3145444989204407,
    -0.07887133210897446,
    -0.18495561182498932,
    -0.09759827703237534,
    0.20752380788326263,
    0.0002660630561877042,
    0.025117194280028343,
    -0.3236020505428314,
    0.21704871952533722,
    -0.20958499610424042,
    -0.30887970328330994,
    -0.39369162917137146,
    -0.4541487991809845,
    -0.3006436824798584,
    0.2732269763946533,
    0.28519636392593384,
    -0.27733004093170166,
    0.09134820848703384,
    0.19789211452007294,
    0.16620713472366333,
    0.15451955795288086,
    -0.16677556931972504,
    -0.11001905798912048,
    0.2350681871175766,
    0.14352057874202728,
    0.08762780576944351,
    -0.14170658588409424,
    -0.2390274554491043,
    -0.06526391208171844,
    0.19815577566623688,
    0.11691268533468246,
    -0.10566674917936325,
    -0.01293251197785139,
    0.1028236523270607,
    0.0663963183760643,
    -0.15577580034732819,
    0.05088368430733681,
    -0.3613817095756531,
    0.035468459129333496,
    -0.2183608114719391,
    -0.07660580426454544,
    0.07465414702892303,
    -0.34323880076408386,
    -0.25135529041290283,
    -0.035666778683662415,
    -0.17220216989517212,
    0.0796608179807663,
    0.11376814544200897,
    -0.26657962799072266,
    -0.10785310715436935,
    0.0011672211112454534,
    0.060230884701013565,
    -0.059616390615701675,
    0.1091652512550354,
    -0.2163013517856598,
    -0.042512599378824234,
    -0.21879377961158752,
    -0.03120023012161255,
    0.26145562529563904,
    0.4099317491054535,
    -0.07657518982887268,
    -0.19645383954048157,
    0.1214444488286972,
    -0.15863333642482758,
    -0.21740862727165222,
    -0.09555281698703766,
    -0.16180285811424255,
    0.3358859717845917,
    -0.007481173612177372,
    0.0592496432363987,
    -0.45026442408561707,
    0.30858924984931946,
    -0.3005770444869995,
    -0.056945886462926865,
    0.3019435703754425,
    0.20108728110790253,
    0.21957458555698395,
    0.03857314586639404,
    0.20800574123859406,
    0.14485691487789154,
    0.1916215568780899,
    -0.0020670725498348475,
    0.3323781192302704,
    -0.3285926878452301,
    0.3763233721256256,
    -0.0013920333003625274,
    0.07155482470989227,
    -0.09482531994581223,
    -0.0708141177892685,
    -0.1718258559703827,
    -0.37243640422821045,
    -0.06825312227010727,
    0.16272546350955963,
    0.14173169434070587,
    -0.14848820865154266,
    0.15826699137687683,
    0.3839591443538666,
    -0.2125476896762848,
    0.07903873175382614,
    0.44199442863464355,
    -0.26676616072654724,
    0.09757755696773529,
    0.13383761048316956,
    0.14373788237571716,
    0.09867347031831741,
    -0.1315346658229828,
    -0.3013248145580292,
    -0.17324237525463104,
    0.21638785302639008,
    0.09092690795660019,
    -0.10350912064313889,
    -0.19982074201107025,
    -0.5378654599189758,
    -0.07663127779960632,
    0.3250696063041687,
    -0.07982280850410461,
    -0.3214959502220154,
    0.2379886358976364,
    -0.058239664882421494,
    0.30418604612350464,
    -0.2900577187538147,
    -0.22941656410694122,
    -0.08184462785720825,
    -0.023101476952433586,
    -0.153031125664711,
    0.1359335333108902,
    -0.1226024180650711,
    -0.1791592389345169,
    0.09988260269165039,
    -0.020172426477074623,
    -0.2852059304714203,
    0.45608699321746826,
    0.22607013583183289,
    -0.13433052599430084,
    -0.14698535203933716,
    -0.043917521834373474,
    -0.1707649827003479,
    -0.18633048236370087,
    -0.06523872166872025,
    0.16834892332553864,
    0.18413908779621124,
    0.2815704941749573,
    0.11103569716215134,
    0.30391865968704224,
    0.283029168844223,
    0.1416444331407547,
    -0.042800698429346085,
    -0.24971401691436768,
    0.23590309917926788,
    0.06522729992866516,
    0.08855345100164413,
    -0.05423424392938614,
    -0.2309979796409607,
    0.1957871913909912,
    -0.23140402138233185,
    0.18335404992103577,
    -0.13024133443832397,
    0.2774726152420044,
    -0.340467631816864,
    0.441804438829422,
    -0.276013046503067,
    0.12379983067512512,
    0.04399506747722626,
    -0.1829381287097931,
    -0.2591503858566284,
    -0.11711062490940094,
    -0.0632048100233078,
    -0.026645071804523468,
    0.18277552723884583,
    0.021059319376945496,
    0.3614908456802368,
    0.06206006184220314,
    -0.07163409888744354,
    -0.041581377387046814,
    -0.17677843570709229,
    -0.131730318069458,
    0.21672207117080688,
    0.38391438126564026,
    0.24665479362010956,
    0.06858744472265244,
    -0.10113944113254547,
    0.11653739959001541,
    -0.3127738833427429,
    -0.20196931064128876,
    -0.16854704916477203,
    0.20115606486797333,
    0.1781543344259262,
    -0.02193872258067131,
    0.1828240305185318,
    -0.14550761878490448,
    -0.2326435148715973,
    0.11454261094331741,
    0.297747939825058,
    0.022496163845062256,
    0.0538192093372345,
    0.3587631285190582,
    0.15320129692554474,
    -0.19066448509693146,
    0.015059860423207283,
    0.0029263466130942106,
    -0.06956949084997177,
    0.1699836105108261,
    -0.06845001876354218,
    0.0017190612852573395,
    0.3263634443283081,
    -0.07113509625196457,
    0.3319341540336609,
    0.010027533397078514,
    0.16950343549251556,
    -0.10483969748020172,
    -0.2998875379562378,
    -0.23045486211776733,
    -0.2758725881576538,
    -0.11985764652490616,
    -0.2686576545238495,
    -0.19980600476264954,
    -0.4138680100440979,
    -0.1077808067202568,
    -0.1492578089237213,
    0.13200101256370544,
    0.36830294132232666,
    -0.08478769659996033,
    0.033339790999889374,
    -0.002336981939151883,
    -0.03442233055830002,
    0.32943832874298096,
    0.03849964961409569,
    -0.24022988975048065,
    -0.10006731003522873,
    0.012672185897827148,
    0.06638535857200623,
    0.06642032414674759,
    0.07391534745693207,
    -0.24832631647586823,
    -0.22292757034301758,
    -0.00998215563595295,
    0.06770025193691254,
    0.0005948301986791193,
    0.11045613139867783,
    -0.013355172239243984,
    -0.3046444356441498,
    0.12517647445201874,
    0.21137303113937378,
    0.2320733666419983,
    0.12761768698692322,
    -0.14024242758750916,
    0.10619757324457169,
    0.004375155083835125,
    -0.2231045663356781,
    0.08461868017911911,
    -0.2950367331504822,
    -0.3721398711204529,
    0.17330145835876465,
    0.23854418098926544,
    -0.5307270288467407,
    0.012168901972472668,
    -0.008676702156662941,
    -0.11905895918607712,
    0.014401458203792572,
    -0.2830154299736023,
    -0.4244348108768463,
    -0.07601073384284973,
    -0.12438468635082245,
    -0.11757797747850418,
    0.3823363482952118,
    0.17712704837322235,
    -0.2841915786266327,
    0.09009560197591782,
    -0.21773451566696167,
    -0.35735413432121277,
    -0.019248757511377335,
    -0.05660978704690933,
    -0.4503512680530548,
    0.17601518332958221,
    -0.07592517882585526,
    -0.017740637063980103,
    0.2308356910943985,
    0.18754231929779053,
    -0.10002169758081436,
    0.19592846930027008,
    -0.08100283145904541,
    0.46763908863067627,
    -0.01018767710775137,
    -0.06349238753318787,
    -0.3370194435119629,
    -0.000698139367159456,
    0.00844316091388464,
    0.05779176577925682,
    -0.07074493914842606,
    0.242868572473526,
    0.3492843508720398,
    0.08455496281385422,
    -0.20705756545066833,
    -0.19167593121528625,
    0.13049881160259247,
    0.475999653339386,
    0.10838420689105988,
    0.014645983465015888,
    0.26804178953170776,
    -0.004578789696097374,
    -0.2304943948984146,
    -0.22115907073020935,
    -0.18102207779884338,
    0.3641524314880371,
    0.1879137009382248,
    -0.2001994103193283,
    0.22903770208358765,
    0.339185506105423,
    0.09152352064847946,
    0.3772352635860443,
    -0.359032541513443,
    -0.20067743957042694,
    -0.29734930396080017,
    0.08637548238039017,
    0.1874120682477951,
    -0.3613969087600708,
    -0.24671083688735962,
    0.3716210424900055,
    0.24372325837612152,
    -0.010151015594601631,
    0.038459960371255875,
    0.2505260705947876,
    0.11547160893678665,
    0.19965754449367523,
    0.02591020241379738,
    -0.11946342885494232,
    0.16225086152553558,
    -0.29998981952667236,
    -0.08956541121006012,
    -0.17742612957954407,
    0.04627487435936928,
    0.11556537449359894,
    -0.21252484619617462,
    0.18583796918392181,
    -0.2050047516822815,
    -0.04747660830616951,
    0.07976546138525009,
    -0.053681228309869766,
    -0.1431625485420227,
    -0.053266748785972595,
    0.2652992010116577,
    -0.3357309401035309,
    0.020267300307750702,
    -0.11011805385351181,
    -0.004514480009675026,
    -0.07781223952770233,
    0.26643383502960205,
    0.19736529886722565,
    -0.16440820693969727,
    0.03900334984064102,
    -0.07384948432445526,
    -0.23138369619846344,
    0.24710531532764435,
    -0.28062722086906433,
    -0.18227963149547577,
    0.2834342420101166,
    0.09329964965581894,
    -0.043751057237386703,
    0.021708974614739418,
    0.1445489078760147,
    -0.04934481903910637,
    -0.008936065249145031,
    -0.11593864858150482,
    -0.14795023202896118,
    0.3682711124420166,
    -0.10305444151163101
  ]
} 

Search Index Definition

{
  "mappings": {
    "dynamic": false,
    "fields": {
      "title": {
        "type": "token"
      },
      "vector": {
        "dimensions": 384,
        "similarity": "cosine",
        "type": "knnVector"
      }
    }
  }
} 

Versions Impacted

This issue was first raised in https://jira.mongodb.org/browse/HELP-52233.

The user experienced the issue on 6.0.11 and in the course of investigating the issue their cluster upgraded to 6.0.12. The issue seems present in at least 6.0.11 and 6.0.12. It is unclear if this impacts other versions.



 Comments   
Comment by Nupoor Kotasthane [ 30/Nov/23 ]

Also, I tried this out with $search as well and I could reproduce it. The issue doesn't seem specific to vector indexes.

Example pipeline and index demonstrating the issue:

[
  {
    "$search": {
      "index": "test-project",
      "equals": {
        "path": "vector",
        "value": 1.1
      }
    }
  },
  {
    "$project": {
      "vector": 0,
      "score": {
        "$meta": "searchScore"
      }
    }
  },
  {
    "$match": {
      "score": {
        "$gt": 0
      }
    }
  }
]

 

{
  "mappings": {
    "dynamic": true
  }
}

 
Example data document:

{
  "_id": 1,
  "vector": [
    1.1,
    2.1
  ]
}

Comment by Nupoor Kotasthane [ 30/Nov/23 ]

Hi henry.weller@mongodb.com , I did try adding a $project stage that excludes the vector field after the $match stage. And that seems to work. 

Comment by Henry Weller [ 30/Nov/23 ]

nupoor.kotasthane@mongodb.com if you were to project out the vector in a second project stage following the $match, does it still work? If so we'll need to update documentation until this bug is fixed to mention this workaround as quite a few folks have asked for post-filtering by score

 

Generated at Thu Feb 08 06:53:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.