[SERVER-55158] Missing objects in request rersponse. Created: 11/Mar/21 Updated: 04/May/21 Resolved: 04/May/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Querying |
| Affects Version/s: | 3.6.22 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Ivan Maireaux | Assignee: | Eric Sedor |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Participants: |
| Description |
|
Hello,
I have an issue when querying a collection using this criteria : {deleted_at: null, event_campaign_optin: true, event_id: ObjectId('5ea92da4a5522a080614d29f'), status: "registered", "label_ids": [ObjectId('6048a3c4b3d10c003c14b22e')]}=> It returns only one Object (instead of 5)
So I tried to add a filter on the ids of expected objects : _id: {"$in": [ObjectId('60377e2172fcdc005b32e9ff'), ObjectId('5efdaeafa62e5400233657a0'), ObjectId('5efca695f820c7144e46ffda'), ObjectId('5ee0cebdb28593001c8bfaf2'), ObjectId('5ed65cd396f090255d089b77')]}
The complete query : {deleted_at: null, event_campaign_optin: true, event_id: ObjectId('5ea92da4a5522a080614d29f'), status: "registered", "label_ids": [ObjectId('6048a3c4b3d10c003c14b22e')], _id: {"$in": [ObjectId('60377e2172fcdc005b32e9ff'), ObjectId('5efdaeafa62e5400233657a0'), ObjectId('5efca695f820c7144e46ffda'), ObjectId('5ee0cebdb28593001c8bfaf2'), ObjectId('5ed65cd396f090255d089b77')]}} => This one returns 5 objects
I do not understand how adding a AND filter on 5 ids could return more results ... I executed this requests on the Atlas interface, same issue form mongoid ODM form the ruby console.
Any idea of what happen ? I have no clue. I thought that the index used for the request could change the result, one request using a sparse index maybe ... so I forced the index to use with : .extras(hint: {deleted_at: 1, event_id: 1, created_at: -1}) (from the ruby console this time, not atlas) And still the same results.
Any clue is welcome. Thanks for your help. |
| Comments |
| Comment by Eric Sedor [ 04/May/21 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Thanks ivan.maireaux@eventmaker.io, I'm going to close this ticket but feel free to reach back out if needed. | |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Ivan Maireaux [ 03/May/21 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Hi Eric,
As the 3.6 reached end of life, we upgraded recently to 4.4 and effectively I can't reproduce anymore.
Thank you very much for your help ! | |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Eric Sedor [ 27/Apr/21 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Hi ivan.maireaux@eventmaker.io and thanks for your patience. I don't have specific information about other possibilities. It was simply my intention to not rule anything out. I've had a chance to review the latest output you provided. If the document with id: ObjectId('605b147862f1bc005ca7bacd') can be found in the _id index but not in the event_id_1_thematics_quartile_1 index, it does suggest the event_id_1_thematics_quartile_1 index doesn't have an entry for the document. I believe that in this case the problem could be related to the size of the thematics_quartile field. In not_returned_document_that_should_havve_been.txt, this looks to be a subdocument with about 43 subfields, like:
This looks like it would be subject to the index key limitation. Can you provide the output of validate on the collection and also compare the other documents you expect to match the query? Do they also have large thematics_quartile subdocuments? Importantly, since MongoDB 3.6 is reaching end of life at the end of this month (April 2021), it will help if you can reproduce the issue on MongoDB 4.0 and rule out the possibility that the data and indexes were created on 3.6 or an earlier version of MongoDB. Is this feasible? | |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Ivan Maireaux [ 07/Apr/21 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Hi Eric, As the queries do not work, our customer has updated to use some other fields and launch his campaign so I can't reproduce the initial issue.
But we had an other similar case today. I uploaded 4 files using the provided link (https://10gen-httpsupload.s3.amazonaws.com/upload_forms/77adcadf-ab72-4d61-9f06-12662007c3b5.html)
In your first answer you wrote : "checking results against different indexes does sound like a good first step. But, there may be other possibilities" Could you tell me more about these other possibilities ? | |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Eric Sedor [ 25/Mar/21 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
HI ivan.maireaux@eventmaker.io; Apologies, I should have asked for explain("executionStats") as it will include more information to help us infer what's being seen in the indexes. Could you re-run those? Totally understood about the sensitivity documents in question. We've a couple of options. Could you either:
| |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Ivan Maireaux [ 16/Mar/21 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Hi Eric, I cannot provide a full content of a document as it contains personal data about people. So I displayed fields that are present into the criteria selector => documents_content-1.txt Query with wrong results explain => wrong_results_query_explain-1.txt Query with filter on expected ids (results ok) explain => query_ok_explain-1.txt
Last week 1 guest was returned instead of 5, today I have 0 objects returned. No update on any of these documents since 2021-03-10 12:26:35 UTC
| |||||||||||||||||||||||||||||||||||||||||||||
| Comment by Eric Sedor [ 15/Mar/21 ] | |||||||||||||||||||||||||||||||||||||||||||||
|
Hi ivan.maireaux@eventmaker.io, checking results against different indexes does sound like a good first step. But, there may be other possibilities, so I'd like to ask for more information. Can you provide:
|