[SERVER-85324] Change representation of empty arrays in cell blocks Created: 17/Jan/24 Updated: 07/Feb/24 Resolved: 07/Feb/24 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Foteini Alvanaki | Assignee: | Backlog - Query Execution |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
|||||||||||||||||||||||||||||||||||||||||||||||
| Assigned Teams: |
Query Execution
|
|||||||||||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | |||||||||||||||||||||||||||||||||||||||||||||||
| Steps To Reproduce: | Create a timeseries collection
Insert a few documents
Count the number of documents that have `a: {$eq: null} `
In master the above query returns 2 (a:null and a:[null, null]) while when we run it with block TypeMatch (PR with the code) it returns 3 (a:null , a:[null, null] and a: []). SBE Plan with block TypeMatch
SBE Plan with scalar TypeMatch
Query used to get the plans : `db.runCommand({explain: {count: "nT", query: { a: {$eq: null} }}})` |
|||||||||||||||||||||||||||||||||||||||||||||||
| Participants: | ||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
In block processing, an array is flattened and represented as a CellBlock (walk array code). When the array is empty since there are no other values we just add Nothing in the block representation. In typeMatch, when we find an element with tag Nothing we just return Nothing and it is the caller's responsibility to decide how to handle it. We have followed the same semantics in block typeMatch. When we query for documents that have null in a field fieldA (e.g. db.runCommand({count: "collection", query: { fieldA: {$eq: null} }})) , we use traverseF with typeMatch as the lambda and then we discard all documents for which typeMatch has returned false. In case fieldA is Nothing, typeMatch does not return false and the document satisfies the query. In case fieldA is an empty array, the scalar version does not run typeMatch, returns false and the document is filtered out. In block processing however, an empty arrays is represented as Nothing, block typeMatch processes it and returns Nothing and since it is not false the document is not filtered out. To test my assumption I changed the default value from Nothing to NumberInt32 and the document with the empty array did not satisfy the query in the block typeMatch. |