[SERVER-75095] Avoid generating extra stages for Nebari shard filter Created: 21/Mar/23  Updated: 29/Oct/23  Resolved: 30/Mar/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Ivan Fefer Assignee: Ivan Fefer
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
Related
Backwards Compatibility: Fully Compatible
Sprint: QE 2023-04-03
Participants:
Linked BF Score: 35

 Description   

Currently we generate 4 extra stages for shard filter in Nebari.

We have all the builtins we need to have a single filter stage.
Also some type checks can be avoided if there is only a single stage.



 Comments   
Comment by Githook User [ 29/Mar/23 ]

Author:

{'name': 'Ivan Fefer', 'email': 'ivan.fefer@mongodb.com', 'username': 'Fefer-Ivan'}

Message: SERVER-75095 Generate single stage for Nebari shard filter
Branch: master
https://github.com/mongodb/mongo/commit/b9cbc751fc5ee4ea921b51590c768e9d7503e748

Comment by Ivan Fefer [ 21/Mar/23 ]

Example of a new shard filter with dotted fields 

[2] filter {
    let [
        l1.0 =
            let [
                l2.0 = (getField(s7, "x") ?: null)
            ]
            in
                if isArray(l2.0)
                then l2.0
                else (getField(l2.0, "x") ?: null)
    ]
    in shardFilter(s6,
        if isArray(l1.0)
        then Nothing
        else makeBsonObj(MakeObjSpec(drop, [], ["x.x"]), Nothing, l1.0)
   )
}
[1] scan s7 s8 none none none none [] @"3c709eda-5bf9-4718-b500-a2ff6b8c9414" true false 

Comment by Ivan Fefer [ 21/Mar/23 ]

Example of shard filter, generated by current master 

[2] filter {shardFilter(s6, s12)}
[2] project [s12 =
    if (!(exists(s9)) || !(exists(s10)))
    then Nothing
    else s11
]
[2] mkbson s11 [_id = s9, i = s10] true false
[2] project [s9 =
    let [
        l1.0 = (getField(s7, "_id") ?: null)
    ]
    in
        if isArray(l1.0)
        then Nothing
        else l1.0
, s10 =
    let [
        l2.0 = (getField(s7, "i") ?: null)
    ]
    in
        if isArray(l2.0)
        then Nothing
        else l2.0
]
[1] scan s7 s8 none none none none [] @"6e230ff9-a281-4224-83d5-987e58a65521" true false 

Same shard filter from my branch

[2] filter {
    let [
        l1.0 = (getField(s7, "_id") ?: null)
        l1.1 = (getField(s7, "i") ?: null)
    ]
    in shardFilter(s6,
        if (isArray(l1.0) || isArray(l1.1))
        then Nothing
        else makeBsonObj(MakeObjSpec(drop, [], ["_id", "i"]), Nothing, l1.0, l1.1)
   )
}
[1] scan s7 s8 none none none none [] @"d5b11fec-b7ce-4af3-a7c4-c68a673d321a" true false 

Generated at Thu Feb 08 06:29:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.