[SERVER-82430] $slice (project) and $slice (aggregation) aggregation returns different results for an array nested in an array of subdocuments Created: 25/Oct/23  Updated: 09/Jan/24  Resolved: 09/Jan/24

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 7.0.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Thorsten Schodde Assignee: Foteini Alvanaki
Resolution: Works as Designed Votes: 0
Labels: aggregation-framework, bug, projection
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File repro.js    
Assigned Teams:
Query Execution
Operating System: ALL
Steps To Reproduce:

This can also be reproduced in the mongo-compas or via C# driver. 

In general, you can follow my description above. Insert the shown document and execute both projections to see the difference.

Sprint: QE 2024-01-08, QE 2024-01-22
Participants:

 Description   

Dear all,

I've the following use case: I've got a model, that includes an array of models that includes an array of models. Here is an example:

 

{
  "_id": {
    "$oid": "6537c348acf272384df46220"
  },
  "MyBoundModelList": [
    {
      "MyInt8": 50,
      "MyBoundModelList": [
        {
          "MyString": "0",
          "MyInt32": 0
        },
        {
          "MyString": "1",
          "MyInt32": 1
        },
        {
          "MyString": "2",
          "MyInt32": 2
        },
        {
          "MyString": "3",
          "MyInt32": 3
        }
      ]
    },
    {
      "MyInt8": 33,
      "MyBoundModelList": [
        {
          "MyString": "0",
          "MyInt32": 0
        },
        {
          "MyString": "1",
          "MyInt32": 1
        },
        {
          "MyString": "2",
          "MyInt32": 2
        }
      ]
    },
    {
      "MyInt8": 41,
      "MyBoundModelList": [
        {
          "MyString": "0",
          "MyInt32": 0
        },
        {
          "MyString": "1",
          "MyInt32": 1
        },
        {
          "MyString": "2",
          "MyInt32": 2
        }
      ]
    },
    {
      "MyInt8": 16,
      "MyBoundModelList": [
        {
          "MyString": "0",
          "MyInt32": 0
        },
        {
          "MyString": "1",
          "MyInt32": 1
        },
        {
          "MyString": "2",
          "MyInt32": 2
        },
        {
          "MyString": "3",
          "MyInt32": 3
        }
      ]
    }
  ]
} 

My goal was to slice the inner list to return just the first element. To achiev this I initially used a "normal" projection: 

{"MyBoundModelList.MyBoundModelList": {$slice:[0,1]}} 

This resulted in the following entity, which, in my opinion, is correct:

{
  "_id": {
    "$oid": "6537c348acf272384df46220"
  },
  "MyBoundModelList": [
    {
      "MyInt8": 50,
      "MyBoundModelList": [
        {
          "MyString": "0",
          "MyInt32": 0
        }
      ]
    },
    {
      "MyInt8": 33,
      "MyBoundModelList": [
        {
          "MyString": "0",
          "MyInt32": 0
        }
      ]
    },
    {
      "MyInt8": 41,
      "MyBoundModelList": [
        {
          "MyString": "0",
          "MyInt32": 0
        }
      ]
    },
    {
      "MyInt8": 16,
      "MyBoundModelList": [
        {
          "MyString": "0",
          "MyInt32": 0
        }
      ]
    }
  ]
}

However, since I had some truble in combination with other include projections, I had to switch to aggregation pipelines to be able to send multiple projections that are applied after each other. Thus, I converted the "normal" projection definition into a pipeline:

new BsonDocument("$project"
    new BsonDocument("MyBoundModelList.MyBoundModelList"
    new BsonDocument("$slice"
    new BsonArray
                {
                    "$MyBoundModelList.MyBoundModelList",
                    0,
                    1
                })))

After executing this pipeline I saw the first different: It just returns the named lists on the path. Well, this was not expected since the "normal" slice projection acts differently but can be handled. The real "bug", at least in my opinion, happens in the inner list. Here the result of that projection:

{
  "_id": {
    "$oid": "6537c348acf272384df46220"
  },
  "MyBoundModelList": [
    {
      "MyBoundModelList": [
        [
          {
            "MyString": "0",
            "MyInt32": 0
          },
          {
            "MyString": "1",
            "MyInt32": 1
          },
          {
            "MyString": "2",
            "MyInt32": 2
          },
          {
            "MyString": "3",
            "MyInt32": 3
          }
        ]
      ]
    },
    {
      "MyBoundModelList": [
        [
          {
            "MyString": "0",
            "MyInt32": 0
          },
          {
            "MyString": "1",
            "MyInt32": 1
          },
          {
            "MyString": "2",
            "MyInt32": 2
          },
          {
            "MyString": "3",
            "MyInt32": 3
          }
        ]
      ]
    },
    {
      "MyBoundModelList": [
        [
          {
            "MyString": "0",
            "MyInt32": 0
          },
          {
            "MyString": "1",
            "MyInt32": 1
          },
          {
            "MyString": "2",
            "MyInt32": 2
          },
          {
            "MyString": "3",
            "MyInt32": 3
          }
        ]
      ]
    },
    {
      "MyBoundModelList": [
        [
          {
            "MyString": "0",
            "MyInt32": 0
          },
          {
            "MyString": "1",
            "MyInt32": 1
          },
          {
            "MyString": "2",
            "MyInt32": 2
          },
          {
            "MyString": "3",
            "MyInt32": 3
          }
        ]
      ]
    }
  ]
} 

The crucial part is the inner list. There, the projection inserted additional list-brakets instead of "slicing away" elements. The result is that the inner list has just one element, as expected, but this element is the original list with all elements. 

 



 Comments   
Comment by Foteini Alvanaki [ 09/Jan/24 ]

thorsten.schodde@bst.group the slice in projection was easy to mis-use. The slice in aggregation is more clear. It gets an array and slices it, it does not do projections.

 

Since this is not a bug I am going to close to this ticket. Please open a new ticket if there are any other issues.

Comment by Thorsten Schodde [ 08/Jan/24 ]

Dear foteini.alvanaki@mongodb.com ,

then I raise the question: Why does the slice aggregation work differently as the slice projection? This is a point that confuses us a bit. The slice project seem to be able to handle it "correctly". 

Thanks in advance!

Thorsten

Comment by Foteini Alvanaki [ 08/Jan/24 ]

Happy New Year thorsten.schodde@bst.group ! The slice in aggregation does not distinguish arrays-of-arrays from arrays-of-objects. So, the result you are getting is expected and not a bug. 

Comment by Thorsten Schodde [ 04/Jan/24 ]

Dear Foteini,

thanks for your help! I will test that, especially in the "general case". Our models are not fixed, meaning, we can have as many lists in lists as required. For that the point-notation was perfect! The mongo-driver or even the mongo itself resolved everything and did the map on its own on every hierarchy-level. I will check how flexible this map projection is.

But in general: I guess you will still fix the bug i reported, right? 

Thanks in advance and kind regards,

Thorsten 

Comment by Foteini Alvanaki [ 20/Dec/23 ]

Hi thorsten.schodde@bst.group ,

in order to get the result you want in aggregation you should use $map which applies an expression to every item in an array. For your example dataset the query should be

 

{$project: {
  "MyBoundModelList": {
    $map: {
      input: "$MyBoundModelList",
      as: "lout",
      in: {"MyInt8": "$$lout.MyInt8",  "MyBoundModelList" : {$slice: ["$$lout.MyBoundModelList", 0, 1]}}
    }
  }
}}
 

 

Comment by Edwin Zhou [ 18/Dec/23 ]

Hi thorsten.schodde@bst.group,

Thank you for your patience while we investigate this issue. I've been able to reproduce this behavior on my end and will pass this forward to the query team for further investigation.

repro.js

Kind regards,
Edwin

Comment by Thorsten Schodde [ 01/Dec/23 ]

Dear all,

just let me know if you need further information or help in reproducing the issue. 

Is there any plan how long this will take? We are currently blocked in upgrading to a newer Mongo-Server version since this bug will result in a crucial lose of functionality in our applications. 

Thanks in advance!

Kind regards,

Thorsten Schodde

Generated at Thu Feb 08 06:49:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.