[SERVER-42770] Ability to see if arrayFilters matched anything Created: 12/Aug/19  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Usability
Affects Version/s: 4.0.12
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Ben Rotz Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Execution
Participants:

 Description   

when using arrayFilters, it is not possible to determine if the full query (including arrayFilters) matched anything.

 

For example, given the document:

use test;
db.test.drop();
db.test.insert({
  name: 'A',
  parents: [
    {
      name: 'AA',
      children: [
        {
          name: 'AAA',
          count: 2
        }, {
          name: 'AAB',
          count: 2
        }
      ]
    }
  ]
});

Now run an update:

db.test.update({name: 'A'}, {
  $set: {
    'parents.$[parentMatch].children.$[childMatch].count': 1
  },
}, {
  arrayFilters: [
    {'parentMatch.name': {$eq: 'AA'}},
    {'childMatch.name': {$eq: 'AAA'}}
  ]
});

So far so good, now let's run another update:

db.test.update({name: 'A'}, {
  $set: {
    'parents.$[parentMatch].children.$[childMatch].count': 1
  },
}, {
  arrayFilters: [
    {'parentMatch.name': {$eq: 'AA'}},
    {'childMatch.name': {$eq: 'AAC'}}
  ],
  upsert: true
});

Has no effect. The upsert: true has no effect because it only applies to the match predicate in the first parameter. My request is not to change the behavior here, I'm just noting for discussion.

 

Running this query in production, it would be impossible to know if the update just simply didn't have nModified > 0 because the arrayFilter didn't match anything, or because the count value was already 1 and so no data modification occurred.

 

 

I can see that the value for nMatched needs to be 1 because it is applied to the document level, not the subdocument level (as arrayFilters can be used across multiple sub documents within a single document, it would be difficult to assert the meaning of this value if it took that into consideration).

 

I'm not sure what the best solution is, but I believe it is a shortcoming.

Perhaps there is additional information returned in writeResult.

Perhaps the update positional operator needs to be expanded so that the arrays deeper than 1 level can be updated. (I think this is a bad idea for other reasons).

 

I like arrayFilters, and it is very useful and extendible, but it is hard to work with when "upsert" functionality is desired.

 

Thank you for your consideration.

 

 

 

 



 Comments   
Comment by Danny Hatcher (Inactive) [ 19/Aug/19 ]

ben@ethika.com, thank you for the in-depth report as well as your workaround. I'll forward this along to our query team to validate whether we want to include something server-side to accomplish the same goal.

Comment by Ben Rotz [ 12/Aug/19 ]

OK, here's a potential answer to my this behavior. While it doesn't actually perform an upsert, at least it provides definitive feedback as whether or not nMatched happened.

This assumes that arrayFilters are being used only to update a single subdocument at a specific level within the main document. If you are using multiple arrayFilters to modify several subdocuments within a single document, obviously that will not work. You need to isolate the update down to just one, so that you can use nMatched in writeResult to determine if the operation matched an existing subdoc or not.

So, you use a combination of arrayFilters AND $elemMatch. Not the addition of the `parents: { $elemMatch...` section

 

db.test.update(
{
  name: 'A',
  parents: {
    $elemMatch: {
      name: 'AA',
      children: {
        $elemMatch: {
          name: 'AAC'
        }
      }
    }
  }
}, {
  $set: {
    'parents.$[parentMatch].children.$[childMatch].count': 1
  },
}, {
  arrayFilters: [
    {'parentMatch.name': {$eq: 'AA'}},
    {'childMatch.name': {$eq: 'AAC'}}
  ]
});

Because the match condition duplicates the conditions in the arrayFilter, nMatched will now correspond to those match conditions as well.

Ideally, there would be a way to just "upsert" the things arrayFilters, but I understand that is more complicated behavior and becomes extremely complicated once you start updating multiple subdocuments within a single subdocument.

 

Generated at Thu Feb 08 05:01:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.