[SERVER-6865] positional operator projection does not work with find and modify Created: 27/Aug/12  Updated: 09/Jul/16  Resolved: 05/May/15

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Aaron Staple Assignee: Max Hirschhorn
Resolution: Duplicate Votes: 3
Labels: query_triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File server6865.js    
Issue Links:
Depends
Duplicate
duplicates SERVER-16063 Rewrite findAndModify Closed
Related
related to SERVER-6868 prevent use of positional projection ... Closed
related to DOCS-6107 Documenting positional operator behav... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Quint Iteration 3.1.2, Quint Iteration 3
Participants:

 Description   

The match details aren't passed in when find and modify calls transform() so positional operator projection doesn't work yet.

c = db.c;
c.drop();
 
c.save( { a:[ 1, 2 ] } );
 
// Does the expected projection.
printjson( c.find( { a:2 }, { 'a.$':1 } ).toArray() );
 
// Doesn't do the expected projection.
printjson( c.findAndModify( { query:{ a:2 }, fields:{ 'a.$':1 }, remove:true } ) );



 Comments   
Comment by J Rassi [ 05/May/15 ]

James: I'm afraid not. If you upgrade to 3.0, you'll have to change your application to project the full array, and come up with some way to emulate the positional projection.

Comment by James Simpson [ 05/May/15 ]

Are there any workarounds for this in 3.0? The fix we were using to add "sort: {}" no longer works in 3.0, which will end up preventing us from upgrading from 2.6 until 3.2 is released.

Comment by J Rassi [ 05/May/15 ]

Re-opened to close as a dup of SERVER-16063.

Comment by Max Hirschhorn [ 05/May/15 ]

This issue was resolved by the rewrite of the findAndModify command in SERVER-16063. The test cases attached to this ticket were incorporated into find_and_modify_server6865.js.

Positional projections are supported on the removed document, as well as the old version of the updated document.

> c.save({_id:1, counter:0, b:[{name:'first', value:1}, {name:'second', value:2}, {name:'third', value:3}]})
WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : 1 })
> c.find({b:{$elemMatch:{name:'first'}}}, {_id:0, counter:1, 'b.$':1})
{ "counter" : 0, "b" : [ { "name" : "first", "value" : 1 } ] }
> c.findAndModify({query:{b:{$elemMatch:{name:'first'}}}, update:{$inc:{counter:1, 'b.$.counter':2}}, fields:{_id:0, counter:1, 'b.$':1}, new:false})
{ "counter" : 0, "b" : [ { "name" : "first", "value" : 1 } ] }
> c.findAndModify({query:{b:{$elemMatch:{name:'first'}}}, remove:true, fields:{_id:0, counter:1, 'b.$':1}})
{ "counter" : 1, "b" : [ { "name" : "first", "value" : 1, "counter": 2 } ] }

Positional projections are still not supported on the new version of the updated document, and the server will respond with an appropriate error message.

> c.findAndModify({query:{b:{$elemMatch:{name:'first'}}}, update:{$inc:{counter:1, 'b.$.counter':2}}, fields:{_id:0, counter:1, 'b.$':1}, new:true})
2015-05-04T23:41:31.844-0400 E QUERY    Error: findAndModifyFailed failed: {
  "ok" : 0,
  "errmsg" : "cannot use a positional projection and return the new document",
  "code" : 2
}
    at Error (<anonymous>)
    at DBCollection.findAndModify (src/mongo/shell/collection.js:651:15)
    at (shell):1:3 at src/mongo/shell/collection.js:651

The positional information (i.e. the index into the array) recorded in the MatchDetails is based on the old version of the updated document. There is currently no mechanism to translate that offset for update operators such as $pull that can modify the index of what array element was matched by the query specification.

Since this ticket was originally opened with regard to positional projections on the document removed by the findAndModify command, I'm marking it as resolved. jim.oleary@10gen.com, feel free to open a new ticket for supporting positional projections on the new version of the updated document.

Comment by J Rassi [ 20/Feb/15 ]

Verified that the attached test fails on 3.0.0-rc8.

This issue should be resolved by SERVER-16063. Linking ticket as "related".

Comment by Vladimir [ 22/Sep/14 ]

In addition to James Simpson comment , if I add `new` option query fails with

"Can't canonicalize query: BadValue Positional projection 'names.$' does not match the query document."

where `names` is property name.

Comment by James Simpson [ 12/Aug/14 ]

Has there been any progress on this? We are running into this same issue and found that adding a sort to the findAndModify (even with just an empty object) causes the query to work correctly.

Comment by James O'Leary [ 06/Nov/13 ]

I'd like to add the following examples where findAndModify fields projection also seems to be broken for $ positional operator (and the record is being updated, not removed):

The following find shows what you would expect:

> c.save( { _id:1, counter : 0, b:[ {name:"first", value:1},{name:"second", value:2}, {name:"third", value:3 }] } );
> printjson( c.find( { b: {$elemMatch : {name:"first"}} }, { _id: 0, counter : 1 , 'b.$':1 } ).toArray() );
[
  {
    "counter": 0,
    "b": [
      {
        "name": "first",
        "value": 1
      }
    ]
  }
]

Running findAndModify with b.$ and new:true results in an array of empty elements (although the top level counter is returned correctly):

> printjson( c.findAndModify({ query:{ b: {$elemMatch : {name:"first"}} }, update:{$inc :{counter : 1, "b.$.counter":1}},fields: { _id:0, counter:1, 'b.$':1 }, new :true }) );
{
  "b": [
    {
      
    },
    {
      
    },
    {
      
    }
  ],
  "counter": 1
}
// running the find again, shows that the '"b.$.counter":1' update worked  
> printjson( c.find( { b: {$elemMatch : {name:"first"}} }, { _id: 0, counter : 1 , 'b.$':1 } ).toArray() );
[
  {
    "b": [
      {
        "counter": 1,
        "name": "first",
        "value": 1
      }
    ],
    "counter": 1
  }
]

Running findAndModify without the positional projection and new:true returns the new record as expected :

> printjson( c.findAndModify({ query:{ b: {$elemMatch : {name:"first"}} }, update:{$inc :{counter : 1, "b.$.counter":1}},new : true}) );
{
  "_id": 1,
  "b": [
    {
      "counter": 2,
      "name": "first",
      "value": 1
    },
    {
      "name": "second",
      "value": 2
    },
    {
      "name": "third",
      "value": 3
    }
  ],
  "counter": 2
}

Generated at Thu Feb 08 03:12:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.