[SERVER-12316] Bulk API downconversion does not return upsertedId with non-ObjectID _id Created: 10/Jan/14  Updated: 10/Dec/14  Resolved: 22/Jan/14

Status: Closed
Project: Core Server
Component/s: Shell
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Christian Amor Kvalheim Assignee: Randolph Tan
Resolution: Won't Fix Votes: 0
Labels: 26qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Operating System: ALL
Steps To Reproduce:

exports['Should correctly perform ordered upsert with custom _id'] = {
  // Add a tag that our runner can trigger on
  // in this case we are setting that node needs to be higher than 0.10.X to run
  requires: {serverType: 'Server'},
  
  // The actual test we wish to run
  test: function(configuration, test) {
    var db = configuration.newDbInstance({w:1}, {poolSize:1, auto_reconnect:false});
    db.open(function(err, db) {
      // Get the collection
      var col = db.collection('batch_write_ordered_ops_8');
      // Initialize the Ordered Batch
      var batch = col.initializeOrderedBulkOp();
 
      // Add some operations to be executed in order
      batch.find({_id:2}).upsert().updateOne({$set: {b:2}});
 
      // Execute the operations
      batch.execute(function(err, result) {
        // Check state of result
        test.equal(1, result.nUpserted);
        test.equal(0, result.nInserted);
        test.equal(0, result.nUpdated);
        test.equal(0, result.nModified);
        test.equal(0, result.nRemoved);
        
        var upserts = result.getUpsertedIds();
        test.equal(1, upserts.length);
        test.equal(0, upserts[0].index);
        test.equal(2, upserts[0]._id);
 
        // Finish up test
        db.close();
        test.done();
      });
    });
  }
}

Participants:

 Description   

When you perform an upsert with a custom _id the upserted field is not returned by GLE meaning the bulk api does not register it correctly as a upsert.

Beneath is the node.js test that replicates this behavior



 Comments   
Comment by Daniel Pasette (Inactive) [ 22/Jan/14 ]

After discussing with renctan and greg_10gen, there is no way to fix this in 2.5.x alone. This is issue is only present when connecting to a 2.4 mongod from a 2.6 mongo shell using the bulk API when doing an upsert on a non-ObjectID _id field and the impact is that the "upsertedId" is blank.

Comment by Christian Amor Kvalheim [ 14/Jan/14 ]

You are probably right about that but we have to at least detect the situation where updatedExisting == false and no upserted is in the GLE and update the nModified and nUpserted counts.

Comment by Randolph Tan [ 13/Jan/14 ]

Not sure if we want to go to that path. On the side note, the above patch does not cover cases where:

1. _id is in update doc:

update({ x: 1 }, { _id: 4 }, true)

2. _id is in $mod

// this example doesn't actually work in 2.4.8... doesn't upsert, and gle gives no errors.
// Saved by the bug? Upserts fine in master
update({ x: 1 }, { $set: { _id: 4 }}, true)

Comment by Christian Amor Kvalheim [ 13/Jan/14 ]

behavior can be emulated in the bulk api and it's what the node driver is doing. it uses the original op's _id field and returns it to emulate write commands for 2.4.

  // We have an upserted field (might happen with a write concern error)
  if(_result.upserted) {
    _results.upserted.push({
        index: _index
      , _id: _result.upserted
    })
  } else if(!_result.updatedExisting && _result.upserted == null && _op.q && _op.q._id) {
    _results.upserted.push({
        index: _index
      , _id: _op.q._id
    })    
  }

Comment by Randolph Tan [ 13/Jan/14 ]

This is a bug in v2.4 and is caused by:

https://github.com/mongodb/mongo/blob/r2.4.9/src/mongo/db/ops/update.h#L39-41

Furthermore, the LastError struct does not support storing non OID _id.

Just to clarify, this is only an issue when calling getLastError on mongod v2.4 and below.

Generated at Thu Feb 08 03:28:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.