Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-91730

Oplog documents do not always have _id field first

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • ALL
    • Hide

      We noticed this in a test that was running jstests/core/query/dbref/ref.js. I can reliably reproduce it on 6.0 and 7.0 by doing the same thing the test does.

      In the mongo shell:

      test> var other = {s: "other thing", n: 1}
test> db.otherthings.save(other)
WriteResult({ "nInserted" : 1 })
test> other.n++
1
test> db.otherthings.save(other)
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

      This generates the following oplog: note that in the insert, the _id is the first field in the o document, and in the second, it's the last field. If you select the document from the collection directly, you (of course) see the _id first.

      local> db.oplog.rs.find({ ns: 'test.otherthings' })
[
  {
    op: 'i',
    ns: 'test.otherthings',
    ui: UUID('411d1bc7-b792-4bec-a428-1fe4886e0577'),
    o: {
      _id: ObjectId('66748a8b8db3d26e101d5665'),
      s: 'other thing',
      n: 1
    },
    o2: { _id: ObjectId('66748a8b8db3d26e101d5665') },
    ts: Timestamp({ t: 1718913675, i: 3 }),
    t: Long('2'),
    v: Long('2'),
    wall: ISODate('2024-06-20T20:01:15.503Z')
  },
  {
    op: 'u',
    ns: 'test.otherthings',
    ui: UUID('411d1bc7-b792-4bec-a428-1fe4886e0577'),
    o: {
      s: 'other thing',
      n: 2,
      _id: ObjectId('66748a8b8db3d26e101d5665')
    },
    o2: { _id: ObjectId('66748a8b8db3d26e101d5665') },
    ts: Timestamp({ t: 1718913693, i: 1 }),
    t: Long('2'),
    v: Long('2'),
    wall: ISODate('2024-06-20T20:01:33.049Z')
  }
]
      Show
      We noticed this in a test that was running jstests/core/query/dbref/ref.js. I can reliably reproduce it on 6.0 and 7.0 by doing the same thing the test does. In the mongo shell: test> var other = {s: "other thing" , n: 1} test> db.otherthings.save(other) WriteResult({ "nInserted" : 1 }) test> other.n++ 1 test> db.otherthings.save(other) WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 }) This generates the following oplog: note that in the insert, the _id is the first field in the o document, and in the second, it's the last field. If you select the document from the collection directly, you (of course) see the _id first. local> db.oplog.rs.find({ ns: 'test.otherthings' }) [ { op: 'i' , ns: 'test.otherthings' , ui: UUID( '411d1bc7-b792-4bec-a428-1fe4886e0577' ), o: { _id: ObjectId( '66748a8b8db3d26e101d5665' ), s: 'other thing' , n: 1 }, o2: { _id: ObjectId( '66748a8b8db3d26e101d5665' ) }, ts: Timestamp({ t: 1718913675, i: 3 }), t: Long ( '2' ), v: Long ( '2' ), wall: ISODate( '2024-06-20T20:01:15.503Z' ) }, { op: 'u' , ns: 'test.otherthings' , ui: UUID( '411d1bc7-b792-4bec-a428-1fe4886e0577' ), o: { s: 'other thing' , n: 2, _id: ObjectId( '66748a8b8db3d26e101d5665' ) }, o2: { _id: ObjectId( '66748a8b8db3d26e101d5665' ) }, ts: Timestamp({ t: 1718913693, i: 1 }), t: Long ( '2' ), v: Long ( '2' ), wall: ISODate( '2024-06-20T20:01:33.049Z' ) } ]

      Sometimes, the oplog contains a document that is not in the same order that it is eventually stored in. For some replace events, the _id field is last in the oplog, but when it's saved to disk, the document is reordered so that the _id field is first.

      Max, in Slack, suggests that the update subsystem generates the oplog entry from the replacement as the client specified and not the change the update execution does.

      Because this affects the change stream in fullDocument mode, we had to work around this in mongosync's application code by adjusting the document to move the _id field first. We would like to remove this workaround!

            Assignee:
            colin.stolley@mongodb.com Colin Stolley
            Reporter:
            michael.mcclimon@mongodb.com Michael McClimon
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: