[SERVER-9787] Oplog translates 'pushAll' to 'set' Created: 27/May/13  Updated: 28/May/13  Resolved: 27/May/13

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 2.4.3
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Brandon Mason Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

OSX, Ubuntu


Operating System: ALL
Steps To Reproduce:

1. chats.insert(

{messages:[]}

)
2. chats.update(

{ _id: ObjectId("51a0249adae44c01b0000003") }

, { '$pushAll': { messages: [

{message: 'hey'}

] } }
3. use local
4. rs.oplog.find()

Participants:

 Description   

I'm watching the oplog for use within my application. I'm noticing some difference in behavior between 2.2.0 and 2.4.3. This is the query that's being run:

chats.update(

{ _id: ObjectId("51a0249adae44c01b0000003") }

) { '$pushAll': { messages: [

{ _id: ObjectId("51a3dc72aead18550a000003"), timestamp: new Date("Mon, 27 May 2013 22:21:38 GMT"), username: 'Bob', message: 'hey' }

] } } {}

It is generated by Mongoose as the result of pushing to a subdocument and calling model.save().

In Mongo 2.2.0 this produces the following oplog entry:

https://gist.github.com/bitmage/5659191

While in Mongo 2.4.3 it produces this:

https://gist.github.com/bitmage/5659209

So it seems the 'pushAll' is being transformed into a 'set' which pushes out the contents of the entire Array.

Can someone familiar with this change comment on the reason for it and if this will be changing back? Definitely doesn't seem good for my case... this would force my application to figure out the delta, and it also results in a lot of unnecessary data being sent around.



 Comments   
Comment by Scott Hernandez (Inactive) [ 28/May/13 ]

The oplog does not capture only the information you are interested in so it is probably not the best tool.

See these related issues to watch/vote-up: SERVER-124, SERVER-3385

Comment by Brandon Mason [ 28/May/13 ]

Thanks Scott, I understand the direction better now. I am currently relying upon the oplog to monitor the state of my DB models, so it sounds like I may want to transition to a different strategy. I suppose I'll look into creating my own out-of-band pubsub for this purpose.

Is there a supported API for triggers on the Mongo roadmap?

Comment by Scott Hernandez (Inactive) [ 27/May/13 ]

This change is required for the oplog entry to be idempotent and fixes issues around corner cases during application, and the initial sync process.

The format of the oplog entries may change as improvements are made so please consider their format internal to the replication system. If you are reading the oplog the only parts which are user consumable should be the operation type and _id. All other data is internal and subject to change at any time.

If you want to keep a changelog of your data it is best this be done outside of the oplog (and replication) as its only requirement is consistent data across replicas.

Generated at Thu Feb 08 03:21:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.