[SERVER-10123] Support Timestamp(0,0) server replace for update (mods) Created: 08/Jul/13  Updated: 10/Dec/14  Resolved: 11/Oct/13

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Scott Hernandez (Inactive) Assignee: Scott Hernandez (Inactive)
Resolution: Duplicate Votes: 6
Labels: timestamp, transformation, update
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
duplicates SERVER-10911 Add $currentDate update modifier Closed
is duplicated by SERVER-10113 Reodering of update clause causes tim... Closed
Related
related to SERVER-1650 Server Side Timestamps Closed
Backwards Compatibility: Fully Compatible
Participants:

 Description   

If a Timestamp(0,0) is sent via an update then do the server replacement of the value.

Currently this behavior is only supported for the first 2 fields during an insert operation, but not updates.



 Comments   
Comment by Scott Hernandez (Inactive) [ 30/Nov/13 ]

If you follow SERVER-10911 you will see that you can use the new $currentDate update modifier in 2.5.3 (dev release) and will be in the 2.6 stable release.

Comment by Jeremy [ 29/Nov/13 ]

What version / release will this feature be in?

Comment by Jeremy [ 11/Oct/13 ]

Great. Can't wait to try it out.

Comment by Scott Hernandez (Inactive) [ 11/Oct/13 ]

Yes, it is the same Timestamp type – http://bsonspec.org/#/specification

Comment by Jeremy [ 11/Oct/13 ]

Hey Scott,

Does $currentDate allow us to create the timestamp + unique increment like the Timestamp(0,0)?

Thanks,

Jeremy

Comment by Scott Hernandez (Inactive) [ 11/Oct/13 ]

We now have $currentDate which allows you to set either a date or timestamp on the server – see SERVER-10911

Comment by Jeremy [ 08/Aug/13 ]

Regarding this: "This doesn't do what you suggest in a sharded environment since you don't get monotonically increasing values over a sharded collection, since there are multiple servers doing writes. Each shard (a replica set) basically will have its own oplog, and ordering of writes local to that shard."

Actually, even in a shared environment, could this feature not still be used just by use the same shard key when incrementing is desired? Yes, it would only write to 1 place per key, but by using different keys for unrelated channels, you could still scale out writes.

Comment by Jeremy [ 30/Jul/13 ]

Thanks Scott,

Understood about sharding--I wouldn't expect sequencing to happen across multi-servers. In our case, there are many more readers than writers, so a single replica set with many secondaries works great...just that we need to improve on this model. I recently discovered that using the MongoTimestamp isn't exactly working 100% anyway since it can still be committed to the db out of order (rare but does happen).

See below where my 'ts' i value is not in order (87 got committed before 86).

{
"ts":

{ "t": 1374700873, "i": 88 }

,
"h": NumberLong("7696705560612586844"),
"v": 2,
"op": "i",
"ns": "rt01.notice",
"o": {
"ts":

{ "t": 1374700873, "i": 87 }

,
"vars":

{ ..object here. },
"_id": "0f882b762b6f9f00f76562caac6550ba"
}
}

{
"ts": { "t": 1374700873, "i": 89 },
"h": NumberLong("-6769405481799817919"),
"v": 2,
"op": "i",
"ns": "rt01.notice",
"o": {
"ts": { "t": 1374700873, "i": 86 },
"vars": { ..object here. }

,
"_id": "0252a912a919c7ebd4c7d0d5020c6e13"
}
}

Comment by Scott Hernandez (Inactive) [ 30/Jul/13 ]

Jeremy,

Replication no longer uses this. We will see if it will fit in the update refactoring or if another modifier is a better solution.

This doesn't do what you suggest in a sharded environment since you don't get monotonically increasing values over a sharded collection, since there are multiple servers doing writes. Each shard (a replica set) basically will have its own oplog, and ordering of writes local to that shard.

Feel free to collect votes and add ideas/feedback here.

Comment by Jeremy [ 29/Jul/13 ]

HOW ABOUT THIS IDEA THAT WOULD ELIMINATE THE MongoTimestamp HACK?

In the oplog, there is a 'ts'. Would it possible to just have a feature that will let us add this 'ts' to our own document instead of generating one with the null hack? If so, that would be a much better and more reliable solution.

Maybe a new operator called '$oplogTimestamp'.

db.test.save({ _id:'test', 'info':'insert and stamp', $oplogTimestamp:

{'ts':1}

);
db.test.find().pretty();
{
"_id": "test",
"info": "insert and stamp",
"ts":

{ "t": 1375116500, "i": 1 }

}

AND IN OPLOG

db.oplog.rs.find().sort({$natural:-1}).limit(1).pretty()
{
"ts" :

{ "t" : 1375116500, "i" : 1 }

,
"h" : NumberLong("8697630143468493520"),
"v" : 2,
"op" : "i",
"ns" : "mydb.test",
"o" : {
"_id" : "test",
"info" : "insert and stamp",
"ts" :

{ "t" : 1375116500, "i" : 1 }

}
}

AND IN UPDATES

db.test.update(

{ _id:'test'}

, { $oplogTimestamp:

{'tsu':1}

} );
db.test.find();
{
"_id": "test",
"info": "insert and stamp",
"ts":

{ "t": 1375116500, "i": 1 }

,
"tsu":

{ "t": 1375116850, "i": 3 }

}

Am I just dreaming here?

Comment by Jeremy [ 20/Jul/13 ]

Hey Scott.

While the notion that this is hack may be true, please consider for a moment why many of us want to use this hack. Just having a '$now' option isn't good enough for a polling mechanism since it would be possible for 2 or more records inserted at the same time to have the same timestamp. What is quite lovely about your MongoTimestamp is that it is guaranteed to be unique (at least in non-sharded setup). Also, it not only provides a centrally generated timestamp, but also gives an order (increment) to these inserted records. This lets me do some neat stuff. For example: if 'ts' is a MongoTimestamp, it lets us make a query such as 'get the next 10 records where 'ts' is greater than last 'ts' received->sort by 'ts'. On response, we can take the last 'ts' and use it for the next 'gte' polling. If we did this just on a time, we would consistently be skipping records (limit 10 gte a $now might give us 10 back but, there could be 3 more after with same timestamp that wouldn't be captured on next polling). Here is our how we use this in our API: http://rtrt.me/docs/api/rest#polling

If your Replication implementation needs this feature, why not let others use it for similar reasons. If you support this better, MongoDB could find many more uses...I can see it becoming a great backend store for a message bus.

A a regular server side timestamp might be OK if you could also provide us with a server-side auto-incrementing option. I've seen people talk about a findAndModify option for this, but doesn't that require some more intense locking? And, getting a timestamp from client isn't very good idea since clients are never 100% in time sync.

Finally, I'm not sure why the ticket says that currently this behavior is only supported for 'inserts' and not 'updates'. You can actually use the 'update' command, but you cannot do a '$set' of a field with this null timestamp and have it re-stamp the field. Being able to '$set' for this, and being able to use this on more than one field at a time is really all I am asking for.

Thanks!

Jeremy

Comment by Scott Hernandez (Inactive) [ 09/Jul/13 ]

Once we have SERVER-1650 ($now/$date) with a update modifier that can set the date/time on the server I'm not sure this is all that important and us just duplicated functionality – currently using a Timestamp is really a hack and relic from the replication implementation and not something meant to be surfaced or used outside of that.

Generated at Thu Feb 08 03:22:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.