[SERVER-39511] $setOnInsert does not replace empty timestamp with current timestamp Created: 11/Feb/19  Updated: 06/Dec/22  Resolved: 29/Jun/19

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: 3.6.2, 4.0.6
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Simon Yarde Assignee: Backlog - Query Team (Inactive)
Resolution: Won't Do Votes: 2
Labels: query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query
Participants:

 Description   

$setOnInsert has different behaviour to $insert, in that MongoDB server does not replace an empty timestamp with the current timestamp value.

 



 Comments   
Comment by Asya Kamsky [ 29/Jun/19 ]

SERVER-40381 implemented support for aggregation expressions to specify update for 4.2. 

You can see some examples here.

4.2 also supports aggregation variables "$$NOW" and "$$CLUSTER_TIME" which can be referenced in your pipeline updates.

Comment by Simon Yarde [ 27/Mar/19 ]

@daniel.moqvist%40knowit.se

I agree this is an entirely reasonable requirement to have of a database like MongoDB. Hopefully this will be addressed soon with the proposed current-time expression.

Whilst I don’t fully understand your use-case, it seems that handling failed updates with an insert might work; especially if the volume of updates is greater than that of inserts, and the through-put performance is acceptable.

Comment by daniel moqvist [ 27/Mar/19 ]

ok, we really need to have an unique ordered "version field" so the "data customers" can get all data that is created since last time they fetched data. $setOnInsert and an empty Timestamp would be great to use for this scenario, so we can do one atomic upsert instead of first do a find to see if we should do an insert or an update.

Comment by Simon Yarde [ 26/Mar/19 ]

@daniel.moqvist%40knowit.se

ObjectId's are not monotonic (do not preserve write order), but they are unique and contain a timestamp with a resolution of one second. ObjectId does "guarantee a unique timestamp on each document"; it's just that your documents will not be totally ordered.

So, whilst Timestamp does create a unique and totally ordered temporal event, it does not provide any more time-resolution than ObjectId. Both ObjectId and Timestamp provide a time-resolution of one second; it's just that ObjectId achieves uniqueness through some semi-randomness, whilst Timestamp achieves uniqueness with a counter that increments on each server write operation in any given second.

Otherwise, the only current option using Timestamp is to separate the findAndModify into an initial write operation and subsequent update operations, and handle any errors.

Comment by daniel moqvist [ 23/Mar/19 ]

Is the timestamp of objectid really guaranteed to be write ordered? The documentation says: 

IMPORTANT

While ObjectId values should increase over time, they are not necessarily monotonic. This is because they:

  • Only contain one second of temporal resolution, so ObjectId values created within the same second do not have a guaranteed ordering

https://docs.mongodb.com/manual/reference/bson-types/#objectid

Comment by Simon Yarde [ 22/Mar/19 ]

@asya I need causal ordering.

Great, this is now picked up in epic 'Expression to get current time'.

@daniel.moqvist%40knowit.se ObjectId's are unique and contain a timestamp component. Timestamp helps when you need to preserve the order of writes.

Comment by daniel moqvist [ 14/Mar/19 ]

We have multiple clients and need to guarantee an unique timestamp on each document. The timestamp type seems perfect for creating unique timestamp fields.

Comment by Asya Kamsky [ 14/Mar/19 ]

> $setOnInsert: {"ts": new Timestamp()}

Why not use timestamp passed in from client?

Comment by daniel moqvist [ 14/Mar/19 ]

I'm trying to do an atomic "find or create" operation to set a unique timestamp in documents created. But if the document already existed it should just return the "old" document. I've tried the following command:

db.getCollection("test").findAndModify({
	query: {
		"hash": 1
	},
	update: {
		$setOnInsert: {
			"ts": new Timestamp()
		}
	},
	new: true,
	upsert: true
})

But unfortunately $setOnInsert does not support to translate Timestamp to current time.

I cannot figure out how to use $currentDate either since the documentation says it is an update command.

Comment by Asya Kamsky [ 25/Feb/19 ]

simony can you explain details of the use case - specifically, is there a reason that you don't want to (or cannot) use a client generated time?  And as far as the format of the field, must it be Timestamp - or could it be ISODate format?

 

Comment by Simon Yarde [ 12/Feb/19 ]

Hi Danny,

Can you retitle this as a feature request to “Set Current Date On Insert for Upsert Ops”?

On reflection I see this is not a bug but a gap, and it is not possible to alter the behaviour of $set or $setOnInsert as it prevents setting an empty timestamp.

Can’t help thinking it would have been better to have a document for the initial insert, and insert rather than set semantics.

Best Regards

Simon

Comment by Simon Yarde [ 12/Feb/19 ]

Hi Danny

I saw that issue, but as you say it doesn’t address the same behaviour, because only the $set operator has an upsert equivalent $setOnInsert.

The equivalent operator required would seem to be $currentDateOnInsert.

I understand the current approach is ‘set on insert’ rather than ‘insert if not exists’, and from this perspective it follows that set does not support timestamp replacement, and that is entirely logical.

However, this does rather seem to leave a hole in what intuitively seems possible from an upsert op.

Best Regards

Simon

Comment by Danny Hatcher (Inactive) [ 11/Feb/19 ]

Hello Simon,

This issue was raised in the past in SERVER-10123 but it was determined at the time that the use case should be covered by the $currentDate operator. However, I will forward this on to our Query team to take a look.

Thank you,

Danny

Generated at Thu Feb 08 04:52:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.