[SERVER-39511] $setOnInsert does not replace empty timestamp with current timestamp Created: 11/Feb/19 Updated: 06/Dec/22 Resolved: 29/Jun/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Write Ops |
| Affects Version/s: | 3.6.2, 4.0.6 |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Simon Yarde | Assignee: | Backlog - Query Team (Inactive) |
| Resolution: | Won't Do | Votes: | 2 |
| Labels: | query-44-grooming | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Query
|
| Participants: |
| Description |
|
$setOnInsert has different behaviour to $insert, in that MongoDB server does not replace an empty timestamp with the current timestamp value.
|
| Comments |
| Comment by Asya Kamsky [ 29/Jun/19 ] | ||||||||||||
|
You can see some examples here. 4.2 also supports aggregation variables "$$NOW" and "$$CLUSTER_TIME" which can be referenced in your pipeline updates. | ||||||||||||
| Comment by Simon Yarde [ 27/Mar/19 ] | ||||||||||||
|
@daniel.moqvist%40knowit.se I agree this is an entirely reasonable requirement to have of a database like MongoDB. Hopefully this will be addressed soon with the proposed current-time expression. Whilst I don’t fully understand your use-case, it seems that handling failed updates with an insert might work; especially if the volume of updates is greater than that of inserts, and the through-put performance is acceptable. | ||||||||||||
| Comment by daniel moqvist [ 27/Mar/19 ] | ||||||||||||
|
ok, we really need to have an unique ordered "version field" so the "data customers" can get all data that is created since last time they fetched data. $setOnInsert and an empty Timestamp would be great to use for this scenario, so we can do one atomic upsert instead of first do a find to see if we should do an insert or an update. | ||||||||||||
| Comment by Simon Yarde [ 26/Mar/19 ] | ||||||||||||
|
@daniel.moqvist%40knowit.se ObjectId's are not monotonic (do not preserve write order), but they are unique and contain a timestamp with a resolution of one second. ObjectId does "guarantee a unique timestamp on each document"; it's just that your documents will not be totally ordered. So, whilst Timestamp does create a unique and totally ordered temporal event, it does not provide any more time-resolution than ObjectId. Both ObjectId and Timestamp provide a time-resolution of one second; it's just that ObjectId achieves uniqueness through some semi-randomness, whilst Timestamp achieves uniqueness with a counter that increments on each server write operation in any given second. Otherwise, the only current option using Timestamp is to separate the findAndModify into an initial write operation and subsequent update operations, and handle any errors. | ||||||||||||
| Comment by daniel moqvist [ 23/Mar/19 ] | ||||||||||||
|
Is the timestamp of objectid really guaranteed to be write ordered? The documentation says: IMPORTANT While ObjectId values should increase over time, they are not necessarily monotonic. This is because they:
https://docs.mongodb.com/manual/reference/bson-types/#objectid | ||||||||||||
| Comment by Simon Yarde [ 22/Mar/19 ] | ||||||||||||
|
@asya I need causal ordering. Great, this is now picked up in epic 'Expression to get current time'. @daniel.moqvist%40knowit.se ObjectId's are unique and contain a timestamp component. Timestamp helps when you need to preserve the order of writes. | ||||||||||||
| Comment by daniel moqvist [ 14/Mar/19 ] | ||||||||||||
|
We have multiple clients and need to guarantee an unique timestamp on each document. The timestamp type seems perfect for creating unique timestamp fields. | ||||||||||||
| Comment by Asya Kamsky [ 14/Mar/19 ] | ||||||||||||
|
> $setOnInsert: {"ts": new Timestamp()} Why not use timestamp passed in from client? | ||||||||||||
| Comment by daniel moqvist [ 14/Mar/19 ] | ||||||||||||
|
I'm trying to do an atomic "find or create" operation to set a unique timestamp in documents created. But if the document already existed it should just return the "old" document. I've tried the following command:
But unfortunately $setOnInsert does not support to translate Timestamp to current time. I cannot figure out how to use $currentDate either since the documentation says it is an update command. | ||||||||||||
| Comment by Asya Kamsky [ 25/Feb/19 ] | ||||||||||||
|
simony can you explain details of the use case - specifically, is there a reason that you don't want to (or cannot) use a client generated time? And as far as the format of the field, must it be Timestamp - or could it be ISODate format?
| ||||||||||||
| Comment by Simon Yarde [ 12/Feb/19 ] | ||||||||||||
|
Hi Danny, Can you retitle this as a feature request to “Set Current Date On Insert for Upsert Ops”? On reflection I see this is not a bug but a gap, and it is not possible to alter the behaviour of $set or $setOnInsert as it prevents setting an empty timestamp. Can’t help thinking it would have been better to have a document for the initial insert, and insert rather than set semantics. Best Regards Simon | ||||||||||||
| Comment by Simon Yarde [ 12/Feb/19 ] | ||||||||||||
|
Hi Danny I saw that issue, but as you say it doesn’t address the same behaviour, because only the $set operator has an upsert equivalent $setOnInsert. The equivalent operator required would seem to be $currentDateOnInsert. I understand the current approach is ‘set on insert’ rather than ‘insert if not exists’, and from this perspective it follows that set does not support timestamp replacement, and that is entirely logical. However, this does rather seem to leave a hole in what intuitively seems possible from an upsert op. Best Regards Simon | ||||||||||||
| Comment by Danny Hatcher (Inactive) [ 11/Feb/19 ] | ||||||||||||
|
Hello Simon, This issue was raised in the past in Thank you, Danny |