[DOCS-12300] Further Specification of BSON Timestamp Guarantees Created: 19/Dec/18  Updated: 30/Oct/23  Resolved: 15/Sep/20

Status: Closed
Project: Documentation
Component/s: bsonspec, Server
Affects Version/s: None
Fix Version/s: Server_Docs_20231030

Type: Improvement Priority: Minor - P4
Reporter: Joël Gähwiler Assignee: Unassigned
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:
Days since reply: 3 years, 21 weeks, 1 day ago

 Description   

Description

The BSON Timestamp type, and the feature that it is set by the server with a sequential id upon insertion, has very interesting properties:

It could be used to let MongoDB generate an ordered sequence of timestamps when inserting documents in a collection (similar to the internal oplog). Later the collection can be sorted on the timestamp in order to restore the order in which documents are inserted. Additionally, one might also watch the collection using change streams (insert events) and further process the inserts in order. Simply said, it would allow MongoDB to be used a streaming platform similar to Kafka and others.

Unfortunately, there is little information on the properties of the Timestamp and its associated guarantees and behaviour in combination with other features. It would be great if you could clear this up and hopefully provide information that would allow building such applications.

Currently the documentation says the following about BSON Timestamps (stripped):

BSON has a special timestamp type for internal MongoDB use and is not associated with the regular Date type. Timestamp values are a 64 bit value where:

  • the first 32 bits are a time_t value (seconds since the Unix epoch)
  • the second 32 bits are an incrementing ordinal for operations within a given second.

Within a single mongod instance, timestamp values are always unique.

In replication, the oplog has a ts field. The values in this field reflect the operation time, which uses a BSON timestamp value.

https://docs.mongodb.com/manual/reference/bson-types/

The following additional information/specification should be provided/defined:

  • Timestamps shall be used by applications to implement advanced patterns (not only internal).
  • The timestamp is guaranteed to be monotonic per server and also cluster in case of a failover.
  • Ordering by the timestamp results in the insertion order in the oplog and matches the order of events consumed using change streams or documents read directly from the oplog collection.
  • Behaviour in relation to sharded collections.
  • MongoDB version since this guarantees are met.

Again, clearing this up would be a huge gain for developers that look at MongoDB as an simple alternative to Kafka and others. With introduction of change streams it seems that this is also an interest from the developers.

Scope of changes

Impact to Other Docs

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)



 Comments   
Comment by Andrew Feierabend (Inactive) [ 15/Sep/20 ]

Disagree. The note just beneath the quoted section of the manual warns against usage of this type:

NOTE

The BSON timestamp type is for internal MongoDB use. For most cases, in application development, you will want to use the BSON date type. See Date for more information.

For those users who still wish to use this BSON data type, we link to http://bsonspec.org/ at the top of this page.

Closing this ticket, but feel free to reopen if you have a strong argument for including this information despite the note. Thank you for raising this ticket.

Generated at Thu Feb 08 08:04:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.