Further Specification of BSON Timestamp Guarantees

Description

The BSON Timestamp type, and the feature that it is set by the server with a sequential id upon insertion, has very interesting properties:

It could be used to let MongoDB generate an ordered sequence of timestamps when inserting documents in a collection (similar to the internal oplog). Later the collection can be sorted on the timestamp in order to restore the order in which documents are inserted. Additionally, one might also watch the collection using change streams (insert events) and further process the inserts in order. Simply said, it would allow MongoDB to be used a streaming platform similar to Kafka and others.

Unfortunately, there is little information on the properties of the Timestamp and its associated guarantees and behaviour in combination with other features. It would be great if you could clear this up and hopefully provide information that would allow building such applications.

Currently the documentation says the following about BSON Timestamps (stripped):

BSON has a special timestamp type for internal MongoDB use and is not associated with the regular Date type. Timestamp values are a 64 bit value where:

• the first 32 bits are a time_t value (seconds since the Unix epoch)
• the second 32 bits are an incrementing ordinal for operations within a given second.

Within a single mongod instance, timestamp values are always unique.

In replication, the oplog has a ts field. The values in this field reflect the operation time, which uses a BSON timestamp value.

https://docs.mongodb.com/manual/reference/bson-types/

The following additional information/specification should be provided/defined:

• Timestamps shall be used by applications to implement advanced patterns (not only internal).
• The timestamp is guaranteed to be monotonic per server and also cluster in case of a failover.
• Ordering by the timestamp results in the insertion order in the oplog and matches the order of events consumed using change streams or documents read directly from the oplog collection.
• Behaviour in relation to sharded collections.
• MongoDB version since this guarantees are met.

Again, clearing this up would be a huge gain for developers that look at MongoDB as an simple alternative to Kafka and others. With introduction of change streams it seems that this is also an interest from the developers.

Scope of changes

Impact to Other Docs

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)