ISSUE DESCRIPTION AND IMPACT

Performing a delete with a non-metaField filter on a time-series collection followed by inserts on an adjacent timeField time range can cause the following for the data inserted after the delete:

• TTL deletion prematurely expires the data.
• Incomplete query results for queries filtering on time.

This may happen if:
A specific index is present at the time of the inserts:

• { <metaField>: 1, <timeField>: 1 } (collections with metaField)
  • This index is created by default in MongoDB versions 7.0 and later.
• { <timeField>: 1 } (collections without metaField)
  • This index is not created by default in any MongoDB version.

The timeField timestamp of documents inserted after the deletion is within the following time range of the deleted document(s):

• Collections with seconds granularity: 2 hours
• Collections with minutes granularity: 2 days
• Collections with hours granularity: 2 months
• Collections with custom bucketing setting: 2*bucketMaxSpanSeconds

This happens because deletes on a time-series collection with a non-metaField filter can update the internal “control.min” timestamp of an underlying bucket making data outside the allowed maximum time span eligible for insertion. This violates the internal clustered _id index used to optimize queries on time-series collections when filtering on time.

DIAGNOSIS AND AFFECTED VERSIONS

MongoDB versions 7.0.0 through 7.0.15 and 8.0.0 through 8.0.3 are affected. Users can determine if they have been impacted by running validate:

Depending on the version used an error or warning is returned with the following message: “Detected one or more documents in this collection incompatible with time-series specifications. For more info, see logs with log id 6698300.”. The server logs “Mismatch between the embedded timestamp [...] in the time-series bucket '_id' field and the timestamp [...] in 'control.min' field.” when an affected bucket is detected.

On MongoDB versions 7.0.0 through 7.0.14 and 8.0.0 through 8.0.4 validate will return false positives on time-series collections containing data with timeField timestamps before 1970-01-01T00:00:00.000Z or after 2038-01-19T03:14:07.000Z.

REMEDIATION AND WORKAROUNDS

The specific index listed above can be hidden or removed to prevent further occurrences of this issue.

Data can be retrieved and reinserted into the collection using a fixed version of MongoDB. A remediation script is available here.