Uploaded image for project: 'Drivers'
  1. Drivers
  2. DRIVERS-747

Deprecate capped collections and use improved TTL as replacement

    • Type: Icon: Task Task
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Component/s: None
    • Labels:
      None
    • Not Needed

      Downstream Change Summary

      null

      Description of Linked Ticket

      Epic Summary

      Problems

      • Capped collections do not replicate properly. They do not replicate record deletes, which can result in data corruption between replica set nodes (documents present on some nodes and not others). Due to the design of mmapV1 capped collections, it is not possible to replicate capped collection deletes for mmapV1.
      • Capped collection performance is not good, despite one of the purported use cases being "to get good performance from an append-only workload".

      Goal

      • Deprecate capped collections and replace them with something that doesn't corrupt replicated data

      Nongoal

      • The special Oplog capped collection does not suffer from the data corruption issue, and already has custom code to make it unlike all other capped collections. It does not need to be converted to a TTL collection.

      Possible Implementation Details

      • Allow TTL index by specifying number of documents or data size as expiration criteria
      • Create logOp for TTL expiration specifying a field and expiration point combo
      • Allow TTL index on _id
      • Make TTL collections with TTL on _id use clustered index
      • Efficiently truncate TTL collections with TTL on _id

            Assignee:
            Unassigned Unassigned
            Reporter:
            backlog-server-pm Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: