Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-94469

Capture pre-images for ONLY specified change events

    • Query Execution

      Currently, pre-images are captured for all updates, replacements and deletes on a given collection.

      Ask: Enable capture of pre-images for ['DELETED', ...] documents only.
      **

      Context: A customer uses a collection for ingestion, where 16-20M documents are ingested every 20 minutes. Documents are indexed with Elastic Search upon insertion, and are removed from Elastic Search upon deletion. However, the ingestion process requires subsequent operations: an average of 30 updates are made on each document, and the customer doesn't want to pay the penalty of generating pre- and post-images on updates.
       

      Impact: Updating a collection of 80GB (20M docs of 4kb each) 30 times per document unnecessarily creates a Change Stream storage requirement of 2.4TB. This prevents the customer's adoption of full document change streams.
       

            Assignee:
            Unassigned Unassigned
            Reporter:
            matthew.royal@mongodb.com Matthew Royal
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated: