-
Type:
Improvement
-
Resolution: Unresolved
-
Priority:
Minor - P4
-
None
-
Affects Version/s: None
-
Component/s: None
-
Query Execution
Currently, pre-images are captured for all updates, replacements and deletes on a given collection.
Ask: Enable capture of pre-images for ['DELETED', ...] documents only.
**
Context: A customer uses a collection for ingestion, where 16-20M documents are ingested every 20 minutes. Documents are indexed with Elastic Search upon insertion, and are removed from Elastic Search upon deletion. However, the ingestion process requires subsequent operations: an average of 30 updates are made on each document, and the customer doesn't want to pay the penalty of generating pre- and post-images on updates.
Impact: Updating a collection of 80GB (20M docs of 4kb each) 30 times per document unnecessarily creates a Change Stream storage requirement of 2.4TB. This prevents the customer's adoption of full document change streams.
- mentioned in
-
Page Loading...