[SERVER-85088] POC: Explore ways we can shrink the generated oplog entries from pipeline-style updates Created: 01/Apr/19  Updated: 12/Jan/24  Resolved: 18/Dec/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Charlie Swanson Assignee: Charlie Swanson
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-40381 Add the ability to specify a pipeline... Closed
Sprint: Query 2019-06-17, Query 2019-12-02, Query 2019-12-16, Query 2019-12-30
Participants:

 Description   

The existing update system has a fairly sophisticated way of tracking damages performed by an update to generate a minimal oplog entry. We have a fairly sophisticated system within aggregation to detect which fields are modified by which stages - we may be able to re-use it to inform oplog entry generation.



 Comments   
Comment by Charlie Swanson [ 18/Dec/19 ]

Just chatted with ian.boros who's going to be working on the design. We have a rough POC here which doesn't handle $unsets which could be interesting and different, but we don't think there's remaining value in this ticket, so we'll call it complete.

Comment by Craig Homa [ 12/Dec/19 ]

Hey charlie.swanson, is there a link to the POC document that you can share?

Comment by Charlie Swanson [ 19/Jun/19 ]

We're going to need to scope this project (PM-1460) before getting started meaningfully so I'm punting this ticket out of the sprint to be revisited after scoping.

Comment by Charlie Swanson [ 30/May/19 ]

We have some initial perf results to consider. I'll gather them and report here.

Generated at Thu Feb 08 06:56:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.