[SERVER-29450] Lower OplogFetcher and CollectionCloner batch sizes in tests by default Created: 05/Jun/17 Updated: 30/Oct/23 Resolved: 28/Aug/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | 4.1.3 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Judah Schvimer | Assignee: | Tess Avitabile (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | neweng | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Sprint: | Repl 2018-09-10 | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
We currently barely test our batching code. Lowering the batch sizes would improve this. |
| Comments |
| Comment by Githook User [ 28/Aug/18 ] |
|
Author: {'name': 'Tess Avitabile', 'email': 'tess.avitabile@mongodb.com', 'username': 'tessavitabile'}Message: |
| Comment by Tess Avitabile (Inactive) [ 27/Aug/18 ] |
|
We have some coverage for multiple batches in our unit tests. See this OplogFetcher test and this CollectionCloner test. For this ticket, I will change the replica_sets_initsync_jscore_passthrough and replica_sets_initsync_static_jscore_passthrough suites to use lower values for bgSyncOplogFetcherBatchSize and initialSyncOplogFetcherBatchSize. I will also update initial_sync_capped_index.js to take advantage of initialSyncOplogFetcherBatchSize in order to use a smaller collection and document size. |
| Comment by Tess Avitabile (Inactive) [ 08/Aug/18 ] |
|
Got it. Then I will leave this in the epic to make sure that not all results are returned in the first batch in our tests. |
| Comment by Judah Schvimer [ 08/Aug/18 ] |
|
I think I was wrong. I still think we could benefit from testing our batching logic more since most of our batch sizes in initial sync and normal oplog fetching are bigger than anything we test. |
| Comment by Tess Avitabile (Inactive) [ 08/Aug/18 ] |
|
judah.schvimer, how would this have helped us find |
| Comment by Gregory McKeon (Inactive) [ 15/May/18 ] |
|
Since we're likely to switch the OplogFetcher and CollectionCloner to use exhaust cursors, we may not do this. |
| Comment by William Schultz (Inactive) [ 10/May/18 ] |
|
Another alternative here is to add a separate passthrough suite that runs with lower batch sizes, if we still want to maintain test coverage of default batch sizes. |
| Comment by Judah Schvimer [ 10/May/18 ] |
|
This may have caught |