[SERVER-53717] Dynamically split large concurrency tasks Created: 12/Jan/21  Updated: 29/Oct/23  Resolved: 13/Jan/21

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.9.0, 4.4.4, 4.2.13

Type: Bug Priority: Major - P3
Reporter: David Bradford (Inactive) Assignee: David Bradford (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4, v4.2
Sprint: DAG 2021-01-25
Participants:
Linked BF Score: 25
Story Points: 1

 Description   

There are several concurrency tasks that have long-running times. We should be able to dynamically split these to bring the wall-clock time down.

Also, these require running on large distros to avoid OOM, but new build variants are being added that don't run them on the right distros. By splitting them dynamically, we can handle that without new build variants needing to know.

The tasks we will split:

  • concurrency_replication_causal_consistency
  • concurrency_sharded_causal_consistency_and_balancer
  • concurrency_sharded_clusterwide_ops_add_remove_shards
  • concurrency_sharded_local_read_write_multi_stmt_txn
  • concurrency_sharded_local_read_write_multi_stmt_txn_with_balancer
  • concurrency_sharded_multi_stmt_txn
  • concurrency_sharded_multi_stmt_txn_with_balancer
  • concurrency_sharded_multi_stmt_txn_with_stepdowns
  • concurrency_sharded_replication
  • concurrency_sharded_replication_no_txns
  • concurrency_sharded_replication_no_txns_with_balancer
  • concurrency_sharded_replication_with_balancer
  • concurrency_sharded_with_stepdowns
  • concurrency_sharded_with_stepdowns_and_balancer


 Comments   
Comment by Githook User [ 03/Feb/21 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53717: Dynamically split large concurrency tasks

(cherry picked from commit 8fc9c9bab5fcfb95ade1acf6c61017f4416d9888)
(cherry picked from commit 1eae571d0ba85ecb98b354c733ea6605eb4fff69)
Branch: v4.2
https://github.com/mongodb/mongo/commit/419db82fea6469b6adddbe5224fa06ba03e65efa

Comment by Githook User [ 13/Jan/21 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53717: Dynamically split large concurrency tasks

(cherry picked from commit 8fc9c9bab5fcfb95ade1acf6c61017f4416d9888)
Branch: v4.4
https://github.com/mongodb/mongo/commit/1eae571d0ba85ecb98b354c733ea6605eb4fff69

Comment by Githook User [ 13/Jan/21 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-53717: Dynamically split large concurrency tasks
Branch: master
https://github.com/mongodb/mongo/commit/8fc9c9bab5fcfb95ade1acf6c61017f4416d9888

Generated at Thu Feb 08 05:31:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.