[SERVER-60823] runCommandWithRetries in JS test framework exceeds JS interpreter recursion limit Created: 19/Oct/21 Updated: 23/Jan/24 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor - P4 |
| Reporter: | Benety Goh | Assignee: | Backlog - Replication Team |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Assigned Teams: |
Replication
|
||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||
| Backport Requested: |
v7.3
|
||||||||||||||||||||||||
| Sprint: | Execution Team 2021-11-29, Execution Team 2021-12-13, Execution Team 2022-01-24 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Linked BF Score: | 173 | ||||||||||||||||||||||||
| Description |
|
In some of our CI suites, the command invocation in tests is overridden to support retries. For some multi-document passthroughs, when there is a need to retry a transaction, the runCommandWithRetries() logic in implicitly_retry_on_background_op_in_progress.js and network_error_and_txn_override.js implicitly recurses into itself with the following repeating stack of function calls. This has the potential to exceed the internal recursion limit in the JS interpreter, leading to the test terminating early.
|
| Comments |
| Comment by Sviatlana Zuiko [ 02/Jun/22 ] | ||
|
benety.goh@mongodb.com and colleagues,
Taking into account that the issue is being hit constantly and is also "Hot due to frequency", can we prioritize the work on SERVER-60823? Thank you! | ||
| Comment by Steven Vannelli [ 10/May/22 ] | ||
|
Moving this ticket to the Backlog and removing the "Backlog" fixVersion as per our latest policy for using fixVersions. | ||
| Comment by Benety Goh [ 14/Apr/22 ] | ||
|
A more sustainable approach to solving the recursion issue may be to address the reentrant nature of the runCommandOverride() function in network_error_and_txn_override.js. This would involve revisiting and refining the solution in | ||
| Comment by Benety Goh [ 14/Apr/22 ] | ||
|
One alternate solution is to address the indirection in overrideRunFunction added in | ||
| Comment by Benety Goh [ 07/Apr/22 ] | ||
|
Testing with a multi-doc transaction test overrides and a modified server that repeatedly fails with a WriteConflictException, the part of the stack that is currently most interesting is:
In retryEntryTransaction, we invoke the top-level DB.runCommand() on each embedded operation in a multi-doc transaction. This has the effect of adding a duplicate stack of test override calls to the recursion on each retry attempt. | ||
| Comment by Githook User [ 10/Mar/22 ] | ||
|
Author: {'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}Message: SERVER-60823 temporarly disable test from txn suites due to recursion issue | ||
| Comment by Dianna Hohensee (Inactive) [ 02/Nov/21 ] | ||
|
At the least, we need the BF to stop happening. | ||
| Comment by Benety Goh [ 19/Oct/21 ] | ||
|
One possible solution is to avoid the use recursion to retry a command - this might be accomplished by having one of the lower level functions in the JS stack return some signal to retry, rather than performing the command invocation itself. |