[SERVER-28547] sharding_continuous_config_stepdown suite doesn't actually step any nodes down Created: 29/Mar/17  Updated: 15/Aug/17  Resolved: 30/Mar/17

Status: Closed
Project: Core Server
Component/s: Sharding, Testing Infrastructure
Affects Version/s: 3.4.2, 3.5.1
Fix Version/s: 3.4.4, 3.5.6

Type: Bug Priority: Critical - P2
Reporter: Max Hirschhorn Assignee: Esha Maharishi (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-28549 pass stepdownDelaySeconds as an argum... Closed
Related
is related to SERVER-26903 ShardingTest is failing non-retriable... Closed
is related to SERVER-28567 assert.throws should validate that th... Backlog
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

diff --git a/jstests/libs/override_methods/sharding_continuous_config_stepdown.js b/jstests/libs/override_methods/sharding_continuous_config_stepdown.js
index 638dc8bb29..2f242591fd 100644
--- a/jstests/libs/override_methods/sharding_continuous_config_stepdown.js
+++ b/jstests/libs/override_methods/sharding_continuous_config_stepdown.js
@@ -90,8 +92,13 @@ tracking: {verbosity: 0} }";
                     print('*** Stepping down ' + primary);
 
                     assert.throws(function() {
-                        var result = primary.adminCommand(
-                            {replSetStepDown: stepdownDelaySeconds, force: true});
+                        try {
+                            var result = primary.adminCommand(
+                                {replSetStepDown: stepdownDelaySeconds, force: true});
+                        } catch (e) {
+                            print("!!! " + tojson(e));
+                            throw e;
+                        }
                         print('replSetStepDown command did not throw and returned: ' +
                               tojson(result));

$ python buildscripts/resmoke.py --executor=sharding_continuous_config_stepdown jstests/sharding/addshard1.js
...
[js_test:addshard1] 2017-03-29T16:46:16.488-0400 *** Stepping down connection to localhost:20011
[js_test:addshard1] 2017-03-29T16:46:16.492-0400 !!! ReferenceError: stepdownDelaySeconds is not defined

Sprint: Sharding 2017-04-17
Participants:

 Description   

As part of trying to reproduce SERVER-28545 outside of Jepsen, I've discovered that the background thread started by the sharding_continuous_config_stepdown.js override triggers a ReferenceError when attempting to run the "replSetStepDown" command. It therefore fails to ever actually send the "replSetStepDown" command to the current primary of the CSRS and causes the assert.throws() to silently succeed.

This defect was introduced by the changes in 76dc485 as part of SERVER-26903 by scoping the stepdownDelaySeconds variable outside of the _continuousPrimaryStepdownFn() function, thereby having the _continuousPrimaryStepdownFn() close over it. The _continuousPrimaryStepdownFn() function is passed to the ScopedThread constructor which executes in a fresh JavaScript context and therefore won't have this variable defined when invoked. Instead, the stepdownDelaySeconds value can be passed as an argument to the _continuousPrimaryStepdownFn() function.

Note: While it is effectively invalid to have a function run by a ScopeThread close over a variable from an outer scope, I have yet to figure out how to make our linters aware of the mongo shell's threading/process model.



 Comments   
Comment by Githook User [ 30/Mar/17 ]

Author:

{u'username': u'EshaMaharishi', u'name': u'Esha Maharishi', u'email': u'esha.maharishi@mongodb.com'}

Message: SERVER-28547 pass stepdownDelaySeconds as an argument to _continuousPrimaryStepdownFn in continuous stepdown suite override
Branch: v3.4
https://github.com/mongodb/mongo/commit/a64111eab10afccd2168e3bd17f520fed5b77376

Comment by Githook User [ 30/Mar/17 ]

Author:

{u'username': u'EshaMaharishi', u'name': u'Esha Maharishi', u'email': u'esha.maharishi@mongodb.com'}

Message: SERVER-28547 pass stepdownDelaySeconds as an argument to _continuousPrimaryStepdownFn in continuous stepdown suite override
Branch: master
https://github.com/mongodb/mongo/commit/9f5c312c9a21590a8ed430ccbc50bf81c915424f

Generated at Thu Feb 08 04:18:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.