[SERVER-20646] stale_clustered.js fails with "node is recovering" error Created: 25/Sep/15  Updated: 15/Oct/15  Resolved: 13/Oct/15

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 3.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: Kaloian Manassiev
Resolution: Done Votes: 0
Labels: disabled-test
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Sharding A (10/09/15), Sharding B (10/30/15)
Participants:

 Comments   
Comment by Githook User [ 13/Oct/15 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-20646 libasync_results_merger needs to depend on libcoreshard

Fixes an issue introduced in 15350b50 where libasync_results_merger
was made to depend on mongo::grid, but the symbol cannot be resolved
at link time.
Branch: master
https://github.com/mongodb/mongo/commit/90cd064713dfcf5c82be07742f7377c83ea0a4f2

Comment by Githook User [ 13/Oct/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20646 Cluster find command per-host retry logic

This change makes the cluster 'find' command to retry on a per-host basis
instead of the entire operation.

Reverts commit c433c8157f988a377c1cf9646078450ecd68c297.
Reverts commit 5ab3290f8796f2143acd5011ab0baae70ed5cece.
Branch: master
https://github.com/mongodb/mongo/commit/66c0dda7e9f97d03383139f7e62e4534934b5ecc

Comment by Githook User [ 13/Oct/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20646 Move remote cursor host targeting inside the AsyncResultsMerger

No functional changes, just moves the resolution of the remote cursor's
host to happen inside the AsyncResultsMerger instead of at the level of
the entire find command. Also hooks the async merger tests with the
sharding test fixture so they can access all objects.
Branch: master
https://github.com/mongodb/mongo/commit/15350b5087dfd09cb2b25a22f422804abc8c2654

Comment by Githook User [ 13/Oct/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20646 Make a common sharding test fixture

This change pulls the catalog manager replica set test fixture into a
common library to be used across all sharding-related unit tests.
Branch: master
https://github.com/mongodb/mongo/commit/40631b8742851654ae6b3d7cbd742404c1479a47

Comment by Githook User [ 13/Oct/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20646 Get rid of CommonErrorCodes

Replace all usages with the standard ErrorCodes enumeration.
Branch: master
https://github.com/mongodb/mongo/commit/25b498934fea1c93ae82823ee13cff0866ed6007

Comment by Githook User [ 12/Oct/15 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-20646 Disable stale_clustered.js in sharding_auth_audit.
Branch: master
https://github.com/mongodb/mongo/commit/c433c8157f988a377c1cf9646078450ecd68c297

Comment by Githook User [ 09/Oct/15 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-20646 Temporarily disable stale_clustered.js.
Branch: master
https://github.com/mongodb/mongo/commit/5ab3290f8796f2143acd5011ab0baae70ed5cece

Comment by Kaloian Manassiev [ 08/Oct/15 ]

The test is failing because the AsyncResultsMerger is not handling a NotMasterOrSecondaryCode (13436) response and is not notifying the targeter.

Comment by Max Hirschhorn [ 08/Oct/15 ]

Still failing:
https://evergreen.mongodb.com/task/mongodb_mongo_master_solaris_64_bit_sharding_0891369a428c69283544d4689883eab4e7e31dd9_15_10_07_22_42_22

[js_test:stale_clustered] 2015-10-08T01:41:40.109+0000 2015-10-08T01:41:40.103+0000 E QUERY    [thread1] Error: command failed: { "ok" : 0, "errmsg" : "node is recovering", "code" : 13436 } : undefined :
[js_test:stale_clustered] 2015-10-08T01:41:40.109+0000 _getErrorWithCode@src/mongo/shell/utils.js:23:13
[js_test:stale_clustered] 2015-10-08T01:41:40.109+0000 doassert@src/mongo/shell/assert.js:13:14
[js_test:stale_clustered] 2015-10-08T01:41:40.109+0000 assert.commandWorked@src/mongo/shell/assert.js:259:5
[js_test:stale_clustered] 2015-10-08T01:41:40.109+0000 DBCommandCursor@src/mongo/shell/query.js:657:5
[js_test:stale_clustered] 2015-10-08T01:41:40.110+0000 DBQuery.prototype._exec@src/mongo/shell/query.js:103:28
[js_test:stale_clustered] 2015-10-08T01:41:40.110+0000 DBQuery.prototype.hasNext@src/mongo/shell/query.js:257:5
[js_test:stale_clustered] 2015-10-08T01:41:40.110+0000 DBQuery.prototype.itcount@src/mongo/shell/query.js:378:13
[js_test:stale_clustered] 2015-10-08T01:41:40.110+0000 @jstests/sharding/stale_clustered.js:81:14
[js_test:stale_clustered] 2015-10-08T01:41:40.110+0000 @jstests/sharding/stale_clustered.js:4:2
[js_test:stale_clustered] 2015-10-08T01:41:40.110+0000 
[js_test:stale_clustered] 2015-10-08T01:41:40.110+0000 failed to load: jstests/sharding/stale_clustered.js

Comment by Githook User [ 07/Oct/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-20646 Move stale_clustered.js to the sharding suite

It doesn't need to be under NoPassthroughWithMongoD. Also changed the
ShardingTest instantiation to use the BSON-based constructor and tightened
some of the checks in order to catch the exact error location.
Branch: master
https://github.com/mongodb/mongo/commit/f381fd0166ddaac9064e43e8308bdf582dd9af3f

Comment by Githook User [ 28/Sep/15 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-20646 Temporarily disable stale_clustered.js
Branch: master
https://github.com/mongodb/mongo/commit/780565d818e82f841ba5f847b315358e519a47ec

Comment by J Rassi [ 28/Sep/15 ]

Kal, do you think it's highly likely that you can push a fix for this early afternoon today? If not, I'd like to disable this test.

Comment by Kaloian Manassiev [ 25/Sep/15 ]

[js_test:stale_clustered] 2015-09-25T08:32:02.727+0000 2015-09-25T08:32:02.726+0000 E QUERY    [thread1] Error: error: { "$err" : "node is recovering", "code" : 13436 } :
[js_test:stale_clustered] 2015-09-25T08:32:02.727+0000 _getErrorWithCode@src/mongo/shell/utils.js:23:13
[js_test:stale_clustered] 2015-09-25T08:32:02.727+0000 DBQuery.prototype.next@src/mongo/shell/query.js:278:1
[js_test:stale_clustered] 2015-09-25T08:32:02.727+0000 DBQuery.prototype.itcount@src/mongo/shell/query.js:372:9
[js_test:stale_clustered] 2015-09-25T08:32:02.727+0000 @jstests/noPassthroughWithMongod/stale_clustered.js:83:35
[js_test:stale_clustered] 2015-09-25T08:32:02.727+0000 
[js_test:stale_clustered] 2015-09-25T08:32:02.727+0000 failed to load: jstests/noPassthroughWithMongod/stale_clustered.js

Generated at Thu Feb 08 03:54:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.