[SERVER-22155] Improve reliability of cleanup_orphaned_cmd_during_movechunk*.js tests Created: 12/Jan/16  Updated: 06/Dec/22  Resolved: 30/Nov/16

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Minor - P4
Reporter: Dianna Hohensee (Inactive) Assignee: [DO NOT USE] Backlog - Sharding Team
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Sharding
Sprint: Sharding 2016-08-29, Sharding 2016-09-19, Sharding 2016-10-10
Participants:
Linked BF Score: 0

 Description   

cleanup_orphaned_cmd_during_movechunk_hashed.js and cleanup_orphaned_cmd_during_movechunk.js are failing sporadically under the stress of continuous config server step down in the sharding_csrs_continuous_config_stepdown_WT test suite.

Log of a failures here: https://logkeeper.mongodb.org/build/56943d13be07c46e4db5ca58/test/56943dc49041300b270834d5, https://logkeeper.mongodb.org/build/569594159041300b271b4362/test/569598b6be07c46e4dc955b4



 Comments   
Comment by Dianna Hohensee (Inactive) [ 30/Nov/16 ]

schwerin new BFs will not come in on this, as the cleanup orphaned JS tests have been blacklisted from the stepdown suite. We do not currently know what the issue is, but the cleanupOrphaned command is unreliable in an environment where config primary stepdowns occur.

Comment by Dianna Hohensee (Inactive) [ 22/Sep/16 ]

Ran several evergreen builds on the stepdown suite with cleanup_orphaned_cmd_during_movechunk*.js running. Finally got a failure that needs to be investigated:

https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_71_ppc64le_sharding_csrs_continuous_config_stepdown_WT_7693fa59c4470db729d85af99fb9cc3b264fa8c8_16_09_16_13_52_28

Comment by Dianna Hohensee (Inactive) [ 20/Apr/16 ]

I'd create a new BF ticket for the sharding_last_stable_mongos suite failure because I don't think it will get addressed here: this ticket has a pretty neglected life.

Comment by Judah Schvimer [ 20/Apr/16 ]

This failed on the sharding_last_stable_mongos suite:
https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_62_64_bit_sharding_last_stable_mongos_cf6b197b57a039b78e3353efa509a987287a1043_16_04_19_19_44_21

[js_test:cleanup_orphaned_cmd_during_movechunk_hashed] 2016-04-19T21:50:47.845+0000 assert: [4] != [2] are not equal : undefined
[js_test:cleanup_orphaned_cmd_during_movechunk_hashed] 2016-04-19T21:50:47.845+0000 doassert@src/mongo/shell/assert.js:15:14
[js_test:cleanup_orphaned_cmd_during_movechunk_hashed] 2016-04-19T21:50:47.846+0000 assert.eq@src/mongo/shell/assert.js:51:5
[js_test:cleanup_orphaned_cmd_during_movechunk_hashed] 2016-04-19T21:50:47.846+0000 @jstests/sharding/cleanup_orphaned_cmd_during_movechunk_hashed.js:27:5
[js_test:cleanup_orphaned_cmd_during_movechunk_hashed] 2016-04-19T21:50:47.846+0000 @jstests/sharding/cleanup_orphaned_cmd_during_movechunk_hashed.js:11:2

Comment by Githook User [ 30/Mar/16 ]

Author:

{u'username': u'DiannaHohensee', u'name': u'Dianna Hohensee', u'email': u'dianna.hohensee@10gen.com'}

Message: Enabling CSRS continuous config stepdown workload in evergreen

Patches:

SERVER-22081 Enable CSRS continuous stepdown workload in evergreen
(cherry picked from commit 5be3e6bcfe523ffdce23099c71ffc23fa40c2560)

SERVER-22511 blacklisting sharding_rs1.js because of config db write without retries
(cherry picked from commit b01e99cf36b81b00def7e53f3ce4a7154770b84a)

SERVER-22081 blacklisting migrateBig.js from the sharding_continuous_config_stepdown suite: it can fail writing to config w/o retries
(cherry picked from commit 57e294758424b0a358294337918a065614cafe4a)

SERVER-22081 blacklist split_large_key.js from config stepdown suite because count() against config server has no retries
(cherry picked from commit ee050e4700001d0f9ebf8aa388c57307ff405574)

SERVER-22081 blacklisting tests from the continuous config stepdown suite that do direct writes (with no retries) against config/admin databases
(cherry picked from commit 920d38ceb07bff0ec18e2f173d942c3b743dea4b)

SERVER-22155 temporarily blacklist cleanup_orphaned_cmd_during_movechunk.js until it's improved
(cherry picked from commit f4f60232b46cc2295119f0c572d7251af4500e5d)

SERVER-22151 typo fix
(cherry picked from commit c298fa69111ec33912c9f2294de8fe8f7703ab35)

SERVER-22155 temporarily blacklist cleanup_orphaned_cmd_during_movechunk_hashed.js until it's improved
(cherry picked from commit 1cb056dc49c1bbd0fd30e5c3634e85f93c1eb3b3)

SERVER-22151 blacklist lagged_config_secondary.js and similar tests from the sharding_csrs_continuous_config_stepdown_WT test suite
(cherry picked from commit c60496946bdf5e723bfa3cfb8937e1ab10d15bee)

Conflicts:
buildscripts/resmokeconfig/suites/sharding_continuous_config_stepdown.yml
jstests/sharding/authCommands.js
jstests/sharding/authConnectionHook.js
jstests/sharding/cleanup_orphaned_cmd_prereload.js
jstests/sharding/coll_epoch_test0.js
jstests/sharding/coll_epoch_test1.js
jstests/sharding/coll_epoch_test2.js
jstests/sharding/covered_shard_key_indexes.js
jstests/sharding/cursor_cleanup.js
jstests/sharding/delete_during_migrate.js
jstests/sharding/exact_shard_key_target.js
jstests/sharding/listDatabases.js
jstests/sharding/mapReduce_inSharded.js
jstests/sharding/mapReduce_inSharded_outSharded.js
jstests/sharding/mapReduce_nonSharded.js
jstests/sharding/mapReduce_outSharded.js
jstests/sharding/mongos_no_detect_sharding.js
jstests/sharding/mongos_no_replica_set_refresh.js
jstests/sharding/multi_mongos2.js
jstests/sharding/recovering_slaveok.js
jstests/libs/override_methods/sharding_continuous_config_stepdown.js
Branch: v3.2
https://github.com/mongodb/mongo/commit/558ee7cc36b81b4eab5d93d3620ee0806a90ae70

Comment by Githook User [ 13/Jan/16 ]

Author:

{u'username': u'DiannaHohensee', u'name': u'Dianna Hohensee', u'email': u'dianna.hohensee@10gen.com'}

Message: SERVER-22155 temporarily blacklist cleanup_orphaned_cmd_during_movechunk.js until it's improved
Branch: master
https://github.com/mongodb/mongo/commit/f4f60232b46cc2295119f0c572d7251af4500e5d

Comment by Githook User [ 12/Jan/16 ]

Author:

{u'username': u'DiannaHohensee', u'name': u'Dianna Hohensee', u'email': u'dianna.hohensee@10gen.com'}

Message: SERVER-22155 temporarily blacklist cleanup_orphaned_cmd_during_movechunk_hashed.js until it's improved
Branch: master
https://github.com/mongodb/mongo/commit/1cb056dc49c1bbd0fd30e5c3634e85f93c1eb3b3

Generated at Thu Feb 08 03:59:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.