[SERVER-36865] make fsm workloads that extend kill_rooted_or.js more robust in the sharded concurrency suites, or blacklist them Created: 24/Aug/18  Updated: 29/Oct/23  Resolved: 25/Feb/19

Status: Closed
Project: Core Server
Component/s: Querying, Sharding
Affects Version/s: None
Fix Version/s: 4.1.9, 4.0.15

Type: Bug Priority: Major - P3
Reporter: Esha Maharishi (Inactive) Assignee: Justin Seyster
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
causes SERVER-64985 typo in kill_rooted_or.js concurrency... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.0
Participants:
Linked BF Score: 8

 Description   

The kill_rooted_or.js workload has a stage that issues createIndexes and dropIndexes, and since the workload uses 10 threads, this means a createIndexes can max out its CannotImplicitlyCreateCollection retry attempts, which causes the workload to fail.

(Each createIndexes attempt creates the collection; this failure happens when the collection keeps getting dropped in between mongos retrying the createIndexes).

After speaking with Dave Storch, assigning to Query to see if they would like to work to make the tests more robust to sharding, or to blacklist the tests from the sharded concurrency suites because of limitations around running concurrent catalog operations in sharding.



 Comments   
Comment by Githook User [ 17/Dec/19 ]

Author:

{'name': 'Justin Seyster', 'email': 'justin.seyster@mongodb.com', 'username': 'jseyster'}

Message: SERVER-36865 Improved kill_rooted_or.js and kill_aggregation.js

Backports the original version of this commit (on branch v4.2), so
that we don't get CannotImplicitlyCreateCollection errors when testing
v4.0.

Unlike in the original commit, this backport does not add a function
to repopulate the test collection after dropping it. The v4.0 version
of this test does not populate the collection in the first place. We
would have to backport that functionality as well if we wanted to
backport the populateCollection function.

(cherry picked from commit 369a2e0432c27b20b106056e2f2e0849cab617f0)
Branch: v4.0
https://github.com/mongodb/mongo/commit/d379a124bd3f22e0970d4ec3028a67f724f25b31

Comment by Justin Seyster [ 11/Oct/19 ]

This test-only change fixes a BF that still occurs in 4.0. It also makes the test more sensitive to potential failures.

Comment by Githook User [ 25/Feb/19 ]

Author:

{'name': 'Justin Seyster', 'email': 'justin.seyster@mongodb.com', 'username': 'jseyster'}

Message: SERVER-36865 Improved kill_rooted_or.js and kill_aggregation.js

In addition to the CannotImplicitlyCreateCollection problem described
in the Jira ticket, these tests were querying an empty collection. The
function that drops the collection was re-creating its indexes but was
not repopulating the actual documents.

This change also fixes a couple of places where exceptions were
getting squelched. Instead, the test now checks for the specific
errors that we expect to see and fails on other errors.

These changes should make the tests more sensitive to future bugs.
Branch: master
https://github.com/mongodb/mongo/commit/369a2e0432c27b20b106056e2f2e0849cab617f0

Comment by Justin Seyster [ 18/Nov/18 ]

That’s a good idea. I think it’s also worth considering rewriting the assertion so that “CannotImplicitlyCreateCollection” is not considered a test-fatal error.

Comment by Esha Maharishi (Inactive) [ 24/Aug/18 ]

One idea is to make the number of retries mongos uses configurable, so that the sharded concurrency suites can set it to be, for example, an order of magnitude higher (100 rather than 10).

Generated at Thu Feb 08 04:44:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.