[SERVER-19630] FSM tests - capability to run against existing clusters Created: 28/Jul/15  Updated: 18/Jan/23  Resolved: 30/Apr/18

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.0.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Jonathan Abrahams Assignee: Robert Guo (Inactive)
Resolution: Done Votes: 0
Labels: tig-concurrency
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-34289 Allow FSM test suites to connect to a... Closed
Related
related to SERVER-72985 FSM workloads which use the connectio... Closed
related to SERVER-22091 Run FSM suites with readConcern/write... Closed
is related to SERVER-19488 FSM tests - standalone tests invoked ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.6
Sprint: TIG 2018-04-23, TIG 2018-05-07
Participants:

 Description   

The FSM test suite is loosely coupled to run its workloads in various modes:

  • standalone
  • replica set
  • sharded cluster

It turns out the FSM workloads are very good test clients for performing a lot of various DB operations. New tests that use FSM workloads would benefit if the FSM workloads could be exercised in those modes. Presently the replica set & sharded cluster modes start up their own mongod and mongos. Adding options to be able to use the FSM workloads against a running replica set or sharded cluster would be worthwhile.



 Comments   
Comment by Githook User [ 30/Apr/18 ]

Author:

{'email': 'robert.guo@10gen.com', 'username': 'guoyr', 'name': 'Robert Guo'}

Message: SERVER-19630 allow FSM tests to connect to an existing cluster
Branch: master
https://github.com/mongodb/mongo/commit/39622745cd5258d40924c8e44be73b5c2e1b4ca4

Comment by Robert Guo (Inactive) [ 27/Apr/18 ]

Capturing Max's comment from the code review for easier future reference.

A couple minor comments. I also wanted to summarize some of the changes that
happened in patch set 13 and beyond in our future selves need to refer back to
this code review.

1. The concurrency_sharded*.yml test suites rely on the namespace for each FSM
workload being unique in order to avoid collection lifecycle issues. An initial
attempt at using the FSM workload's name as the database prefix was made, but it
was ultimately dropped in favor of using a global incrementing counter in
fsm_workload_test.py as the database name for certain FSM workloads grew beyond
64 characters otherwise.
https://github.com/mongodb/mongo/blob/b57eee5a295ede1fd67299dc9990c272c1f66ea...

2. The sharded_mergeChunks_partitioned.js, sharded_moveChunk_partitioned.js, and
sharded_splitChunk_partitioned.js FSM workloads implicitly assumed that there is
only data in the "config.chunks" collection for the collection associated with
the FSM workload that's executing. This assumption was violated by having
resmoke_runner.js not call the cleanupWorkloadData() function in order to have
data for resmoke.py to run the data consistency checks on. The FSM workloads and
the chunks.js helper were updated to plumb the "ns" field into the queries they
perform on the "config.chunks" collection to eliminate this assumption.

3. The concurrency_sharded_causal_consistency*.yml test suites use the
set_read_preference_secondary.js override in order to avoid using
readPreference=

Unknown macro: {mode}

for the "admin" and "config" databases in a
sharded cluster because we're running with a 1-node CSRS. However, changing
Mongo.prototype.runCommand() to inject a read preference now means that we're
using readPreference=

Unknown macro: {mode}

for the connections in the connection
cache when we had only been setting the read preference on the `db.getMongo()`
connection before. We can use the _DelegatingDriverSession object to share the
same session for different Mongo connection objects to ensure that we're reading
from a secondary only at a time after the effect of the sharding metadata
operations have been applied.

Comment by Jonathan Abrahams [ 16/Nov/15 ]

Improvements of this nature require that SERVER-21485 is implemented. The cluster object (cluster.js) has too many dependencies on ReplSetTest & ShardingTest, such that it depends on those objects to permit a connection to bind to them. Otherwise, there will be a lot of duplicate code.

Generated at Thu Feb 08 03:51:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.