[SERVER-29908] Libraries db/s/sharding and db/query/query are directly cyclic Created: 28/Jun/17  Updated: 30/Oct/23  Resolved: 13/Aug/18

Status: Closed
Project: Core Server
Component/s: Build, Querying, Sharding
Affects Version/s: None
Fix Version/s: 4.0.3, 4.1.2

Type: Bug Priority: Critical - P2
Reporter: Andrew Morrow (Inactive) Assignee: Kaloian Manassiev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-35849 Remove dependency of the write comman... Closed
depends on SERVER-36164 Decouple ScopedCollectionMetadata fro... Closed
is depended on by SERVER-30955 dynamically linked mongod fails to st... Closed
is depended on by SERVER-32677 Segmentation fault converting Replica... Closed
is depended on by SERVER-33912 Remove --disable-warnings-as-errors f... Closed
is depended on by SERVER-33889 Remove commands from embedded that sh... Closed
is depended on by SERVER-30815 Migrate library dependencies to LIBDE... Closed
is depended on by SERVER-35078 Build embedded SDK with bitcode Closed
is depended on by SERVER-36084 Remove sharding runtime from embedded Closed
Duplicate
is duplicated by SERVER-34786 service_context_d is not an independe... Closed
Related
related to SERVER-29907 Dependency tangle within query subsystem Closed
related to SERVER-53182 [v3.6] Omit "versions" from {sharding... Closed
is related to SERVER-29909 Libraries db/db_raii and db/s/shardin... Closed
is related to SERVER-30815 Migrate library dependencies to LIBDE... Closed
is related to SERVER-36889 Explore removing the index catalog SC... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.0
Sprint: Sharding 2018-02-12, Sharding 2018-02-26, Sharding 2018-03-12, Sharding 2018-04-09, Sharding 2018-04-23, Sharding 2018-05-07, Sharding 2018-05-21, Sharding 2018-06-04, Sharding 2018-06-18, Sharding 2018-07-02, Sharding 2018-07-16, Sharding 2018-07-30, Sharding 2018-08-13, Sharding 2018-08-27
Participants:
Linked BF Score: 0

 Description   

The libraries db/s/sharding and db/query/query are directly cyclic. This pulls the sharding subsytem into the dependency tangle identified in SERVER-29907. It also introduces an indirect dependency cycle with db/query/internal_plans, since db/s/sharding depends on db/query/internal_plans, which is also and already cyclic with db/query/query.



 Comments   
Comment by Githook User [ 19/Sep/18 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-32198 Get rid of CollectionShardingState::collectionIsSharded

(cherry picked from commit 80de0da37b00dbeed576b28a842cb172b6714358)

SERVER-35773 Remove references to the CatalogCache from MetadataManager

(cherry picked from commit 2aa65a86193e5d38934a4f2d6b0a8298e2432485)

SERVER-32198 Add support for an optional `vWanted` to StaleConfigInfo

(cherry picked from commit 60559a00b81293184922b3418a8e56610edf8dd9)

SERVER-36054 Remove ScopedCollectionMetadata's operator bool

(cherry picked from commit c9c340ad6e9e1f33cb001a8375c62d6b16138c74)

SERVER-36054 Remove more unused methods from CSS/MetadataManager

(cherry picked from commit ca04f5bcf9bfa73c9162b3a77225c997c6deec8a)

SERVER-36116 Get rid of CollectionShardingState::resetAll

(cherry picked from commit db1cc80d13d203b6351f5510f7756cc1c7bfc0ea)

SERVER-36054 Get rid of unused methods from CollectionShardingState

(cherry picked from commit 884d232473dca72e0872f0e540d4c3108c1e0b3d)

SERVER-36164 Decouple ScopedCollectionMetadata from MetadataManager

(cherry picked from commit d91262c4a2ed7d94923c3b1c5ff5d208aa981c73)

SERVER-29908 Move CollectionShardingState under sharding_api_d

(cherry picked from commit e491e284e8066929c8272c96a3128241ab481be8)

SERVER-29908 Remove ShardingState::appendInfo

Expose the ShardingState properties and move the appendInfo logic to be
entirely inside the 'getShardingState' function, which is its only
consumer.

(cherry picked from commit 24e411d5cd7f64c5b4da25a351529cd1873284b8)

SERVER-29908 Move 'updateConfigServerOpTimeFromMetadata' out of ShardingState

(cherry picked from commit 7a97557ce5bf74dc2b663762b7a5ffb9c958d580)

SERVER-29908 Move all runtime logic out of ShardingState

... and move it into a ShardingInitializationMongoD class, which is
responsible for driving the sharding-awareness of the node and setting
it onto ShardingState.

Also gets rid of the 'sharding' library, so there is no more library
dependency cycle.

(cherry picked from commit 200c3dc58410d8b3287a2075cc9b2ad085100e83)

SERVER-29908 Fold the 'sharding_connection_hook' library into 'sharding_initialization'

... and also remove dependency of MongoS on the replication coordinator

(cherry picked from commit fab6864f4edcae7bb304f79e601f1f62cc376a77)
Branch: v4.0
https://github.com/mongodb/mongo/commit/27e0275301eed05bea3d65c766dbe76ee1da9b8a

Comment by Gregory McKeon (Inactive) [ 16/Aug/18 ]

kaloian.manassiev this is the ticket: BACKPORT-2674.

Can you pick that up, or does schwerin need to do it?

Comment by Githook User [ 15/Aug/18 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-29908 Fold the 'sharding_connection_hook' library into 'sharding_initialization'

... and also remove dependency of MongoS on the replication coordinator
Branch: master
https://github.com/mongodb/mongo/commit/fab6864f4edcae7bb304f79e601f1f62cc376a77

Comment by Githook User [ 13/Aug/18 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-29908 Move all runtime logic out of ShardingState

... and move it into a ShardingInitializationMongoD class, which is
responsible for driving the sharding-awareness of the node and setting
it onto ShardingState.

Also gets rid of the 'sharding' library, so there is no more library
dependency cycle.
Branch: master
https://github.com/mongodb/mongo/commit/200c3dc58410d8b3287a2075cc9b2ad085100e83

Comment by Githook User [ 01/Aug/18 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-29908 Move 'updateConfigServerOpTimeFromMetadata' out of ShardingState
Branch: master
https://github.com/mongodb/mongo/commit/7a97557ce5bf74dc2b663762b7a5ffb9c958d580

Comment by Githook User [ 01/Aug/18 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-29908 Remove ShardingState::appendInfo

Expose the ShardingState properties and move the appendInfo logic to be
entirely inside the 'getShardingState' function, which is its only
consumer.
Branch: master
https://github.com/mongodb/mongo/commit/24e411d5cd7f64c5b4da25a351529cd1873284b8

Comment by Githook User [ 27/Jul/18 ]

Author:

{'username': 'kaloianm', 'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com'}

Message: SERVER-29908 Fix lint error
Branch: master
https://github.com/mongodb/mongo/commit/2f358ee8e4ca9f6b5a92e4face04246117dea8c4

Comment by Githook User [ 27/Jul/18 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-29908 Move CollectionShardingState under sharding_api_d
Branch: master
https://github.com/mongodb/mongo/commit/e491e284e8066929c8272c96a3128241ab481be8

Comment by Andy Schwerin [ 08/Jul/18 ]

I'm happy to work with henrik.edin to hand off sooner, if platforms likes. Things might move faster if I acted primarily as reviewer, given my schedule. Fundamentally, the range deleter and its three intertwined data structures need to be untangled, but it's getting easier.

Comment by Gregory McKeon (Inactive) [ 06/Jul/18 ]

schwerin Platforms is worried about this being a blocker for Mobile GA - are we able to hand this off to henrik.edin if we haven't made headway by July 27th?

Comment by Githook User [ 23/Apr/18 ]

Author:

{'email': 'schwerin@mongodb.com', 'username': 'amschwerin', 'name': 'Andy Schwerin'}

Message: SERVER-29908 Move OpObserver and MigrationSourceManager logic out of CollectionShardingState.

Makes MigrationSourceManager a decoration on CollectionShardingState,
not a member and moves the op observer behavior from
CollectionShardingState to free functions in shard_server_op_observer.h/cpp.
Branch: master
https://github.com/mongodb/mongo/commit/046740799031ca275dc3e9a5e25c4d1581ab88fb

Comment by Githook User [ 13/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move the migration critical section out of MigrationSourceManager
Branch: master
https://github.com/mongodb/mongo/commit/296f242fedb66eff99861f4fa3cda1ffc552ad9d

Comment by Githook User [ 13/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move more libraries from sharding into sharding_api_d/sharding_runtime_d
Branch: master
https://github.com/mongodb/mongo/commit/a1b194059660d78afba0d8750231f48e4f77f2a6

Comment by Githook User [ 08/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Fix failing ReplSetDistLockManagerTest
Branch: master
https://github.com/mongodb/mongo/commit/ab171e26ad78df728a4e30bf1b999bebf3535eb2

Comment by Githook User [ 07/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Fix broken logical_time_validator_test
Branch: master
https://github.com/mongodb/mongo/commit/7cde272a1f7dc876214969b03dfe43d1fc5497e8

Comment by Githook User [ 07/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Fix broken keys_collection_manager_sharding_test
Branch: master
https://github.com/mongodb/mongo/commit/eb90269e893cd978d880c403d344343a9b9eeacd

Comment by Githook User [ 07/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Update renamed test's name in the continuous stepdown suite's excluded files
Branch: master
https://github.com/mongodb/mongo/commit/720bcdfa893dd12a668d64b8d2faf6858ca23d4b

Comment by Githook User [ 07/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move OpObserver callbacks out of CollectionShardingState
Branch: master
https://github.com/mongodb/mongo/commit/083647f38662195653b87b6a79ae1183d269f910

Comment by Githook User [ 05/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move ChunkSplitter to the `sharding_runtime_d` library
Branch: master
https://github.com/mongodb/mongo/commit/c207a0ced18bba58a8ad1b08df85b1e0a0b136f9

Comment by Githook User [ 01/Mar/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move some mongos commands to separate .cpp files

No functional changes, just cleanup of commands_public.cpp
Branch: master
https://github.com/mongodb/mongo/commit/74824705de288dc46a691a517e9c17e060b042e3

Comment by Githook User [ 26/Feb/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move some cluster commands to separate files
Branch: master
https://github.com/mongodb/mongo/commit/5caa675c36e71dae1f95a70f46d2c59878760247

Comment by Githook User [ 22/Feb/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move stale shard version handling out of ShardingState
Branch: master
https://github.com/mongodb/mongo/commit/3157be3048cdeb676579ed0d860d8416cb8c4667

Comment by Githook User [ 13/Feb/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Create skeleton libraries for the sharding subsystem

Splits the sharding sybsystem into 3 main libraries, which are currently
mostly empty, but we will start moving code from all the other existing
libraries into them.

The libraries are:
s/sharding_api, db/s/sharding_api and db/s/sharding_runtime
Branch: master
https://github.com/mongodb/mongo/commit/f8af7207e2359a119b29b6d4e4c6945cc0bf1a8a

Comment by Githook User [ 13/Feb/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: Revert "SERVER-29908 Create skeleton libraries for the sharding subsystem"

This reverts commit da12466c2f109ada2d487db9c6fd92200f5b6b1d.
Branch: master
https://github.com/mongodb/mongo/commit/81a18c3927610b3023895088998e56798b348a0b

Comment by Githook User [ 13/Feb/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Create skeleton libraries for the sharding subsystem

Splits the sharding sybsystem into 3 main libraries, which are currently
mostly empty, but we will start moving code from all the other existing
libraries into them.

The libraries are:
s/sharding_api, db/s/sharding_api and db/s/sharding_runtime
Branch: master
https://github.com/mongodb/mongo/commit/da12466c2f109ada2d487db9c6fd92200f5b6b1d

Comment by Githook User [ 12/Feb/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Consolidate some sharding libraries to help break dependency cycles
Branch: master
https://github.com/mongodb/mongo/commit/e6fef1f93c6daf736b69f40cc4c0c92b4df75b4b

Comment by Githook User [ 06/Feb/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move sharding_catalog_manager and tests under db/s/config

Also get rid of sharding_catalog_test_fixture since it doesn't provide
much value.
Branch: master
https://github.com/mongodb/mongo/commit/0851ee0434ba5352561a204f368a062d660c8882

Comment by Githook User [ 02/Feb/18 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-29908 Move sharding_task_executor under mongo/s
Branch: master
https://github.com/mongodb/mongo/commit/721d2547c6c2883b522740dc2b7ff420aeebb7e9

Comment by Andrew Morrow (Inactive) [ 11/Jan/18 ]

The work done in SERVER-30815 in commit https://github.com/mongodb/mongo/commit/9d1f323a131ba5e3e1af79973ae79910de10a957 resolved SERVER-29909 and SERVER-29907. The effect of that work was to introduce a new library db/query_exec. Currently the db/query_exec library depends on db/s/sharding. However, this dependency should almost certainly be reversed, such that db/s/sharding can use the facilities from db/query_exec. As it currently stands, this is not possible, since db/query_exec makes calls directly to concrete classes in db/s/sharding. The work for this ticket is to decouple the two libraries.

Note that this is the last remaining library dependency cycle within the codebase. Addressing it will leave us with a DAG for library dependencies.

Generated at Thu Feb 08 04:22:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.