[SERVER-19855] Operations that convey shard version information must include a minimum optime Created: 10/Aug/15  Updated: 25/Jan/17  Resolved: 18/Sep/15

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.1.7
Fix Version/s: 3.1.9

Type: Bug Priority: Major - P3
Reporter: Andy Schwerin Assignee: Spencer Brody (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-20025 mongos fails to start if CSRS primary... Closed
is depended on by SERVER-20494 Enable reading from CSRS secondaries Closed
Related
related to SERVER-19736 Add opTime invariant checks when upda... Closed
is related to SERVER-20285 Unify the formats used by the find an... Closed
is related to SERVER-20498 mongos should convey config optime in... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

Change the read preference in the CSRS catalog manager implementation to "secondary preferred" and run the sharding test suite.

Sprint: Sharding 8 08/28/15, Sharding 9 (09/18/15), Sharding A (10/09/15)
Participants:
Linked BF Score: 0

 Description   

When a setShardVersion command, find command or write command that includes shard version information is sent to a shard server from another node (typically a mongos), it must also include the an optime on the config server no earlier than the earliest one in which that shard version information became available. Otherwise, a shard server refreshing its view of the sharding metadata from a secondary might read too-old metadata, and not see the new shard version.



 Comments   
Comment by Githook User [ 24/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 Include config server optime even on command failure responses from mongod
Branch: master
https://github.com/mongodb/mongo/commit/3a7b1a9800f75706f35a290a8dee198bb29e3366

Comment by Githook User [ 18/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 Update config server optime based on response metadata from running commands through DBClientInterface connections
Branch: master
https://github.com/mongodb/mongo/commit/455d5e0cc5aa9649b12b25f827efdf96b4a7c360

Comment by Githook User [ 18/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 Update config server optime based on response metadata from running commands against shards through the ShardRegistry
Branch: master
https://github.com/mongodb/mongo/commit/d6c1bb1838d434c9ff354467ba8f831ef379c954

Comment by Githook User [ 18/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 SERVER-20425 Make ShardRegistry responsible for tracking the config server optime.
Branch: master
https://github.com/mongodb/mongo/commit/a3ef7c1a233148861b3c8ada4849f5763d26d5f8

Comment by Githook User [ 18/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 Change ShardingRequestMetadata to ConfigServerRequestMetadata to parallel ConfigServerResponseMetadata
Branch: master
https://github.com/mongodb/mongo/commit/9b4cf35eed13f9661db7735cd942b2f43aed136c

Comment by Githook User [ 17/Sep/15 ]

Author:

{u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}

Message: SERVER-19855 Make shards append last known config opTime on command metadata response
Branch: master
https://github.com/mongodb/mongo/commit/85cfca58ffe522f13118d715532f7eb8a45ad0c1

Comment by Githook User [ 17/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19905 SERVER-19855 Put config server optime into its own subobject when sending with commands
Branch: master
https://github.com/mongodb/mongo/commit/3619498a9f4ff383cdd546a320abbec11a511bc6

Comment by Githook User [ 17/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 Advance catalog manager optime on shards based on information sent from mongos
Branch: master
https://github.com/mongodb/mongo/commit/0e0a4e5e8a70ce1f6208c613d2897186a0b8a1c3

Comment by Githook User [ 17/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 SERVER-20285 Make write commands send shard version information at the top level
Branch: master
https://github.com/mongodb/mongo/commit/e44bf3dc351624ff26968a8006dfa385ffc83516

Comment by Githook User [ 17/Sep/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 Make ShardForceVersionOkModeBlock use the OperationShardVersion instead of the ShardedConnectionInfo
Branch: master
https://github.com/mongodb/mongo/commit/9c9356b7887b3fb633203a229007a2b4e225d12f

Comment by Githook User [ 25/Aug/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 Use ShardConnection for migrations and splits

This reverts commit 169a7ca486099a93e8d3ea19cc556dc2c55fcd0e.
Branch: master
https://github.com/mongodb/mongo/commit/0d61a386620b9403b5fd19f276ded78ba766c2e8

Comment by Githook User [ 25/Aug/15 ]

Author:

{u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}

Message: SERVER-19855 Do not perform shard version checking where not necessary

This reverts commit 045cd1070cae1e7827255850c2fe35194e48b24e.
Branch: master
https://github.com/mongodb/mongo/commit/d64c98deed2440ae0f3700dc373bf8b97886030c

Comment by Githook User [ 22/Aug/15 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: Revert "SERVER-19855 Use ShardConnection for migrations and splits"

This reverts commit fdc2aec174d2facc5edeacd2b7f4c078836f1dfb.
Branch: master
https://github.com/mongodb/mongo/commit/169a7ca486099a93e8d3ea19cc556dc2c55fcd0e

Comment by Githook User [ 22/Aug/15 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: Revert "SERVER-19855 Do not perform shard version checking where not necessary"

This reverts commit 31716d2ae526d82d7d36464f6c9fae8b9f38542f.
Branch: master
https://github.com/mongodb/mongo/commit/045cd1070cae1e7827255850c2fe35194e48b24e

Comment by Githook User [ 21/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: Revert "Revert "SERVER-19855 Use ShardConnection for migrations and splits""

This reverts commit bae1d256c52d42e2e095d127f5bbec94abb68e05.
Branch: master
https://github.com/mongodb/mongo/commit/fdc2aec174d2facc5edeacd2b7f4c078836f1dfb

Comment by Githook User [ 21/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-19855 Do not perform shard version checking where not necessary

The code in RangeDeleter and sharding migration and split uses
AutoGetCollectionForRead, which has the side effect of checking for shard
version, based on the context. This causes problems in the cases where we
are transmitting shard version information as part of the context, in
particular during the migration cases.

This change gets rid of these checks and replaces them with plain lock
retrieval.
Branch: master
https://github.com/mongodb/mongo/commit/31716d2ae526d82d7d36464f6c9fae8b9f38542f

Comment by Githook User [ 21/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: Revert "SERVER-19855 Use ShardConnection for migrations and splits"

This reverts commit f8cdaf32b6c44528123d8ffef7ea801a2f611745.
Branch: master
https://github.com/mongodb/mongo/commit/bae1d256c52d42e2e095d127f5bbec94abb68e05

Comment by Githook User [ 21/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-19855 Use ShardConnection for migrations and splits

This ensures that prior to sending the operation on the wire, the host
will be primed to know that it is part of a sharded cluster.
Branch: master
https://github.com/mongodb/mongo/commit/f8cdaf32b6c44528123d8ffef7ea801a2f611745

Comment by Githook User [ 18/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-19855 Remove 'shardName' field from write commands' metadata

It is not necessary, because it already goes over an initialized
connection, which would have performed the shard name validation.
Branch: master
https://github.com/mongodb/mongo/commit/beff0ee6a7a044bc44953a5bd383479d749e8471

Comment by Githook User [ 17/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-19855 Include min OpTime with shard version

This change adds the OpTime of the chunk manager as of the time its cached
metadata was loaded along with each versioned request sent from MongoS.
This includes write commands and the setShardVersion command.

The OpTime is only sent as part of this change. There will be a follow-up
change to add code to interpret this information on the MongoD side.
Branch: master
https://github.com/mongodb/mongo/commit/5be3721da7af1fc5258e9f1753e51324cdd83092

Comment by Githook User [ 14/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: Revert "SERVER-19855 Include min OpTime with shard version"

This reverts commit cec89b290906adb7d36507d20d619aaa5a6f6c16.
Branch: master
https://github.com/mongodb/mongo/commit/1cee34ee3907b7652abdfc7f75ca27ef421a46c4

Comment by Githook User [ 14/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-19855 Include min OpTime with shard version

This change adds the OpTime of the chunk manager as of the time its cached
metadata was loaded along with each versioned request sent from MongoS.
This includes write commands and the setShardVersion command.

The OpTime is only sent as part of this change. There will be a follow-up
change to add code to interpret this information on the MongoD side.
Branch: master
https://github.com/mongodb/mongo/commit/cec89b290906adb7d36507d20d619aaa5a6f6c16

Comment by Githook User [ 14/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-19855 Make write_ops have their own SConscript

Also fixes some includes and dependencies.
Branch: master
https://github.com/mongodb/mongo/commit/914c12f4c7e850cacc6292bc330101579a0627ef

Comment by Githook User [ 13/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-19855 Move dropCollection implementation out of CatalogManager
Branch: master
https://github.com/mongodb/mongo/commit/312fe328972341c071516b2e92227325471293ea

Comment by Githook User [ 12/Aug/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-19855 Parsing/serialization logic for OpTime

This change pulls the OpTime serialization/deserialization logic to be
part of the class.
Branch: master
https://github.com/mongodb/mongo/commit/2482d4387b6a0fcdf825e07d7f5260041a89f3f6

Generated at Thu Feb 08 03:52:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.