[SERVER-18849] find/getMore against config server content cannot use new host targeting logic Created: 05/Jun/15  Updated: 19/Sep/15  Resolved: 10/Sep/15

Status: Closed
Project: Core Server
Component/s: Querying, Sharding
Affects Version/s: None
Fix Version/s: 3.1.8

Type: Task Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: David Storch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: Quint 9 09/18/15
Participants:

 Description   

In MongoDB 3.2 with the switch to supporting config servers as replica sets we will use a uniform host targeting logic to figure out to which host to send a particular operation. This applies to both operations coming from the CatalogManager and operations coming through connections (i.e., queries, updates, etc to the config and admin databases).

This logic will not work in the transient state where customers are upgrading to config servers as replica sets, because they will have a MongoS version 3.2 backed by a legacy (3-host) config server. The shard targeting logic is not intended to work against such legacy config servers and for this reason, only legacy-style queries and update should be supported against the config or admin databases.

It is acceptable behaviour to have the find/getMore commands fail if run against 3.2 MongoS still running with a legacy confing server triplet.

As part of the find/getMore commands work we need to write tests to ensure this upconversion does not happen.



 Comments   
Comment by David Storch [ 18/Sep/15 ]

david.golden, yes, in master right now mongos will always return maxWireVersion 4, regardless of the config server mode. This was part of the following commit:

Message: SERVER-20194 SERVER-18849 add support for querying SCCC mode config servers in the new mongos query path
Branch: master
https://github.com/mongodb/mongo/commit/1422edf755dba283ca300365977e379ddb75a4a7

These are packaged into the same change because adding support for querying SCCC mode config servers in the new find command path implies full find command support, regardless of config server mode.

Comment by David Golden [ 18/Sep/15 ]

A mongod will return a maxWireVersion of 4 to indicate support for the find and
getMore commands. In constrast, mongos will only return maxWireVersion of 4 if
configured with CSRS, as full find/getMore command support is predicated on
config server as a replica set.

I thought this was fixed so that 3.2 mongos always returns maxWireVersion 4 but I don't see that mentioned in a commit message. Is there a commit for that?

David

Comment by Githook User [ 17/Sep/15 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-18849 SERVER-20194 Fix null dereference in ClusterFind
Branch: master
https://github.com/mongodb/mongo/commit/815f7e0f2ebe698061cbe494ed2184c2ab35eedc

Comment by Githook User [ 12/Sep/15 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-18849 SERVER-20194 Temporary fix for a circ. dependency in s/

Temporarily addresses a libmongoscore <=> libcluster_query circular
dependency by library-izing store_possible_cursor.cpp with the
'incomplete' LIBDEPS tag. When CursorCache is deleted, this tag can
be removed.
Branch: master
https://github.com/mongodb/mongo/commit/3105f6403fabbdbcc18acc19dc390525310c3d42

Comment by Githook User [ 10/Sep/15 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-20194 SERVER-18849 add support for querying SCCC mode config servers in the new mongos query path
Branch: master
https://github.com/mongodb/mongo/commit/1422edf755dba283ca300365977e379ddb75a4a7

Comment by David Storch [ 24/Aug/15 ]

Remaining work on this ticket is to ensure that OP_QUERY style find against config server content is not diverted to the new mongos query path unless the config server mode has been upgraded to CSRS.

EDIT: Instead of diverting queries against SCCC config servers to the old query path, the new plan is to add support for querying SCCC config to the new query path. This means that mongos can unconditionally return a maxWireVersion of 4, regardless of the config server mode. It also means that a find command targeting the config shard should always work as expected.

CC david.golden jesse christkv behackett

Comment by Githook User [ 24/Aug/15 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-18849 only send wire version for find/getMore commands if config server mode is CSRS

A mongod will return a maxWireVersion of 4 to indicate support for the find and
getMore commands. In constrast, mongos will only return maxWireVersion of 4 if
configured with CSRS, as full find/getMore command support is predicated on
config server as a replica set.
Branch: master
https://github.com/mongodb/mongo/commit/dd15f9c211f5bc40b6fa8a7cd44350d3aeb87d89

Comment by David Storch [ 20/Aug/15 ]

Any sharding test which explicitly issues queries over config server content will currently fail with a segfault when --readMode commands is enabled:

[js_test:balance_tags1] 2015-08-20T18:33:11.840-0400  m30999| 2015-08-20T18:33:11.835-0400 F -        [conn1] Invalid access at address: 0
[js_test:balance_tags1] 2015-08-20T18:33:11.841-0400  m30999| 2015-08-20T18:33:11.841-0400 F -        [conn1] Got signal: 11 (Segmentation fault).
[js_test:balance_tags1] 2015-08-20T18:33:11.842-0400  m30999|
[js_test:balance_tags1] 2015-08-20T18:33:11.842-0400  m30999|  0xba58cb 0xba523d 0x7fa36eec2340 0xaecc58 0xa7ef01 0xaf3408 0xaf3966 0xb00177 0xaf2005 0x6156e5 0xb555ac 0x7fa36eeba182 0x7fa36ebe747d
[js_test:balance_tags1] 2015-08-20T18:33:11.842-0400  m30999| ----- BEGIN BACKTRACE -----
[js_test:balance_tags1] 2015-08-20T18:33:11.843-0400  m30999| {"backtrace":[{"b":"400000","o":"7A58CB"},{"b":"400000","o":"7A523D"},{"b":"7FA36EEB2000","o":"10340"},{"b":"400000","o":"6ECC58"},{"b":"400000","o":"67EF01"},{"b":"400000","o":"6F3408"},{"b":"400000","o":"6F3966"},{"b":"400000","o":"700177"},{"b":"400000","o":"6F2005"},{"b":"400000","o":"2156E5"},{"b":"400000","o":"7555AC"},{"b":"7FA36EEB2000","o":"8182"},{"b":"7FA36EAED000","o":"FA47D"}],"processInfo":{ "mongodbVersion" : "3.1.7", "gitVersion" : "7d7f4fb3b6f6a171eacf53384053df0fe728db42", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.13.0-46-generic", "version" : "#75-Ubuntu SMP Tue Feb 10 15:24:04 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "09669B29AE8FC1517040B7FA176E287E653040FE" }, { "b" : "7FFFAB296000", "elfType" : 3, "buildId" : "88E7559031E488BC215236F4181BD1FAF9A458F0" }, { "b" : "7FA36F9F6000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "1D76B71E905CB867B27CEF230FCB20F01A3178F5" }, { "b" : "7FA36F7EE000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "92FCF41EFE012D6186E31A59AD05BDBB487769AB" }, { "b" : "7FA36F5EA000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "C1AE4CB7195D337A77A3C689051DABAA3980CA0C" }, { "b" : "7FA36F2E6000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "19EFDDAB11B3BF5C71570078C59F91CF6592CE9E" }, { "b" : "7FA36F0D0000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "8D0AA71411580EE6C08809695C3984769F25725B" }, { "b" : "7FA36EEB2000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "9318E8AF0BFBE444731BB0461202EF57F7C39542" }, { "b" : "7FA36EAED000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "30C94DC66A1FE95180C3D68D2B89E576D5AE213C" }, { "b" : "7FA36FCFC000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "9F00581AB3C73E3AEA35995A0C50D24D59A01D47" } ] }}
[js_test:balance_tags1] 2015-08-20T18:33:11.844-0400  m30999|  mongos(_ZN5mongo15printStackTraceERSo+0x2B) [0xba58cb]
[js_test:balance_tags1] 2015-08-20T18:33:11.844-0400  m30999|  mongos(+0x7A523D) [0xba523d]
[js_test:balance_tags1] 2015-08-20T18:33:11.844-0400  m30999|  libpthread.so.0(+0x10340) [0x7fa36eec2340]
[js_test:balance_tags1] 2015-08-20T18:33:11.844-0400  m30999|  mongos(_ZN5mongo11ClusterFind8runQueryEPNS_16OperationContextERKNS_14CanonicalQueryERKNS_21ReadPreferenceSettingEPSt6vectorINS_7BSONObjESaISA_EE+0x9F8) [0xaecc58]
[js_test:balance_tags1] 2015-08-20T18:33:11.844-0400  m30999|  mongos(+0x67EF01) [0xa7ef01]
[js_test:balance_tags1] 2015-08-20T18:33:11.845-0400  m30999|  mongos(_ZN5mongo7Command22execCommandClientBasicEPNS_16OperationContextEPS0_RNS_11ClientBasicEiPKcRNS_7BSONObjERNS_14BSONObjBuilderE+0x2A8) [0xaf3408]
[js_test:balance_tags1] 2015-08-20T18:33:11.845-0400  m30999|  mongos(_ZN5mongo7Command20runAgainstRegisteredEPNS_16OperationContextEPKcRNS_7BSONObjERNS_14BSONObjBuilderEi+0x106) [0xaf3966]
[js_test:balance_tags1] 2015-08-20T18:33:11.845-0400  m30999|  mongos(_ZN5mongo8Strategy15clientCommandOpEPNS_16OperationContextERNS_7RequestE+0x817) [0xb00177]
[js_test:balance_tags1] 2015-08-20T18:33:11.845-0400  m30999|  mongos(_ZN5mongo7Request7processEPNS_16OperationContextEi+0x425) [0xaf2005]
[js_test:balance_tags1] 2015-08-20T18:33:11.845-0400  m30999|  mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0x65) [0x6156e5]
[js_test:balance_tags1] 2015-08-20T18:33:11.846-0400  m30999|  mongos(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x21C) [0xb555ac]
[js_test:balance_tags1] 2015-08-20T18:33:11.846-0400  m30999|  libpthread.so.0(+0x8182) [0x7fa36eeba182]
[js_test:balance_tags1] 2015-08-20T18:33:11.846-0400  m30999|  libc.so.6(clone+0x6D) [0x7fa36ebe747d]
[js_test:balance_tags1] 2015-08-20T18:33:11.846-0400  m30999| -----  END BACKTRACE  -----

The particular failure mode is that runQueryWithoutRetrying() in cluster_find.cpp is dereferencing a null RemoteCommandTargeter (i.e. the config shard does not have a targeter).

Generated at Thu Feb 08 03:48:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.