[SERVER-18813] ClusterPipelineCmd can fail to throw RecvStaleConfigException when merging cursors Created: 03/Jun/15  Updated: 19/Sep/15  Resolved: 03/Jun/15

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Sharding
Affects Version/s: 3.1.3
Fix Version/s: 3.1.4

Type: Bug Priority: Major - P3
Reporter: Randolph Tan Assignee: Adam Midvidy
Resolution: Done Votes: 0
Labels: UT
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File testshard1.log    
Issue Links:
Related
is related to SERVER-18814 ClusterPipelineCmd calls find on $cmd... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Platform 4 06/05/15
Participants:

 Description   

As ClusterPipelineCmd does not use runCommand to run aggregation commands on shards, it must correctly handle SendStaleConfig exception. The ClusterPipelineCmd::aggPassthrough calls PipelineCommand::aggRunCommand then checks if the response indicates the need to throw a RecvStaleConfigException. However, the run() method makes a bare call to aggRunCommand when merging cursors. Due to the format of command errors changing (SERVER-18664), the StaleConfigError is getting returned back to the client.

Was:

 m30000| 2015-06-03T13:09:07.989-0400 W -        [conn3] DBException thrown :: caused by :: 13388 [aggShard.ts1] shard version not ok: version mismatch detected for aggShard.ts1, stored major version 14 does not match received 13 ( ns : aggShard.ts1, received : 13|1||556f346b4f7b317b44b97613, wanted : 14|0||556f346b4f7b317b44b97613, send )
 m30000| 2015-06-03T13:09:08.005-0400 I -        [conn3]
 m30000|  0xfd9616 0xf750c0 0x891eeb 0xee5e83 0x9f1862 0x9f1acc 0x9a2911 0x9f3024 0x9f474d 0x9f50da 0xac9f6f 0xacc79f 0x81133a 0xf9bef6 0x7f5316cf073d 0x7f5316a66d1d
 m30000| ----- BEGIN BACKTRACE -----
 m30000| {"backtrace":[{"b":"400000","o":"BD9616"},{"b":"400000","o":"B750C0"},{"b":"400000","o":"491EEB"},{"b":"400000","o":"AE5E83"},{"b":"400000","o":"5F1862"},{"b":"400000","o":"5F1ACC"},{"b":"400000","o":"5A2911"},{"b":"400000","o":"5F3024"},{"b":"400000","o":"5F474D"},{"b":"400000","o":"5F50DA"},{"b":"400000","o":"6C9F6F"},{"b":"400000","o":"6CC79F"},{"b":"400000","o":"41133A"},{"b":"400000","o":"B9BEF6"},{"b":"7F5316CEA000","o":"673D"},{"b":"7F5316993000","o":"D3D1D"}],"processInfo":{ "mongodbVersion" : "3.1.4-pre-", "gitVersion" : "b9ed79089841c5c05086a490c1323514f342bb42", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "2.6.32-431.3.1.el6.x86_64", "version" : "#1 SMP Fri Jan 3 21:39:27 UTC 2014", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000" }, { "b" : "7FFF3E4FF000", "elfType" : 3 }, { "b" : "7F531789A000", "path" : "/lib64/librt.so.1", "elfType" : 3 }, { "b" : "7F5317696000", "path" : "/lib64/libdl.so.2", "elfType" : 3 }, { "b" : "7F5317396000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3 }, { "b" : "7F5317113000", "path" : "/lib64/libm.so.6", "elfType" : 3 }, { "b" : "7F5316F05000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3 }, { "b" : "7F5316CEA000", "path" : "/lib64/libpthread.so.0", "elfType" : 3 }, { "b" : "7F5316993000", "path" : "/lib64/libc.so.6", "elfType" : 3 }, { "b" : "7F5317AA3000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3 } ] }}
 m30000|  mongod(_ZN5mongo15printStackTraceERSo+0x26) [0xfd9616]
 m30000|  mongod(_ZN5mongo11DBException13traceIfNeededERKS0_+0x180) [0xf750c0]
 m30000|  mongod(_ZN5mongo20StaleConfigExceptionC1ERKSsS2_iNS_12ChunkVersionES3_+0x2AB) [0x891eeb]
 m30000|  mongod(_ZN5mongo27ensureShardVersionOKOrThrowERKSs+0x463) [0xee5e83]
 m30000|  mongod(_ZN5mongo24AutoGetCollectionForRead5_initERKSsNS_10StringDataE+0x52) [0x9f1862]
 m30000|  mongod(_ZN5mongo24AutoGetCollectionForReadC2EPNS_16OperationContextERKSs+0x11C) [0x9f1acc]
 m30000|  mongod(_ZN5mongo15PipelineCommand3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderE+0x231) [0x9a2911]
 m30000|  mongod(_ZN5mongo7Command3runEPNS_16OperationContextERKNS_7BSONObjERKNS_3rpc16RequestInterfaceEPNS6_21ReplyBuilderInterfaceE+0x424) [0x9f3024]
 m30000|  mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE+0xF8D) [0x9f474d]
 m30000|  mongod(_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE+0x21A) [0x9f50da]
 m30000|  mongod(+0x6C9F6F) [0xac9f6f]
 m30000|  mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x81F) [0xacc79f]
 m30000|  mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0xBA) [0x81133a]
 m30000|  mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x276) [0xf9bef6]
 m30000|  libpthread.so.0(+0x673D) [0x7f5316cf073d]
 m30000|  libc.so.6(clone+0x6D) [0x7f5316a66d1d]
 m30000| -----  END BACKTRACE  -----
 m30000| 2015-06-03T13:09:08.005-0400 I COMMAND  [conn3] assertion while executing command 'aggregate' on database 'aggShard' :13388 [aggShard.ts1] shard version not ok: version mismatch detected for aggShard.ts1, stored major version 14 does not match received 13 ( ns : aggShard.ts1, received : 13|1||556f346b4f7b317b44b97613, wanted : 14|0||556f346b4f7b317b44b97613, send )
2015-06-03T13:09:08.009-0400 E QUERY    [thread1] Error: stale config in runCommand ( ns : aggShard.ts1, received : 0|0||000000000000000000000000, wanted : 0|0||000000000000000000000000, recv )
    at DB.runCommand (src/mongo/shell/db.js:116:32)
    at DBCollection._dbCommand (src/mongo/shell/collection.js:110:21)
    at DBCollection.<anonymous> (src/mongo/shell/collection.js:1283:18)
    at DBCollection.aggregate (src/mongo/shell/collection.js:1286:15)
    at jstests/aggregation/testshard1.js:272:12
    at jstests/aggregation/testshard1.js:274:2 at src/mongo/shell/db.js:126
failed to load: jstests/aggregation/testshard1.js



 Comments   
Comment by Githook User [ 03/Jun/15 ]

Author:

{u'username': u'amidvidy', u'name': u'Adam Midvidy', u'email': u'amidvidy@gmail.com'}

Message: SERVER-18813 move check for stale config in to aggRunCommand
Branch: master
https://github.com/mongodb/mongo/commit/99167a78d937ceaf7917f1f759d5291cbe2f5b66

Generated at Thu Feb 08 03:48:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.