[SERVER-18283] Cleanup the sharding map/reduce implementation Created: 30/Apr/15  Updated: 06/Dec/22  Resolved: 16/Dec/19

Status: Closed
Project: Core Server
Component/s: MapReduce, Sharding
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: [DO NOT USE] Backlog - Sharding Team
Resolution: Duplicate Votes: 1
Labels: FewerMongosPipelines
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-42677 M/R Agg: Remove old mongos MR code Closed
Related
Assigned Teams:
Sharding
Participants:

 Description   

The MongoS implementation of map/reduce should be moved to a separate file and it should also be split into smaller stages from the huge monolithic piece of code that it currently is.



 Comments   
Comment by Nicholas Zolnierz [ 14/Oct/19 ]

This will go away as part of PM-766.

Comment by Scott Hernandez (Inactive) [ 05/May/15 ]

As discussed in person, we should fix-up things to be consistent and use "long long" everywhere in our code.

So from the commented code above we can do this:

                splitInfo = cmdObj["splitInfo"].numberLong();

and store Config::splitInfo as "long long" as well.

Comment by Githook User [ 04/May/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-18283 Move sharded aggregation command to separate file
Branch: master
https://github.com/mongodb/mongo/commit/347672f8d6f89956f86f5f33361e9be85062e2bd

Comment by Githook User [ 04/May/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-18283 Profile and map/reduce commands should be in separate files
Branch: master
https://github.com/mongodb/mongo/commit/9cd0baeecb91880ff0d46cc0a6d999fd9ea2827d

Comment by Scott Hernandez (Inactive) [ 04/May/15 ]

Here is an example of the failure: https://evergreen.mongodb.com/task/mongodb_mongo_master_osx_108_d2db2f3827a762daf5b3af1611525f977ef1b551_15_05_04_14_10_08_sharding_osx_108

From: mr.cpp line 270 –

splitInfo = cmdObj["splitInfo"].Int();

 m30001| 2015-05-04T11:45:11.030-0400 I -        [conn5] Assertion: 13111:wrong type for field (splitInfo) 18 != 16
 m30001| ----- BEGIN BACKTRACE -----
 m30001|  mongod(_ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE+0x39) [0x1045cae39]
 m30001|  mongod(_ZN5mongo10logContextEPKc+0x15C) [0x10457bfac]
 m30001|  mongod(_ZN5mongo11msgassertedEiPKc+0x199) [0x1045685b9]
 m30001|  mongod(_ZN5mongo11msgassertedEiRKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE+0x1A) [0x10456841a]
 m30001|  mongod(_ZNK5mongo11BSONElement3chkEi+0x23D) [0x103e6e2fd]
 m30001|  mongod(_ZN5mongo2mr6ConfigC2ERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEERKNS_7BSONObjE+0x361) [0x103fc4ea1]
 m30001|  mongod(_ZN5mongo2mr16MapReduceCommand3runEPNS_16OperationContextERKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEERNS_7BSONObjEiRSA_RNS_14BSONObjBuilderE+0xFF) [0x103fd1cff]
 m30001|  mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEERNS_7BSONObjEiRSA_RNS_14BSONObjBuilderE+0x2A) [0x104029ada]
 m30001|  mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderE+0xC17) [0x10402a8e7]
 m30001|  mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEi+0x497) [0x10402b4c7]
 m30001|  mongod(_ZN5mongo11runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEi+0x29) [0x10402b9e9]
 m30001|  mongod(_ZN5mongoL15receivedCommandEPNS_16OperationContextERKNS_15NamespaceStringERNS_6ClientERNS_10DbResponseERNS_7MessageE+0x2AD) [0x1040ff4bd]
 m30001|  mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x4DB) [0x1040fcc1b]
 m30001|  mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0xBB) [0x103e50a4b]
 m30001|  mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x282) [0x10458c882]
 m30001|  mongod(_ZN5boost12_GLOBAL__N_112thread_proxyEPv+0x8A) [0x1045ffe4a]
 m30001|  libsystem_c.dylib(_pthread_start+0x147) [0x7fff90d027a2]
 m30001|  libsystem_c.dylib(thread_start+0xD) [0x7fff90cef1e1]

Comment by Githook User [ 04/May/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: Revert "SERVER-18283 Profile and map/reduce commands should be in separate files"

This reverts commit d2db2f3827a762daf5b3af1611525f977ef1b551.
Branch: master
https://github.com/mongodb/mongo/commit/935a95b2a5aed52c9b5a04c566a3878208dd3bdf

Comment by Githook User [ 04/May/15 ]

Author:

{u'username': u'kaloianm', u'name': u'Kaloian Manassiev', u'email': u'kaloian.manassiev@mongodb.com'}

Message: SERVER-18283 Profile and map/reduce commands should be in separate files

This change moves the sharding implementation of these two commands to
their own source files. There are no functional changes.
Branch: master
https://github.com/mongodb/mongo/commit/d2db2f3827a762daf5b3af1611525f977ef1b551

Generated at Thu Feb 08 03:47:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.