Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-35590

GlobalWrite during MapReduce: outType: reduce nonAtomic: true

    XMLWordPrintable

    Details

    • Type: Question
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Won't Fix
    • Affects Version/s: 3.6.5
    • Fix Version/s: None
    • Component/s: MapReduce
    • Labels:
      None

      Description

      I check the source code in the file mr.cpp and I can see

      that a Map-Reduce with nonAtomic: true and outType: reduce can generate a Global Lock.

      In the comment in the code is written: "This must be global because we may write across different databases."

      My question is how can a map reduce will write across different databases, if it is needed to specified the output database of the map reduce?

      This Global locks has a lot of influence in the performance of  another operations that are I am running (make them slow).

      Code (mr.cpp):

      if (_config.outputOptions.outType == Config::REDUCE) {
       // reduce: apply reduce op on new result and existing one
       BSONList values;
       
      {
       const auto count = collectionCount(opCtx, _config.tempNamespace, callerHoldsGlobalLock);
       stdx::lock_guard<Client> lk(*opCtx->getClient());
       curOp->setMessage_inlock(
       "m/r: reduce post processing", "M/R Reduce Post Processing Progress", count);
       }
       unique_ptr<DBClientCursor> cursor = _db.query(_config.tempNamespace.ns(), BSONObj());
       while (cursor->more()) {
       // This must be global because we may write across different databases.
       {color:#f6c342}Lock::GlobalWrite lock(opCtx);{color}
       BSONObj temp = cursor->nextSafe();
       BSONObj old;
       
       

        Attachments

          Activity

            People

            Assignee:
            asya Asya Kamsky
            Reporter:
            rribeiro Rui Ribeiro
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: