Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22167

Failed to insert document larger than 256k

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.3, 3.3.2
    • Component/s: Storage
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:
    • Steps To Reproduce:
      Hide

      Use node.js MongoDB driver cannot reproduce the problem. It seems use bulkwrite works fine.

      With POCO MongoDB connector to insert a document larger than 256k can reproduce the problem.

      Show
      Use node.js MongoDB driver cannot reproduce the problem. It seems use bulkwrite works fine. With POCO MongoDB connector to insert a document larger than 256k can reproduce the problem.
    • Sprint:
      Integration 10 (02/22/16)

      Description

      Error:

      Collection::insertDocument got document without _id for ns...

      We are using POCO MongoDB connector to write/read MongoDB.
      http://pocoproject.org/docs-1.6.0/Poco.MongoDB.html

      It works fine 2.x, 3.0 and 3.1 version. But failed to insert large document above 256k from MongoDB 3.2 version.

      The problem we found is because mongo does not generate index ID so the exception happened in Collection::insertDocuments:

      for (auto it = begin; it != end; it++) {
              if (hasIdIndex && (*it)["_id"].eoo()) {
                  return Status(ErrorCodes::InternalError,
                                str::stream() << "Collection::insertDocument got "
                                                 "document without _id for ns:" << _ns.ns());
              }
       
              auto status = checkValidation(txn, *it);
              if (!status.isOK())
                  return status;
          }
      

      From debugging we found the insert process in insertMulti function go directly to insertMultiSingletons which use insertDocument without generating index ID. It seems not right. Only document larger than 256K has the problem because insertVectorMaxBytes is 256k.

      I have attached a test file written by POCO C++ to replicate this problem.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: