Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22078

Remove term list limits for text index in FCV 4.2

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Major - P3
    • Resolution: Done
    • 3.0.1, 3.1.0
    • 4.1.2
    • MMAPv1, Text Search
    • None
    • Fully Compatible
    • Storage NYC 2018-08-13, Storage NYC 2018-08-27

    Description

      SERVER-8431 restricted inserting a document into a collection with a text index under the following circumstances:

      • The number of unique terms in the fields indexed by the text index is greater than 400,000.
      • The total size of those terms is greater than 4MB.

      This was done to avoid causing the server to shut down from generating a group commit larger than 512MB. However, we have since increased the journaling capacity limit to 2000MB as part of SERVER-17501. We should considering raising these limits to allows users to store more text data in a single document.

      Steps to reproduce

      db.foo.drop();
      		 
      db.foo.createIndex({a: 'text'});
      		 
       
      		 
      var termLength = 16;
      		 
       
      		 
      var str = '';
      		 
      for (var i = 0; i < (5 * 1024 * 1024 / termLength); i++) {
      		 
          if (i > 0) {
      		 
              str += ' ';
      		 
          }
      		 
          str += i.zeroPad(termLength);
      		 
      }
      		 
       
      		 
      assert.writeOK(db.foo.insert({a: str}));


      assert: write failed with error: {
      		 
          "nInserted" : 0,
      		 
          "writeError" : {
      		 
              "code" : 16733,
      		 
              "errmsg" : "trying to index text where term list is too big, max is 4mb _id: ObjectId('568d68d5dd2e24af1002e247')"
      		 
          }
      		 
      }
      		 
      _getErrorWithCode@src/mongo/shell/utils.js:23:13
      		 
      doassert@src/mongo/shell/assert.js:13:14
      		 
      assert.writeOK@src/mongo/shell/assert.js:414:9

      Attachments

        Issue Links

          is documented by

          Task - A task that needs to be done. DOCS-11991 Docs for SERVER-22078: Remove term list limits for text index in FCV 4.2

          • Major - P3 - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-8431 Large update to text index can cause group commit to exceed max size

          • Major - P3 - Major loss of function.
          • Closed

          Task - A task that needs to be done. SERVER-36440 Completely remove code and tests about text index limit

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-17501 Increase journalling capacity limits

          • Critical - P2 - Crashes, loss of data, severe memory leak.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-3372 Allow indexing fields of arbitrary length

          • Major - P3 - Major loss of function.
          • Closed

          Activity

            People

              xiangyu.yao@mongodb.com Xiangyu Yao (Inactive)
              max.hirschhorn@mongodb.com Max Hirschhorn
              Votes:
              2 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: