Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36087

Executing $text statements in conjunction with 'count' or 'sort' provokes Out Of Memory

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.6.6
    • Component/s: Querying
    • None
    • ALL
    • Hide

      1) Create a database with 1 collection

      2) Insert 20 million documents in that collection

      3) documents contain:

      • ID field
      • Text field (with a text index) 'text'
      • Date field (with an index) 'start'

      4) Text field has the value 'andreas.estermann@dynatrace.com' for all documents. Date field values are distributed over time.

      5) Issue the following queries in the mongodb shell:

      db.collection.find({"text": "andreas.estermann@dynatrace.com"})
      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).count()
      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25)
      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25).sort({"start": -1})
      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }})
      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).count()
      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25)
      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25).sort({"start": -1})

      6) Find the following results for the queries

      db.collection.find({"text": "andreas.estermann@dynatrace.com"})

      -> executes within milliseconds, result correct, expected

      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).count()

      -> executes within about 1 minute, result correct, expected

      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25)

      -> executes within milliseconds, result correct, expected

      db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25).sort({"start": -1})

      -> executes within milliseconds, result correct, expected

      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }})

      -> executes within milliseconds, result correct, expected

      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).count()

      -> crashes server because of excessive memory consumption, not expected

      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25)

      -> executes within milliseconds, result correct, expected

      db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25).sort({"start": -1})

      -> crashes server because of excessive memory consumption, not expected

      Show
      1) Create a database with 1 collection 2) Insert 20 million documents in that collection 3) documents contain: ID field Text field (with a text index) 'text' Date field (with an index) 'start' 4) Text field has the value 'andreas.estermann@dynatrace.com' for all documents. Date field values are distributed over time. 5) Issue the following queries in the mongodb shell: db.collection.find({"text": "andreas.estermann@dynatrace.com"}) db.collection.find({"text": "andreas.estermann@dynatrace.com"}).count() db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25) db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25).sort({"start": -1}) db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}) db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).count() db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25) db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25).sort({"start": -1}) 6) Find the following results for the queries db.collection.find({"text": "andreas.estermann@dynatrace.com"}) -> executes within milliseconds, result correct, expected db.collection.find({"text": "andreas.estermann@dynatrace.com"}).count() -> executes within about 1 minute, result correct, expected db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25) -> executes within milliseconds, result correct, expected db.collection.find({"text": "andreas.estermann@dynatrace.com"}).limit(25).sort({"start": -1}) -> executes within milliseconds, result correct, expected db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}) -> executes within milliseconds, result correct, expected db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).count() -> crashes server because of excessive memory consumption, not expected db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25) -> executes within milliseconds, result correct, expected db.collection.find({$text: {$search: "\"andreas.estermann@dynatrace.com\"" }}).limit(25).sort({"start": -1}) -> crashes server because of excessive memory consumption, not expected

      Executing a full text search and then executing additional functions on the result set seems to result in MongoDB using all the server memory which ultimately leads to a crash.

      The machine used to reproduce the issue only has 4 GB of memory. However, the same behavior can be observed on our production machines that have a lot more memory. It just takes longer (up to 40 minutes) until the memory is used up and the server crashes.

        1. diagnostics.tar.gz
          44.82 MB
        2. metrics.2018-08-13T09-48-27Z-00000
          252 kB
        3. metrics.2018-08-13T13-11-05Z-00000
          583 kB
        4. metrics.2018-08-13T14-39-37Z-00000
          405 kB
        5. metrics.interim
          46 kB
        6. metrics.interim
          42 kB
        7. mongod.log
          541 kB
        8. mongod.log
          71 kB

            Assignee:
            david.storch@mongodb.com David Storch
            Reporter:
            estermann@qumram.com Andreas Estermann
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: