Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-89408

WindowStage estimated memory should not be negative

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 8.1.0-rc0, 8.0.0-rc3
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Query Execution
    • Fully Compatible
    • ALL
    • v8.0
    • Hide

      Query

      [
              {
                  $setWindowFields: {
                      sortBy: {
                          "num": -1
                      },
                      output: {
                          "minStr": {
                              $min: "$str"
                          }
                      }
                  }
              }
          ]
      

      Data

      [
          {_id: 0, "num": 10, "str": "ABCDEFGHIJK" },
          {_id: 1, "num": 3, "str": "ABCDE"},
          {_id: 2, "num": 5, "str": "AB"},
      ]
      
      Show
      Query [ { $setWindowFields: { sortBy: { "num" : -1 }, output: { "minStr" : { $min: "$str" } } } } ] Data [ {_id: 0, "num" : 10, "str" : "ABCDEFGHIJK" }, {_id: 1, "num" : 3, "str" : "ABCDE" }, {_id: 2, "num" : 5, "str" : "AB" }, ]
    • 166

      The window stage estimates the memory Y that a window frame of size X will need using linear regression and the formula Y = aX + b. It computes the coefficients a and b incrementally sampling the documents that are added in the window. Every time a new document is added in the window the estimated memory is computed and compared against a threshold.

      The problem exists only in the case when the window is not removable. In this case, the samples consider the size of the frame (i.e the total number of documents) but the memory of the last seen document only. When the memory of the new documents is smaller than the memory of the previous documents then the memory and the window frame size are seen to be reverse proportional, i.e. the window frame increases and the memory decreases, and the linear regression formula gives negative results. 

      The implementation of the memory estimation should be updated to correctly consider the memory in the case of the non removable windows.

            Assignee:
            foteini.alvanaki@mongodb.com Foteini Alvanaki
            Reporter:
            foteini.alvanaki@mongodb.com Foteini Alvanaki
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: