Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-59505

Time-series query on mixed, nested measurements can miss some events

    • Fully Compatible
    • ALL
    • v5.1, v5.0
    • QO 2021-09-06, QO 2021-10-04, QO 2021-10-18, QO 2021-11-01, QO 2021-11-15
    • 120

      Normally when a measurement field 'x' contains an array, the 'control.min.x' field on the bucket contains the min for each position in the array.

      > db.events.find()
      { x: [0, 2] }
      { x: [3, 1] }
      
      > db.system.buckets.events.find()
      {
        control: {
          min: {x: [0, 1]},
          max: {x: [3, 2]},
        },
        ...
      }
      

      But if 'x' also contains non-arrays, then 'control.min.x' or 'control.max.x' will be a non-array:

      > db.events.find()
      { x: [0, 2] }
      { x: [3, 1] }
      { x: 10 }
      
      > db.system.buckets.events.find()
      {
        control: {
          min: {x: 10},
          max: {x: [3, 2]},
        },
        ...
      }
      

      The predicate pushdowns don't account for this, so "multikey" queries can incorrectly exclude this bucket:

      > db.events.find({ x: {$lt: 5} })
      (no results)
      
      // because internally:
      > db.system.buckets.events.find({$expr: {$lt: ["$control.min.x", 5]}})
      

      A similar thing can happen if 'x' is a mixture of objects and non-objects:

      > db.events.find()
      { "time" : ..., "x" : ISODate("1970-01-01T00:00:00Z"), "_id" : ... }
      { "time" : ..., "x" : { "y" : ISODate("2021-01-02T11:59:59Z") }, "_id" : ...) }
      
      > db.events.find({ 'x.y': {$gt: ISODate('2000-01-01')} })
      (no results)
      

      This happens because although 'control.max.x' is the max of 'x', 'control.max.x.y' is not the max of 'x.y'. ('control.max.x.y' is 'missing', but 'missing' < ISODate.)

            Assignee:
            samuel.mercier@mongodb.com Sam Mercier
            Reporter:
            david.percy@mongodb.com David Percy
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'slack.nextup.jira:slack-integration-plus'. Please contact your Jira administrators.