Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-94

Tailable Cursor on the Oplog Does not Respect skip() or sort()

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 1.4
    • Affects Version/s: 1.3
    • Component/s: None
    • Labels:
    • Environment:
      node v0.10.17, mongo 2.2.0 || 2.4.8

      I'm trying to find a way to establish a tailable cursor on the oplog which will ignore all previous events, and only capture deltas from this point forward.

      I'm able to do this using the 'ts' field by looking for anything > current time, however it seems that this incurs a performance penalty of about 250ms. This SO answer says I should be using the natural order instead:

      http://stackoverflow.com/questions/15389325/index-on-ts-field-in-oplog-rs-is-not-updated

      From here, I've been experimenting to find out how I can filter out the results. I tried reverse sorting by natural:

      sort.javascript
      cursor = oplog.find({}, connOpts).sort({$natural: -1})
      cursor = oplog.find({}, connOpts).sort({$natural: 1}) 
      

      Whether I do 1 or -1, this makes no difference. When I call cursor.nextObject I get records starting with:

      record.json
      {
        ts: { _bsontype: 'Timestamp', low_: 1, high_: 1362604692 },
        h: 0,
        op: 'n',
        ns: '',
        o: { msg: 'initiating set' }
      }
      

      Which clearly is the first record that was ever recorded in the oplog.

      I tried running a count first, and then skipping that many records:

      skip.javascript
      oplog.find({}).count(function(err, count) {
        cursor = oplog.find({}, connOpts).skip(count);
        cursor.nextObject(function(err, obj) {
          console.log(obj);
        });
      })
      

      In my case the skip should be moving past about 244910 records, however the skip() appears to have no effect. I still get the first record from the oplog.

      I'm at a loss here as to how I can achieve the behavior I want without incurring the performance penalty of a non-indexed query. Again, ideally I'd like to not have to use skip at all... it would be nice to not retrieve all the records I don't need.

      Thanks,
      Brandon

            Assignee:
            christkv Christian Amor Kvalheim
            Reporter:
            bitmage Brandon Mason
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: