Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.6
    • Component/s: Querying
    • Labels:
    • Environment:
      All
    • Backwards Compatibility:
      Fully Compatible

      Description

      We've decided to implement a few bit-test query operators:

      $bitsAllSet - matches if the input bit positions are all 1
      $bitsAllClear - matches if the input bit positions are all 0
      $bitsAnySet - matches if any of the bit positions are 1
      $bitsAnyClear - matches if any of the bit positions are 0

      These operators can take an array of bit positions, a 64-bit number bitmask, or a BinData bitmask. For the number and BinData bitmasks, the bit positions to check are the positions of 1's in their binary representation.

      The operators will only match against int32, int64, doubles, and binary data. Other types will not match.

      Note that bit position 0 means the least significant bit.

      An example using the array of bit positions syntax:

      db.foo.insert( { a: 43 } ); // Binary form: 101011 (leading zeroes truncated)
      db.foo.find( { a: { $bitsAllSet: [0, 1, 3] } }, { _id: 0, a: 1 } );
      Result: { a: 43 }
      

      An example using the bitmask syntax:

      db.foo.insert( { a: 43 } ); // Binary form: 101011 (leading zeroes truncated)
      db.foo.find( { a: { $bitsAllSet: 9 } }, { _id: 0, a: 1 } ); // Binary form of the bitmask: 1001
      Result: { a: 43 }
      

        Issue Links

          Activity

          Hide
          tpneumat Jeremy added a comment -

          Glad to see progress here, but without using an index, will this perform any better than the previous solution of using the $where. e.g. '$where'=>'!(this.code & 8)’?

          Bitwise operators that don't use indexes will not help us at all. Will stick to array list of values until this happens.

          Show
          tpneumat Jeremy added a comment - Glad to see progress here, but without using an index, will this perform any better than the previous solution of using the $where. e.g. '$where'=>'!(this.code & 8)’? Bitwise operators that don't use indexes will not help us at all. Will stick to array list of values until this happens.
          Hide
          asya Asya Kamsky added a comment - - edited

          Since $where involves JavaScript and these operators would run on the server natively, $where will likely be slower in all cases.

          Don't forget that in cases where there are other conditions in addition to the bitwise operator, those conditions can use available indexes, meanwhile if using array of list values is working well for you, there should be no reason to switch.

          In addition, $where cannot be used in some cases where the bitwise operators can be used (i.e. aggregation framework $match phase, in nested documents ie $elemMatch).

          Show
          asya Asya Kamsky added a comment - - edited Since $where involves JavaScript and these operators would run on the server natively, $where will likely be slower in all cases. Don't forget that in cases where there are other conditions in addition to the bitwise operator, those conditions can use available indexes, meanwhile if using array of list values is working well for you, there should be no reason to switch. In addition, $where cannot be used in some cases where the bitwise operators can be used (i.e. aggregation framework $match phase, in nested documents ie $elemMatch).
          Hide
          nifan nifan added a comment -

          IMO the array of list values is a workaround in itself for these cases which we would like to get rid of.

          An array will not work with capped collections, since adding a flag will change the size of the document.
          In non-capped collections it may trigger a relocation or (with powerof2 now default) at the very least a rewrite of the document.

          Which should all performance significantly worse then an in-place update on an int or byte field.

          The index would probably be much more compact then indexing a list of things and querying by bitmask probably faster then doing a ["foo", "bar", "egg"] in some-list for each document.

          For my use cases I'm with Jeremy; I really like the new operators but they are not very helpful if they cannot use an index.

          Show
          nifan nifan added a comment - IMO the array of list values is a workaround in itself for these cases which we would like to get rid of. An array will not work with capped collections, since adding a flag will change the size of the document. In non-capped collections it may trigger a relocation or (with powerof2 now default) at the very least a rewrite of the document. Which should all performance significantly worse then an in-place update on an int or byte field. The index would probably be much more compact then indexing a list of things and querying by bitmask probably faster then doing a ["foo", "bar", "egg"] in some-list for each document. For my use cases I'm with Jeremy; I really like the new operators but they are not very helpful if they cannot use an index.
          Hide
          asya Asya Kamsky added a comment -

          Regarding index use by $bitwise query operators, please create a new ticket requesting this new feature. Keep in mind that since currently in MongoDB Indexes hold the values that are searched for it would not be a trivial task to implement it.

          nifan discussion of performance of different ways of implementing this probably fits better on mongodb-user google group rather than in this ticket.

          Show
          asya Asya Kamsky added a comment - Regarding index use by $bitwise query operators, please create a new ticket requesting this new feature. Keep in mind that since currently in MongoDB Indexes hold the values that are searched for it would not be a trivial task to implement it. nifan discussion of performance of different ways of implementing this probably fits better on mongodb-user google group rather than in this ticket.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'coollog', u'name': u'Qingyang Chen', u'email': u'qingyang.chen@10gen.com'}

          Message: SERVER-3518 Tests for full code coverage in bit test query operators
          Branch: master
          https://github.com/mongodb/mongo/commit/d09c418c8b5fb5ccc7c2864cac8367d1f849bef1

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'coollog', u'name': u'Qingyang Chen', u'email': u'qingyang.chen@10gen.com'} Message: SERVER-3518 Tests for full code coverage in bit test query operators Branch: master https://github.com/mongodb/mongo/commit/d09c418c8b5fb5ccc7c2864cac8367d1f849bef1

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              1 week, 5 days ago
              Date of 1st Reply:

              Agile