Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-1475

{field: {$type:"array"}} should return documents where "field" is an array

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 1.4.4
    • Fix Version/s: 3.5.7
    • Component/s: Querying
    • Labels:
      None
    • Environment:
      Ubuntu 10.04(lucid), x64
    • Backwards Compatibility:
      Minor Change
    • Operating System:
      ALL
    • Sprint:
      Query 2017-05-08, Query 2017-05-29

      Description

      Operator $type does not finds array (type=4) elements.

      It tested db, 'aList' field is array. There are records with empty arrays and not empty.
      But request to count elemets by type = array brings zero (0) results;
      >db.mydata.count({ aList: {$type: 4 } }); //type = Array
      0

      Meantime, the following check shows that there are arrays, and som of them are non-empty:
      > var obj = db.mydata.count({ aList: {$type:3} }); //type = Object
      > obj
      33276

      > var zero = db.mydata.count({ aList: {$size: 0 } }); //empty arrays
      > zero
      421251
      > var all = db.mydata.count({ }); //all records
      > all
      454527
      > all - zero // number of non- empty arrays
      33276

      As you see, obj == (all - zero) - all arrays are treated as Objects(type=3)!
      Expected: array fileds should be treated as Arrays (type=4).

        Issue Links

          Activity

          Hide
          tomzahn Thomas Zahn added a comment - - edited

          That's a nice work-around, however it doesn't work for nested arrays.
          Consider these arrays:

          mongos> db.test1.find()
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebeef"), "a" : [  7,  10 ] }
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef0"), "a" : [  1,  2,  3 ] }
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef1"), "a" : [  [  10,  11 ],  [  20,  21 ] ] }
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef2"), "a" : [  [  1 ],  [  2 ],  [  3 ] ] }
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef3"), "a" : [  [ ] ] }

          Suppose one now wanted to explicitly find those documents where a specific element of a (e.g. a.0) is again an array, i.e. a 2-dimensional array. Unfortunately, only $size seems to work on a.0, whereas $elemMatch fails:

          mongos> db.test1.find({$or: [{"a.0": {$size: 0}}, {"a.0": {$elemMatch: {$exists: true}}}]})
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef3"), "a" : [  [ ] ] }
          mongos> db.test1.find({"a.0": {$size: 0}})
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef3"), "a" : [  [ ] ] }
          mongos> db.test1.find({"a.0": {$elemMatch: {$exists: true}}})
          mongos>

          This seems related to the following reported bug in $elemMatch: SERVER-1264.
          So, yes, a new $isArray operator would be great.

          Show
          tomzahn Thomas Zahn added a comment - - edited That's a nice work-around, however it doesn't work for nested arrays. Consider these arrays: mongos> db.test1.find() { "_id" : ObjectId("52fcf0e3f067bdb1472ebeef"), "a" : [ 7, 10 ] } { "_id" : ObjectId("52fcf0e3f067bdb1472ebef0"), "a" : [ 1, 2, 3 ] } { "_id" : ObjectId("52fcf0e3f067bdb1472ebef1"), "a" : [ [ 10, 11 ], [ 20, 21 ] ] } { "_id" : ObjectId("52fcf0e3f067bdb1472ebef2"), "a" : [ [ 1 ], [ 2 ], [ 3 ] ] } { "_id" : ObjectId("52fcf0e3f067bdb1472ebef3"), "a" : [ [ ] ] } Suppose one now wanted to explicitly find those documents where a specific element of a (e.g. a.0 ) is again an array, i.e. a 2-dimensional array. Unfortunately, only $size seems to work on a.0 , whereas $elemMatch fails: mongos> db.test1.find({$or: [{"a.0": {$size: 0}}, {"a.0": {$elemMatch: {$exists: true}}}]}) { "_id" : ObjectId("52fcf0e3f067bdb1472ebef3"), "a" : [ [ ] ] } mongos> db.test1.find({"a.0": {$size: 0}}) { "_id" : ObjectId("52fcf0e3f067bdb1472ebef3"), "a" : [ [ ] ] } mongos> db.test1.find({"a.0": {$elemMatch: {$exists: true}}}) mongos> This seems related to the following reported bug in $elemMatch : SERVER-1264 . So, yes, a new $isArray operator would be great.
          Hide
          theshuss aurelien lambert added a comment -

          The javascript workaround is as said not efficient, and then non javascript workaround requires an $or at the root, which makes it tricky if one needs to check if several fields are arrays.

          Wouldn't it be more simple if the $all:[] could do it ? It would look like

          > db.test1.find()
          { "a" : [ 0 ] }
          { "a" : [ ] }
          { "a" : 0 }
          > db.test1.find({a : {$all : []}})
          { "a" : [ 0 ] }
          { "a" : [ ] }

          Isn't it simple ?

          Show
          theshuss aurelien lambert added a comment - The javascript workaround is as said not efficient, and then non javascript workaround requires an $or at the root, which makes it tricky if one needs to check if several fields are arrays. Wouldn't it be more simple if the $all:[] could do it ? It would look like > db.test1.find() { "a" : [ 0 ] } { "a" : [ ] } { "a" : 0 } > db.test1.find({a : {$all : []}}) { "a" : [ 0 ] } { "a" : [ ] } Isn't it simple ?
          Hide
          asya Asya Kamsky added a comment -

          As of 3.4 the proposed workaround works fine on two dimensional arrays too:

          db.test1.find({$or: [{"a.0": {$size: 0}}, {"a.0": {$elemMatch: {$exists: true}}}]})
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef1"), "a" : [ [ 10, 11 ], [ 20, 21 ] ] }
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef2"), "a" : [ [ 1 ], [ 2 ], [ 3 ] ] }
          { "_id" : ObjectId("52fcf0e3f067bdb1472ebef3"), "a" : [ [ ] ] }
          Fetched 3 record(s) in 2ms
          

          Show
          asya Asya Kamsky added a comment - As of 3.4 the proposed workaround works fine on two dimensional arrays too: db.test1.find({$or: [{"a.0": {$size: 0}}, {"a.0": {$elemMatch: {$exists: true}}}]}) { "_id" : ObjectId("52fcf0e3f067bdb1472ebef1"), "a" : [ [ 10, 11 ], [ 20, 21 ] ] } { "_id" : ObjectId("52fcf0e3f067bdb1472ebef2"), "a" : [ [ 1 ], [ 2 ], [ 3 ] ] } { "_id" : ObjectId("52fcf0e3f067bdb1472ebef3"), "a" : [ [ ] ] } Fetched 3 record(s) in 2ms
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

          Message: SERVER-1475 Make $type:'array' match outer arrays.

          Prior to this change, the semantics of $type were that it
          only matches documents with nested arrays, e.g. {x: {$type:
          'array'}} would match

          {x: [1, [2, 3]]}

          but not

          {x: [1, 2, 3]}

          . This is inconsistent with the matching semantics for
          other query predicate operators, which apply both to the
          individual array elements and the array as a whole.

          The new behavior is that single-level arrays as well as
          nested arrays will match {$type:'array'}. This is a breaking
          change for applications which rely on the old behavior.
          Branch: master
          https://github.com/mongodb/mongo/commit/c6d0c0dd01c328d8ed1a5744485b29826b1fb2c9

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'} Message: SERVER-1475 Make $type:'array' match outer arrays. Prior to this change, the semantics of $type were that it only matches documents with nested arrays, e.g. {x: {$type: 'array'}} would match {x: [1, [2, 3]]} but not {x: [1, 2, 3]} . This is inconsistent with the matching semantics for other query predicate operators, which apply both to the individual array elements and the array as a whole. The new behavior is that single-level arrays as well as nested arrays will match {$type:'array'}. This is a breaking change for applications which rely on the old behavior. Branch: master https://github.com/mongodb/mongo/commit/c6d0c0dd01c328d8ed1a5744485b29826b1fb2c9
          Hide
          david.storch David Storch added a comment - - edited

          This change will affect users on 3.4.x who have partial indexes whose partialFilterExpression has a $type:"array" expression and wish to upgrade to 3.6. As part of the upgrade process, such users must rebuild any such partial index when first starting a 3.6 node. Failing to do so could cause the user to experience a mix of the old and new $type:"array" semantics. We should clearly document this wrinkle in the 3.6 compatibility notes so that users who may be affected can take the appropriate action.

          Note that this also applies to $type:4, since 4 is the type code for BSON arrays.

          Show
          david.storch David Storch added a comment - - edited This change will affect users on 3.4.x who have partial indexes whose partialFilterExpression has a $type:"array" expression and wish to upgrade to 3.6. As part of the upgrade process, such users must rebuild any such partial index when first starting a 3.6 node. Failing to do so could cause the user to experience a mix of the old and new $type:"array" semantics. We should clearly document this wrinkle in the 3.6 compatibility notes so that users who may be affected can take the appropriate action. Note that this also applies to $type:4 , since 4 is the type code for BSON arrays.

            People

            • Votes:
              14 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                  Agile