Uploaded image for project: 'PHP Legacy Driver'
  1. PHP Legacy Driver
  2. PHP-163

Group doesn't work correctly with initial integers and mongo.native_long = 1 in 64bit environments

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Gone away
    • Affects Version/s: 1.0.9
    • Fix Version/s: None
    • Component/s: pecl-mongo
    • Labels:
      None
    • Environment:
      64bit linux
      Sharded mongo 1.6.3 environment with 1.0.9 php driver
    • # Replies:
      5
    • Last comment by Customer:
      true

      Description

      If you have mongo.native_long set to 1 in 64bit environment, group will fail in reduce because initial value is not number but object and all values are concatenaed into string and not added together:

      ["count"]=> string(1003) "[object
      Object]2874742968653174343283283040123398253734093635923323133340483295743339733494073112782785112259601970261727

      Passing $initial = array('count' => new MongoInt64(0)); doesn't seem to help and the end result is the same.

      This can be worked around by passing $initial = array('count' => 0.0); but that prevents us from using 64bit integers and falling back to floating point values.

      In 32bit environment, this seems to work correctly at least with 1.0.8 and 1.0.10 drivers.

        Activity

        Hide
        kristina Kristina Chodorow (Inactive) added a comment -

        Can you give the full group expression you're using? I'm having a hard time reproducing this. (Also, you might want to try the 1.7 branch, there were some bugs in JS's handling of longs that have been fixed but not backported to 1.6.)

        Show
        kristina Kristina Chodorow (Inactive) added a comment - Can you give the full group expression you're using? I'm having a hard time reproducing this. (Also, you might want to try the 1.7 branch, there were some bugs in JS's handling of longs that have been fixed but not backported to 1.6.)
        Hide
        jalava Jalmari Raippalinna added a comment -

        I fetched some code i had in my SVN that could help you, it's was bit a mess to find something pasteable..

        $keys = new MongoCode('function(doc) { return { "adId": doc._id.adId}}');
        $cond = array('_id.date'=> array('$gte' => new \MongoDate($from), '$lt' => new \MongoDate($to)), '_id.adId' => array('$in'=> $adIds));
        $initial = array('count' => 0);
        $reduce = "function(doc, prev)

        { prev.count+=doc.value.count; }

        ";
        $reduce = new MongoCode($reduce);
        $impressionData = $impressionCollection->group(
        $keys,
        $initial,
        $reduce,
        $cond
        );

        I was bit surprised myself that this suddenly happened. I've converted into using map reduce already in all of my code as group doesn't yield well in 1.6.x

        If initial value is chanced from 0 to 0.0 it worked just fine.

        Show
        jalava Jalmari Raippalinna added a comment - I fetched some code i had in my SVN that could help you, it's was bit a mess to find something pasteable.. $keys = new MongoCode('function(doc) { return { "adId": doc._id.adId}}'); $cond = array('_id.date'=> array('$gte' => new \MongoDate($from), '$lt' => new \MongoDate($to)), '_id.adId' => array('$in'=> $adIds)); $initial = array('count' => 0); $reduce = "function(doc, prev) { prev.count+=doc.value.count; } "; $reduce = new MongoCode($reduce); $impressionData = $impressionCollection->group( $keys, $initial, $reduce, $cond ); I was bit surprised myself that this suddenly happened. I've converted into using map reduce already in all of my code as group doesn't yield well in 1.6.x If initial value is chanced from 0 to 0.0 it worked just fine.
        Hide
        shift8 Tom Maiaroto added a comment -

        I also have this problem under MongoDB version 1.8.1 and PHP PECL driver version 1.2.1
        I was not able to work around it by specifying something like $initial => array('count' => 0.0) or even specifically casting like: $initial => array('count' => (float)0.0) etc.

        I was able to in the JavaScript function within $reduce, use if/else with typeof() ... If it was an object, use something like: count += variable.floatApprox and if it was a number: += variable and if undefined, I just said for my needs += 0;

        Hope this helps someone else...I would just leave it at .floatApprox and be done with it except for the fact that I test locally on a 32bit VM and run live on a 64bit server =(
        Oh well. I hope the if/else doesn't hurt performance too bad.

        Show
        shift8 Tom Maiaroto added a comment - I also have this problem under MongoDB version 1.8.1 and PHP PECL driver version 1.2.1 I was not able to work around it by specifying something like $initial => array('count' => 0.0) or even specifically casting like: $initial => array('count' => (float)0.0) etc. I was able to in the JavaScript function within $reduce, use if/else with typeof() ... If it was an object, use something like: count += variable.floatApprox and if it was a number: += variable and if undefined, I just said for my needs += 0; Hope this helps someone else...I would just leave it at .floatApprox and be done with it except for the fact that I test locally on a 32bit VM and run live on a 64bit server =( Oh well. I hope the if/else doesn't hurt performance too bad.
        Hide
        derick Derick Rethans added a comment -

        Hi Tom and Jalmari,

        we're going through all the issues to see if they're still relevant. Can you confirm or deny whether this issue is still blocking you from anything?

        cheers,
        Derick

        Show
        derick Derick Rethans added a comment - Hi Tom and Jalmari, we're going through all the issues to see if they're still relevant. Can you confirm or deny whether this issue is still blocking you from anything? cheers, Derick
        Hide
        jalava Jalmari Raippalinna added a comment -

        Not a blocker any more, all code from us have been converted into map reduces.

        We don't currently use any group calls to our database, because they were blocking our reads and too heavy.

        Cheers,
        Jalmari

        Show
        jalava Jalmari Raippalinna added a comment - Not a blocker any more, all code from us have been converted into map reduces. We don't currently use any group calls to our database, because they were blocking our reads and too heavy. Cheers, Jalmari

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              5 years, 3 weeks, 3 days ago
              Date of 1st Reply: