Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major - P3 Major - P3
    • Resolution: Fixed
    • Affects Version/s: 1.6.3
    • Fix Version/s: 1.7.3
    • Component/s: Write Ops
    • Labels:
      None
    • Environment:
      MongoDB v1.6.3, 64-bit builds on 64-bit Linux (Ubuntu 10.4)
    • Backport:
      No
    • Operating System:
      ALL
    • # Replies:
      5
    • Last comment by Customer:
      true

      Description

      I'm having an issue that I can only assume is a 32-bit integer overflow issue. I'm running on 64-bit Linux boxes with the 64-bit builds of MongoDB v1.6.3.

      Here's the deal:

      I have a collection used for logging and aggregate counters. I'm using an upsert with $inc to increment these counters. When the counters get over 2^31, it rolls over to negative.

      I've been able to reproduce it with the ruby driver:

      require 'rubygems'
      require 'mongo'

      mongo = Mongo::Connection.new "localhost"
      db = mongo.db "test"

      db.collection('junk').drop
      (0..20).each do |i|
      db.collection('junk').update({:name=>'luke'}, {"$inc"=>{:total=>123456789}}, {:upsert=>true})
      puts db.collection('junk').find({}, {:fields=>{:_id=>0}}).to_a.inspect
      end

      and that outputs: (I've bolded the line where it flips negative)

      [

      {"name"=>"luke", "total"=>123456789}

      ]
      [

      {"name"=>"luke", "total"=>246913578}

      ]
      [

      {"name"=>"luke", "total"=>370370367}

      ]
      [

      {"name"=>"luke", "total"=>493827156}

      ]
      [

      {"name"=>"luke", "total"=>617283945}

      ]
      [

      {"name"=>"luke", "total"=>740740734}

      ]
      [

      {"name"=>"luke", "total"=>864197523}

      ]
      [

      {"name"=>"luke", "total"=>987654312}

      ]
      [

      {"name"=>"luke", "total"=>1111111101}

      ]
      [

      {"name"=>"luke", "total"=>1234567890}

      ]
      [

      {"name"=>"luke", "total"=>1358024679}

      ]
      [

      {"name"=>"luke", "total"=>1481481468}

      ]
      [

      {"name"=>"luke", "total"=>1604938257}

      ]
      [

      {"name"=>"luke", "total"=>1728395046}

      ]
      [

      {"name"=>"luke", "total"=>1851851835}

      ]
      [

      {"name"=>"luke", "total"=>1975308624}

      ]
      [

      {"name"=>"luke", "total"=>2098765413}

      ]
      [

      {"name"=>"luke", "total"=>-2072745094}

      ]
      [

      {"name"=>"luke", "total"=>-1949288305}

      ]
      [

      {"name"=>"luke", "total"=>-1825831516}

      ]
      [

      {"name"=>"luke", "total"=>-1702374727}

      ]

      I suspect that initially MongoDB is storing the integer using the 4-byte integer data type, but when my upsert increments it, it isn't converted to the 8-byte integer.

      Just for fun, I tried upserting a large number and it indeed is stored just fine (I suspect it is created as an 8-byte integer):

      db.collection('junk').drop
      db.collection('junk').update({:name=>'luke'}, {"$inc"=>{:total=>9999999999999}}, {:upsert=>true})
      puts db.collection('junk').find({}, {:fields=>{:_id=>0}}).to_a.inspect

      and that outputs (which is clearly an 8-byte integer):

      [

      {"name"=>"luke", "total"=>9999999999999}

      ]

        Activity

        Hide
        auto
        added a comment -

        Author:

        {'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}

        Message: test for SERVER-2005
        http://github.com/mongodb/mongo/commit/4e5c46df588084198a777d34e18c62de2910ca72

        Show
        auto
        added a comment - Author: {'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'} Message: test for SERVER-2005 http://github.com/mongodb/mongo/commit/4e5c46df588084198a777d34e18c62de2910ca72
        Hide
        Nathan Ehresman
        added a comment -

        Will this affect query snapshots? According to the link below, $inc wont change the document size:

        http://www.mongodb.org/display/DOCS/How+to+do+Snapshotted+Queries+in+the+Mongo+Database

        Show
        Nathan Ehresman
        added a comment - Will this affect query snapshots? According to the link below, $inc wont change the document size: http://www.mongodb.org/display/DOCS/How+to+do+Snapshotted+Queries+in+the+Mongo+Database
        Hide
        Eliot Horowitz
        added a comment -

        Once fixed - when it needs to overflow - the size will have to change as it will go from int to long

        Show
        Eliot Horowitz
        added a comment - Once fixed - when it needs to overflow - the size will have to change as it will go from int to long
        Hide
        auto
        added a comment -

        Author:

        {'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}

        Message: fix $inc overflow on int SERVER-2005
        http://github.com/mongodb/mongo/commit/2dd5f90b5d2ae403abbf895316f08262600df380

        Show
        auto
        added a comment - Author: {'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'} Message: fix $inc overflow on int SERVER-2005 http://github.com/mongodb/mongo/commit/2dd5f90b5d2ae403abbf895316f08262600df380
        Hide
        Nathan Ehresman
        added a comment -

        The documentation should be changed on the following link:

        http://www.mongodb.org/display/DOCS/How+to+do+Snapshotted+Queries+in+the+Mongo+Database

        It currently says "This is most important when doing a find-and-update loop that changes the size of documents that are returned ($inc does not change size)." With the fix on this ticket, it is no longer true that $inc does not change the BSON document size.

        Show
        Nathan Ehresman
        added a comment - The documentation should be changed on the following link: http://www.mongodb.org/display/DOCS/How+to+do+Snapshotted+Queries+in+the+Mongo+Database It currently says "This is most important when doing a find-and-update loop that changes the size of documents that are returned ($inc does not change size)." With the fix on this ticket, it is no longer true that $inc does not change the BSON document size.

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              3 years, 25 weeks, 1 day ago
              Date of 1st Reply: