[SERVER-2005] Integer overflow with upsert Created: 25/Oct/10  Updated: 12/Jul/16  Resolved: 02/Nov/10

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: 1.6.3
Fix Version/s: 1.7.3

Type: Bug Priority: Major - P3
Reporter: Luke Ehresman Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

MongoDB v1.6.3, 64-bit builds on 64-bit Linux (Ubuntu 10.4)


Operating System: ALL
Participants:

 Description   

I'm having an issue that I can only assume is a 32-bit integer overflow issue. I'm running on 64-bit Linux boxes with the 64-bit builds of MongoDB v1.6.3.

Here's the deal:

I have a collection used for logging and aggregate counters. I'm using an upsert with $inc to increment these counters. When the counters get over 2^31, it rolls over to negative.

I've been able to reproduce it with the ruby driver:

require 'rubygems'
require 'mongo'

mongo = Mongo::Connection.new "localhost"
db = mongo.db "test"

db.collection('junk').drop
(0..20).each do |i|
db.collection('junk').update({:name=>'luke'}, {"$inc"=>{:total=>123456789}}, {:upsert=>true})
puts db.collection('junk').find({}, {:fields=>{:_id=>0}}).to_a.inspect
end

and that outputs: (I've bolded the line where it flips negative)

[

{"name"=>"luke", "total"=>123456789}

]
[

{"name"=>"luke", "total"=>246913578}

]
[

{"name"=>"luke", "total"=>370370367}

]
[

{"name"=>"luke", "total"=>493827156}

]
[

{"name"=>"luke", "total"=>617283945}

]
[

{"name"=>"luke", "total"=>740740734}

]
[

{"name"=>"luke", "total"=>864197523}

]
[

{"name"=>"luke", "total"=>987654312}

]
[

{"name"=>"luke", "total"=>1111111101}

]
[

{"name"=>"luke", "total"=>1234567890}

]
[

{"name"=>"luke", "total"=>1358024679}

]
[

{"name"=>"luke", "total"=>1481481468}

]
[

{"name"=>"luke", "total"=>1604938257}

]
[

{"name"=>"luke", "total"=>1728395046}

]
[

{"name"=>"luke", "total"=>1851851835}

]
[

{"name"=>"luke", "total"=>1975308624}

]
[

{"name"=>"luke", "total"=>2098765413}

]
[

{"name"=>"luke", "total"=>-2072745094}

]
[

{"name"=>"luke", "total"=>-1949288305}

]
[

{"name"=>"luke", "total"=>-1825831516}

]
[

{"name"=>"luke", "total"=>-1702374727}

]

I suspect that initially MongoDB is storing the integer using the 4-byte integer data type, but when my upsert increments it, it isn't converted to the 8-byte integer.

Just for fun, I tried upserting a large number and it indeed is stored just fine (I suspect it is created as an 8-byte integer):

db.collection('junk').drop
db.collection('junk').update({:name=>'luke'}, {"$inc"=>{:total=>9999999999999}}, {:upsert=>true})
puts db.collection('junk').find({}, {:fields=>{:_id=>0}}).to_a.inspect

and that outputs (which is clearly an 8-byte integer):

[

{"name"=>"luke", "total"=>9999999999999}

]



 Comments   
Comment by Nathan Ehresman [ 03/Nov/10 ]

The documentation should be changed on the following link:

http://www.mongodb.org/display/DOCS/How+to+do+Snapshotted+Queries+in+the+Mongo+Database

It currently says "This is most important when doing a find-and-update loop that changes the size of documents that are returned ($inc does not change size)." With the fix on this ticket, it is no longer true that $inc does not change the BSON document size.

Comment by auto [ 02/Nov/10 ]

Author:

{'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}

Message: fix $inc overflow on int SERVER-2005
http://github.com/mongodb/mongo/commit/2dd5f90b5d2ae403abbf895316f08262600df380

Comment by Eliot Horowitz (Inactive) [ 26/Oct/10 ]

Once fixed - when it needs to overflow - the size will have to change as it will go from int to long

Comment by Nathan Ehresman [ 26/Oct/10 ]

Will this affect query snapshots? According to the link below, $inc wont change the document size:

http://www.mongodb.org/display/DOCS/How+to+do+Snapshotted+Queries+in+the+Mongo+Database

Comment by auto [ 26/Oct/10 ]

Author:

{'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}

Message: test for SERVER-2005
http://github.com/mongodb/mongo/commit/4e5c46df588084198a777d34e18c62de2910ca72

Generated at Thu Feb 08 02:58:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.