[SERVER-5908] update push override the last element Created: 23/May/12  Updated: 07/Mar/14  Resolved: 18/Jun/12

Status: Closed
Project: Core Server
Component/s: Internal Code, Write Ops
Affects Version/s: 2.0.4
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Simon Assignee: Ben Becker
Resolution: Won't Fix Votes: 0
Labels: update
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ubuntu 12.04

Linux simon-1 3.2.0-23-generic-pae #36-Ubuntu SMP Tue Apr 10 22:19:09 UTC 2012 i686 i686 i386 GNU/Linux


Attachments: File 5908.js     File bug.lua    
Operating System: ALL
Participants:

 Description   

I wrote a lua driver(https://github.com/bigplum/lua-resty-mongol) based on openresty(ngx_lua) project. There is an issue: http://forum.nginx.org/read.php?2,226716,226744#msg-226744

1. use lua-resty-mongol inster an embeded array, mongo shell shows ok.
The "undefined" value comes from that lua table index beginning from 1.

local t = {}
table.insert(t,

{a = "aa"}

)
table.insert(t,

{b = "bb"}

)
r, err = col:insert(name="dog",n="10",tab=t, nil, true)

> db.test.find()
{ "_id" : ObjectId("4fbc97e10853d55c767302f8"), "tab" : [ undefined,

{ "a" : "aa" }

,

{ "b" : "bb" }

], "name" : "dog", "n" : "10" }

2. use mongo shell update to push a new value into tab, the last element of tab is overrided.

> db.test.update(

{name:"dog"}

, {$push:{tab:"a5"}})
> db.test.find()
{ "_id" : ObjectId("4fbc97e10853d55c767302f8"), "n" : "10", "name" : "dog", "tab" : [ undefined,

{ "a" : "aa" }

, "a5" ] }

So, is this bug is related with the "undefined" value?



 Comments   
Comment by Simon [ 19/Jun/12 ]

Hi Ben Becker

Thanks for the comments. I've modify the driver to pack index 0 into the BSON string as I commented at May 25.

https://github.com/bigplum/lua-resty-mongol/commit/e1594ca019a655b14a843267abd3a5905b69fefa

Comment by Ben Becker [ 18/Jun/12 ]

Simon,

Please feel free to reopen this issue if you have further questions/concerns. Also, feel free to open a new issue to add more robust array index verification to the server.

Comment by Ben Becker [ 18/Jun/12 ]

Hi Simon,

The issue with inserting an undefined value into an array was indeed due to the array index difference you mentioned. The BSON spec requires keys be in ascending numeric order, but the logic in to_bson() does not enforce this.

You may want to consider changing bson.lua ~L188, which checks for a zero-based array, and simply enforce 0-based sequential array values in this function. I believe from_bson()/read_document() will need a similar translation to ensure table indices start at 1. IIRC, the luamongo driver takes a similar approach (see bson_to_array() and lua_append_bson() in utils.cpp).

Regarding the behavior where $push'ing a value overwrites the last array value when an 'undefined' value is present, this definitely seems undesirable. That being said, the issue is due to array indexing of malformed BSON, and the behavior technically makes sense when you consider the original array indexes are off by one.

We could add a check for the index base, but full verification of the array index sequence could be expensive. Ideally the driver should enforce array index order at creation time (as the driver needs to build the BSON array anyway).

Best Regards,
Ben

Comment by siddharth.singh@10gen.com [ 15/Jun/12 ]

Hi Simon,

Can you please send your lua test script, the one that you used to repro this bug.

Thanks.

Comment by siddharth.singh@10gen.com [ 15/Jun/12 ]

I am not sure using the lua driver will help but I am willing to give it a try. The effect of using the lua driver is only with regards to Step 1, where it enters an undefined value to the array. In my script, at the end of line 4, the array looks exactly the way it would if I were using the lua driver. Step 2 (where you actually see the override behavior) is performed from the mongo shell anyways so I don't think it should affect anything.

Anyways, I will update the issue after I have tried it with the lua driver.

Comment by Simon [ 13/Jun/12 ]

JS array index is begining from 0, so the js script could not reproduce this issue. You should try the lua driver.

Comment by siddharth.singh@10gen.com [ 12/Jun/12 ]

I checked both against the 2.0.4 version and the current master and $push seems to work correctly. I used the mongo shell and then followed the same steps as you outlined in the ticket. To be really sure I verified both with the 'undefined' and 'null' values. I have attached a script that you can try as well.

Comment by Simon [ 12/Jun/12 ]

The latest version of lua-resty-mongol has fixed this issue. But I think may be a bug in mongodb kernel. I can pack an old version to reproduce this issue.

https://github.com/downloads/bigplum/lua-resty-mongol/lua-resty-mongol.80cd614a4e.tar.gz

Comment by siddharth.singh@10gen.com [ 11/Jun/12 ]

Hi Simon,

I just tested push with array values and it works fine. Not sure what you meant when you said "is this bug is related with the "undefined" value?" Was it an issue with your driver that you patched ? Can you please confirm.

Thanks.

Comment by Simon [ 25/May/12 ]

I've patched my driver, add tab[0]=nil into the LUA array. So the insert operation would produce a null value instead of "undefined" value.

{ "_id" : ObjectId("4fbc97e10853d55c767302f8"), "tab" : [ null,

{ "a" : "aa" }

,

{ "b" : "bb" }

], "name" : "dog", "n" : "10" }

And the push issue is also fixed now.

Generated at Thu Feb 08 03:10:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.