[DOCS-923] Document driver support for duplicate BSON keys Created: 21/Dec/12  Updated: 30/Oct/23  Resolved: 03/Jan/13

Status: Closed
Project: Documentation
Component/s: manual
Affects Version/s: None
Fix Version/s: Server_Docs_20231030

Type: Task Priority: Major - P3
Reporter: Stennie Steneker (Inactive) Assignee: Sam Kleinman (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-1606 Oplog entries contain repeated fields... Closed
Related
is related to SERVER-7871 Updates create $set operations in opl... Closed
Participants:
Days since reply: 11 years, 6 weeks, 6 days ago

 Description   

Most MongoDB drivers have a native representation of BSON documents as a dictionary or hash structure that does not support duplicate keys.

This can be an issue when trying to read from the oplog; SERVER-1606

It might be helpful to have a page documenting driver support & example usage.



 Comments   
Comment by auto [ 03/Jan/13 ]

Author:

{u'date': u'2013-01-03T23:12:31Z', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: DOCS-923: duplicate field name note
Branch: master
https://github.com/mongodb/docs/commit/ab22a79332125722d229258ce358249e7fc7c3fe

Comment by Dave Curylo [ 28/Dec/12 ]

Thanks Scott. toMap().entrySet() seems to give me the data.

I saw the change to the oplog format is scheduled for 2.5 (prod-ready 2.6) which I'm assuming is a long way off given the amount of time it took to go from 2.0 to 2.2. So I won't be stuck on 2.2.0 forever, I'm going through the pain now of converting my code from C# to Java and using IKVM to run the Java code under .NET so I don't have to deploy under two runtimes since the rest of my application is .NET. The more information about these sorts of discrepancies between drivers that can be known up front, the better, as this one really bit us.

Thanks again for your help.

Comment by Scott Hernandez (Inactive) [ 27/Dec/12 ]

Dave, if you use the entrySet() method then you will get back all pairs, with their actual data. This is not the purpose of that class but because of the way it is implemented it works. I also have another implementation which will combine multiple repeats like {$set:

{a:1}

, $set:{b:1}} into {$set:{a:1, b:1}} which can be used in java, but is not generally useful nor efficient since it has to extra work to track fields at each level.

We will document this behavior and we are going to change the oplog format (probably collecting all fields in each repeated field into one) to produce something easily consumable to all languages in a version in the short-term future (currently scheduled for 2.5.x I believe).

Comment by Dave Curylo [ 27/Dec/12 ]

LazyDBObject gets past the error and lets you parse these oplog documents, but it appears that it will replace the values as it parses the document and finds elements that repeat the same key. As a result, you'll get a partial document, with the value for the duplicate key that was set last. This isn't optimal, but a broken parser is better than no parser at all.

Scott, I think it would be helpful if these driver limitations were documented. While the oplog format is internal, Eliot said that this is valid BSON data, and only the C++ driver is capable of handling it. It seems possible that users would store data in a document using one driver that couldn't be read by other drivers. The oplog appears to be one case of this occurring, but if anyone could do that, then knowing the capabilities of other drivers in this regard would be very helpful.

Comment by Scott Hernandez (Inactive) [ 22/Dec/12 ]

We should not document this as it is internal and we should simply wait till it is fixed. If anything, and note that using anything other than C++ is a problem is enough.

Stephen, have you tested that the LazyDBObject works in all case, or even common ones? I don't think it does what you think it does.

Generated at Thu Feb 08 07:39:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.