[CSHARP-1426] Oplog tail BsonDocument deserialization Created: 29/Sep/15  Updated: 05/Apr/19  Resolved: 05/Apr/16

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: 1.10.1
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Chris Robison Assignee: Unassigned
Resolution: Done Votes: 0
Labels: question
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

We are running into an interesting issue. We are tailing the Mongo oplog and thought that BsonDocument would just be a catch all. But, it turns out that another product in our company has started using the same Mongo instance and they have set their C# driver to use the standard UUID format. It looks like BsonDocument is trying to deserialize the BinData using our C# settings. We get the following:

The GuidRepresentation for the reader is CSharpLegacy, which requires the binary sub type to be UuidLegacy, not UuidStandard.

MongoDB.Bson.IO.BsonBinaryReader.ReadBinaryData()
at MongoDB.Bson.Serialization.Serializers.BsonBinaryDataSerializer.Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonBaseSerializer.Deserialize(BsonReader bsonReader, Type nominalType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonValueSerializer.Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonBaseSerializer.Deserialize(BsonReader bsonReader, Type nominalType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonDocumentSerializer.Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonBaseSerializer.Deserialize(BsonReader bsonReader, Type nominalType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonValueSerializer.Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonBaseSerializer.Deserialize(BsonReader bsonReader, Type nominalType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonDocumentSerializer.Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
at MongoDB.Bson.Serialization.Serializers.BsonBaseSerializer.Deserialize(BsonReader bsonReader, Type nominalType, IBsonSerializationOptions options)
at MongoDB.Driver.Internal.MongoReplyMessage`1.ReadBodyFrom(BsonBuffer buffer)
at MongoDB.Driver.Internal.MongoConnection.ReceiveMessage[TDocument](BsonBinaryReaderSettings readerSettings, IBsonSerializer serializer, IBsonSerializationOptions serializationOptions)
at MongoDB.Driver.MongoCursorEnumerator`1.GetReply(MongoConnection connection, MongoRequestMessage message)
at MongoDB.Driver.MongoCursorEnumerator`1.GetFirst()
at MongoDB.Driver.MongoCursorEnumerator`1.MoveNext()

Is there a way to have BsonDocument just give us what's in the database without having to deserialize anything or to just ignore uuid setting?



 Comments   
Comment by Robert Stam [ 30/Sep/15 ]

BsonBinaryData has two primary properties:

1. Bytes
2. SubType

For pure binary data that's all that matters.

However, if the SubType is 3 or 4, a third property comes into play:

3. GuidRepresentation

When you deserialize a BsonBinaryData value the GuidRepresentation is remembered from whatever the BsonBinaryReaderSettings said the GuidRepresentation for that data source is.

When you write BsonBinaryData to a BsonBinaryWriter we want to make sure that the bytes are written in the correct order, as specified by the GuidRepresentation. This might involve converting from one representation to another. However, if the BsonBinaryData value has a GuidRepresentation of Unspecified the serializer doesn't know what to do, because the GuidRepresentation is Unspecified.

If you're going to work with mixed GuidRepresentations you're going to have to be extra careful that all the settings are configured correctly for all readers and writers, and if you read Guids with a setting of Unspecified and later want to write them out, you're going to have to figure out what the actual GuidRepresentation is and convert the values that have an Unspecified GuidRepresentation to values that have one of the other GuidRepresentations (making sure that the values don't get mangled in the process).

Comment by Chris Robison [ 30/Sep/15 ]

We tried setting up our own reader but were still running into problems with the Guid stuff. Do we need to convert the BsonDocument back into bson byte array to get that kind of thing to work?

We still get this: Cannot serialize BsonBinaryData with GuidRepresentation Unspecified to destination with GuidRepresentation CSharpLegacy.

Does BsonBinaryData not keep track of the original sub type?

Comment by Robert Stam [ 30/Sep/15 ]

You won't be able to use AsGuid because AsGuid relies on the BsonBinaryData instance having been "tagged" with the GuidRepresentation that was in effect when the binary data was read. If you use GuidRepresentation Unspecified then AsGuid doesn't know what representation it should use.

Instead you'll have to use ToGuid(guidRepresentation) and pass in the correct representation. You'll have to figure out which representation is the correct one (perhaps by knowing which collection the data is in).

BsonSerializer.Deserialize is just a helper method for setting up the appropriate BsonReader and reading from it. You can write your own helper method that sets up the appropiate GuidRepresentation in the reader settings. You can model it after the existing Deserialize methods which start here:

https://github.com/mongodb/mongo-csharp-driver/blob/v1.x/MongoDB.Bson/Serialization/BsonSerializer.cs#L88

The upgrade path from the legacy format to the standard format is to rewrite the documents using the new GuidRepresentation. If you use Guids as your _ids you can't directly update the document (because the _id value can't be changed), so in that case you have to delete the old document and insert a new one.

Comment by Chris Robison [ 29/Sep/15 ]

In the 1.x driver, it looks like it's not passing through the subtype in the BsonBinaryReader correctly (well, I suppose it is, but we need it to assume C# representation when the legacy bin data is encountered). It only matches the standard format and then uses whatever is being specified in the reader settings.

As a side note, is there an upgrade path to go from the legacy format to the standard format?

Comment by Chris Robison [ 29/Sep/15 ]

It looks like we also can't use BsonSerializer.Deserialize for the embedded docs because of that setting.

Comment by Chris Robison [ 29/Sep/15 ]

This works to a point. We can get the BsonDocument to come down now. But, we can't use the AsGuid property any more on BsonValue, which does make sense. So we've had to find a workaround for that.

Comment by Robert Stam [ 29/Sep/15 ]

Try setting the GuidRepresentation of the MongoCollectionSettings to Unspecified.

That should disable this check.

Generated at Wed Feb 07 21:39:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.