[CSHARP-4199] Change stream exception on update notification after some hours of usage Created: 07/Jun/22  Updated: 27/Oct/23  Resolved: 30/Jun/22

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: 2.11.4
Fix Version/s: None

Type: Question Priority: Unknown
Reporter: Mario Vernari Assignee: James Kovacs
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Summary

After some hours of usage, the access to the UpdateDescription property of the change stream update notification gives a format exception.

MongoDB driver version 2.11.4 against Atlas Shared Free 5.0.8.

How to Reproduce

Not easy to reproduce, but it happens regularly after some hours of usage. Once happened while debugging in Visual Studio and it seems the simple access to the UpdateDescription property of an update notification gives an exception (I've checked in the Watch pane).

The exception stack is the following:

      System.FormatException: Invalid field name: "truncatedArrays".
         at MongoDB.Driver.ChangeStreamUpdateDescriptionSerializer.DeserializeValue(BsonDeserializationContext context, BsonDeserializationArgs args)
         at MongoDB.Bson.Serialization.Serializers.SealedClassSerializerBase`1.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
         at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
         at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
         at MongoDB.Bson.Serialization.BsonSerializationInfo.DeserializeValue(BsonValue value)
         at MongoDB.Bson.Serialization.BsonDocumentBackedClass.GetValue[T](String memberName, T defaultValue)
         at MongoDB.Driver.ChangeStreamDocument`1.get_UpdateDescription()
         at Cet.Core.Web.MongoDB.MongoChangeStreamAdapter.Convert(String room, ChangeStreamDocument`1 change) in C:\DotNet18\Cet.Core.Web\Cet.Core.Web\MongoDB\Listeners\MongoChangeStreamAdapter.cs:line 134
         at Cet.Core.Web.MongoDB.MongoChangeStreamAdapter.ListenCollectionAsync(IReadOnlyMongoListenerConnectionOptions observedCollection, Action`1 handler, ILogger logger, CancellationToken stoppingToken) in C:\DotNet18\Cet.Core.Web\Cet.Core.Web\MongoDB\Listeners\MongoChangeStreamAdapter.cs:line 98

Additional Background

Here is the code where the exception happens. I'll patch by catching and ignoring the case, but it means for the application to miss a notification.

            __            var client = new MongoClient(observedCollection.ConnectionString);
            var database = client.GetDatabase(observedCollection.DatabaseName);
            var collection = database.GetCollection<BsonDocument>(observedCollection.CollectionName);
                while (stoppingToken.IsCancellationRequested == false)
                {
                    ChangeStreamOptions opts = new();
                    using var cursor = await collection.WatchAsync(opts, stoppingToken);
                    while (await cursor.MoveNextAsync(stoppingToken))
                    {
                        foreach (ChangeStreamDocument<BsonDocument> change in cursor.Current)
                        {
                            handler(
                                Convert(room, change)
                                );
{{                        }}}
{{                    }}}
{{                }}}
        // ...{{        }}

        private static IMongoChangeStreamDocument Convert(
            string room,
            ChangeStreamDocument<BsonDocument> change
            )
        {
            ChangeStreamUpdateDescription? updateDescription = null;
            switch (change.OperationType)
            {
                case ChangeStreamOperationType.Insert:
                    break;                case ChangeStreamOperationType.Update:
                    updateDescription = change.UpdateDescription;
                    break;                case ChangeStreamOperationType.Delete:
                    break;
{{            }}}
            return new MongoChangeStreamDocumentMessage(
                room,
                MongoHelpers.OidToString(change.DocumentKey)
                )
            {
                OperationType = change.OperationType,
                FullDocument = change.FullDocument,
                UpdateDescription = updateDescription,
            };
{{        }}}



 Comments   
Comment by PM Bot [ 30/Jun/22 ]

There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information.

Comment by James Kovacs [ 15/Jun/22 ]

Hi, vernarim@libero.it,

We are glad that you were able to find a solution and thank you for your question. MongoDB does advertise its supported wire versions via the hello command (formerly called isMaster). It will reply with a minWireVersion and maxWireVersion. Drivers know their supported wire versions. For example, the 2.14.0 driver - which requires MongoDB 3.6+ - has a minWireVersion of 6. If you try to connect to a MongoDB 3.4 server using the 2.14.0 driver, you would receive an error stating that the wire versions aren't compatible as the server's maxWireVersion is 5 and the driver's minWireVersion is 6.

So far so good. The question now becomes why doesn't the driver raise an exception if the server's maxWireVersion is higher than the driver's? The reason is that a lot of times it doesn't matter. As long as you're not using new features introduced in later versions of MongoDB, older clients will most often work. And if you try to use a new server feature with an old driver, you will quickly know whether it works or not. You encountered a bit of an unfortunate edge case where you start an operation (a change stream) and you might not see a change stream event with the new truncatedArrays field for hours. Depending on your workload, you may never see this field. Thus we made the choice to rely on human-readable compatibility matrices rather than not allowing developers to use older drivers with newer servers.

We are investing in better forward compatibility guarantees with features like the MongoDB Stable API as well as making our APIs more tolerant of future changes such as DRIVERS-1995 / CSHARP-4036.

Please let us know if you have any additional questions.

Sincerely,
James

Comment by Boris Dogadov [ 09/Jun/22 ]

Hi vernarim@libero.it, thank you for your question.
As you discovered, this issue (CSHARP-3212) was resolved in 2.12.0 driver version.

.Net driver has versions range validation. We will look deeper into what the expected behaviour should be in such case.

Comment by Mario Vernari [ 09/Jun/22 ]

Okay, the problem seems solved. It is the driver version, which is not compatible with the Atlas MongoDB version. I upgraded it to version 2.13.3 and now everything seems working fine. I had to limit to that version because the same app must connect to an old Mongo 3.4: this driver is the only able to fit practically any MongoDB version.

Just a question: why don't you implement the compatibility matrix inside the driver, so that it'll raise an exception SOON when the server doesn't fit? it isn't nice allowing the connection, then having strange behaviors after the app is released.

Comment by Mario Vernari [ 07/Jun/22 ]

Isn't a way to edit the post? the code seems now full of garbage...

Generated at Wed Feb 07 21:47:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.