[CSHARP-750] Improve performance of serialization/deserialization of types serialized as BSON arrays Created: 05/Jun/13  Updated: 27/Jul/13  Resolved: 13/Jul/13

Status: Closed
Project: C# Driver
Component/s: Performance
Affects Version/s: 1.8.1
Fix Version/s: 1.8.2

Type: Improvement Priority: Minor - P4
Reporter: Dmitry Naumov Assignee: Robert Stam
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 Professional SP1 x64
.NET 4.0


Attachments: JPEG File LookupSerializer.jpg    
Backwards Compatibility: Fully Compatible

 Description   

When serializing array of value types there is no need to lookup for serializer for every item, because it is homogeneous. This saves us some cpu cycles but most benefit may be achieved when multiple threads are doing serialization, like in my multi-threaded insert benchmark. The suggest fix allows to avoid extra-contention on ReaderWriterLockSlim inside BsonSerializer.

				if (typeof(T).IsValueType)
				{
					var serializer = BsonSerializer.LookupSerializer(typeof(T));
					foreach (var item in array)
					{
						serializer.Serialize(bsonWriter, typeof (T), item, options);
					}
				}
				else
				{
					foreach (var item in array)
					{
						BsonSerializer.Serialize(bsonWriter, typeof (T), item, itemSerializationOptions);
					}
				}



 Comments   
Comment by auto [ 09/Jul/13 ]

Author:

{u'username': u'rstam', u'name': u'rstam', u'email': u'robert@10gen.com'}

Message: CSHARP-750: Added optimizations for ThreeDimensionalArraySerializer and TwoDimensionalArraySerializer.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/eb055575db6a1c05c9ff4784f883c2a9b88737f0

Comment by auto [ 09/Jul/13 ]

Author:

{u'username': u'rstam', u'name': u'rstam', u'email': u'robert@10gen.com'}

Message: CSHARP-750: Optimize serialization of enumerable types.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/060902ad10a7a2de1f1e4f96aa545db99d3f3162

Comment by auto [ 09/Jul/13 ]

Author:

{u'username': u'DmitryNaumov', u'name': u'Dmitry Naumov', u'email': u'dnaumov@paladyne.com'}

Message: CSHARP-750: Fixed array of value type serialization performance issue
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/3ccfbb1666e02888f562c7823ebf558620e6115a

Comment by Dmitry Naumov [ 17/Jun/13 ]

I've updated pull request, now it contains optimizations for both serialization and deserialization. I also thought about covering such case:
class Dog : Animal {}
and serializing Animal[] where no nulls and all items are of type Dog. It will require additional pass through array, which is not a big deal, but I was concerned about code complexity growth and decided that such rare case won't payback it's optimization.

Comment by Chad Kreimendahl [ 13/Jun/13 ]

I suppose I'm speaking more of DEserialization, but bunched it with "serialization" as a whole. Our mongo machines are physically separate from our web servers. The mongo servers never really use more than 4-5% cpu [each], even when half a dozen web servers are being load tested.

When we run the load test with a performance profiler enabled on the web site (IIS8), the vast majority of the CPU time used is in deserialization of objects coming out of mongo. We utilize some in-memory short-term caching to get around having to re-deserialize the same objects multiple times on a request, but still see that as the most major bottleneck.

In all honesty, we know it's always going to be the largest resource hog, which is why even minor improvements in its performance will go a very long way to reducing our need to scale. I was only suggesting that some of the recent improvements made to other libraries for performance could be re-used [if legal] to improve the deserialization performance of the mongo.C# driver.

Interestingly enough, Json.NET supports BSON, so it may be relatively easy to integrate by wrapping it with your customizations, so that future performance improvements to Json.NET automatically end up working in your code.

It's really just an idea I had. There may be several reasons [political, security], that you don't want to simply integrate with a non-mongo-controlled third party open source library. Fortunately it's got an MIT license, so it can be copied without much headache.

Comment by Robert Stam [ 13/Jun/13 ]

You certainly could do that. We're not going to be working on this just yet, so you have time to submit a new pull request.

Thanks!

Comment by Dmitry Naumov [ 13/Jun/13 ]

Robert,
suppose I have time and will to do it, should I improve my pull request, add what you've mentioned above plus apply same improvement to deserialization side?

Comment by Dmitry Naumov [ 13/Jun/13 ]

Chad,
you're right about serialization as most CPU utilizing task, but imho it really matters on server-side inside mongod which is written on C++ not C#. So how can we use advantages on Json.Net?

Comment by Chad Kreimendahl [ 12/Jun/13 ]

Is it worthwhile to look at what newtonsoft/json.net has done recently in their serialization code that has dramatically improved its performance beyond where serviceStack is? Because mongo is so performant, our bottleneck (or location requiring most scalability) is our web site. Serialization is the vast majority of the CPU utilization, so focusing here and incorporating what others have done (as you guys have done very well in the past), could be good to do again.

Comment by Robert Stam [ 11/Jun/13 ]

Thanks for the additional comments. We are looking to centralize all array-like serialization into a common base class, at which point we can make as many optimizations as we want without having to copy and paste them to all the other array-like serializers (the 10 or so listed earlier).

We can definitely optimize value types separately from reference types. We won't forget about the Utf8Encoding change either. The plan is to merge your pull request and make further changes from there.

Comment by Dmitry Naumov [ 11/Jun/13 ]

A few things to mention:

1. for-loop is penalized by boundaries check on each iteration.
2. Don't forget another part of fix - avoiding per-call instantiation of UTF8Encoding
3. As far as I remember code, same optimization can be performed on deserialization call
4. I know GetType() is fast, but why not to avoid extra calls...

Comment by Robert Stam [ 11/Jun/13 ]

A better version of the alternative change proposed two comments ago is:

var itemNominalType = typeof(T);
var itemNominalTypeSerializer = BsonSerializer.LookupSerializer(itemNominalType);
 
bsonWriter.WriteStartArray();
foreach (var item in array)
{
    IBsonSerializer itemSerializer;
    if (item.GetType() == itemNominalType)
    {
        itemSerializer = itemNominalTypeSerializer;
    }
    else
    {
        itemSerializer = BsonSerializer.LookupSerializer(item.GetType());
    }
    itemSerializer.Serialize(bsonWriter, itemNominalType, item, itemSerializationOptions);
}
bsonWriter.WriteEndArray();

This version emphasizes that what we're doing is optimizing the selection of the serializer, and only has one call to Serialize (using whatever serializer was chosen).

It also abandons the numeric for loop in favor of foreach. Presumably someone thought that for was faster than foreach, but I did some benchmarks and they are virtually identical, so the simpler form is preferable.

Comment by Robert Stam [ 11/Jun/13 ]

There are several other serializers that would benefit from a similar optimization:

EnumerableSerializer
EnumerableSerializer<T>
QueueSerializer
QueueSerializer<T>
ReadOnlyCollectionSerializer<T>
StackSerializer
StackSerializer<T>
ThreeDimensionalArraySerializer<T>
TwoDimensionalArraySerializer<T>

Comment by Robert Stam [ 11/Jun/13 ]

A similar change based on the same idea is:

var itemNominalType = typeof(T);
var itemNominalTypeSerializer = BsonSerializer.LookupSerializer(itemNominalType);
 
bsonWriter.WriteStartArray();
for (int index = 0; index < array.Length; index++)
{
    var item = array[index];
    if (item.GetType() == itemNominalType)
    {
        itemNominalTypeSerializer.Serialize(bsonWriter, itemNominalType, item, itemSerializationOptions);
    }
    else
    {
        BsonSerializer.Serialize(bsonWriter, typeof(T), array[index], itemSerializationOptions);
    }
}
bsonWriter.WriteEndArray();

This alternative would benefit homogenous arrays of all types, whether they are value types or not.

Comment by Dmitry Naumov [ 10/Jun/13 ]

This issue was hidden by SERVER-9754 where disk i/o was bottleneck, please consider it if you run your own benchmarks.

Comment by Craig Wilson [ 05/Jun/13 ]

Thanks Dmitry. We'll definitely take a look.

Comment by Dmitry Naumov [ 05/Jun/13 ]

With another fix removing per-call creation of UTF8Encoding, the numbers are even better:

Threads Unpatched Patched
1 11883 10250
2 14198 10424
3 19870 10643
4 19930 11220
5 20025 11611
6 20892 12446
Comment by Dmitry Naumov [ 05/Jun/13 ]

https://github.com/mongodb/mongo-csharp-driver/pull/159

Benchmark results for inserting document with array of 1000 ints:

Threads Unpatched Patched
1 11883 10918
2 14198 11554
3 19870 11752
4 19930 12878
5 20025 14029
6 20892 15750
Generated at Wed Feb 07 21:37:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.