[JAVA-505] Performance regression due to increased memory allocation Created: 13/Jan/12  Updated: 18/Jun/12  Resolved: 30/Jan/12

Status: Closed
Project: Java Driver
Component/s: None
Affects Version/s: 2.7.2
Fix Version/s: 2.7.3

Type: Bug Priority: Major - P3
Reporter: Antoine Girbal Assignee: Jeffrey Yemin
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends

 Description   

use MongoPerfTest.java to reproduce

antoine@agAntec:~/tengen/support$ java -cp "mongo_2-6-5.jar:." MongoPerfTest
insert: 17945ms
find: 4804ms
antoine@agAntec:~/tengen/support$ java -cp "mongo_2-7-0.jar:." MongoPerfTest
insert: 18852ms
find: 6587ms

inserts are marginally slower (about 5%) but queries are 37.5% slower.
It seems to have introduced by following commit:

commit 66c54120c56f9329b392264ec5efba0b2a292217
Author: Brendan W. McAdams <brendan@10gen.com>
Date: Fri Aug 19 16:05:22 2011 -0400

JAVA-421 - LazyBSONObject exhibits certain behavioral breakages

  • Fully functioning implemented LazyBSONObject, fixed string decodes
  • Test class now validates all known decodable types for correctness
  • Now works correctly, fixes known issues.
  • For performance, sanity and memory usage replaced all arraycopy calls
    with ByteBuffer usage which should help overall longterm behavior


 Comments   
Comment by auto [ 20/Jan/12 ]

Author:

{u'login': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}

Message: JAVA-505: Made encoder creation work just like decoder creation
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/88b3406f2fc841b22ec2aa4208942b64876d20f1

Comment by auto [ 20/Jan/12 ]

Author:

{u'login': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}

Message: JAVA-505: using cached DBDecoder in most cases to avoid excessive memory allocation caused by too many instances of DefaultDBDecoder being created
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/807c797be4212c7dbbff891aa177843edc6c5ab1

Comment by Jeffrey Yemin [ 19/Jan/12 ]

To keep things dead simple for a patch release, I'm not going to add any pooling of custom decoders just yet.

Comment by Jeffrey Yemin [ 19/Jan/12 ]

The previous behavior was to cache an instance of the decoder with each instance of DBPort (which are themselves cached in the connection pool). It can't be a singleton because DefaultDBDecoder is not thread-safe.

Comment by Leo Kim [ 19/Jan/12 ]

Can the DefaultDBDecoder.FACTORY return a DefaultDBDecoder singleton?

Comment by Antoine Girbal [ 19/Jan/12 ]

right ideally if using the default decoder, there should be 1 global.
That's the way it was..
Only if using a custom decoder would we need one per call (though could be pooled)

Comment by Jeffrey Yemin [ 19/Jan/12 ]

Commit 66c54120c56f9329b392264ec5efba0b2a292217 (fix for JAVA-421) is half the problem. The performance regression in that commit is that several large buffers are instantiated every time an instance of BSONInput is instantiated (two 1k byte arrays, one 16k byte array, and 1 16k char array). In practice an instance of BSONInput is instantiated every time a document is decoded. It's easy to roll back that change such that the arrays are again instantiated in BasicBSONDecoder. That part is done.

But after rolling back this change, there's a second half of the problem, in commit 673beed75fc347054328afaa2841d43410ee0f28. In the commit, the driver is creating many more decoders. Instead of creating one per DBPort, it's creating one per call/say. This is a lot more work for the garbage collector.

This commit will also have to be rolled back or modified.

Comment by auto [ 17/Jan/12 ]

Author:

{u'login': u'bwmcadams', u'name': u'Brendan W. McAdams', u'email': u'brendan@10gen.com'}

Message: JAVA-505 / JAVA-421 - Regression in performance of Java Driver should be rolled back
Branch: release-2.7.x
https://github.com/mongodb/mongo-java-driver/commit/9910c355ba80a85ff4bfed8cf691f089eb4eb6de

Comment by auto [ 17/Jan/12 ]

Author:

{u'login': u'bwmcadams', u'name': u'Brendan W. McAdams', u'email': u'brendan@10gen.com'}

Message: JAVA-505 / JAVA-421 - Regression in performance of Java Driver should be rolled back
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/3f927463b0383586ac5df2c7403a370622460aa3

Generated at Thu Feb 08 08:52:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.