Java Driver
  1. Java Driver
  2. JAVA-505

Performance regression due to increased memory allocation

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major - P3 Major - P3
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.7.3
    • Component/s: None
    • Labels:
      None
    • Operating System:
      ALL
    • # Replies:
      9
    • Last comment by Customer:
      false

      Description

      use MongoPerfTest.java to reproduce

      antoine@agAntec:~/tengen/support$ java -cp "mongo_2-6-5.jar:." MongoPerfTest
      insert: 17945ms
      find: 4804ms
      antoine@agAntec:~/tengen/support$ java -cp "mongo_2-7-0.jar:." MongoPerfTest
      insert: 18852ms
      find: 6587ms

      inserts are marginally slower (about 5%) but queries are 37.5% slower.
      It seems to have introduced by following commit:

      commit 66c54120c56f9329b392264ec5efba0b2a292217
      Author: Brendan W. McAdams <brendan@10gen.com>
      Date: Fri Aug 19 16:05:22 2011 -0400

      JAVA-421 - LazyBSONObject exhibits certain behavioral breakages

      • Fully functioning implemented LazyBSONObject, fixed string decodes
      • Test class now validates all known decodable types for correctness
      • Now works correctly, fixes known issues.
      • For performance, sanity and memory usage replaced all arraycopy calls
        with ByteBuffer usage which should help overall longterm behavior

        Activity

        Hide
        auto
        added a comment -

        Author:

        {u'login': u'bwmcadams', u'name': u'Brendan W. McAdams', u'email': u'brendan@10gen.com'}

        Message: JAVA-505 / JAVA-421 - Regression in performance of Java Driver should be rolled back
        Branch: master
        https://github.com/mongodb/mongo-java-driver/commit/3f927463b0383586ac5df2c7403a370622460aa3

        Show
        auto
        added a comment - Author: {u'login': u'bwmcadams', u'name': u'Brendan W. McAdams', u'email': u'brendan@10gen.com'} Message: JAVA-505 / JAVA-421 - Regression in performance of Java Driver should be rolled back Branch: master https://github.com/mongodb/mongo-java-driver/commit/3f927463b0383586ac5df2c7403a370622460aa3
        Hide
        auto
        added a comment -

        Author:

        {u'login': u'bwmcadams', u'name': u'Brendan W. McAdams', u'email': u'brendan@10gen.com'}

        Message: JAVA-505 / JAVA-421 - Regression in performance of Java Driver should be rolled back
        Branch: release-2.7.x
        https://github.com/mongodb/mongo-java-driver/commit/9910c355ba80a85ff4bfed8cf691f089eb4eb6de

        Show
        auto
        added a comment - Author: {u'login': u'bwmcadams', u'name': u'Brendan W. McAdams', u'email': u'brendan@10gen.com'} Message: JAVA-505 / JAVA-421 - Regression in performance of Java Driver should be rolled back Branch: release-2.7.x https://github.com/mongodb/mongo-java-driver/commit/9910c355ba80a85ff4bfed8cf691f089eb4eb6de
        Hide
        Jeff Yemin
        added a comment -

        Commit 66c54120c56f9329b392264ec5efba0b2a292217 (fix for JAVA-421) is half the problem. The performance regression in that commit is that several large buffers are instantiated every time an instance of BSONInput is instantiated (two 1k byte arrays, one 16k byte array, and 1 16k char array). In practice an instance of BSONInput is instantiated every time a document is decoded. It's easy to roll back that change such that the arrays are again instantiated in BasicBSONDecoder. That part is done.

        But after rolling back this change, there's a second half of the problem, in commit 673beed75fc347054328afaa2841d43410ee0f28. In the commit, the driver is creating many more decoders. Instead of creating one per DBPort, it's creating one per call/say. This is a lot more work for the garbage collector.

        This commit will also have to be rolled back or modified.

        Show
        Jeff Yemin
        added a comment - Commit 66c54120c56f9329b392264ec5efba0b2a292217 (fix for JAVA-421 ) is half the problem. The performance regression in that commit is that several large buffers are instantiated every time an instance of BSONInput is instantiated (two 1k byte arrays, one 16k byte array, and 1 16k char array). In practice an instance of BSONInput is instantiated every time a document is decoded. It's easy to roll back that change such that the arrays are again instantiated in BasicBSONDecoder. That part is done. But after rolling back this change, there's a second half of the problem, in commit 673beed75fc347054328afaa2841d43410ee0f28. In the commit, the driver is creating many more decoders. Instead of creating one per DBPort, it's creating one per call/say. This is a lot more work for the garbage collector. This commit will also have to be rolled back or modified.
        Hide
        Antoine Girbal
        added a comment -

        right ideally if using the default decoder, there should be 1 global.
        That's the way it was..
        Only if using a custom decoder would we need one per call (though could be pooled)

        Show
        Antoine Girbal
        added a comment - right ideally if using the default decoder, there should be 1 global. That's the way it was.. Only if using a custom decoder would we need one per call (though could be pooled)
        Hide
        Leo Kim
        added a comment -

        Can the DefaultDBDecoder.FACTORY return a DefaultDBDecoder singleton?

        Show
        Leo Kim
        added a comment - Can the DefaultDBDecoder.FACTORY return a DefaultDBDecoder singleton?
        Hide
        Jeff Yemin
        added a comment -

        The previous behavior was to cache an instance of the decoder with each instance of DBPort (which are themselves cached in the connection pool). It can't be a singleton because DefaultDBDecoder is not thread-safe.

        Show
        Jeff Yemin
        added a comment - The previous behavior was to cache an instance of the decoder with each instance of DBPort (which are themselves cached in the connection pool). It can't be a singleton because DefaultDBDecoder is not thread-safe.
        Hide
        Jeff Yemin
        added a comment -

        To keep things dead simple for a patch release, I'm not going to add any pooling of custom decoders just yet.

        Show
        Jeff Yemin
        added a comment - To keep things dead simple for a patch release, I'm not going to add any pooling of custom decoders just yet.
        Hide
        auto
        added a comment -

        Author:

        {u'login': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}

        Message: JAVA-505: using cached DBDecoder in most cases to avoid excessive memory allocation caused by too many instances of DefaultDBDecoder being created
        Branch: master
        https://github.com/mongodb/mongo-java-driver/commit/807c797be4212c7dbbff891aa177843edc6c5ab1

        Show
        auto
        added a comment - Author: {u'login': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'} Message: JAVA-505 : using cached DBDecoder in most cases to avoid excessive memory allocation caused by too many instances of DefaultDBDecoder being created Branch: master https://github.com/mongodb/mongo-java-driver/commit/807c797be4212c7dbbff891aa177843edc6c5ab1
        Hide
        auto
        added a comment -

        Author:

        {u'login': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'}

        Message: JAVA-505: Made encoder creation work just like decoder creation
        Branch: master
        https://github.com/mongodb/mongo-java-driver/commit/88b3406f2fc841b22ec2aa4208942b64876d20f1

        Show
        auto
        added a comment - Author: {u'login': u'jyemin', u'name': u'Jeff Yemin', u'email': u'jeff.yemin@10gen.com'} Message: JAVA-505 : Made encoder creation work just like decoder creation Branch: master https://github.com/mongodb/mongo-java-driver/commit/88b3406f2fc841b22ec2aa4208942b64876d20f1

          People

          • Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              2 years, 13 weeks, 5 days ago
              Date of 1st Reply: