[JAVA-601] Throughput mongo java driver Versus mongo C driver Created: 12/Jul/12  Updated: 25/Jun/13  Resolved: 14/Sep/12

Status: Closed
Project: Java Driver
Component/s: Performance
Affects Version/s: 2.8.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Sushil Thampy Assignee: Jeffrey Yemin
Resolution: Duplicate Votes: 0
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux 2.6.32-220.el6.x86_64
Red Hat Enterprise Linux Server release 6.2
java version "1.7.0_05"
Mongo java driver 2.8.0
mongodb-linux-x86_64-2.0.6
App server has 32-cores, hyperthreading, 192gb ram


Attachments: File MongoTest.java     File README     Text File mongotest.c    
Issue Links:
Depends
depends on JAVA-610 Make DBCursor finalizer optional and ... Closed
Related

 Description   

The heap usage in the java driver is causing about 30% of the time to be spent on garbage collection reducing throughput. Worsens with increased number of threads and/or increased heap size. The mongo C driver, in comparison is able to sustain a good thoughput. Sample tests attached.



 Comments   
Comment by Sushil Thampy [ 09/Aug/12 ]

I used a workaround which removed the finalizer ..however the final change is this one: JAVA-610

Comment by Jon Hoffman [ 09/Aug/12 ]

Hi Sushil, do you have a fix you can share?

Comment by Sushil Thampy [ 24/Jul/12 ]

Jeff indicated that GC might be slower collecting instances of com.mongodb.DBApiLayer$Result, because that class has overrides for the finalize method. We tried a test removing the finalize method, and the improvements seen are very good. [ used a ycsb read only workload to compare ]. The time spent in garbage collection went down to less than 1% of the overall execution time (from the 25-30% earlier). The overall reported throughput and average latency is also much better (improvements of about 25-30%).

Comment by Jeffrey Yemin [ 12/Jul/12 ]

The Java driver has been optimized over the years to minimize unnecessary heap usage, but there's only so much that can be done. See for example this excellent paper comparing performance of C+/Java/Go/Scala: https://days2011.scala-lang.org/sites/days2011/files/ws3-1-Hundt.pdf. One of the things they point out is the dramatic affect of GC tuning on the overall performance of Java, but even after tuning the C+ application is significantly faster.

Also, to make the comparison farer, you should comment out the println in the Java program (which, by the way, doesn't compile as is).

If you have further suggestions for reducing memory consumption, please open a pull request.

Generated at Thu Feb 08 08:52:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.