[SERVER-6572] Performance issue Created: 24/Jul/12  Updated: 11/Jul/16  Resolved: 14/Aug/12

Status: Closed
Project: Core Server
Component/s: Performance
Affects Version/s: 2.2.0-rc0
Fix Version/s: 2.2.0-rc1

Type: Bug Priority: Major - P3
Reporter: Eric Milkie Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 4
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-6728 B-tree scans are significantly slower... Closed
Related
Operating System: ALL
Participants:

 Description   

A read/write test shows significant performance degradation between 2.0 and 2.2



 Comments   
Comment by auto [ 09/Aug/12 ]

Author:

{u'date': u'2012-08-08T22:41:59-07:00', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-6572 - use pthread specifics on os x for performance
Branch: master
https://github.com/mongodb/mongo/commit/78de2819dca377af2e4c26b1160832336a573126

Comment by auto [ 09/Aug/12 ]

Author:

{u'date': u'2012-08-08T18:22:29-07:00', u'email': u'tad@10gen.com', u'name': u'Tad Marshall'}

Message: SERVER-6572 fix Windows warning

Explicitly make the target of the bit shift 'long long' to avoid
MSVC warning: "src\mongo\db\record.cpp(242) : warning C4334:
'<<' : result of 32-bit shift implicitly converted to 64 bits
(was 64-bit shift intended?)".
Branch: master
https://github.com/mongodb/mongo/commit/4e7a01cb208120a7ff956d8ae7d37b8c00a36c71

Comment by auto [ 08/Aug/12 ]

Author:

{u'date': u'2012-08-08T15:34:03-07:00', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-6572 - fix windows
Branch: master
https://github.com/mongodb/mongo/commit/f69d15b42a46326aabc436077c047adfa6fa6061

Comment by auto [ 08/Aug/12 ]

Author:

{u'date': u'2012-08-08T15:29:47-07:00', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-6572 - can't use enum on windows
Branch: master
https://github.com/mongodb/mongo/commit/a3dd87187cc8788788dc1fe51b71acdfc18b1ffe

Comment by auto [ 08/Aug/12 ]

Author:

{u'date': u'2012-08-08T14:56:33-07:00', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-6572 - don't use a class so we can use simple thread locals
Branch: master
https://github.com/mongodb/mongo/commit/e09439b53ea25f7df38b06c159e78d97e0e9f310

Comment by auto [ 08/Aug/12 ]

Author:

{u'date': u'2012-08-08T13:59:44-07:00', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-6572 use a thread local cache for better perforamnce in record caching
needs work on os x
Branch: master
https://github.com/mongodb/mongo/commit/368c55d0f63a2914ec6c0b6cd30fdaafd1b01090

Comment by auto [ 08/Aug/12 ]

Author:

{u'date': u'2012-08-08T13:50:56-07:00', u'email': u'schwerin@10gen.com', u'name': u'Andy Schwerin'}

Message: Revert "SERVER-6572 use a thread local cache for better perforamnce in record caching"

This reverts commit f7fdee2e22b37c20a2999fe579111f0fb1f5eb91.
Branch: master
https://github.com/mongodb/mongo/commit/e6d8466bc18b4b6c686a745276e745b69abc6ebd

Comment by auto [ 08/Aug/12 ]

Author:

{u'date': u'2012-08-08T13:47:24-07:00', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-6572 use a thread local cache for better perforamnce in record caching
needs work on osx
Branch: master
https://github.com/mongodb/mongo/commit/f7fdee2e22b37c20a2999fe579111f0fb1f5eb91

Comment by Ben Becker [ 03/Aug/12 ]

The biggest discrepancy between v2.0 and v2.2 appears to be due to the MemoryTracking code. This is most evident when querying for large result sets. The worst-case test I came up with was simply querying for 200,000 small documents.

Dataset

> db.x.find()
{ "_id" : ObjectId("501c0b0573a6ea925a901c26"), "a" : 0 }
{ "_id" : ObjectId("501c0b0573a6ea925a901c27"), "a" : 1 }
...
> db.x.count()
200000

Tests

> benchRun( { ops : [{ns:'test.x', op:'find', query:{a:{$gte:1}}}] , host : 'localhost' , db : 'test' , parallel : 2 , seconds : 30 } )
> benchRun( { ops : [{ns:'test.x', op:'find', query:{a:{$gte:1}}}] , host : 'localhost' , db : 'test' , parallel : 1 , seconds : 30 } )

Results
Threads v2.0 v2.2-rc1-pre v2.2-rc1-pre (No MemTracking*)
1 12q/s, 24gm/s 8q/s, 16gm/s 11q/s, 22gm/s
2 21q/s, 42gm/s 13q/s, 26gm/s 21q/s, 41gm/s

*No MemTracking simply means:

-    bool Record::MemoryTrackingEnabled = true;
+    bool Record::MemoryTrackingEnabled = false;

Profiler Results

Perftools has been troublesome due to libunwind/pthread, so these results were taken from sysprof. The default build of v2.2-rc1-pre indicated the following notable call times:

mongo::ps::Rolling::access():            6.9% CPU
mongo::ps::hash():                       5.15% CPU
mongo::Record::likelyInPhysicalMemory(): 4.5% CPU

Final Notes

The DEV block in likelyInPhysicalMemory() really throws things off. I think we should get rid.

Generated at Thu Feb 08 03:12:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.