[SERVER-12679] Regression of queries that project a large number of fields Created: 11/Feb/14  Updated: 06/Dec/22  Resolved: 22/Aug/19

Status: Closed
Project: Core Server
Component/s: Performance, Querying
Affects Version/s: 2.5.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Davide Italiano Assignee: Backlog - Query Team (Inactive)
Resolution: Won't Do Votes: 1
Labels: 26qa, Performance, query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File murmur_vs_oldhash.png     File projection.js    
Assigned Teams:
Query
Operating System: ALL
Steps To Reproduce:

Linux ip-10-0-0-85 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Participants:

 Description   

Some of my benchmarks show the following (it happens both in the single and multi-threaded case scenario). When the number of fields to project is small the regression is not noticeable, I'll try to investigate a bit more on the root cause.
Minimal test case attached.

Master (as per today)

connecting to: test
9071
8931
8937
8940
8980

2.4.9

connecting to: test
8183
8065
8097
8165
8081



 Comments   
Comment by Charlie Swanson [ 22/Aug/19 ]

This is an old report of a performance regression that we do not plan on spending time investigating.

Comment by Davide Italiano [ 19/Feb/14 ]

I modified the script to change the number of fields that are projected (starting from 5, up to 500).
It looks like the growth of the function is linear in both cases. The two functions are equivalent up to ~ 80 fields projected, after which Murmur is beaten by the other hash. Screenshot attached.

Comment by Davide Italiano [ 11/Feb/14 ]

I had time to do some profiling.

(pprof) top 5
Total: 4288 samples
     696  16.2%  16.2%      696  16.2% __nss_hosts_lookup
     587  13.7%  29.9%      591  13.8% boost::scoped_array::operator[] (inline)
     514  12.0%  41.9%     3854  89.9% mongo::ProjectionExec::transform@952f70
     459  10.7%  52.6%      621  14.5% MurmurHash3_x64_128
     273   6.4%  59.0%      298   6.9% StringData (inline)
     249   5.8%  64.8%      581  13.5% mongo::BSONObjIterator::next
     194   4.5%  69.3%     2449  57.1% mongo::ProjectionExec::append
     162   3.8%  73.1%      273   6.4% fmix (inline)
     158   3.7%  76.8%      755  17.6% mongo::UnorderedFastKeyTable::Area::find (inline)
     157   3.7%  80.4%      370   8.6% mongo::BSONElement::size

Major bottleneck is lookup in the StringMap (FieldMap)

     .      .  482:
     .      .  483:         // Skip if the field name matches a computed $meta field.
     .      .  484:         // $meta projection fields can exist at the top level of
     .      .  485:         // the result document and the field names cannot be dotted.
    41     74  486:         if (_meta.find(elt.fieldName()) != _meta.end()) {
     .      .  487:             return Status::OK();
     .      .  488:         }
     .      .  489:
    35   1903  490:         FieldMap::const_iterator field = _fields.find(elt.fieldName());
     2     18  491:         if (field == _fields.end()) {
    16     28  492:             if (_include) {
     .      .  493:                 bob->append(elt);

Apparently this commit is responsible for the performance drop, reverting it I'm completely able to come back to the 2.4.9 numbers
https://github.com/mongodb/mongo/commit/22424987d2f0df32671998f0c7574adcfa07bf32

Generated at Thu Feb 08 03:29:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.