[SERVER-44657] Variables class does not behave as an associative map Created: 15/Nov/19  Updated: 29/Oct/23  Resolved: 20/Mar/20

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.7.0

Type: Bug Priority: Major - P3
Reporter: Martin Neupauer Assignee: Anton Korshunov
Resolution: Fixed Votes: 0
Labels: afz, qexec-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:
Linked BF Score: 20

 Description   

See the related BF for the context. The problem is in Variables class. It's interface seems like an associative map from Id (64 bit int) to Value (ValueAndState).
However, internally it uses std::vector instead of a real map. So when we call setValue(100000, ...) it actually allocates a vector of 100000 elements and then _valueList[100000] = ...
Even though there is only 1 variable the underlying vector is 100000 elements big!!! The vector then gets constructed/destructed many times in the $graphlookup hence it takes forever to execute.
The tangential question is why the Id goes up so much in the graphlookup query. The partial answer is that we do not reset the IdGenerator for every subpipeline we construct. It was deemed more risky to reset if back to 0 than switching to a proper map in Variables class.



 Comments   
Comment by Githook User [ 20/Mar/20 ]

Author:

{'name': 'Anton Korshunov', 'username': 'antkorsh', 'email': 'anton.korshunov@mongodb.com'}

Message: SERVER-44657 Variables class does not behave as an associative map
Branch: master
https://github.com/mongodb/mongo/commit/a6740f45374acb4171edc920240c51218754fb0a

Comment by Martin Neupauer [ 20/Dec/19 ]

I'll replace std::vector<ValueAndState> _valueList; with stdx::unordered_map<Id, ValueAndState> _valueList; in the Variables class and it should do the trick.

Comment by David Storch [ 21/Nov/19 ]

martin.neupauer can you take a look at this tomorrow for BF friday?

Comment by Craig Homa [ 19/Nov/19 ]

We plan to look for a fix on a future BF Friday.

Generated at Thu Feb 08 05:06:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.