[SERVER-44487] Only pull looked-up fields into DocumentStorage Created: 07/Nov/19  Updated: 29/Oct/23  Resolved: 13/Nov/19

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 4.3.2

Type: Improvement Priority: Major - P3
Reporter: David Storch Assignee: David Storch
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Problem/Incident
Related
is related to SERVER-44482 Value duplicates fields in the cache ... Closed
Backwards Compatibility: Fully Compatible
Sprint: Query 2019-11-18
Participants:
Linked BF Score: 0

 Description   

During query execution, when we need to lookup a Value inside a Document, there are two possible code paths:

  1. The Value already resides in DocumentStorage.
  2. We need to create the Value by reading the backing BSON object.

The second case requires a linear scan of the BSON in order to find a BSONElement with the correct field name. During this scan, we currently eagerly convert all of the BSONElements we encounter into Values which reside inside the DocumentSource cache. This is done to ensure that only one forwards scan of the BSON object is necessary, avoiding potential quadratic performance required to look up n fields in a Document. However, our performance tests show that this eager conversion of BSONElement to Value is costly. In many cases, it is likely wasted work, since we might never need to read the field again later on. In order to improve performance, we should only pull the fields that the caller actually looked up into DocumentStorage.

In the future, if we find that repeated scans of the backing BSON are expensive, we can implement query analysis in order to compute a set of fields that are likely to be looked up later, and then pull these only these fields into the document's cache.



 Comments   
Comment by Githook User [ 13/Nov/19 ]

Author:

{'name': 'David Storch', 'username': 'dstorch', 'email': 'david.storch@mongodb.com'}

Message: SERVER-44487 Only construct Values in DocumentStorage cache for looked-up fields.
Branch: master
https://github.com/mongodb/mongo/commit/5ae506f4fa83c27b74a38ee6c2f58fbaf680c25d

Generated at Thu Feb 08 05:06:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.