[SERVER-80255] Refactor lookup_unionWith_subpipeline_local_read.js to avoid asserting on exact number of local/remote reads Created: 18/Aug/23  Updated: 29/Oct/23  Resolved: 12/Oct/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Ben Shteinfeld Assignee: Ben Shteinfeld
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-80254 Refactor lookup_unionWith_subpipeline... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.1
Participants:
Linked BF Score: 113

 Description   

lookup_unionWith_subpipeline_local_read.js is structured to execute pipelines with subpipelines in various topologies and inspect profiler entries to assert that a certain number of the subpipeline invocations where performed as remote vs local reads. However there are an increasing number of complex race conditions which affect the exact number of local vs remote reads that are done, becoming a cause of non-trivial BFs. But in the majority of cases, we really only care whether local reads were performed or not, we don't care exactly how many there were.

The task of this ticket is to refactor the test to avoid asserting on an exact number of local/remote reads, making it more robust.



 Comments   
Comment by Githook User [ 12/Oct/23 ]

Author:

{'name': 'Ben Shteinfeld', 'email': 'ben.shteinfeld@mongodb.com', 'username': 'bshteinfeld'}

Message: SERVER-80255 Refactor lookup local read test to be more robust

`lookup_unionWith_subpipeline_local_read.js` is currently structured to
execute aggregations with subpipelines in various toplogoies/options and
query the profiler to assert that a certain number of reads for the
subpipeline occured as "local reads" vs "remote reads". Due to many race
conditions, the exact number might vary from run to run. This patch
refactors the test to instead assert on whether local/remote reads were
done on a particular shard.
Branch: master
https://github.com/mongodb/mongo/commit/fead90944aa779b19b488c558672f24762f35f2a

Comment by Ivan Fefer [ 25/Aug/23 ]

I would like to have some tests cover this optimization: https://github.com/mongodb/mongo/blob/6cf7867d79341b141162bbc086217706161cfdd7/src/mongo/db/pipeline/sharded_agg_helpers.cpp#L1811

When the data, required for a lookup is local, don't create remote cursors

Comment by Ben Shteinfeld [ 24/Aug/23 ]

Can you please be more specific as to what coverage is lost?

Comment by Ivan Fefer [ 24/Aug/23 ]

Doing this will lead to loss of coverage of local read optimization for $lookup.
We would like to still have it. Maybe we should introduce more simple test that checks local/remote reads exactly?

Comment by Ivan Fefer [ 24/Aug/23 ]

Doing this will lead to loss of coverage of local read optimization for $lookup.
We would like to still have it. Maybe we should introduce more simple test that checks local/remote reads exactly?

Generated at Thu Feb 08 06:43:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.