[SERVER-29449] Explain of find command does not transform query for shards Created: 05/Jun/17  Updated: 17/Nov/23

Status: Backlog
Project: Core Server
Component/s: Querying, Sharding
Affects Version/s: 3.4.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Joe Caswell Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: query-44-grooming, quick-win, storch
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-82471 Make sure explain.find issued on mong... Backlog
is duplicated by SERVER-32555 executionStats.nReturned incorrect fo... Closed
is duplicated by SERVER-33895 Not needed SORT_KEY_GENERATOR in aggr... Closed
Related
related to SERVER-32563 explain output for sharded count comm... Backlog
related to SERVER-34338 find explain in legacy query path on ... Closed
related to SERVER-82218 Unify explain and execution path in s... Closed
is related to SERVER-34736 Make ClusterFind::explain directly ca... Closed
Assigned Teams:
Query Optimization
Operating System: ALL
Steps To Reproduce:

Starting with a sharded collection, in this example, the people.json from MongoDB University:

mongos> db.version();
3.4.4
mongos> db.people.find().itcount();
50474
mongos> db.people.find().skip(1000).itcount();
49474
mongos> var stats = db.people.find().explain("executionStats");
mongos> print("nReturned: " + stats.executionStats.nReturned);var total=0;stats.executionStats.executionStages.shards.forEach(function(s){print(s.shardName+" nReturned: "+s.executionStages.nReturned); total += s.executionStages.nReturned;}); print("Sum of shards: " + total);
nReturned: 50474
shard0000 nReturned: 12130
shard0001 nReturned: 13125
shard0002 nReturned: 25219
Sum of shards: 50474
mongos> var stats = db.people.find().skip(1000).explain("executionStats");
mongos> print("nReturned: " + stats.executionStats.nReturned);var total=0;stats.executionStats.executionStages.shards.forEach(function(s){print(s.shardName+" nReturned: "+s.executionStages.nReturned); total += s.executionStages.nReturned;}); print("Sum of shards: " + total);
nReturned: 47474
shard0000 nReturned: 11130
shard0001 nReturned: 12125
shard0002 nReturned: 24219
Sum of shards: 47474

Sprint: Query 2018-05-21
Participants:

 Description   

This can lead to various problems such as:

  1. the following query does not include a SORT_KEY_GENERATOR stage:

    db.sharded.explain().find().sort({a: 1});
    

    This is because the explain path on mongos diverges from the regular, non-explain path pretty early. In particular, the non-explain path will transform the query sent to the shards in transformQueryForShards. This transformation does not happen in the explain path, and causes us to leave off the projection {$sortKey: 1} which will signal to the shards that we intend to merge the sorted output on mongos.

  2. When using .explain("executionStats") on a query that includes a skip phase on sharded collection, the skip phase is applied separately at each queried shard. The result is each individual shard's nReturned is reduced by the skip_count, so the overall nReturned is reduced by (skip_count * shard_count).
  3. An "executionStats" explain with a limit is subject to a similar bug, as described in SERVER-32555.


 Comments   
Comment by Githook User [ 06/Nov/19 ]

Author:

{'name': 'Charlie Swanson', 'username': 'cswanson310', 'email': 'charlie.swanson@mongodb.com'}

Message: SERVER-44208 Update SERVER-32555 TODOs

The ticket was closed as a duplicate of SERVER-29449 which is still
unresolved. So the TODOs should remain but are stale.
Branch: master
https://github.com/mongodb/mongo/commit/a0f929889db6566a92200ac3f1f430f1d042862a

Comment by Charlie Swanson [ 25/Apr/18 ]

The previous description of this ticket was describing a symptom of a more general problem, also experienced in SERVER-33895. I've generalized the title/description, and moved this back into Needs Triage for re-evaluation.

Generated at Thu Feb 08 04:20:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.