[SERVER-61969] Simplify exact_shard_key_target.js so that it does not depend on orphan writing Created: 09/Dec/21  Updated: 29/Oct/23  Resolved: 07/Jan/22

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 5.3.0

Type: Improvement Priority: Major - P3
Reporter: Antonio Fuschetto Assignee: Antonio Fuschetto
Resolution: Fixed Votes: 0
Labels: sharding-wfbf-day
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: Sharding EMEA 2022-01-10
Participants:

 Description   

As a consequence of PM-2423, write operations to orphan documents have no effect on the actual data, leaving it unchanged on disk and waiting for range deleter to do its job.

The exact_shard_key_target.js test verifies that shard key targeted update/delete operations go to exactly one shard when targeted by nested shard keys. This is currently based on explicit insertion of orphan documents, requiring now additional logic to properly handle the new behavior of writes to orphans.

Evaluate a new implementation in order to achieve the same test's goals without using orphan documents.

Refer to this comment for more information on the requirement.



 Comments   
Comment by Githook User [ 07/Jan/22 ]

Author:

{'name': 'Antonio Fuschetto', 'email': 'antonio.fuschetto@mongodb.com', 'username': 'afuschetto'}

Message: SERVER-61969 Simplify exact_shard_key_target.js so that it does not depend on orphan writing
Branch: master
https://github.com/mongodb/mongo/commit/1a99f108fe8ef92ac35b66e22e92fa316193504a

Comment by Antonio Fuschetto [ 07/Jan/22 ]

We eventually found a third option that allowed us to avoid using orphaned documents in this test.

Using the explain function in the write operations, we are able to identify the shards actually involved in the operation thus excluding that it is broadcasted to all shards. For example:

var explainOutput = coll.explain().remove({a: {b: 1}}, {justOne: true});
assert.eq(1, explainOutput.queryPlanner.winningPlan.shards.length);

Comment by Antonio Fuschetto [ 04/Jan/22 ]

This test verifies that, on sharded collections using nested shard keys, some targeted operations using the object notation (e.g. {a: {b: 1}}) are forwarded to the proper shard. Before MongoDB version 2.6, indeed, this type of operations were improperly broadcasted to all shards (see SERVER-14138). To test targeted write operations, the test environment needs sharded collections with orphaned documents so that the test can verify these remain unchanged.

After recent changes to prevent write operations to orphaned documents (see SERVER-59832), these conditions are equally satisfied regardless of the fix introduced in version 2.6.

I therefore see two options:

  1. The specific tests that verify that orphans are not changed (2 out of 4 tests) could be removed when the feature flag to prevent writes to orphans will be enabled by default. At the present, instead, it could make sense to run these tests (as they are) when the feature flag is disabled.
  2. Adapt these 2 tests so that they can work as originally designed with orphaned documents. This means to explicitly remove them after the test run as they have been explicitly inserted.

In order to preserve the semantic of the test, which also depicts the problem that has been fixed, I opt for the second solution. This also works well taking into account upcoming changes where direct writes to orphans (always allowed) will not generate events on change streams.

Generated at Thu Feb 08 05:53:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.