[SERVER-39081] createIndexes/shardCollection race condition in agg_out.js Created: 18/Jan/19  Updated: 29/Oct/23  Resolved: 18/Jan/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.1.7
Fix Version/s: 4.1.8

Type: Bug Priority: Major - P3
Reporter: James Wahlin Assignee: James Wahlin
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:
Linked BF Score: 16

 Description   

agg_out.js will occasionally attempt to create a shard key index on a shard output collection and then shard the collection. This operation can be performed by multiple threads concurrently. The index is required for collection sharding as the collection may have data.

When 2 threads do attempt his concurrently, it is possible for the shardCollection command to fail due to missing shard key index. This can happen when:

Thread 1: Calls createIndex() to build the shard key index. Index build is started.
Thread 2: Calls createIndex(), which returns successfully as an index specification has been created for the in-progress index build.
Thread 2: Attempts to shard the collection. The shardCollection command fails as it does not see the in-progress index build (as it does not call listIndexes with the {includeIndexBuilds: true} option).

We can fix this by moving index creation for the output collection to the FSM setup function.



 Comments   
Comment by Githook User [ 05/Feb/19 ]

Author:

{'name': 'James Wahlin', 'email': 'james@mongodb.com', 'username': 'jameswahlin'}

Message: SERVER-38387 agg_out.js concurrency workload always fails to shard the output collection

(cherry picked from commit 9ed4ddae8b3c2aa7f876bdbfce00523c2b2a118c)

SERVER-39081 Fix createIndexes/shardCollection race condition in agg_out.js

(cherry picked from commit aad17aeba520f5c421716cb2d740057be96ce61a)
Branch: v4.0
https://github.com/mongodb/mongo/commit/0aee7dae5a62206232223614cc95cb371e69070a

Comment by Githook User [ 04/Feb/19 ]

Author:

{'name': 'James Wahlin', 'email': 'james@mongodb.com', 'username': 'jameswahlin'}

Message: SERVER-38387 agg_out.js concurrency workload always fails to shard the output collection

(cherry picked from commit 9ed4ddae8b3c2aa7f876bdbfce00523c2b2a118c)

SERVER-39081 Fix createIndexes/shardCollection race condition in agg_out.js

(cherry picked from commit aad17aeba520f5c421716cb2d740057be96ce61a)
Branch: v3.6
https://github.com/mongodb/mongo/commit/683afc244ee5104372190d29c6905a868cb6a78d

Comment by Githook User [ 18/Jan/19 ]

Author:

{'username': 'jameswahlin', 'email': 'james@mongodb.com', 'name': 'James Wahlin'}

Message: SERVER-39081 Fix createIndexes/shardCollection race condition in agg_out.js
Branch: master
https://github.com/mongodb/mongo/commit/aad17aeba520f5c421716cb2d740057be96ce61a

Generated at Thu Feb 08 04:50:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.