[SERVER-46048] Prepared transactions should block if any single-phase index builds are in progress Created: 07/Feb/20  Updated: 29/Oct/23  Resolved: 11/Mar/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.4.0-rc0, 4.7.0

Type: Bug Priority: Major - P3
Reporter: Louis Williams Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4
Sprint: Execution Team 2020-02-24, Execution Team 2020-03-09, Execution Team 2020-03-23
Participants:

 Description   

Prepared transactions and two-phase index builds can coexist during secondary oplog application (see SERVER-43638), however single-phase index builds cannot (see SERVER-38588). 

The logic to that determines whether to block checks the system's ability to support two-phase index builds. This is not completely correct, because it may be possible for index builds to operation in single-phase mode even if two-phase index builds are supported.

This will lead to potential index inconsistencies in line with SERVER-38588.



 Comments   
Comment by Githook User [ 11/Mar/20 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 block prepared transactions on single-phased index builds on secondaries

(cherry picked from commit 3af11010c3efe00f4fc3304de7a6b101db218e60)
Branch: v4.4
https://github.com/mongodb/mongo/commit/bd00f8ebf1025840f95fa144fbd1d8b9cad1f8ab

Comment by Githook User [ 11/Mar/20 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 remove unnecessary text log search from js test

(cherry picked from commit db557f20e03c91590ab3f6178bdcce2d327c7235)
Branch: v4.4
https://github.com/mongodb/mongo/commit/cdb9c945a6e31eb0e0e97b249199953a87d9f905

Comment by Githook User [ 11/Mar/20 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 IndexBuildsCoordinator::inProgForCollection() allows filtering on single/two phase index builds in a collection

(cherry picked from commit 588531744dcb3da7ab811da297cb53d62a21add6)
Branch: v4.4
https://github.com/mongodb/mongo/commit/83aa04ce7935f1a97389d14cae1ec26dcb32584d

Comment by Githook User [ 11/Mar/20 ]

Author:

{'name': 'Benety Goh', 'username': 'benety', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 IndexBuildsCoordinator::awaitNoIndexBuildInProgressForCollection() supports waiting on a one type of index builds (single/two phased)

(cherry picked from commit 0d0a968cad781532ab3f01acf690fda387fc5c47)
Branch: v4.4
https://github.com/mongodb/mongo/commit/0edc43bcd6fc5adfd8330347270f73757aa8031f

Comment by Githook User [ 11/Mar/20 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 add IndexBuildsCoordinator::_filterIndexBuilds_inlock()

(cherry picked from commit b1c7fcda2ec1fbb77430fb66898561ba59e86413)
Branch: v4.4
https://github.com/mongodb/mongo/commit/270e7e2f28d0e1c09318f49c3d2f839a1c3871d0

Comment by Githook User [ 10/Mar/20 ]

Author:

{'name': 'Benety Goh', 'username': 'benety', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 block prepared transactions on single-phased index builds on secondaries
Branch: master
https://github.com/mongodb/mongo/commit/3af11010c3efe00f4fc3304de7a6b101db218e60

Comment by Githook User [ 10/Mar/20 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 remove unnecessary text log search from js test
Branch: master
https://github.com/mongodb/mongo/commit/db557f20e03c91590ab3f6178bdcce2d327c7235

Comment by Githook User [ 10/Mar/20 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 IndexBuildsCoordinator::inProgForCollection() allows filtering on single/two phase index builds in a collection
Branch: master
https://github.com/mongodb/mongo/commit/588531744dcb3da7ab811da297cb53d62a21add6

Comment by Githook User [ 10/Mar/20 ]

Author:

{'name': 'Benety Goh', 'username': 'benety', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 IndexBuildsCoordinator::awaitNoIndexBuildInProgressForCollection() supports waiting on a one type of index builds (single/two phased)
Branch: master
https://github.com/mongodb/mongo/commit/0d0a968cad781532ab3f01acf690fda387fc5c47

Comment by Githook User [ 10/Mar/20 ]

Author:

{'name': 'Benety Goh', 'username': 'benety', 'email': 'benety@mongodb.com'}

Message: SERVER-46048 add IndexBuildsCoordinator::_filterIndexBuilds_inlock()
Branch: master
https://github.com/mongodb/mongo/commit/b1c7fcda2ec1fbb77430fb66898561ba59e86413

Comment by Benety Goh [ 05/Mar/20 ]

I looked at the internal subsystems that still build indexes outside the IndexBuildsCoordinator and these systems use the two-phased index build method:

As a defensive measure, the check to block prepared transactions should still be modified to check for the presence of single-phased index builds managed by the IndexBuildsCoordinator.

Comment by Eric Milkie [ 05/Mar/20 ]

Do we still have a single-phase path today on secondaries for non-empty collections?

If so, we'll need a way to determine if in-progress index builds are single-phase (I think we can query the index builds manager for this info?), and then use the code that is in 4.2 to block prepared transactions on secondaries:
https://github.com/mongodb/mongo/commit/c9bdb7e64265bf36cb3ca81edd8cb36ed0589a04

Generated at Thu Feb 08 05:10:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.