[SERVER-39532] Handle commitQuorum via createIndexes command or setIndexCommitQuorum command on a standalone/embedded node Created: 12/Feb/19  Updated: 29/Oct/23  Resolved: 04/Feb/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.3.4

Type: Improvement Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Suganthi Mani
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2020-01-13, Execution Team 2020-01-27, Execution Team 2020-02-10
Participants:

 Description   

Since we're adding a 'commitQuorum' field to the createIndexes command to enable simultaneous index builds, we need to explore whether we should assert/warn/ignore if the 'commitQuorum' field is also present when running the createIndexes command in standalone mode. Similarly, how the setIndexCommitQuorum command should behave on a standalone node.

UPDATE: we've settled on error'ing on receipt of commitQuorum settings on standalones.

Note: internally we can set whatever we want, this is just defining the user interface.



 Comments   
Comment by Githook User [ 04/Feb/20 ]

Author:

{'name': 'Suganthi Mani', 'username': 'smani87', 'email': 'suganthi.mani@mongodb.com'}

Message: SERVER-39532 standalones doesn't support commitQuorum.
Branch: master
https://github.com/mongodb/mongo/commit/6bcdb9197f45c6791740cf696f9174974b114ecd

Comment by Eric Milkie [ 04/Feb/20 ]

For number 1, it was a long discussion in the design over whether to do it this way. I believe we decided to keep them separate because it would be easier to understand the concepts as separate, especially when we introduce the ability to change the quorum on a running index build (it doesn't make sense to change the write concern on a running build).
For number 2, commitQuorum shouldn't be restricted in the way you describe.
For number 3, the design has been approved in its current state, which is to use an API where the user can provide an explicit override of the commitQuorum for each index build. We should reopen the design for review to discuss changing this further, as the design reviewers will want to weigh in.

Comment by Suganthi Mani [ 04/Feb/20 ]

Questions/concerns to milkie and benety.goh

My understanding is that createIndex cmd would support a field named "commitQuorum". For e.g.

   { createIndexes : "bar",
     indexes : [ { ns : "test.bar", key : { x : 1 }, name: "x_1" } ],
     commitQuorum: "majority" }

1) Are we really planning to support the "commitQuorum" to accept to any number of data bearing nodes (like writeConcern). If so, why can't we just use the writeConcern options as commitQuorum Value?

  • By this we can create less confusion to users and we can avoid adding the new field "commitQuorum" to "createIndexes" cmd.

2) In case, if commitQuorum is going to restrict the values to only 0 and "majority", I would like the field to be renamed as majorityCommitQuorum: on|off to reduce the confusion for users.

3) We mainly have this commit quorum to solve the majority commit point lag due to slow secondaries. And, I consider it as an internal optimization. I strongly feel, createIndexes should not have an explicit commitQuorum field. By default, the majority commit quorum will be on. In case, if the user wants to turn off this majority commit quorum feature (index build commit can happen only if majority nodes finished their index build draining phase), I would expect them to use the server parameter "enableIndexBuildMajorityCommitQuorum". If you think, this can lead to some issue. I would expect at least option #2.

Note to myself: Currently we allow commitQuorum to set to any number > replica set size.

Generated at Thu Feb 08 04:52:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.