[DRIVERS-644] Hybrid index build Created: 28/Apr/19  Updated: 27/Oct/23  Resolved: 13/May/19

Status: Closed
Project: Drivers
Component/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Ian Whalen (Inactive) Assignee: David Golden
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-40926 createIndexes should return IndexBuil... Closed
Related
related to SERVER-39323 Don't consider in-progress indexes wh... Closed
Server Compat: 4.1

 Description   
Epic Summary

Summary

Combine the best qualities of foreground and background index builds to make a new type of index build, replacing the current index build types.

Motivation

Foreground index builds hold an exclusive database lock for the duration of the build, without yielding. This is one of the few operations on the server to hold a lock for a long period of time, and this causes various problems with the system. However, foreground index builds produce more efficient index structures; we want to preserve this outcome in the new Hybrid Index Build.

Documentation

Scope Document
Design Document



 Comments   
Comment by David Golden [ 13/May/19 ]

All necessary server tickets were implemented.

Comment by Shane Harvey [ 07/May/19 ]

david.golden is correct that drivers won't have to do anything if SERVER-40927 is implemented.

Currently, there is a subtle behavior change when an application attempts to build the same index in parallel. I outlined two potential breaking changes in this comment; one in mongomirror and one in GridFS. If SERVER-40927 is not implemented then we may need to workaround this issue in GridFS (by retrying createIndexes after IndexBuildAlreadyInProgress errors). Whether that issue is practical enough to warrant a change in all drivers needs some more thought.

Comment by Bernie Hackett [ 07/May/19 ]

shane.harvey, for drivers the only changes here are test related, right? I haven't seen any reports of test failures outside of PyMongo.

IIRC MongoMirror needs some work to deal with this change.

Comment by David Golden [ 06/May/19 ]

I spoke too soon.  I realized that shane.harvey found and reported SERVER-39323 about breakage caused by the server changing behavior of createIndexes.  Depending on whether SERVER-40926 or SERVER-40927 are completed, drivers may have to do additional error checking on createIndexes.

Generated at Thu Feb 08 08:22:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.