[DOCS-13797] [SERVER] Limit the number of simultaneous index builds Created: 28/Jul/20  Updated: 13/Nov/23  Resolved: 01/Sep/22

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: 4.4.0, 5.0.0, 6.0.0
Fix Version/s: 4.4.0-rc0, 4.7.0, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Dave Cuthbert (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by DOCS-13809 Document maxNumActiveUserIndexBuilds ... Closed
Documented
documents SERVER-47155 Limit the number of simultaneous inde... Closed
Related
Participants:
Days since reply: 1 year, 22 weeks, 1 day ago
Epic Link: DOCSP-12974

 Description   

From SERVER-47155


Description

Downstream Change Summary

Existing, undocumented behavior as of 4.2.3 and 4.4.0:
In an attempt to bound memory usage and resource utilization, the server limits on the number of concurrent index builds started by a user on a primary node to 3.

New behavior (as of 4.4.0 only):
Any index builds started over the limit will block until the number of concurrent index builds drops below the limit. The log message "Too many index builds running simultaneously, waiting until the number of active index builds is below the threshold" (ID 4715500) will be logged when the limit is reached and an index build has blocked.

This default limit can be raised with the maxNumActiveUserIndexBuilds setParameter. This can be changed at startup or runtime.

Description of Linked Ticket

Each index build is allowed to use up to 200MB of memory outside of the WT cache by default. This is controlled by maxIndexBuildMemoryUsageMegabytes. In an attempt to bound memory usage and reduce WT cache pressure, we limit the number of concurrent index builds started by a user on a primary node.

Any index builds started over the limit will block until the number of concurrent index builds drops below the limit. The log message "Too many index builds running simultaneously, waiting until the number of active index builds is below the threshold" (ID 4715500) will be logged when the limit is reached and an index build has blocked.

This default limit can be raised with the maxNumActiveUserIndexBuilds setParameter. This can be changed at startup or runtime.

Original Description

Both the primary and secondary nodes will have an unlimited thread pool size. This is done to allow secondary nodes to startup as many index builders as necessary in order to prevent scheduling deadlocks during initial sync or oplog application.

When commands are run from user connections that need to create indexes, those commands will hang until there are less than 3 running index builder threads, or until the operation is interrupted.

 

Scope of changes

Impact to Other Docs

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)



 Comments   
Comment by Githook User [ 06/Sep/22 ]

Author:

{'name': 'Dave Cuthbert', 'email': '69165704+davemungo@users.noreply.github.com', 'username': 'davemungo'}

Message: DOCS-13797 limit simultaneous index builds (#1649)

  • Review feedback
  • Review feedback
  • Review feedback
  • Review feedback
  • Review feedback
Comment by Githook User [ 01/Sep/22 ]

Author:

{'name': 'Dave Cuthbert', 'email': '69165704+davemungo@users.noreply.github.com', 'username': 'davemungo'}

Message: DOCS-13797 BACKPORT (#1749)
Branch: v4.4
https://github.com/10gen/docs-mongodb-internal/commit/b30745c04d0b02fa96771c591000ecb751033fd6

Comment by Githook User [ 01/Sep/22 ]

Author:

{'name': 'Dave Cuthbert', 'email': '69165704+davemungo@users.noreply.github.com', 'username': 'davemungo'}

Message: DOCS-13797 BACKPORT (#1748)
Branch: v5.0
https://github.com/10gen/docs-mongodb-internal/commit/c44fbb54d04ae1bc9292d76c3cab5f4514cb9aa4

Comment by Githook User [ 31/Aug/22 ]

Author:

{'name': 'Dave Cuthbert', 'email': '69165704+davemungo@users.noreply.github.com', 'username': 'davemungo'}

Message: DOCS-13797 limit simultaneous index builds (#1649)

  • Review feedback
  • Review feedback
  • Review feedback
  • Review feedback
  • Review feedback
Comment by Eric Milkie [ 29/Jul/20 ]

This ticket is automatically generated by someone checking the "Docs" checkbox in the Downstream Changes tab of SERVER-47155. It fills in this ticket with the contents of the Downstream Changes box appearing on that same tab. In general we haven't been using that particular workflow for SERVER tickets for projects; instead we do it via the PM ticket (which has a similar checkbox). In this case however, the SERVER ticket wasn't part of a project and was standalone, and we just forgot to request that it get documented.

Generated at Thu Feb 08 08:08:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.