[SERVER-43664] Speedup WiredTiger storage engine startup for many tables by optimizing WiredTigerUtil::setTableLogging() Created: 26/Sep/19 Updated: 29/Oct/23 Resolved: 03/Sep/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 4.7.0, 4.4.2, 4.2.11 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Louis Williams | Assignee: | Gregory Wlodarek |
| Resolution: | Fixed | Votes: | 3 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||
| Backport Requested: |
v4.4, v4.2
|
||||||||||||||||||||||||||||||||
| Sprint: | Execution Team 2019-12-30, Execution Team 2020-09-07 | ||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||
| Linked BF Score: | 43 | ||||||||||||||||||||||||||||||||
| Description |
|
Even with these optimizations, given a large number of collections, we could still benefit from parallelizing the startup procedure. It is currently single-threaded and mostly CPU-bound, spending almost all CPU time-per-collection in setTableLogging(). This uses an expensive "metadata:create" cursor to determine whether or not a table is logged. |
| Comments |
| Comment by Githook User [ 27/Oct/20 ] | ||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}Message: (cherry picked from commit 8e64b07e3bb363347ee2c11a56aba873365ed74a) | ||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 12/Oct/20 ] | ||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}Message: Revert " This reverts commit 97fdfbefdc6841d0b07d0bf54a28c86c70ca5e19. | ||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 03/Oct/20 ] | ||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}Message: (cherry picked from commit 8e64b07e3bb363347ee2c11a56aba873365ed74a) | ||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 03/Oct/20 ] | ||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}Message: (cherry picked from commit 8e64b07e3bb363347ee2c11a56aba873365ed74a) | ||||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 07/Sep/20 ] | ||||||||||||||||||||||||||||||||||||
|
Thanks gregory.wlodarek - that is a great improvement! | ||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 03/Sep/20 ] | ||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}Message: | ||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 03/Sep/20 ] | ||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}Message: | ||||||||||||||||||||||||||||||||||||
| Comment by Gregory Wlodarek [ 02/Sep/20 ] | ||||||||||||||||||||||||||||||||||||
|
The change for this is now in the commit queue, so I thought I'd follow up with a summary of the results. I ran some basic performance results on the time spent starting up and shutting down mongod averaged out over ten executions. On master:
On v4.4:
Difference:
| ||||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 05/Jan/20 ] | ||||||||||||||||||||||||||||||||||||
|
milkie that sounds like a good idea to me. If we make a change here I think we should also check to make sure that opening an old database with a new version gets a useful error message, and doesn't either try to proceed without updating the logging settings or (worse) corrupting the database. I remember that it was a bit of a journey to get to this solution with upgrade/downgrade considerations (I think | ||||||||||||||||||||||||||||||||||||
| Comment by Eric Milkie [ 30/Dec/19 ] | ||||||||||||||||||||||||||||||||||||
|
After examining this behavior, I think we should work on removing the blanket check for table logging instead of trying to speed it up. I believe we only need to go through all the tables when making an actual change to logging, and we can simply assume the logging is set correctly otherwise. The logging setting changes when switching between replica set and standalone, and when starting a replica set member in standalone maintenance mode. |