[SERVER-33494] WT SizeStorer never deletes old entries Created: 26/Feb/18 Updated: 20/Nov/23 Resolved: 13/Nov/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 7.3.0-rc0, 7.2.0-rc2, 7.0.5, 6.0.13, 5.0.24 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Daniel Gottlieb (Inactive) | Assignee: | Gregory Noma |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||
| Assigned Teams: |
Storage Execution
|
||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||
| Backport Requested: |
v7.2, v7.0, v6.0, v5.0
|
||||||||||||||||||||||||||||||||
| Sprint: | Execution Team 2023-11-13, Execution Team 2023-11-27 | ||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||
| Linked BF Score: | 152 | ||||||||||||||||||||||||||||||||
| Description |
|
The WTSizeStorer is used to keep fast counts for each collection in the system. Creating a collection adds an `ident` -> `count` mapping, but those key/value pairs are never deleted. RTT makes this problem worse. When rolling forward from the stable point -> common point re-creates a collection, it will get a new `ident`, thus leaking the previous ident in the table. |
| Comments |
| Comment by Githook User [ 16/Nov/23 ] |
|
Author: {'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}Message: (cherry picked from commit b4e5daaeb51920b0074661c87dc3b3765431e04e) |
| Comment by Githook User [ 16/Nov/23 ] |
|
Author: {'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}Message: (cherry picked from commit b4e5daaeb51920b0074661c87dc3b3765431e04e) |
| Comment by Githook User [ 16/Nov/23 ] |
|
Author: {'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}Message: (cherry picked from commit b4e5daaeb51920b0074661c87dc3b3765431e04e) |
| Comment by Githook User [ 15/Nov/23 ] |
|
Author: {'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}Message: (cherry picked from commit b4e5daaeb51920b0074661c87dc3b3765431e04e) |
| Comment by Githook User [ 13/Nov/23 ] |
|
Author: {'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}Message: |
| Comment by Eric Milkie [ 10/Nov/23 ] |
|
Does either startup-repair or the validate command audit the entries in this table? If not, should they? At least startup-repair could make sure that any orphaned entries get deleted. |
| Comment by Gregory Noma [ 10/Nov/23 ] |
|
Since this ticket is focusing only on removing size storer entries upon collection drop, I filed SERVER-83120 to look into some one-time cleanup mechanism. |
| Comment by Eric Milkie [ 17/Oct/23 ] |
|
Perhaps back in 2018 things worked differently when Dan G originally wrote the details for this ticket. From what Louis says, I think we should change the title and description to match today's behavior. |
| Comment by Louis Williams [ 17/Oct/23 ] |
|
The size storer, as it exists currently behaves as follows:
milkie@mongodb.com I'm not aware of where we would be leaking memory. The problem as I understand it is that we never clean up old entries in the table when we drop collections. But that memory goes away, and it never fills up space in the future, except maybe on WT pages of the size storer. Is there something I'm missing here? |
| Comment by Eric Milkie [ 17/Oct/23 ] |
|
I thought we only leaked memory, not the actual records in the table that mirrors the in-memory structure? In which case, there is no garbage to collect at startup? |
| Comment by Steven Vannelli [ 17/Oct/23 ] |
|
| Comment by Kyle Suarez [ 02/Mar/18 ] |
|
Is there a way to clean up the SizeStorer? |