[SERVER-31216] Mark internal collections in metadata so tools don't dump them Created: 22/Sep/17 Updated: 06/Dec/22 Resolved: 12/Sep/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Internal Code |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | David Golden | Assignee: | Backlog - Storage Execution Team |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||
| Assigned Teams: |
Storage Execution
|
||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||||||||||
| Description |
|
As we add internal-use collections such as the sessions collection or a future transactions collection, tools like mongodump will wind up dumping these collections. If internal-use collection can be annotated in collection options with some flag that tools can detect, then tools can omit such collections from dump operations without needing users to remember to specific exclusions or for the tools to maintain a blacklist of collections to omit. |
| Comments |
| Comment by Ryan Chipman [ 09/Jun/22 ] | |||||||||||
| Comment by Kyle Suarez [ 09/Jun/22 ] | |||||||||||
|
Flagging this for scheduling per a Slack thread related to | |||||||||||
| Comment by Ian Whalen (Inactive) [ 15/Nov/18 ] | |||||||||||
|
craig.homa to organize the above meeting. | |||||||||||
| Comment by Asya Kamsky [ 11/Jul/18 ] | |||||||||||
|
matt.lord seth.payne this seems to span server, platform and tools - I think the three of us should sync up about how to best tackle this (and which team should own the project).
| |||||||||||
| Comment by David Golden [ 01/Mar/18 ] | |||||||||||
|
Another category is things that should trigger aborting oplog replication – such as an FCV change, which we handle by stopping on any modification of admin.system.version. | |||||||||||
| Comment by Asya Kamsky [ 01/Mar/18 ] | |||||||||||
|
It seems that various tools that get collection catalog would want to know which DBs/collections to exclude. mongodump, mongoexport are obvious ones, there is also mongomirror and possibly others. Are there special collections which do not start with system. prefix? Enumeration of special (updating based on related TOOLS ticket):
Are there others? david.golden can someone from tools review how special collections need to be logically grouped (are there other things than "don't dump this" that a tool may need to know)? | |||||||||||
| Comment by Daniel Gottlieb (Inactive) [ 13/Oct/17 ] | |||||||||||
|
Assigning to asya to investigate who are the stakeholders on which collections they consider special. | |||||||||||
| Comment by Ian Whalen (Inactive) [ 29/Sep/17 ] | |||||||||||