[SERVER-31216] Mark internal collections in metadata so tools don't dump them Created: 22/Sep/17  Updated: 06/Dec/22  Resolved: 12/Sep/22

Status: Closed
Project: Core Server
Component/s: Internal Code
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: David Golden Assignee: Backlog - Storage Execution Team
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-5972 Cannot drop system.js collection Closed
is depended on by SERVER-5955 Forbid renaming system collections Closed
Related
related to TOOLS-2222 Mongodump should ignore config.transa... Closed
related to SERVER-67147 Allow system.profile to be dropped wh... Closed
is related to SERVER-27147 Do not initial sync data written with... Closed
is related to TOOLS-1797 mongodump must skip collections used ... Closed
is related to SERVER-23546 Collection::requiresIdIndex may be in... Closed
is related to TOOLS-1636 mongodump fails when capped collectio... Waiting (Blocked)
Assigned Teams:
Storage Execution
Participants:
Case:

 Description   

As we add internal-use collections such as the sessions collection or a future transactions collection, tools like mongodump will wind up dumping these collections.

If internal-use collection can be annotated in collection options with some flag that tools can detect, then tools can omit such collections from dump operations without needing users to remember to specific exclusions or for the tools to maintain a blacklist of collections to omit.



 Comments   
Comment by Ryan Chipman [ 09/Jun/22 ]

cc tim.fogarty@mongodb.com

Comment by Kyle Suarez [ 09/Jun/22 ]

Flagging this for scheduling per a Slack thread related to SERVER-67147. Since this seems potentially related to a flag in the catalog, sending this to Storage Execution to please take a look. (Also tagging ryan.chipman@mongodb.com for visibility on the Tools side.)

Comment by Ian Whalen (Inactive) [ 15/Nov/18 ]

craig.homa to organize the above meeting.

Comment by Asya Kamsky [ 11/Jul/18 ]

matt.lord  seth.payne this seems to span server, platform and tools - I think the three of us should sync up about how to best tackle this (and which team should own the project).

 

Comment by David Golden [ 01/Mar/18 ]

Another category is things that should trigger aborting oplog replication – such as an FCV change, which we handle by stopping on any modification of admin.system.version.

Comment by Asya Kamsky [ 01/Mar/18 ]

It seems that various tools that get collection catalog would want to know which DBs/collections to exclude. mongodump, mongoexport are obvious ones, there is also mongomirror and possibly others.

Are there special collections which do not start with system. prefix?

Enumeration of special (updating based on related TOOLS ticket):

local.*
config.transactions
config.system.keys
admin.system.keys
*.system.namespaces
*.system.indexes (but this is allowed in the oplog)
*.system.profile
*.system.views
*.system.js
admin.system.version
config.system.sessions

Are there others?

david.golden can someone from tools review how special collections need to be logically grouped (are there other things than "don't dump this" that a tool may need to know)?

Comment by Daniel Gottlieb (Inactive) [ 13/Oct/17 ]

Assigning to asya to investigate who are the stakeholders on which collections they consider special.

Comment by Ian Whalen (Inactive) [ 29/Sep/17 ]

acm milkie should this end up on Platforms or Storage?

Generated at Thu Feb 08 04:26:20 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.