[SERVER-26091] gridfs deduplication Created: 13/Sep/16  Updated: 06/Dec/22  Resolved: 19/Sep/16

Status: Closed
Project: Core Server
Component/s: GridFS
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor - P4
Reporter: Edik Mkoyan Assignee: Backlog - Storage Execution Team
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Storage Execution
Participants:

 Description   

With the original storage engine it was easy to organize incremental or differential backups, as the old files weren't tend to change. We also could choose which collection to backup, as one file meant one collection.
With the wire tiger we need much more backup space, even if one byte is save somewhere, many files are affected and so hashes are changed...

Today we have discovered, that we have 560 copies of the same heavy file, and probably having deduplication functionality on gridfs is possible, or even easier then thinking of filesystem deduplication, that could also reduce the RAM amount mongo uses to cache the data from storage, so we want to ask to add that functionality. Thanks a lot,
at TUMO Center for Creative technologies we love you very much.



 Comments   
Comment by Ian Whalen (Inactive) [ 19/Sep/16 ]

Hi Edik, we've discussed this potential feature on the Integration Team and decided that we do not plan on pursuing it for MongoDB.

There are various options outside of MongoDB to achieve deduplication or efficiently backup MongoDB databases, including with MongoDB Cloud Manager.

Comment by Kelsey Schubert [ 13/Sep/16 ]

Hi edikmkoyan,

Thank you for the feature request. I'm marking this ticket to be considered by our Integration Team.

I see that a similar request was posted many years ago on our user group. I would recommend considering whether Mathias's suggestion works for your use case.

Kind regards,
Thomas

Generated at Thu Feb 08 04:11:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.