[SERVER-28402] Dropping collection not releasing diskspace on secondary Created: 20/Mar/17  Updated: 31/May/17  Resolved: 21/Apr/17

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.2.12
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Frederick Cheung Assignee: Kelsey Schubert
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

amazon linux 2016.03


Attachments: Text File mongod.log    
Issue Links:
Duplicate
duplicates SERVER-26870 Sometimes collection data file is not... Closed
Operating System: ALL
Participants:

 Description   

We're encountering some odd behaviour with an unsharded replica set. The primary is running 3.2.10, the secondary was originally also running 3.2.10, but is now running 3.2.12 (I upgraded to see if the issue went away).

Our usage pattern is that every day we create a smallish number of collections (~13), each containing a few million documents. When we've finished inserting the data for a collection, we drop the correspondind collection from 2 days ago (in theory we could drop all collections except for the ones just created, but we like to have the ability to rollback to the old data). Apart from one small collection with some bookkeeping data, all the collections in this database have this lifecycle of 2-3 days

Occasionally some of these collection drops don't free up disk space on the secondary. Logically the data is the same on the primary and the secondary (ie show collections shows the same output, db.stats() shows very similar numbers), however there the free space on the secondary does not increase and I can observe more files in the mongodb data directory compared to the primary.

Stopping mongodb consistently fixes the issue - the extra files are removed and disk space returns to the same level as the primary. This happens sporadically (sometimes not at all for several weeks, although it seems to be happening a lot at the moment), I have not yet identified what triggers this behaviour. It has never occurred on the primary.

Assuming this isn't a known issue, what data can I provide to assist tracking down this issue. There is nothing that relevant looking in the mongodb log at the point of the mongod stop. I've attached the last few minutes immediately before I stopped mongod - you can see it dropping two collections



 Comments   
Comment by Kelsey Schubert [ 21/Apr/17 ]

Hi fcheung,

Since we haven't heard back from you for some time, I'm going to close this ticket as a duplicate of SERVER-26870. If the issue is not resolved by upgrading to MongoDB 3.4.3 or later, please let us know and we will reopen the ticket.

Thank you,
Thomas

Comment by Eric Milkie [ 26/Mar/17 ]

To aid implementation, the fix for SERVER-26870 utilized some of the code already written for the NoopWriter functionality.

Comment by Frederick Cheung [ 25/Mar/17 ]

OK, I'll see what I can do. Still fuzzy on how a fix to something that doesn't exist in 3.2 (NoopWriter) could fix a problem in 3.2 though.

Comment by Kelsey Schubert [ 25/Mar/17 ]

Hi fcheung,

Unfortunately, this fix was built on the NoopWriter (SERVER-23892), which does not exist in v3.2. Therefore, we cannot backport it to MongoDB 3.2. Please note that a MongoDB 3.4.3 release candidate is currently available for download, and we currently anticipate the production release of MongoDB 3.4.3 next week.

I understand your concerns about a major version upgrade. However, given your responses, I expect SERVER-26870 to correct this behavior, and I do not see a way to progress this investigation without confirming this fix resolves your issue.

Thank you,
Thomas

Comment by Frederick Cheung [ 25/Mar/17 ]

Thanks, that sounds promising & explains the random nature of it. I'm slightly reticent to do a major version upgrade just to test this out - are there any plans to backport what appears to be a very small change?

Comment by Kelsey Schubert [ 24/Mar/17 ]

Hi fcheung,

Thanks for reporting this behavior. This appears to be a duplicate of SERVER-26870. Would you please upgrade to latest version of MongoDB 3.4 and confirm that the issue is resolved?

Kind regards,
Thomas

Generated at Thu Feb 08 04:18:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.