[SERVER-12758] CleanupOldData Created: 18/Feb/14  Updated: 10/Dec/14  Resolved: 30/May/14

Status: Closed
Project: Core Server
Component/s: Performance, Sharding, Storage
Affects Version/s: 2.4.6
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Steffen Assignee: Thomas Rueckstiess
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

We use Mongodb 2.4.6
We added a new shard to our cluster. Each shard is a replica set with 3 nodes (2 powerful nodes and one backup hidden secondary)
We use tag aware sharding.
The new shard was added to a collection which was on 2 shards only before. So the data was balanced from 2 old nodes to the new one. 3 in total for this collection. Chunk size per shard is now equal (1172 each). On these shards there is only one collection (tag aware sharding).

But on one of the 2 old shards we see a lot cleanup messages in the log, thus we have high IO on this server

Tue Feb 18 11:15:33.860 [cleanupOldData-52f25c698cd98919080ca194] moveChunk deleted 124451 documents for database.collection from { targetUid: -5991322687277590967 } -> { targetUid: -5985241970436000175 }
Tue Feb 18 11:15:33.860 [cleanupOldData-52f25e478cd98919080ca199] moveChunk starting delete for: database.collection from { targetUid: -5985241970436000175 } -> { targetUid: -5981234632543141839 }

It seems the the cleaner goes through the whole hashed shard key range.
1. Is there a way to skip this cleanup with compaction, repair or full resync?

We also disabled "_secondaryThrottle" afterwards on the balancer setting, but it seems that this is not recognized on the replicaset

Tue Feb 18 06:07:23.792 [cleanupOldData-52f24e398cd98919080ca17b] Helpers::removeRangeUnlocked time spent waiting for replication: 1151188ms

2. Is there a way to disable it afterwards?



 Comments   
Comment by Thomas Rueckstiess [ 30/May/14 ]

Hi Steffen,

The log messages you posted look like normal behavior of chunk migrations. The cleanup cannot be skipped with the methods you mention, the data needs to be removed from the donor shard of the migration.

As for your question about _secondaryThrottle: The documentation mentions:

The effects of changing the _secondaryThrottle value may not be immediate. To ensure an immediate effect, stop the balancer and restart it with the selected value of _secondaryThrottle. See Manage Sharded Cluster Balancer for details.

The SERVER project is for reporting bugs or feature suggestions for the MongoDB server.

For MongoDB-related support discussion please post on the mongodb-users group (http://groups.google.com/group/mongodb-user) or Stack Overflow with the mongodb tag.

Regards,
Thomas

Generated at Thu Feb 08 03:29:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.