[SERVER-28363] When shard empty collection I have noticed that ChunkManager loads chunks for every other sharded collection in the database Created: 17/Mar/17 Updated: 31/Oct/17 Resolved: 22/Mar/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 3.2.4 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Clive Hill | Assignee: | Mark Agarunov |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Participants: | |||||||||
| Description |
|
I was investigating an issue whereby creating a new collection was taking around 100 seconds. When I looked at the mongos logs I noticed that there was message of following type for every sharded collection in the database: ChunkManager: time to load chunks for ddp_rel.timeseries.58c142b39340386c4d618779: 249ms sequenceNumber: 12117 version: 3|1||58c142b72810c82e0b145d5f based on: (empty) (I have attached log). After investigating I realised that my shard tag was incorrect because it only had the collection name and not the full namespace including the database name. After that, it worked better as each ChunkManager statement was taking 1ms. However, I am concerned that if we have > 10,000 collections (which is quite feasible as this is a time series database with collection per time series) it will still take a long time. Can someone explain to me why this is happening? It states "based on: (empty)" but I did testing by loading dummy data, and it still happens even if the sharded collection is not empty. Any explanation would be greatly appreciated, and if possible a way to prevent this happening. As FYI, we use sharded tags so that the data is pinned to location and shard. |
| Comments |
| Comment by Ramon Fernandez Marina [ 06/Apr/17 ] |
|
Hi EvilChill, please see my comment on Thanks, |
| Comment by Clive Hill [ 06/Apr/17 ] |
|
This: Are there any other steps other than upgrading that need to be done? |
| Comment by Clive Hill [ 06/Apr/17 ] |
|
Hi Kal, I upgraded to MongoDb 3.4.3, and updated the Java driver to 3.4.2 today and tested with sharded collection, and I am still seeing the behavior. I've just re-read your comment "*plan *backports to 3.4". Can you advise as to when this will be done? Best regards, Clive |
| Comment by Kaloian Manassiev [ 22/Mar/17 ] |
|
Hi EvilChill, The behaviour you are experiencing is one of the side effects of the metadata cache refresh bug described in We have fixed it in master and plan backports to 3.4. Please watch Best regards, |