[SERVER-17474] Mongos doesn't see new sharded collection Created: 05/Mar/15  Updated: 06/Dec/22  Resolved: 07/Mar/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.6.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Migalin Danila Assignee: [DO NOT USE] Backlog - Sharding Team
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-14394 Create initial chunks directly on shards Closed
Assigned Teams:
Sharding
Operating System: ALL
Participants:

 Description   

I have sharded cluster and several mongos instances connected to it.
After significant growth of one of the non-sharded collection on initial shard, it was sharded with key {_id: 1}, splitted to dozens of chunks and completely moved by sh.moveChunk() from initial shard.

After that, half of mongos instances began to see whole collection as sharded, but some mongos still thinks, than collection is not sharded and is empty.

Examples:
'good' mongos:

root@mpfs1h:~# mongo mpfs
MongoDB shell version: 2.6.5
connecting to: mpfs
mpfs(obj: 14160349610; size: 12740/13085/18850 Gb)> db.deleted_preview_mids.count()
49215220
mpfs(obj: 14160350020; size: 12740/13085/18850 Gb)> db.deleted_preview_mids.stats()
{
        "sharded" : true,
        "systemFlags" : 1,
        "userFlags" : 1,
        "ns" : "mpfs.deleted_preview_mids",
        "count" : 49215252,
        "numExtents" : 150,
        "size" : 11811660480,
        "storageSize" : 14662729728,
        "totalIndexSize" : 3586108064,
        "indexSizes" : {
                "_id_" : 3586108064
        },
        "avgObjSize" : 240,
        "nindexes" : 1,
        "nchunks" : 66,
        "shards" : {
                "see full doc here" : "http://pastebin.com/uk8jDLxh"
        },
        "ok" : 1
}

'bad' mongos:

root@mpfs1g:~# mongo mpfs
MongoDB shell version: 2.6.5
connecting to: mpfs
mpfs(obj: 14160106632; size: 12740/13085/18850 Gb)> db.deleted_preview_mids.find()
mpfs(obj: 14160114836; size: 12740/13085/18850 Gb)> db.deleted_preview_mids.count()
0
mpfs(obj: 14160116026; size: 12740/13085/18850 Gb)> db.deleted_preview_mids.stats()
{
        "sharded" : false,
        "primary" : "disk1",
        "ns" : "mpfs.deleted_preview_mids",
        "count" : 0,
        "size" : 0,
        "storageSize" : 2140639232,
        "numExtents" : 20,
        "nindexes" : 1,
        "lastExtentSize" : 560488448,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 1,
        "totalIndexSize" : 8176,
        "indexSizes" : {
                "_id_" : 8176
        },
        "ok" : 1,
        "$gleStats" : {
                "lastOpTime" : Timestamp(1425537763, 69),
                "electionId" : ObjectId("54f60db899569dad97618eb1")
        }
}
mpfs(obj: 14160439248; size: 12740/13085/18850 Gb)> use config
switched to db config
config(obj: 252371; size: 0/0/0 Gb)> db.chunks.find({ns: 'mpfs.deleted_preview_mids'}).count()
66

After 'bad' mongos restart, everything becomes OK.



 Comments   
Comment by Kaloian Manassiev [ 07/Mar/19 ]

With the changes done in 3.2 to make all CRUD operations attach shard version instead of going through ShardConnection and with the changes in 4.0.6 to make shards drive the collection sharding process, this should no longer be an issue.

Comment by Migalin Danila [ 05/Mar/15 ]

flushRouterConfig helps:

mongos> db.deleted_preview_mids.count()
0
mongos> db.adminCommand({flushRouterConfig:1})
{ "flushed" : true, "ok" : 1 }
<pause>
mongos> db.deleted_preview_mids.count()
59243913

Comment by Randolph Tan [ 05/Mar/15 ]

Next time this happens again, try calling the { flushRouterConfig: 1 } before restarting.

Comment by Daniel Pasette (Inactive) [ 05/Mar/15 ]

Hi Migalin, "last commenter" in this case only means that the fixVersion was flipped to "debugging with submitter"
An engineer is looking at this issue now.

Comment by Migalin Danila [ 05/Mar/15 ]

Uhh, sorry, I see at ticket's header

Last commenter:	Ramon Fernandez

But doesn't see any comment, looks like I doesn't have an access to them

Generated at Thu Feb 08 03:44:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.