-
Type: Bug
-
Resolution: Cannot Reproduce
-
Priority: Major - P3
-
None
-
Affects Version/s: 1.6.5
-
Component/s: Stability
-
None
-
Environment:Linux CentOs 5
-
Linux
While testing sharding , i can see the data loss.
trace from shell
> db.contacts.count()
1700000
> for (var i = 1; i <= 500000; i++) db.contacts.save(
);
> db.contacts.count()
2199999
> db.contacts.count()
2199999
> db.contacts.count()
2203454
> db.contacts.count()
2199999
> db.contacts.count()
2199999
ERRORs from monos logs
Thu Dec 16 14:41:39 [conn1] autosplitting demo_contacts.contacts size: 1048648 shard: ns:demo_contacts.contacts at: shard0000:localhost:30001 lastmod: 101|31 min:
{ aid: 4.0, contact_id: 676362.0 }max:
{ aid: 5.0, contact_id: 1283.0 }on:
{ aid: 4.0, contact_id: 680372.0 }(splitThreshold 1048576)
Thu Dec 16 14:41:39 [conn1] ERROR: splitIfShould failed: locking namespace failed
Thu Dec 16 14:41:39 [conn1] autosplitting demo_contacts.contacts size: 1048648 shard: ns:demo_contacts.contacts at: shard0000:localhost:30001 lastmod: 101|31 min:
max:
{ aid: 5.0, contact_id: 1283.0 }on:
{ aid: 4.0, contact_id: 681072.0 }(splitThreshold 1048576)
Thu Dec 16 14:41:39 [conn1] ERROR: splitIfShould failed: locking namespace failed
Thu Dec 16 14:41:39 [conn1] autosplitting demo_contacts.contacts size: 1048648 shard: ns:demo_contacts.contacts at: shard0000:localhost:30001 lastmod: 101|31 min:
max:
{ aid: 5.0, contact_id: 1283.0 }on:
{ aid: 4.0, contact_id: 681771.0 }(splitThreshold 1048576)
Thu Dec 16 14:41:39 [conn1] ERROR: splitIfShould failed: locking namespace failed
Thu Dec 16 14:41:39 [conn1] autosplitting demo_contacts.contacts size: 1048648 shard: ns:demo_contacts.contacts at: shard0000:localhost:30001 lastmod: 101|31 min:
max:
{ aid: 5.0, contact_id: 1283.0 }on:
{ aid: 4.0, contact_id: 682471.0 }(splitThreshold 1048576)
Thu Dec 16 14:41:39 [conn1] ERROR: splitIfShould failed: locking namespace failed
Thu Dec 16 14:41:39 [conn1] autosplitting demo_contacts.contacts size: 1048648 shard: ns:demo_contacts.contacts at: shard0000:localhost:30001 lastmod: 101|31 min:
max:
{ aid: 5.0, contact_id: 1283.0 }on:
{ aid: 4.0, contact_id: 683170.0 }(splitThreshold 1048576)
Thu Dec 16 14:41:39 [conn1] ERROR: splitIfShould failed: locking namespace failed
Thu Dec 16 14:41:
Mongd log from one of the shard
Thu Dec 16 14:43:05 [conn12] query admin.$cmd ntoreturn:1 command: { moveChunk: "demo_contacts.contacts", from: "localhost:30001", to: "localhost:30002", min:
{ aid: 4.0, contact_id: 249176.0 }, max:
{ aid: 4.0, contact_id: 253273.0 }, shardId: "demo_contacts.contacts-aid_4.0contact_id_249176.0", configdb: "localhost:20001" } reslen:53 1084msThu Dec 16 14:43:07 [conn9] Assertion: 13388:[demo_contacts.contacts] shard version not ok in Client::Context: your version is too old ns: demo_contacts.contacts global: 116|1 client: 106|1
0x540c7e 0x713d07 0x5fbe40 0x79aa58 0x797596 0x798538 0x5fb7e5 0x60029f 0x7074ba 0x70aaf6 0x82691b 0x83a4b0 0x3d28e0673d 0x3d286d3f6d ./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x1de) [0x540c7e]
./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo6Client7Context11_finishInitEb+0x1b7) [0x713d07] ./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo8runCountEPKcRKNS_7BSONObjERSs+0xc0) [0x5fbe40]
./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo8CmdCount3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0xa8) [0x79aa58] ./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xa16) [0x797596]
./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x798) [0x798538]
./mongodb-linux-x86_64-1.6.5/bin/mongod(ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x35) [0x5fb7e5] ./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0x1bbf) [0x60029f]
./mongodb-linux-x86_64-1.6.5/bin/mongod [0x7074ba]
./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x14d6) [0x70aaf6] ./mongodb-linux-x86_64-1.6.5/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x30b) [0x82691b]
./mongodb-linux-x86_64-1.6.5/bin/mongod(thread_proxy+0x80) [0x83a4b0]
/lib64/libpthread.so.0 [0x3d28e0673d]
/lib64/libc.so.6(clone+0x6d) [0x3d286d3f6d]
Thu Dec 16 14:43:10 [conn12] got movechunk: { moveChunk: "demo_contacts.contacts", from: "localhost:30001", to: "localhost:30002", min:
, max:
{ aid: 4.0, contact_id: 256729.0 }, shardId: "demo_contacts.contacts-aid_4.0contact_id_253273.0", configdb: "localhost:20001" }
Thu Dec 16 14:43:11 [conn12] _recvChunkStatus : { active: true, ns: "demo_contacts.contacts", from: "localhost:30001", min:
, max:
{ aid: 4.0, contact_id: 256729.0 }, s