[SERVER-4877] Iinserting on mogod get error "bad offset:0 accessing file: /sata1/shard9/00030212.0 - consider repairing database code : 16 13440 n : 16 0" Created: 06/Feb/12 Updated: 11/Mar/12 Resolved: 11/Mar/12 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 2.1.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | jitendra | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 1 |
| Labels: | sharding | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Run 10 mongod shards, 1 config server,1 mongos with journal option enabled. Inserting on mongodb by mongodb c driver.System restart and try to insert data on mongodb by mongodb c driver it give error in getlasterror mongod log are below Fri Feb 3 23:51:03 [conn1] insert 00030212.Database 103ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms please give me solution for this problem. . |
||
| Operating System: | Linux |
| Participants: |
| Comments |
| Comment by Eliot Horowitz (Inactive) [ 06/Feb/12 ] |
|
That wasn't my question. |
| Comment by jitendra [ 06/Feb/12 ] |
|
wipe data is solution for this.this is risky. |
| Comment by Eliot Horowitz (Inactive) [ 06/Feb/12 ] |
|
Did you wipe the data but leave the journal? |
| Comment by jitendra [ 06/Feb/12 ] |
|
ya, there was an old journal with an unclean shutdown, after restart first insert give the error. |
| Comment by Eliot Horowitz (Inactive) [ 06/Feb/12 ] |
|
Looks like there was an old journal with an unclean shutdown. |
| Comment by jitendra [ 06/Feb/12 ] |
|
Fri Feb 3 23:40:04 [conn2] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:40:04 [conn2] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:40:04 [conn2] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:40:04 [conn2] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:40:04 [conn2] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:40:04 [conn2] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:40:04 [conn2] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:40:05 [conn2] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:40:05 [conn2] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:40:05 [conn2] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:40:05 [conn2] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:40:06 [conn2] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:40:06 [conn2] command admin.$cmd command: { getlasterror: 1 }ntoreturn:1 reslen:67 0ms
Fri Feb 3 23:45:28 BackgroundJob starting: DataFileSync Fri Feb 3 23:45:29 [initandlisten] flushing directory /sata1/shard9 Fri Feb 3 23:49:37 [initandlisten] Accessing: local for the first time Fri Feb 3 23:50:59 [conn1] run command admin.$cmd { setShardVersion: "", init: true, configdb: "192.168.50.113:30000", serverID: ObjectId('4f2c2581300f707ca5d3c4db'), authoritative: true }Fri Feb 3 23:50:59 [conn1] command: { setShardVersion: "", init: true, configdb: "192.168.50.113:30000", serverID: ObjectId('4f2c2581300f707ca5d3c4db'), authoritative: true }Fri Feb 3 23:50:59 [conn1] entering shard mode for connection ntoreturn:1 reslen:51 0ms Fri Feb 3 23:50:59 [conn2] runQuery called admin.$cmd { writebacklisten: ObjectId('4f2c2581300f707ca5d3c4db') }Fri Feb 3 23:50:59 [conn2] run command admin.$cmd { writebacklisten: ObjectId('4f2c2581300f707ca5d3c4db') }Fri Feb 3 23:50:59 [conn1] run command admin.$cmd { setShardVersion: "00030212.Database", configdb: "192.168.50.113:30000", version: Timestamp 18000|0, serverID: ObjectId('4f2c2581300f707ca5d3c4db'), shard: "shard0008", shardHost: "192.168.50.113:60000" }Fri Feb 3 23:50:59 [conn2] command: { writebacklisten: ObjectId('4f2c2581300f707ca5d3c4db') }Fri Feb 3 23:50:59 [conn1] command: { setShardVersion: "00030212.Database", configdb: "192.168.50.113:30000", version: Timestamp 18000|0, serverID: ObjectId('4f2c2581300f707ca5d3c4db'), shard: "shard0008", shardHost: "192.168.50.113:60000" }Fri Feb 3 23:50:59 [conn1] command admin.$cmd command: { setShardVersion: "00030212.Database", configdb: "192.168.50.113:30000", version: Timestamp 18000|0, serverID: ObjectId('4f2c2581300f707ca5d3c4db'), shard: "shard0008", shardHost: "192.168.50.113:60000" } ntoreturn:1 reslen:162 0ms Fri Feb 3 23:50:59 [conn1] run command admin.$cmd { setShardVersion: "00030212.Database", configdb: "192.168.50.113:30000", version: Timestamp 18000|0, serverID: ObjectId('4f2c2581300f707ca5d3c4db'), authoritative: true, shard: "shard0008", shardHost: "192.168.50.113:60000" }Fri Feb 3 23:50:59 [conn1] command: { setShardVersion: "00030212.Database", configdb: "192.168.50.113:30000", version: Timestamp 18000|0, serverID: ObjectId('4f2c2581300f707ca5d3c4db'), authoritative: true, shard: "shard0008", shardHost: "192.168.50.113:60000" }Fri Feb 3 23:50:59 [conn1] creating new connection to:192.168.50.113:30000 ntoreturn:1 reslen:57 2ms Fri Feb 3 23:51:00 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:00 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:00 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:00 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:00 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:01 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:01 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:01 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:01 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:01 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:01 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:02 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:02 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:02 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:02 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:02 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:02 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:02 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:02 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:67 0ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms Fri Feb 3 23:51:03 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:03 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms Fri Feb 3 23:51:04 [conn1] run command admin.$cmd { getlasterror: 1 }Fri Feb 3 23:51:04 [conn1] command admin.$cmd command: { getlasterror: 1 } ntoreturn:1 reslen:165 0ms |
| Comment by Scott Hernandez (Inactive) [ 06/Feb/12 ] |
|
Can you attach the logs from the server for the time after it restarted (when your manually pressed the reset button on the computer)? |
| Comment by jitendra [ 06/Feb/12 ] |
|
This was with new data.I have not disabled journaling on the shard. |
| Comment by Scott Hernandez (Inactive) [ 06/Feb/12 ] |
|
Was this only with new data, or did you have old data from before using 2.1.0? Have you disabled journaling on the shards? How did the system restart? Was it a clean shutdown or some system error? |