Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-933

move migrate handling to mognod WAS: temp oplog issues WAS: Assertion error when moving chunks

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 1.5.4
    • Affects Version/s: None
    • Component/s: Sharding
    • None
    • Environment:
      mongodb 1.1.1, java driver 0.11, ubuntu 9.04

      I've got an 8-machine mongodb cluster set up, with 2 mongodb servers running on each machine (so 16 mongod processes total, plus a config server running on a 9th machine). I've enabled sharding on a collection, using the "time" field as a key (I initially tried sharding on the _id field, but when I did that the chunks would never split i.e. no matter how much data I inserted, there was only ever one entry in config.chunks).

      When I start inserting data into this collection, I can see in config.chunks that additional chunks are being created on different machines throughout the cluster, as expected. But after a while, it stops creating chunks on new shards and just keeps creating chunks on the same shard over and over. From looking at the logs it seems like its trying to move chunks and failing, but I'm inserting things in order (i.e. the "time" field is always the current time), so I'm not sure why it's even trying to move chunks (shouldn't it just create a new chunk and start inserting things there?).

      Here's what I'm seeing in the mongos log:

      Fri Oct 9 18:13:28 autosplitting test.foo size: 62662123 shard: shard ns:test.foo shard: mongod1:10001 min:

      { time: MinKey }

      max:

      { time: MaxKey }

      ...
      Fri Oct 9 18:13:28 moving chunk (auto): shard ns:test.foo shard: mongod1:10001 min:

      { time: MinKey }

      max:

      { time: new Date(1255112014526) }

      to: mongod2.com:10000 #objcets: 0
      Fri Oct 9 18:13:28 moving chunk ns: test.foo moving chunk: shard ns:test.foo shard: mongod1:10001 min:

      { time: MinKey }

      max:

      { time: new Date(1255112014526) }

      mongod1:10001 -> mongod2:10000

      And after a while:

      Fri Oct 9 18:28:32 Assertion: moveAndCommit failed: movechunk.start failed:

      { errmsg: "", ok: 0.0 }

      Fri Oct 9 18:28:32 UserException: moveAndCommit failed: movechunk.start failed:

      { errmsg: "", ok: 0.0 }

            Assignee:
            eliot Eliot Horowitz (Inactive)
            Reporter:
            mfoemmel Matthew Foemmel
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: