Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-1989

Splitting big chunk caused assertion failed

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 1.6.3
    • Fix Version/s: 1.7.2
    • Component/s: Sharding
    • Labels:
      None
    • Environment:
      debian lenny with bpo kernel (2.6.32), ext4 filesystem, 32GB RAM
    • Operating System:
      Linux

      Description

      while trying to split a chunk manually, the server responded with an assertion error and data retrieval was not possible.

      > db.runCommand({split:"storage.object_interaction", middle:{_id:877070867}})

      {
      "assertion" : "saving chunks failed. cmd: { applyOps: [ { op:
      \"u\", b: true, ns: \"config.chunks\
      ", o: { _id: \"storage.object_interaction-_id_832352546\", lastmod:
      Timestamp 179000|4, ns: \"storage.objec
      t_interaction\", min:

      { _id: 832352546 }

      , max:

      { _id: 832352546.0 }

      ,
      shard: \"shard0001\" }, o2:

      { _id: \"s torage.object_interaction-_id_832352546\" }

      }, { op: \"u\", b: true,
      ns: \"config.chunks\", o: { _id: \"sto
      rage.object_interaction-_id_832352546.0\", lastmod: Timestamp 179000|
      5, ns: \"storage.object_interaction\",
      min:

      { _id: 832352546.0 }

      , max:

      { _id: 832352547.0 }

      , shard:
      \"shard0001\" }, o2:

      { _id: \"storage.object_ interaction-_id_832352546.0\" }

      }, { op: \"u\", b: true, ns:
      \"config.chunks\", o: { id: \"storage.object
      interaction-_id_877070867\", lastmod: Timestamp 179000|6, ns:
      \"storage.object_interaction\", min:

      { _id: 8 77070867 }

      , max:

      { _id: 877070867.0 }

      , shard: \"shard0001\" }, o2:

      { _id: \"storage.object_interaction-_id_ 877070867\" }

      }, { op: \"u\", b: true, ns: \"config.chunks\", o:
      { _id: \"storage.object_interaction-_id_877070867.0\", lastmod:
      Timestamp 179000|7, ns: \"storage.object_interaction\", min:

      { _id: 877070867.0 }

      , max:

      { _id: 877217527 }

      , shard: \"shard0001\" }, o2:

      { _id: \"storage.object_interaction-_id_877070867.0\" }

      } ],
      preCondition: [ { ns: \"config.chunks\", q: { query:

      { ns: \"storage.object_interaction\" }

      , orderby:

      { lastmod: -1 }

      }, res:

      { lastmod: Timestamp 179000|3 }

      } ] } result: { got: { _id:
      \"storage.object_interaction-_id_832352546\", lastmod: Timestamp
      179000|4, ns: \"storage.object_interaction\", min:

      { _id: 832352546 }

      ,
      max:

      { _id: 832352546.0 }

      , shard: \"shard0001\" }, whatFailed: { ns:
      \"config.chunks\", q: { query:

      { ns: \"storage.object_interaction\" }

      ,
      orderby:

      { lastmod: -1 }

      }, res:

      { lastmod: Timestamp 179000|3 }

      },
      errmsg: \"pre-condition failed\", ok: 0.0 }",
      "assertionCode" : 13327,
      "errmsg" : "db assertion failure",
      "ok" : 0
      }

      > db.runCommand({split:"storage.object_interaction", find:{_id:854711706}})
      {
      "assertion" : "Couldn't load a valid config for
      storage.object_interaction after 3 tries. Giving up",
      "assertionCode" : 13282,
      "errmsg" : "db assertion failure",
      "ok" : 0
      }

      > db.object_interaction.findOne({_id:854711706})
      Fri Oct 22 00:04:24 uncaught exception: error

      { "$err" : "assertion s/chunk.cpp:768", "code" : 0 }

      I'm attaching the cunklist (db.printShardingStatus as well as a mongos log file that contain the events)

      What can i do in order to split the chunk? right now the index is not even on 2 of the shard servers.

        Attachments

        1. chunklist
          213 kB
        2. mongos.log
          32 kB

          Activity

            People

            Assignee:
            alerner Alberto Lerner
            Reporter:
            infin1ty Erez Zarum
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: