Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-3652

Sometimes got "invalid chunk config minObj" or "Assertion: 8070:couldn't find a chunk aftry retry which should be impossible extracted" when there're multiple mongos

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Icon: Major - P3 Major - P3
    • None
    • 1.8.2
    • Querying, Sharding
    • Centos 5 64bit
    • Linux

    Description

      I am using a sharding environment(replicaset) with multiple mongos. Sometimes when I perform query or findAndModify a collection with shardkey, there's only ONE mongos worked well, while the others will complains as below. The only way is to restart all the other mongos to fix it. However it may happen again.

      Query:
      Aug 19 07:09:56 dal-tsa04 Fri: Aug 19 07:09:55 [conn60192] Assertion: 13507:invalid chunk config minObj:

      { id: "newbie" }

      Aug 19 07:09:56 dal-tsa04 0x5204fa: 0x618f45 0x63a673 0x66841c 0x67d187 0x580b7c 0x6a50a0 0x35baa0673d 0x35b9ed3f6d
      Aug 19 07:09:56 dal-tsa04 /usr/bin/mongos(_ZN5mongo11msgassertedEiPKc+0x12a): [0x5204fa]
      Aug 19 07:09:56 dal-tsa04 /usr/bin/mongos(_ZN5mongo12ChunkManager17getShardsForQueryERSt3setINS_5ShardESt4lessIS2_ESaIS2_EERKNS_7BSONObjE+0x8c5): [0x618f45]
      Aug 19 07:09:56 dal-tsa04 /usr/bin/mongos(_ZN5mongo13ShardStrategy7queryOpERNS_7RequestE+0x2f3): [0x63a673]
      Aug 19 07:09:56 dal-tsa04 /usr/bin/mongos(_ZN5mongo7Request7processEi+0x29c): [0x66841c]
      Aug 19 07:09:56 dal-tsa04 /usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x77): [0x67d187]
      Aug 19 07:09:56 dal-tsa04 /usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x34c): [0x580b7c]
      Aug 19 07:09:56 dal-tsa04 /usr/bin/mongos(thread_proxy+0x80): [0x6a50a0]
      Aug 19 07:09:56 dal-tsa04 /lib64/libpthread.so.0: [0x35baa0673d]
      Aug 19 07:09:56 dal-tsa04 /lib64/libc.so.6(clone+0x6d): [0x35b9ed3f6d]
      Aug 19 07:09:56 dal-tsa04 Fri: Aug 19 07:09:55 [conn60192] AssertionException in process: invalid chunk config minObj:

      { id: "newbie" }

      FindAndModify:

      Aug 19 07:10:00 dal-tsa04 Fri: Aug 19 07:10:00 [conn60177] Assertion: 8070:couldn't find a chunk aftry retry which should be impossible extracted:

      { id: "david" }

      Aug 19 07:10:00 dal-tsa04 0x5204fa: 0x60adfd 0x65663c 0x57df0c 0x635f52 0x66841c 0x67d187 0x580b7c 0x6a50a0 0x35baa0673d 0x35b9ed3f6d
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(_ZN5mongo11msgassertedEiPKc+0x12a): [0x5204fa]
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(_ZN5mongo12ChunkManager9findChunkERKNS_7BSONObjE+0x49d): [0x60adfd]
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(_ZN5mongo15dbgrid_pub_cmds16FindAndModifyCmd3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x37c): [0x65663c]
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(_ZN5mongo7Command20runAgainstRegisteredEPKcRNS_7BSONObjERNS_14BSONObjBuilderE+0x67c): [0x57df0c]
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(_ZN5mongo14SingleStrategy7queryOpERNS_7RequestE+0x262): [0x635f52]
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(_ZN5mongo7Request7processEi+0x29c): [0x66841c]
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(_ZN5mongo21ShardedMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x77): [0x67d187]
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x34c): [0x580b7c]
      Aug 19 07:10:00 dal-tsa04 /usr/bin/mongos(thread_proxy+0x80): [0x6a50a0]
      Aug 19 07:10:00 dal-tsa04 /lib64/libpthread.so.0: [0x35baa0673d]
      Aug 19 07:10:00 dal-tsa04 /lib64/libc.so.6(clone+0x6d): [0x35b9ed3f6d]

      Attachments

        Activity

          People

            scotthernandez Scott Hernandez (Inactive)
            edwardwei Edward Wei
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: