Assertion failure false src/mongo/db/structure/btree/key.cpp 433 on remove operation

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Done
    • Priority: Major - P3
    • 2.6.10
    • Affects Version/s: 2.6.9
    • Component/s: Index Maintenance
    • None
    • Environment:
      OS: Windows Server 2008 R2 Standard SP1 64bit
      CPU: Intel(R) Core(TM)2 Quad CPU Q9550 @2.83GHz
      RAM: 6.00GB
      MongoDB Version: 2.6.5 and 2.6.7
    • Fully Compatible
    • Windows
    • Hide
      1. execute "test.bat".
      2. wait until "test.bat" finished.
        (about 1hr on my PC environment(refer to "Environment")))
      3. find error log by executing following command. 
         find /N "key.cpp" mongo.log
      Show
      execute "test.bat". wait until "test.bat" finished. (about 1hr on my PC environment(refer to "Environment"))) find error log by executing following command. find /N "key.cpp" mongo.log
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      I got this assertion error when execute remove operation.
      You can get the error by running the attached tool.
      Could you investigate this error using the tool?

      After doing procedure(refer to "Steps To Reproduce"), you can find the error log like below.

      testTenant2.testBucket0 Assertion failure false src\mongo\db\structure\btree\key.cpp 433
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\util\stacktrace.cpp(169)                             mongo::printStackTrace+0x43
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\util\log.cpp(127)                                    mongo::logContext+0x9c
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\util\assert_util.cpp(115)                            mongo::verifyFailed+0x14a
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\structure\btree\key.cpp(433)                      mongo::KeyV1::toBson+0x34e
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\index\btree_interface.cpp(168)                    mongo::BtreeInterfaceImpl<mongo::BtreeData_V1>::keyAt+0x16b
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\index\btree_index_cursor.cpp(175)                 mongo::BtreeIndexCursor::getKey+0x5d
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\exec\index_scan.cpp(314)                          mongo::IndexScan::checkEnd+0x1a1
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\exec\index_scan.cpp(150)                          mongo::IndexScan::work+0x6f
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\exec\fetch.cpp(82)                                mongo::FetchStage::work+0x99
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\exec\keep_mutations.cpp(60)                       mongo::KeepMutationsStage::work+0x55
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\query\plan_executor.cpp(91)                       mongo::PlanExecutor::getNext+0x16b
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\ops\delete_executor.cpp(154)                      mongo::DeleteExecutor::execute+0x866
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\batch_executor.cpp(1201)  mongo::multiRemove+0x589
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\batch_executor.cpp(887)   mongo::WriteBatchExecutor::execRemove+0xad
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\batch_executor.cpp(743)   mongo::WriteBatchExecutor::bulkExecute+0x29b
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\batch_executor.cpp(249)   mongo::WriteBatchExecutor::executeBatch+0xb6c
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\write_commands.cpp(146)   mongo::WriteCmd::run+0x1e6
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1385)                              mongo::_execCommand+0x5e
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1592)                              mongo::Command::execCommand+0xf09
2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1724)                              mongo::_runCommands+0x4a7
2015-01-22T15:38:00.045+0900 [conn65] testTenant2.testBucket0 
2015-01-22T15:38:00.045+0900 [conn65] remove testTenant2.testBucket0 query: { _id: ObjectId('54c09ac7a68d879e3aaf086a'), var1: "str1" } ndeleted:0 keyUpdates:0 exception: assertion src\mongo\db\structure\btree\key.cpp:433 code:8 numYields:0 locks(micros) w:694341 694ms

      Frequency of occurrence is:

      • 4 in 4 times (tool execute times)

      I attached the tool written above:

      • test.bat
        ... Execute "OperationCycle.js" many times at the same time.
      • OperationCycle.js
        ... Repeat executing Mongo operations(insert, findOne, update, remove).
      • Setting.js
        ... Set parameters.
        You don't need to modify these files.

      This tool means:
      Several users execute Mongo Operations(insert, find, update, remove) to several colletions in several databases at the same time.

      I think the important point to occur this error is to execute index scan stage in remove operation by setting "two" items in remove operation's query.
      For example: 

      db.coll.remove({ "_id" : testId , "key": "val"}

      (setting two items : "_id" and "key")

        1. linux_repro.zip
          1 kB
        2. linux_repro.zip
          1 kB
        3. OperationCycle.js
          2 kB
        4. reproduction.zip
          21.84 MB
        5. setting.js
          0.4 kB
        6. test.bat
          0.2 kB

            Assignee:
            Geert Bosch
            Reporter:
            Yuki Okamoto
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: