Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17018

Assertion failure false src/mongo/db/structure/btree/key.cpp 433 on remove operation

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.6.10
    • Affects Version/s: 2.6.9
    • Component/s: Index Maintenance
    • Labels:
      None
    • Environment:
      OS: Windows Server 2008 R2 Standard SP1 64bit
      CPU: Intel(R) Core(TM)2 Quad CPU Q9550 @2.83GHz
      RAM: 6.00GB
      MongoDB Version: 2.6.5 and 2.6.7
    • Fully Compatible
    • Windows
    • Hide
      1. execute "test.bat".
      2. wait until "test.bat" finished.
        (about 1hr on my PC environment(refer to "Environment")))
      3. find error log by executing following command.
         find /N "key.cpp" mongo.log
        
      Show
      execute "test.bat". wait until "test.bat" finished. (about 1hr on my PC environment(refer to "Environment"))) find error log by executing following command. find /N "key.cpp" mongo.log

      I got this assertion error when execute remove operation.
      You can get the error by running the attached tool.
      Could you investigate this error using the tool?

      After doing procedure(refer to "Steps To Reproduce"), you can find the error log like below.

      testTenant2.testBucket0 Assertion failure false src\mongo\db\structure\btree\key.cpp 433
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\util\stacktrace.cpp(169)                             mongo::printStackTrace+0x43
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\util\log.cpp(127)                                    mongo::logContext+0x9c
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\util\assert_util.cpp(115)                            mongo::verifyFailed+0x14a
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\structure\btree\key.cpp(433)                      mongo::KeyV1::toBson+0x34e
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\index\btree_interface.cpp(168)                    mongo::BtreeInterfaceImpl<mongo::BtreeData_V1>::keyAt+0x16b
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\index\btree_index_cursor.cpp(175)                 mongo::BtreeIndexCursor::getKey+0x5d
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\exec\index_scan.cpp(314)                          mongo::IndexScan::checkEnd+0x1a1
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\exec\index_scan.cpp(150)                          mongo::IndexScan::work+0x6f
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\exec\fetch.cpp(82)                                mongo::FetchStage::work+0x99
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\exec\keep_mutations.cpp(60)                       mongo::KeepMutationsStage::work+0x55
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\query\plan_executor.cpp(91)                       mongo::PlanExecutor::getNext+0x16b
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\ops\delete_executor.cpp(154)                      mongo::DeleteExecutor::execute+0x866
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\batch_executor.cpp(1201)  mongo::multiRemove+0x589
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\batch_executor.cpp(887)   mongo::WriteBatchExecutor::execRemove+0xad
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\batch_executor.cpp(743)   mongo::WriteBatchExecutor::bulkExecute+0x29b
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\batch_executor.cpp(249)   mongo::WriteBatchExecutor::executeBatch+0xb6c
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\commands\write_commands\write_commands.cpp(146)   mongo::WriteCmd::run+0x1e6
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1385)                              mongo::_execCommand+0x5e
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1592)                              mongo::Command::execCommand+0xf09
      2015-01-22T15:38:00.045+0900 [conn65] mongod.exe  ...\src\mongo\db\dbcommands.cpp(1724)                              mongo::_runCommands+0x4a7
      2015-01-22T15:38:00.045+0900 [conn65] testTenant2.testBucket0 
      2015-01-22T15:38:00.045+0900 [conn65] remove testTenant2.testBucket0 query: { _id: ObjectId('54c09ac7a68d879e3aaf086a'), var1: "str1" } ndeleted:0 keyUpdates:0 exception: assertion src\mongo\db\structure\btree\key.cpp:433 code:8 numYields:0 locks(micros) w:694341 694ms
      

      Frequency of occurrence is:

      • 4 in 4 times (tool execute times)

      I attached the tool written above:

      • test.bat
        ... Execute "OperationCycle.js" many times at the same time.
      • OperationCycle.js
        ... Repeat executing Mongo operations(insert, findOne, update, remove).
      • Setting.js
        ... Set parameters.
        You don't need to modify these files.

      This tool means:
      Several users execute Mongo Operations(insert, find, update, remove) to several colletions in several databases at the same time.

      I think the important point to occur this error is to execute index scan stage in remove operation by setting "two" items in remove operation's query.
      For example:

      db.coll.remove({ "_id" : testId , "key": "val"}
      

      (setting two items : "_id" and "key")

        1. OperationCycle.js
          2 kB
        2. setting.js
          0.4 kB
        3. test.bat
          0.2 kB
        4. reproduction.zip
          21.84 MB
        5. linux_repro.zip
          1 kB
        6. linux_repro.zip
          1 kB

            Assignee:
            geert.bosch@mongodb.com Geert Bosch
            Reporter:
            y-okamoto Yuki Okamoto
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: