Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-3067

Can't kill indexing operations

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: 1.8.0
    • Fix Version/s: 2.3.1
    • Component/s: Index Maintenance
    • Labels:
      None
    • Environment:
      Ubuntu 10.04.2
      Linux 2.6.32-30-generic #59-Ubuntu SMP Tue Mar 1 21:30:46 UTC 2011 x86_64 GNU/Linux
    • Operating System:
      ALL

      Description

      Overview:

      This ticket adds support for using killop to interrupt a client initiated foreground index build in progress. When the index build is killed, all resources related to the index are cleaned up and the index is removed from system.indexes. An error response is provided on the connection that initiated the index build. A foreground index build will similarly be killed if mongod is shut down while the build is in progress.

      Index builds that are not directly initiated by an external client cannot be interrupted in this manner. For example, map reduce and reindex (as well as other commands) build indexes as part of their internal implementations and cannot be interrupted currently.

      Aaron

      ----------------------

      I accidentally started a foreground index build on 300 million records. I followed the documentation and tried to kill it, but was unsuccessful. This resulted in significant downtime.

      http://www.mongodb.org/display/DOCS/Viewing+and+Terminating+Current+Operation

      {
      "opid" : 710799555,
      "active" : true,
      "lockType" : "write",
      "waitingForLock" : false,
      "secs_running" : 5260,
      "op" : "insert",
      "ns" : "production.system.indexes",
      "query" : {

      },
      "client" : "10.122.166.155:57729",
      "desc" : "conn",
      "msg" : "index: (2/3) btree bottom up 192158787/298486401 64%"
      }

      I issued the command:
      db.killOp(710799555)

      The console printed out that it was trying to kill the process, but it never did. Once the master finally completed, after over an hour, we had to wait for the replica set slaves to also build the index. The overall outage window lasted several hours but could have been avoided if I could kill the job.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              aaron Aaron Staple
              Reporter:
              aaron.westendorf Aaron Westendorf
              Participants:
              Votes:
              12 Vote for this issue
              Watchers:
              25 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: