Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24035

balancer does not respect active window

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Duplicate
    • 3.0.5
    • None
    • Sharding
    • None
    • ALL

    Description

      We just turned the balancer back on for a cluster where it has been off for many months. Typically, we run the balancer 24/7, but in this case we can only run it for a few hours a day. I configured the balancer active window to be between 23:00 and 6:00, per the example given here: https://docs.mongodb.org/v3.0/tutorial/manage-sharded-cluster-balancer/#schedule-the-balancing-window

      However, I noticed that the balancer was running even outside of these hours, and actually caused severe issues to our site this morning. I've reproduced the steps here:

      mongos> db.settings.find({ _id : "balancer" })
      { "_id" : "balancer", "_secondaryThrottle" : false, "_waitForDelete" : true, "activeWindow" : { "start" : "23:00", "stop" : "6:00" }, "stopped" : true }
      mongos> db.locks.find({ _id : "balancer" }, { "state" : 1 })
      { "_id" : "balancer", "state" : 0 }
      mongos> new Date().toLocaleString()
      Tue May 03 2016 21:05:18 GMT+0000 (UTC)
      mongos> db.settings.update({ _id : "balancer" }, { $set : { stopped : false } }, { upsert: true })
      mongos> db.locks.find({ _id : "balancer" }, { "state" : 1 })
      { "_id" : "balancer", "state" : 2 }
      mongos> new Date().toLocaleString()
      Tue May 03 2016 21:06:25 GMT+0000 (UTC)
      mongos> db.settings.update({ _id : "balancer" }, { $set : { stopped : true } }, { upsert: true })
      mongos> new Date().toLocaleString()
      Tue May 03 2016 21:06:56 GMT+0000 (UTC)
      mongos> db.locks.find({ _id : "balancer" }, { "state" : 1 })
      { "_id" : "balancer", "state" : 0 }
      

      As you can see, the time when I ran those commands was just after 21:00, which should not be inside the active window. However, after turning the balancer on, it immediately started migrating chunks. Is there something I'm missing?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              dai@foursquare.com Dai Shi
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: