Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24035

balancer does not respect active window

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.0.5
    • Component/s: Sharding
    • Labels:
      None
    • ALL

      We just turned the balancer back on for a cluster where it has been off for many months. Typically, we run the balancer 24/7, but in this case we can only run it for a few hours a day. I configured the balancer active window to be between 23:00 and 6:00, per the example given here: https://docs.mongodb.org/v3.0/tutorial/manage-sharded-cluster-balancer/#schedule-the-balancing-window

      However, I noticed that the balancer was running even outside of these hours, and actually caused severe issues to our site this morning. I've reproduced the steps here:

      mongos> db.settings.find({ _id : "balancer" })
      { "_id" : "balancer", "_secondaryThrottle" : false, "_waitForDelete" : true, "activeWindow" : { "start" : "23:00", "stop" : "6:00" }, "stopped" : true }
      mongos> db.locks.find({ _id : "balancer" }, { "state" : 1 })
      { "_id" : "balancer", "state" : 0 }
      mongos> new Date().toLocaleString()
      Tue May 03 2016 21:05:18 GMT+0000 (UTC)
      mongos> db.settings.update({ _id : "balancer" }, { $set : { stopped : false } }, { upsert: true })
      mongos> db.locks.find({ _id : "balancer" }, { "state" : 1 })
      { "_id" : "balancer", "state" : 2 }
      mongos> new Date().toLocaleString()
      Tue May 03 2016 21:06:25 GMT+0000 (UTC)
      mongos> db.settings.update({ _id : "balancer" }, { $set : { stopped : true } }, { upsert: true })
      mongos> new Date().toLocaleString()
      Tue May 03 2016 21:06:56 GMT+0000 (UTC)
      mongos> db.locks.find({ _id : "balancer" }, { "state" : 1 })
      { "_id" : "balancer", "state" : 0 }
      

      As you can see, the time when I ran those commands was just after 21:00, which should not be inside the active window. However, after turning the balancer on, it immediately started migrating chunks. Is there something I'm missing?

            Assignee:
            Unassigned Unassigned
            Reporter:
            dai@foursquare.com Dai Shi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: