Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-12145

authSchemaUpgradeStep attempts to proceed forever if config server is not upgraded yet

    XMLWordPrintableJSON

Details

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • 2.5.5
    • None
    • Security

    Description

      i tried running the auth upgrade process in the situation where one of the shards is still on 2.4 to see what might happen if someone proceeds with the upgrade steps, but has forgotten (or missed) a binary upgrade of one of the mongods.

      It looks like in some cases, the upgrade fails but the command still returns

      {"ok":1}

      so running the command in a while loop (as recommended here http://docs.mongodb.org/master/release-notes/2.6-upgrade/#upgrade-from-mongodb-2-4-user-authorization-model-to-mongodb-2-5-x-model) will make it spin forever until killed.

      script to reproduce:

      var st1 = new ShardingTest({shards:1, mongos:2, other:{mongosOptions:{binVersion:"2.5"}, shardOptions:{binVersion:MongoRunner.versionIterator(["2.5", "2.4"])}, configOptions:{binVersion:"2.5"}}})
      print("STOPPIN BALANCER")
      st1.stopBalancer()
      print("STOPPED BALANCER, DOING UPGRADE")
      MongoRunner.stopMongos(st1.s0)
      st1.s0 = MongoRunner.runMongos({restart:st1.s0, binVersion:"2.5", upgrade:""})
       
      //this will spin forever, because res.ok always returns 1, but the command actually seems to be failing 
      do {
      	res = st1.s0.getDB("admin").runCommand({authSchemaUpgradeStep: 1});
      	print(tojson(res));
      } while (res.ok && !res.done);

      The logs say:

       m30999| 2013-12-17T15:02:32.748-0500 [conn1] distributed lock 'authorizationData/Michaels-MacBook-Pro.local:30999:1387310551:16807' acquired, ts : 52b0add83ffc79855ae4c201
       m30999| 2013-12-17T15:02:32.748-0500 [conn1] Auth schema upgrade erasing contents of admin.system.backup_users
       m30999| 2013-12-17T15:02:32.749-0500 [conn1] Auth schema upgrade backing up admin.system.users into admin.system.backup_users
       m30999| 2013-12-17T15:02:32.749-0500 [conn1] scoped connection to localhost:30000 not being returned to the pool
       m30999| 2013-12-17T15:02:32.749-0500 [conn1] Auth schema upgrade dropping indexes from admin.system.new_users
       m30000| Tue Dec 17 15:02:32.749 [conn1014] end connection 127.0.0.1:58454 (7 connections now open)
       m30000| Tue Dec 17 15:02:32.749 [initandlisten] connection accepted from 127.0.0.1:58455 #1015 (8 connections now open)
       m30000| Tue Dec 17 15:02:32.749 [conn1015] CMD: dropIndexes admin.system.new_users
       m30999| 2013-12-17T15:02:32.749-0500 [conn1] warning: Auth schema upgrade failed to drop indexes on admin.system.new_users (UnknownError can't drop system ns)
       m30999| 2013-12-17T15:02:32.749-0500 [conn1] Auth schema upgrade erasing contents of admin.system.new_users
       m30999| 2013-12-17T15:02:32.749-0500 [conn1] Auth schema upgrade creating needed indexes of admin.system.new_users
       m30000| Tue Dec 17 15:02:32.749 [conn1015] build index admin.system.new_users { user: 1, db: 1 }
       m30000| Tue Dec 17 15:02:32.750 [conn1015] build index done.  scanned 0 total records. 0 secs
       m30999| 2013-12-17T15:02:32.750-0500 [conn1] Auth schema upgrade processing schema version 1 users from database admin
       m30999| 2013-12-17T15:02:32.750-0500 [conn1] scoped connection to localhost:30000 not being returned to the pool
       m30999| 2013-12-17T15:02:32.750-0500 [conn1] Auth schema upgrade processing schema version 1 users from database config
       m30000| Tue Dec 17 15:02:32.750 [conn1015] end connection 127.0.0.1:58455 (7 connections now open)
       m30000| Tue Dec 17 15:02:32.750 [initandlisten] connection accepted from 127.0.0.1:58456 #1016 (8 connections now open)
       m30999| 2013-12-17T15:02:32.750-0500 [conn1] scoped connection to localhost:30000 not being returned to the pool
       m30000| Tue Dec 17 15:02:32.751 [conn1016] end connection 127.0.0.1:58456 (7 connections now open)
       m30000| Tue Dec 17 15:02:32.751 [initandlisten] connection accepted from 127.0.0.1:58457 #1017 (8 connections now open)
       m30999| 2013-12-17T15:02:32.751-0500 [conn1] distributed lock 'authorizationData/Michaels-MacBook-Pro.local:30999:1387310551:16807' unlocked.
      { "done" : false, "ok" : 1 }

      This repeats with each iteration of the while loop, ad infinitum.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mikeo@mongodb.com Michael O'Brien
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: