[SERVER-9516] Upgrade/downgrade support for new schema for user and role data Created: 30/Apr/13  Updated: 21/Sep/18  Resolved: 13/Nov/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 2.5.4

Type: Task Priority: Major - P3
Reporter: Andy Schwerin Assignee: Andy Schwerin
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File upgradeusers.js    
Issue Links:
Depends
is depended on by DOCS-1934 Document: Upgrade/downgrade support f... Closed
is depended on by SERVER-10669 Write tests for user upgrade process Closed
Related
related to SERVER-17412 remove authSchemaUpgrade command Closed
is related to SERVER-8391 Pre-flight upgrade tool to check for ... Closed
Backwards Compatibility: Major Change
Participants:

 Description   

Role and user information for all databases will now be stored in system collections in the admin database, and manipulated exclusively via commands per SERVER-6246 and SERVER-9515.

The db.system.users collections will be deprecated, with data migrated from them to the new collection schema as part of the 2.4->2.6 upgrade process.



 Comments   
Comment by Andy Schwerin [ 13/Nov/13 ]

When working with a replicaset, it is only necessary to upgrade/downgrade the primary.

When upgrading with a sharded cluster, first connect to a single mongos and run the upgrade procedure. This upgrades the config servers and all mongoses. Next, connect to the primary replicaset member of each shard, and run the upgrade, to upgrade that replicaset.

When downgrading a sharded cluster, one may downgrade the cluster first or the shards first. Order does not matter.

Comment by Andy Schwerin [ 13/Nov/13 ]

Downgrade is best achieved by the following procedure, assuming that one has left the contents of *.system.users for * != admin intact.

  1. Copy the contents of admin.system.users to admin.system.new_users.
  2. Run the following update of the version document for the auth schema

    admin.system.version.update({ _id: "authSchema" }, { $set: { currentVersion: 2 } })

  3. Remove all documents from admin.system.users.
  4. Copy the contents of admin.system.backup_users, created during upgrade, to admin.system.users.
  5. Run the following update of the version document for the auth schema

    admin.system.version.update({ _id: "authSchema" }, { $set: { currentVersion: 1 } })

Comment by Andy Schwerin [ 13/Nov/13 ]

The attached ugpradeusers.js can be loaded and the upgradeUsers() function can then be called to perform an upgrade.

load("upgradeusers.js");
upgradeUsers(
    db.getMongo(), // Connection object
    {}  // Write concern object.
);

Comment by Andy Schwerin [ 13/Nov/13 ]

Upgrade achieved via by running the new authSchemaUpgradeStep command, as follows, while connected as a user with userAdminAnyDatabase:

var res;
do {
    res = db.getSiblingDB("admin").runCommand({authSchemaUpgradeStep: 1});
} while (res.ok && !res.done);
print(tojson(res));

Comment by Githook User [ 13/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Allow AuthorizationManager to lazily detect that an upgrade happened under it.
Branch: master
https://github.com/mongodb/mongo/commit/94749ae43fa3c8bbd532ad46f29dbbf063de9eb3

Comment by Githook User [ 13/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Add location codes to fasserts.
Branch: master
https://github.com/mongodb/mongo/commit/3b742225a7db2ad9a2e10dbe8efefbc0d7a6757b

Comment by Githook User [ 13/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 User objects must be immutable after first returned from AuthManager.

The mechanism for probing schemaVersion24 users was not respecting this requirement
before this patch, which would have led to concurrency errors.

Also, with this change the internalSecurity.user is never stored in the cache, and
its reference count is never altered.
Branch: master
https://github.com/mongodb/mongo/commit/c2036cf47e4b771bc455a1a3c02256b541874f4f

Comment by Githook User [ 13/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Acquire auth update lock in upgrade command, not in upgradeSchemaStep() method.
Branch: master
https://github.com/mongodb/mongo/commit/cf79f24096a017c2290f7524997813b266e09d71

Comment by auto [ 12/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Fix use-after-free in unit test.
Branch: master
https://github.com/mongodb/mongo/commit/e2f1d1b750d6dca8680c7792fbd008bce9a1aa88

Comment by auto [ 12/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Add missing assert codes.
Branch: master
https://github.com/mongodb/mongo/commit/f4946ff4166a830e204ab14653d88996542770b4

Comment by auto [ 12/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Introduce command to take one step in the user schema upgrade process.
Branch: master
https://github.com/mongodb/mongo/commit/cdc26b4613526d0dc9e389c581cb8243955a76e1

Comment by auto [ 12/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Implement AuthorizationManager::upgradeSchemaStep().
Branch: master
https://github.com/mongodb/mongo/commit/bd4ee11df9d41eaeceed2ef2bf5454276ab8cfa5

Comment by auto [ 12/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Report document changes via the logOp infrastructure in AuthzManagerExternalStateMock.
Branch: master
https://github.com/mongodb/mongo/commit/11db6a808ca368c5b596ff0e4066de599b912439

Comment by auto [ 11/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Reinitialize user and role data on replicaset rollback.
Branch: master
https://github.com/mongodb/mongo/commit/4e4a746769a899bbfb2daaf262dfb341da09e24b

Comment by auto [ 08/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Provide stub implementation of AuthorizationManager::upgradeSchemaStep

Unbreaks the windows compile.
Branch: master
https://github.com/mongodb/mongo/commit/ea1d6a8a9e21a70ff57bda7ca47197ab58d3e7a9

Comment by auto [ 08/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Remove canDelegate and hasRole from auth docs in unit test.
Branch: master
https://github.com/mongodb/mongo/commit/e392f087336daac43b2e52ffc83a5be7663feef6

Comment by auto [ 08/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Factor out common code from mock & mongod implementations of AuthzManagerExternalState.
Branch: master
https://github.com/mongodb/mongo/commit/33f16ec7ed5faf0f5bcf8e6677447a8024f0e7f7

Comment by auto [ 08/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Remove unused methods from AuthzManagerExternalState.
Branch: master
https://github.com/mongodb/mongo/commit/d0fa8b74df7c4a5d1ac897110610d6582f17556b

Comment by auto [ 08/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Change schema upgrade interface in AuthorizationManager.

Update the unit tests to reflect the new interface, and various other
changes that occurred while some tests were disabled.
Branch: master
https://github.com/mongodb/mongo/commit/3bb685079761a085b4ab7e7047251ef06fe62532

Comment by auto [ 08/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Remove dead code from auth_index_d module.
Branch: master
https://github.com/mongodb/mongo/commit/0ba4291e45869b93709f3137b859e265b773f480

Comment by auto [ 05/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Enhance privileges of builtin roles to handle auth upgrade system collections.

This commit lets userAdminAnyDatabase role run listDatabases, since it was the
only AnyDatabase role that couldn't.

It also uses constants for auth collections in RoleGraph.

Finally, it grants access to new_users and backup_users.
Branch: master
https://github.com/mongodb/mongo/commit/c18dfd852cb222a61f6393ebac6e6edf05c52386

Comment by auto [ 05/Nov/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Give userAdminAnyDatabase more privs for v2.4 users; in support of upgrade.
Branch: master
https://github.com/mongodb/mongo/commit/f7bdc39d13a4ee8415698dfd63a95961f7e80062

Comment by auto [ 31/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Legalize admin.system.new_users and admin.system.backup_users collections.
Branch: master
https://github.com/mongodb/mongo/commit/60b4b2ccebcace38b844ccfaf7cd4979d769dbd2

Comment by auto [ 31/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 connectionStatus should return authenticatedUsers as

{user: <string>, db: <string>}

.

Previously, it reported users as

{ user: <string>, userSource: <string> }

Branch: master
https://github.com/mongodb/mongo/commit/655598aba916197c260a88cc49978faf9b51a048

Comment by auto [ 31/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Update admin.system.version as part of user management commands.
Branch: master
https://github.com/mongodb/mongo/commit/01afc9b2b1290f16c10c8d591affbdbe157a6d18

Comment by auto [ 31/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Stop doing system.users index upgrade implicitly.

The right place to upgrade the admin.system.users indexes is during the explicit
2.4->2.6 upgrade process, not on opening the admin database as it was in the
2.2->2.4 upgrade process.

Furthermore, since system indexes are only created on system collection creation, now,
there's no need to check the auth schema version when creating indexes on admin.system.users.
If the collection did not exist before, the schema version must be 2.6.
Branch: master
https://github.com/mongodb/mongo/commit/ab09b4d6369c8623d4d56248da899e685782d908

Comment by auto [ 31/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Make getUserDescription in mongod look in the right collection during upgrade.
Branch: master
https://github.com/mongodb/mongo/commit/3fcc1206716d8a89b91a932b58fae059b8f750e6

Comment by auto [ 30/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Disable user and role management commands when schema is too old.

User and role management commands are only allowed when the schema version
is schemaVersion26Final (3). The usersInfo and rolesInfo commands are not
allowed when the schema version is schemaVersion24 (1). This commit enforces
those rules.
Branch: master
https://github.com/mongodb/mongo/commit/e2b0c88ed2d50962d2f433fcfb85cb0262c16950

Comment by auto [ 29/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Fix fassert number, add missing comment.
Branch: master
https://github.com/mongodb/mongo/commit/a704c10f10d236b1ff1bc48549896ec6d9dd2ca3

Comment by auto [ 29/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Fix concurrency error around AuthorizationManager::CacheGuard::endFetchPhase().

That method should not clear the _isThisGuardInFetchPhase field. Rather, the
fassert ensuring preconditions for isSameCacheGeneration() should recognize the
true "post fetch phase" condition, that _isThisGuardInFetchPhase is true and
that _lock.owns_lock() is also true.
Branch: master
https://github.com/mongodb/mongo/commit/4c21ddbe80d4e77c5f6ca6b4df405f4921c1000b

Comment by auto [ 29/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Fix concurrency error around AuthorizationManager::CacheGuard::endFetchPhase().

That method should not clear the _isThisGuardInFetchPhase field. Rather, the
fassert ensuring preconditions for isSameCacheGeneration() should recognize the
true "post fetch phase" condition, that _isThisGuardInFetchPhase is true and
that _lock.owns_lock() is also true.
Branch: master
https://github.com/mongodb/mongo/commit/87f8f4437ed282eeced26d8d7ff5fda5f3db43cd

Comment by auto [ 25/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: Revert "SERVER-9516 Write version document to admin.system.version as needed in mongod."

This reverts commit 3d61067308f7f526ec00d79bcbb158d31fd413fa, which breaks multi-version
because it replicates writes to admin.system.version at startup.
Branch: master
https://github.com/mongodb/mongo/commit/774860337ea71d1f8b4711971dc9fa61115cd003

Comment by auto [ 25/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Fix fassert codes.
Branch: master
https://github.com/mongodb/mongo/commit/63f43a42b916ead71cce107a76754e032cbc06cf

Comment by auto [ 25/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Write version document to admin.system.version as needed in mongod.

At startup, for standalone mongod and master. Upon election to primary for replicaset members.
Branch: master
https://github.com/mongodb/mongo/commit/3d61067308f7f526ec00d79bcbb158d31fd413fa

Comment by auto [ 25/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Do not alter indexes on admin.system.users at startup unless using schemaVersion26Final.

Older schema versions may have incompatible documents in admin.system.users.
Branch: master
https://github.com/mongodb/mongo/commit/b74dec9a8a8d8b2ec8771148f3edc6aa575db246

Comment by auto [ 25/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Introduce symbolic constants for authz schema versions, renumber them.
Branch: master
https://github.com/mongodb/mongo/commit/d56278b9d8dbecd2a6ca041c8bd07000dfcf015f

Comment by auto [ 25/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Support schemaV24 users, for use prior to auth schema upgrade.

To enable this support, one must change the _version field when constructing the
AuthorizationManager, which is not possible in this change set. A separate
change will introduce proper version detection and writing of
admin.system.version, at which point this functionality can be enabled.
Branch: master
https://github.com/mongodb/mongo/commit/ea84e033a8c61d492e322cd7de69331aa09058d9

Comment by auto [ 18/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Eliminate declaration of unimplemented function, update comment.
Branch: master
https://github.com/mongodb/mongo/commit/4f86462b86c1c2ee06495c93181e345819930dcc

Comment by auto [ 18/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Support for determining and examining the version of the auth schema in durable storage.
Branch: master
https://github.com/mongodb/mongo/commit/d88fa196ad24d2061ae29e608106b2d075485bdb

Comment by auto [ 18/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Mark admin.system.version as a legal namespace.
Branch: master
https://github.com/mongodb/mongo/commit/ee36f600438b349a2325bb12dcdaa7ea238053e7

Comment by auto [ 18/Oct/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Relax type requirements for second parameter to sequenceContains().
Branch: master
https://github.com/mongodb/mongo/commit/86ea362635929038c827616d26c07ef0d70fb709

Comment by auto [ 04/Sep/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Fix authorization_manager_test reference management problem.
Branch: master
https://github.com/mongodb/mongo/commit/e6ed9e9a4fb5e662bc5aa20a158ddeb1fea00ded

Comment by auto [ 04/Sep/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Changes for enterprise module.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/26eb63096f6f2dc00824fccb184b6c9d75f0cf44

Comment by auto [ 03/Sep/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Initialize local variable to satisfy a compiler.

Newer compilers correctly identify that durableVersion in this patch is always
initialized before use, but some older compilers cannot perform a sufficient
analysis, and issue a spurious warning.
Branch: master
https://github.com/mongodb/mongo/commit/805c07c3bb33cae573d90d748ffc5dc9d396f2cd

Comment by auto [ 03/Sep/13 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-9516 Logic of system.users schema upgrade process.

Includes AuthorizationManagerExternalState interface changes and implementation
in the mock, plus unit tests.
Branch: master
https://github.com/mongodb/mongo/commit/5fb2c2c35426d907057bccf6ffefec0493897af6

Generated at Thu Feb 08 03:20:39 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.