[SERVER-12077] authSchemaUpgradeStep error message when running on 2.5.4 mongos with 2.4 config server could be better Created: 12/Dec/13 Updated: 11/Jul/16 Resolved: 19/Feb/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Security, Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 2.6.0-rc0 |
| Type: | Improvement | Priority: | Minor - P4 |
| Reporter: | Valeri Karpov | Assignee: | Spencer Brody (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | 26qa | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Participants: | |||||||||
| Description |
|
Output now looks like: Auth schema upgraded failed with UnknownError cannot update system collection: admin.system.version q: { _id: "authSchema" }u: { $set: { currentVersion: 1 }} This could be more clear and indicate that there's a config server that you forgot to upgrade. |
| Comments |
| Comment by Valeri Karpov [ 18/Feb/14 ] | ||||||
|
spencer LGTM! ---------- m30999| 2014-02-18T18:17:14.946-0500 [conn1] distributed lock 'authorizationData/specter.local:30999:1392765432:16807' acquired, ts : 5303e9fada09571b96365e03 ---------- | ||||||
| Comment by Githook User [ 13/Feb/14 ] | ||||||
|
Author: {u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}Message: | ||||||
| Comment by Spencer Brody (Inactive) [ 13/Feb/14 ] | ||||||
|
Looks like the command is returning the right result object but nothing is being logged by the server to indicate that the upgrade failed. | ||||||
| Comment by Valeri Karpov [ 13/Feb/14 ] | ||||||
|
So I'm running https://github.com/10gen/QA/blob/master/QA-424/sharded_upgrade_with_24_configsvr.js against the binary from http://mci.10gen.com/ui/build/mongodb_mongo_master_osx_108_db7b5a93e9bdfee1826a91b94a9f8b904be89813_14_02_13_03_03_10 and the only output that I get from the authSchemaUpgrade is: m30999| 2014-02-13T16:17:33.594-0500 [conn1] distributed lock 'authorizationData/specter.local:30999:1392326251:16807' acquired, ts : 52fd366dc7de95733d8d1d70 The command does fail, which is good, but I don't see any indication that I'm doing something wrong in the logs. | ||||||
| Comment by Githook User [ 13/Feb/14 ] | ||||||
|
Author: {u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}Message: | ||||||
| Comment by Spencer Brody (Inactive) [ 12/Feb/14 ] | ||||||
|
With the change I just pushed, now when I try this I get:
valeri.karpov, can you test again with the change I just put in? | ||||||
| Comment by Githook User [ 12/Feb/14 ] | ||||||
|
Author: {u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}Message: This improves the error message when you try to run the authSchemaUpgrade on a system with an | ||||||
| Comment by Githook User [ 12/Feb/14 ] | ||||||
|
Author: {u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}Message: This improves the error message when you try to run the authSchemaUpgrade on a system with an | ||||||
| Comment by Valeri Karpov [ 12/Feb/14 ] | ||||||
|
I re-ran my tests and right now I don't get any error message or other indication that something went wrong, other than authSchemaVersion is still 1. The only errors I see are: m30999| 2014-02-12T10:41:50.693-0500 [conn1] warning: Auth schema upgrade failed to drop indexes on admin.system.new_users (Unknown error code dropIndexes failed) | ||||||
| Comment by Spencer Brody (Inactive) [ 07/Feb/14 ] | ||||||
|
Taking back to update the function that checks the versions of the cluster before doing an upgrade to include checking the config servers. | ||||||
| Comment by Greg Studer [ 07/Feb/14 ] | ||||||
|
In general it's not an error to write to different versions of config servers at the write level - it shouldn't/didn't fail because one of the servers was 2.4, it failed because the write result was different on one of the servers. We could attempt to make it an option to only write to particular versions and fail if that's not possible - I see this as having limited general value right now though (maybe later). It seems like this is something that should be caught when you check the versions of everything in the cluster before doing the final upgrade step? | ||||||
| Comment by Spencer Brody (Inactive) [ 06/Feb/14 ] | ||||||
|
The fact that the message says the write was not consistent makes me a bit nervous... Also, it'd be nice if the clusterWrite operation could indicate that the reason it failed was because one of the config servers was still running 2.4. Assigning to Greg for triage. | ||||||
| Comment by Spencer Brody (Inactive) [ 05/Feb/14 ] | ||||||
|
I believe this behavior has changed since this ticket was filed due to the switch to using cluster write operations. The new return value is:
I think any more detailed error message would have to come from the clusterWrite code, not the auth code. valeri.karpov, can you confirm that the behavior has changed from what you initially observed when this was filed? | ||||||
| Comment by Andy Schwerin [ 12/Dec/13 ] | ||||||
|
The AuthzManagerExternalStateS should be extracting codes from failed operations, when available, rather than using UnknownError. This would allow us to check the code and put a custom message in at a higher level. |