[SERVER-8039] Authentication repeatedly fails after major network issue Created: 27/Dec/12 Updated: 08/Mar/13 Resolved: 07/Jan/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Networking, Security |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Anton V. Volokhov | Assignee: | Spencer Brody (Inactive) |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Participants: |
| Description |
|
Hello! Wed Dec 26 20:02:37 [conn1183] authenticate db: partition-validator { authenticate: 1, user: "auto", nonce: "5b3de430aaacf9a5", key: "6eb58aa336bfc1eef5779a04761f83d3" }Wed Dec 26 20:02:37 [conn1183] auth: couldn't find user auto, partition-validator.system.users Wed Dec 26 20:02:56 [conn1186] auth: couldn't find user auto, partition-validator.system.users Network started about 19:00 and was fixed exactly in 20:00, authentication was ok before 20:00, during network fail too. Some logs from the interval 19:00-20:00 update: { $set: { state: 1, who: "cs-mongodb01g:27000:1355404365:864101839:Balancer:636453333", process: "cs-mongodb01g:27000:13 55404365:864101839", when: new Date(1356536930010), why: "doing balance round", ts: ObjectId('50db1c61326d9641ed1f7228') }} gle1: { updatedExisting: true, n: 1, connectionId: 36369, waited: 34, err: nu ll, ok: 1.0 }gle2: { updatedExisting: false, n: 0, connectionId: 38633, waited: 18, err: null, ok: 1.0 }Wed Dec 26 19:48:51 [Balancer] lock update lost, lock 'balancer/cs-mongodb01g:27000:1355404365:864101839' not propagated. |
| Comments |
| Comment by Anton V. Volokhov [ 07/Jan/13 ] | ||
|
Thanks for help, anyway. | ||
| Comment by Spencer Brody (Inactive) [ 07/Jan/13 ] | ||
|
Unfortunately, I don't believe we have enough information here to understand exactly what happened. If this happens again, please increase the logging verbosity on the mongos and the mongod with the problems by running
and attach the logs to this ticket, after re-opening it, before you re-create the user. | ||
| Comment by Anton V. Volokhov [ 07/Jan/13 ] | ||
|
Sorry, they died yesterday, due to logrotate configuration. I have only those, I've sent in a first post. Wed Dec 26 20:02:37 [conn1183] auth: couldn't find user auto, partition-validator.system.users | ||
| Comment by Spencer Brody (Inactive) [ 07/Jan/13 ] | ||
|
Can you please attach the full logs from the mongos and the primary of the shard in question from the time when authentication was failing? | ||
| Comment by Anton V. Volokhov [ 07/Jan/13 ] | ||
|
Recreating the user via db.addUser("auto","foo") fixed the problem. It was the equally the same when the user was corrupted. | ||
| Comment by Spencer Brody (Inactive) [ 07/Jan/13 ] | ||
|
What's the current status of this? Are you still having problems authenticating to users in this database on this machine? If so, can you attach the output of
run against mongos as well as the node that's having the authentication issues? | ||
| Comment by Anton V. Volokhov [ 05/Jan/13 ] | ||
|
Only mongos and isolated node were restarted. Same behavior experienced both through mongos and replica's master. Didn't test on replicas's secondary, unfortunately. | ||
| Comment by Spencer Brody (Inactive) [ 04/Jan/13 ] | ||
|
Hmm, that seems strange. Have you tried restarting this node since the network problems? Are you connecting to the mongos or a mongod directly? If the latter, do you have the same problem with all nodes in the replica set, or just the one that had the network outage? | ||
| Comment by Anton V. Volokhov [ 27/Dec/12 ] | ||
|
So, I couldn't login via shell. When I logon as admin, I saw this user "auto" in db.system.users collection, hash for this username was exactly the same as in the other databases, where this use/pwd pair exists. Logging in the other databases was successful. |