[SERVER-10138] Assertion failure when trying to auth on replset connection before user is replicated (auth_repl.js) Created: 09/Jul/13  Updated: 11/Jul/16  Resolved: 12/Jul/13

Status: Closed
Project: Core Server
Component/s: Internal Client
Affects Version/s: 2.5.0
Fix Version/s: 2.5.1

Type: Bug Priority: Major - P3
Reporter: Matt Kangas Assignee: Randolph Tan
Resolution: Done Votes: 0
Labels: buildbot
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

buildbot: OS X 10.5 64-bit


Issue Links:
Related
is related to SERVER-9980 Add write concern to user management ... Closed
Operating System: ALL
Participants:

 Description   

Summary of bug:

When using the C++ driver (or the mongo shell, which uses the driver) to authenticate on a replica set connection, an assertion will fail when the driver tries to authenticate the user before it gets replicated to the secondary.

Buildbot error:

http://buildbot.mongodb.org/builders/OS%20X%2010.5%2064-bit/builds/5556
http://buildbot.mongodb.org/builders/OS%20X%2010.5%2064-bit/builds/5556/steps/test_11/logs/stdio
http://buildlogs.mongodb.org/OS%20X%2010.5%2064-bit/builds/5556/test/sharding/auth_repl.js

{ "user" : "a", "readOnly" : false }
 m31001| Mon Jul  8 21:44:15.886 [repl writer worker 1] build index admin.system.users { _id: 1 }
 m31000| Mon Jul  8 21:44:15.886 [conn18]  authenticate db: test { authenticate: 1, nonce: "8ff1b65414b46bca", user: "a", key: "ae3ba84fc7997ecdcc40d180ba8ed59c" }
 m31001| Mon Jul  8 21:44:15.887 [repl writer worker 1] build index done.  scanned 0 total records. 0 secs
 m31001| Mon Jul  8 21:44:15.887 [repl writer worker 1] build index admin.system.users { user: 1, userSource: 1 }
 m31001| Mon Jul  8 21:44:15.888 [repl writer worker 1] build index done.  scanned 0 total records. 0 secs
 m31001| Mon Jul  8 21:44:15.888 [conn6]  authenticate db: test { authenticate: 1, nonce: "b005991f1a1b5dd6", user: "a", key: "8233ba10ff78ed256f6599ab0da197e8" }
 
 m31001| Mon Jul  8 21:44:15.889 [conn6] auth: couldn't find user a@test, test.system.users
 
Mon Jul  8 21:44:15.889   Assertion failure _lastSlaveOkConn->isFailed() src/mongo/client/dbclient_rs.cpp 1484
 
 
0x1000f830b 0x1000d563c 0x10005debd 0x1000ce9ee 0x1000b86d0 0x1001a4b70 0xdb7efd06362 
 m31001| Mon Jul  8 21:44:15.889 [repl writer worker 1] build index test.system.users { _id: 1 }
 m31001| Mon Jul  8 21:44:15.890 [repl writer worker 1] build index done.  scanned 0 total records. 0 secs
 m31001| Mon Jul  8 21:44:15.890 [repl writer worker 1] build index test.system.users { user: 1, userSource: 1 }
 m31001| Mon Jul  8 21:44:15.890 [repl writer worker 1] build index done.  scanned 0 total records. 0 secs
 0   mongo                               0x00000001000f830b _ZN5mongo15printStackTraceERSo + 43
 1   mongo                               0x00000001000d563c _ZN5mongo12verifyFailedEPKcS1_j + 284
 2   mongo                               0x000000010005debd _ZN5mongo18DBClientReplicaSet5_authERKNS_7BSONObjE + 225
 3   mongo                               0x00000001000ce9ee _ZN5mongo9mongoAuthEPNS_7V8ScopeERKN2v89ArgumentsE + 302
 4   mongo                               0x00000001000b86d0 _ZN5mongo7V8Scope10v8CallbackERKN2v89ArgumentsE + 116
 5   mongo                               0x00000001001a4b70 _ZN2v88internalL21Builtin_HandleApiCallENS0_12_GLOBAL__N_116BuiltinArgumentsILNS0_21BuiltinExtraArgumentsE1EEEPNS0_7IsolateE + 464
 6   ???                                 0x00000db7efd06362 0x0 + 15083653587810
Error: 0 assertion src/mongo/client/dbclient_rs.cpp:1484
assert: [1] != [0] are not equal : undefined
Error: Printing Stack Trace
    at printStackTrace (src/mongo/shell/utils.js:37:15)
    at doassert (src/mongo/shell/assert.js:6:5)
    at Function.assert.eq (src/mongo/shell/assert.js:32:5)
    at /data/buildslaves/OS_X_105_64bit/mongo/jstests/sharding/auth_repl.js:36:8
Mon Jul  8 21:44:15.893 [1] != [0] are not equal : undefined at src/mongo/shell/assert.js:7
failed to load: /data/buildslaves/OS_X_105_64bit/mongo/jstests/sharding/auth_repl.js



 Comments   
Comment by auto [ 12/Jul/13 ]

Author:

{u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}

Message: SERVER-10138 Temporary fix for auth_repl.js failure
Branch: master
https://github.com/mongodb/mongo/commit/7bafcc73b71bfd364786f6faf4401d345d714eba

Comment by Matt Kangas [ 12/Jul/13 ]

Ren, it looks like this test failed again on OS X 10.5 64-bit. Different issue? It still appears to be an authorization problem.

OS X 10.5 64-bit Build #5569 (revision 30a130776b43c283fc8c38c48a9478e935a69bc2)

http://buildbot.mongodb.org/builders/OS%20X%2010.5%2064-bit/builds/5569/steps/test_11/logs/stdio
http://buildlogs.mongodb.org/OS%20X%2010.5%2064-bit/builds/5569/test/sharding/auth_repl.js

----
Sending an authorized query that should be ok
----
 m31002| Fri Jul 12 01:48:38.338 [initandlisten] connection accepted from 127.0.0.1:50819 #5 (5 connections now open)
 m31001| Fri Jul 12 01:48:38.338 [conn6] end connection 127.0.0.1:50817 (4 connections now open)
 m31002| Fri Jul 12 01:48:38.341 [repl writer worker 1] build index admin.system.users { _id: 1 }
 m31002| Fri Jul 12 01:48:38.342 [repl writer worker 1] build index done.  scanned 0 total records. 0 secs
 m31002| Fri Jul 12 01:48:38.342 [repl writer worker 1] build index admin.system.users { user: 1, userSource: 1 }
 m31002| Fri Jul 12 01:48:38.342 [repl writer worker 1] build index done.  scanned 0 total records. 0 secs
 m31002| Fri Jul 12 01:48:38.343 [conn5]  authenticate db: test { authenticate: 1, nonce: "e93f2232703f18b4", user: "a", key: "0485ebd12b2588af7c5056ea93dbf590" }
 m31002| Fri Jul 12 01:48:38.343 [conn5] auth: couldn't find user a@test, test.system.users
Fri Jul 12 01:48:38.343 warning: cached auth failed for set: testReplSet db: test user: a
 m31002| Fri Jul 12 01:48:38.343 [conn5] assertion 16550 not authorized for query on test.user ns:test.user query:{ query: {}, $readPreference: { mode: "secondary" } }
 m31002| Fri Jul 12 01:48:38.344 [repl writer worker 1] build index test.system.users { _id: 1 }
Fri Jul 12 01:48:38.344 error: { "$err" : "not authorized for query on test.user", "code" : 16550 } at src/mongo/shell/query.js:131
failed to load: /data/buildslaves/OS_X_105_64bit/mongo/jstests/sharding/auth_repl.js

Comment by auto [ 09/Jul/13 ]

Author:

{u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}

Message: SERVER-10138 Assertion failure when trying to auth on replset connection before user is replicated
Branch: master
https://github.com/mongodb/mongo/commit/8a0d36615ae385734fa5bb5e980fccaa372cc695

Comment by Matt Kangas [ 09/Jul/13 ]

Linux 64-bit debug dur off Build #2479

http://buildbot.mongodb.org/builders/Linux%2064-bit%20debug%20dur%20off/builds/2479/steps/test_11/logs/stdio
http://buildlogs.mongodb.org/Linux%2064-bit%20debug%20dur%20off/builds/2479/test/sharding/auth_repl.js

{ "user" : "a", "readOnly" : false }
Tue Jul  9 08:54:11.509   Assertion failure _lastSlaveOkConn->isFailed() src/mongo/client/dbclient_rs.cpp 1484
0x9a6491 0x9893bd 0x96fa6a 0x8de793 0x8c00c0 0x961220 0x941d7c 0xac5262 0xabff97 0xabff68 0x350780d06362 
 m31001| Tue Jul  9 08:54:11.508 [conn6]  authenticate db: test { authenticate: 1, nonce: "456fdf5e3d431b55", user: "a", key: "6f467ae72f468c5dac94a4b9b70f55b4" }
 m31001| Tue Jul  9 08:54:11.508 [conn6] auth: couldn't find user a@test, test.system.users
 m31001| Tue Jul  9 08:54:11.511 [repl writer worker 1] build index test.system.users { _id: 1 }
 m31001| Tue Jul  9 08:54:11.512 [repl writer worker 1] build index done.  scanned 0 total records. 0 secs
 m31001| Tue Jul  9 08:54:11.512 [repl writer worker 1] build index test.system.users { user: 1, userSource: 1 }
 m31001| Tue Jul  9 08:54:11.513 [repl writer worker 1] build index done.  scanned 0 total records. 0 secs
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo(_ZN5mongo15printStackTraceERSo+0x29) [0x9a6491]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo(_ZN5mongo10logContextEPKc+0x5e) [0x9893bd]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo(_ZN5mongo12verifyFailedEPKcS1_j+0x124) [0x96fa6a]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo(_ZN5mongo18DBClientReplicaSet5_authERKNS_7BSONObjE+0x10d) [0x8de793]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo(_ZN5mongo20DBClientWithCommands4authERKNS_7BSONObjE+0x30) [0x8c00c0]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo(_ZN5mongo9mongoAuthEPNS_7V8ScopeERKN2v89ArgumentsE+0x3ed) [0x961220]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo(_ZN5mongo7V8Scope10v8CallbackERKN2v89ArgumentsE+0x136) [0x941d7c]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo [0xac5262]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo [0xabff97]
 /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongo [0xabff68]
 [0x350780d06362]
Tue Jul  9 08:54:11.514 
***aborting after verify() failure as this is a debug/test build

spencer did you fix this with d067cc353 or 02435802ba ?

Generated at Thu Feb 08 03:22:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.