[SERVER-9315] local.slaves is not correctly maintained in authenticated system. Created: 11/Apr/13  Updated: 11/Jul/16  Resolved: 23/Apr/13

Status: Closed
Project: Core Server
Component/s: Replication, Security
Affects Version/s: 2.4.1
Fix Version/s: 2.4.4, 2.5.0

Type: Bug Priority: Minor - P4
Reporter: Zachary Anker Assignee: Spencer Brody (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

2.6.35.14-97.44.amzn1.x86_64


Issue Links:
Related
related to SERVER-4473 duplicate key error on local.slaves Closed
Operating System: Linux
Steps To Reproduce:

I was able to duplicate this on another unrelated replica set when doing a 2.2.3 -> 2.4.0 upgrade, but another unrelated install has no issues.

Participants:

 Description   

I noticed earlier today that our replica set has been triggering user asserts. From looking at MMS, this started immediately after doing a 2.2.2 -> 2.4.0 upgrade. When I changed the log level to 10 and grepped against the logs for "assert", I get (repeatedly):

Thu Apr 11 01:08:21.951 [slaveTracking] User Assertion: 16538:not authorized for upsert on local.slaves
Thu Apr 11 01:08:21.951 [slaveTracking] update local.slaves keyUpdates:0 exception: not authorized for upsert on local.slaves code:16538 0ms

When looking on the primary server, it's missing the local.slaves collection. Restarting the primary or making it step down doesn't fix it, it's never recreated. All the commands such as rs.status() or rs.reconfig() work fine. As far as I can tell, the only issue this causes that it means the assert number isn't very useful to look at.



 Comments   
Comment by auto [ 13/May/13 ]

Author:

{u'date': u'2013-04-22T17:10:56Z', u'name': u'Spencer T Brody', u'email': u'spencer@10gen.com'}

Message: SERVER-9315 Make sure local.slaves collection can be maintained in a system with security enabled
Branch: v2.4
https://github.com/mongodb/mongo/commit/834f226a115f71c7f57d202efc2bf99dcc86509f

Comment by Spencer Brody (Inactive) [ 23/Apr/13 ]

Commit: https://github.com/mongodb/mongo/commit/86734df5eb25a9151ada55de2ddbb1a97b526791

Comment by Spencer Brody (Inactive) [ 22/Apr/13 ]

Problem is that the system thread doing the updates to local.slaves wasn't using GodScope to ensure that it had the access required to perform the update. This problem has always existed - this isn't new to 2.4 - however the problem is a bit more visible in 2.4 as in older versions an authorized write would not trigger a user assertion, it would just return an "unauthorized" message to the next GLE. In 2.4, we now assert whenever an operation fails because it is unauthorized, thus increasing the user assertions counter in serverStatus() and adding a message to the logs.

Comment by Spencer Brody (Inactive) [ 22/Apr/13 ]

Reproduced and have a fix on the way.

Comment by Zachary Anker [ 16/Apr/13 ]

Running with both, results:

{
	"argv" : [
		"/usr/bin/mongod",
		"-f",
		"/etc/mongod.conf"
	],
	"parsed" : {
		"auth" : "true",
		"config" : "/etc/mongod.conf",
		"dbpath" : "/media/ephemeral/mongod",
		"fork" : "true",
		"keyFile" : "/etc/mongod/key",
		"logappend" : "true",
		"logpath" : "/var/log/mongo/mongod.log",
		"nohttpinterface" : "true",
		"nojournal" : "true",
		"port" : 29000,
		"replSet" : "[set name]"
	},
	"ok" : 1
}

Comment by Stennie Steneker (Inactive) [ 16/Apr/13 ]

Hi Zachary,

Are you running with --auth or --keyfile?

Can you confirm the current command-line options with:

  db.getSiblingDB('admin').runCommand({'getCmdLineOpts':1});

Thanks,
Stephen

Generated at Thu Feb 08 03:20:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.