[SERVER-27027] authorization failed (error 13) with Java 3.3 Created: 14/Nov/16  Updated: 13/Feb/17  Resolved: 13/Feb/17

Status: Closed
Project: Core Server
Component/s: Security
Affects Version/s: 3.2.10
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kamal Kalra Assignee: Kelsey Schubert
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File mongodb.log    
Operating System: ALL
Participants:

 Description   

I'm trying to upgrade from MongoDB 3.0.0 to 3.2.10 on a Mac and continuously get an authentication error when trying to update a document (via a Java driver). Updating from the shell works fine. I've tried dropping the db and re-creating it in 3.2.1 and re-creating the user and role but regardless of the approach, Mongo still throws an exception. Reverting back to 3.0.0 works.



 Comments   
Comment by Kelsey Schubert [ 13/Feb/17 ]

Hi kamalkalra,

Thank you for the additional information. Unfortunately, we have not been able to determine the root cause of this issue. Since a workaround has been identified, I am going to close this ticket. If we receive additional reports of this issue, we will continue to investigate.

Thank you,
Thomas

Comment by Kamal Kalra [ 29/Dec/16 ]

Well the first thing I see wrong in your test case is the update statement. This failure happens on a $setOnInsert (see original stack trace). I can replicate the failure every time with the following code (on MongoDB 3.2.10 w/ Java driver 3.3.0):

public class MongoTest {
	public static void main(String[] args) {
		MongoClient mc = getMongoClient();
		MongoCollection coll = mc.getDatabase("TestDB").getCollection("TestCollection");
 
 
		BasicDBObject queryObj = new BasicDBObject("email", "myemail@somewhere.com");
		BasicDBObject updateObj = new BasicDBObject("email", "myemail@somewhere.com");
		BasicDBObject setObj = new BasicDBObject("$setOnInsert", updateObj);
 
		UpdateOptions options = new UpdateOptions();
		options.upsert(true);
 
		coll.updateOne(queryObj, setObj, options);
	}
 
	private static MongoClient getMongoClient()  {
		List<ServerAddress> addresses = new ArrayList<>();
		addresses.add(new ServerAddress("127.0.0.1", 27017));
 
		String userName = "User1";
		String password = "user1password";
 
		List<MongoCredential> credentials = new ArrayList<>();
		credentials.add(MongoCredential.createCredential(userName, "TestUser", password.toCharArray()));
		credentials.add(MongoCredential.createCredential(userName, "TestUser2", password.toCharArray()));
		credentials.add(MongoCredential.createCredential(userName, "TestUser3", password.toCharArray()));
		credentials.add(MongoCredential.createCredential(userName, "TestUser4", password.toCharArray()));
 
		return new MongoClient(addresses, credentials);
	}
}

Comment by Jeffrey Yemin [ 28/Dec/16 ]

Hi Kamal,

Thanks for continuing to work with us.

I tried again to re-create the scenario and still can't reproduce the issue. This time I created the four users in a fresh install of 3.0.12, then dropped in the 3.2.10 binary. This Java program, using the 3.3.0 driver, ran successfully against both server versions:

import com.mongodb.MongoClient;
import com.mongodb.ServerAddress;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.model.UpdateOptions;
import com.mongodb.client.model.Updates;
import com.mongodb.client.result.UpdateResult;
import org.bson.Document;
 
import static com.mongodb.MongoCredential.createCredential;
import static java.util.Arrays.asList;
import static java.util.Collections.singletonList;
 
public class SERVER27027 {
    public static void main(String[] args) {
        String userName = "nubi";
        String password = "nubi";
 
        MongoClient client = new MongoClient(singletonList(new ServerAddress()), asList(
                createCredential(userName, "nubi1", password.toCharArray()),
                createCredential(userName, "nubi2", password.toCharArray()),
                createCredential(userName, "nubi3", password.toCharArray()),
                createCredential(userName, "nubi4", password.toCharArray())));
 
        MongoCollection<Document> collection = client.getDatabase("nubi4").getCollection("test");
 
        UpdateResult result = collection.updateOne(new Document(), Updates.set("x", 1), new UpdateOptions().upsert(true));
 
        System.out.println(result);
    }
}

The users were created like this:

use nubi1
db.createUser({user : "nubi", pwd : "nubi", roles : [{role : "readWrite", db : "nubi1"}] })
use nubi2
db.createUser({user : "nubi", pwd : "nubi", roles : [{role : "readWrite", db : "nubi2"}] })
use nubi3
db.createUser({user : "nubi", pwd : "nubi", roles : [{role : "readWrite", db : "nubi3"}] })
use nubi4
db.createUser({user : "nubi", pwd : "nubi", roles : [{role : "readWrite", db : "nubi4"}] })

A couple of things you can do to get us more information:

1. Run mongod with verbose logging enabled (-vvv) and attach two sets of server log snippets, one with a successful run against 3.0.12 and one with a failing run against 3.2.10
2. Post the output of the following shell command: db.getSiblingDB("admin").system.version.find()

Thanks,
Jeff

Comment by Kamal Kalra [ 28/Dec/16 ]

Ok. I'm happy to provide additional information (as long as I have the current DB setup).

Comment by Jeffrey Yemin [ 28/Dec/16 ]

As I indicated in an earlier comment I did test this and was unable to reproduce the error. So to get to the bottom of it we'd probably require more input from you to determine the difference in our respective environments.

Comment by Kamal Kalra [ 28/Dec/16 ]

Well, I can live with this solution. It's easier to manage. So from my end, I'm considering this issue closed. Regardless, I think you guys should add this scenario to your test cases.

Comment by Jeffrey Yemin [ 28/Dec/16 ]

Kamal,

Glad to hear that you arrived at a solution, though not the root cause of the original issue. Would you like to keep investigating or should we close this?

Comment by Kamal Kalra [ 28/Dec/16 ]

Creating the user in the admin DB worked! But the approach to create the user was exactly the same as the one in the other DBs so I'm not sure why it works in one DB and not another. Regardless, it's a solution for now...

Thanks.

Comment by Kamal Kalra [ 28/Dec/16 ]

Ok, that makes sense. The users (for "User" and "NubiDo") have been in place since the early MongoDB 2.X releases. I'll try migrating them to the admin DB.

The most recent stack trace is from the clean install of MongoDB 3.2.10. The earlier stack trace and log files were from the upgrade (from 3.0 to 3.2.10).

Here's the output from the shell commands:

{
	"_id" : "User.NubiDo",
	"user" : "NubiDo",
	"db" : "User",
	"roles" : [
		{
			"role" : "readWrite",
			"db" : "User"
		}
	]
}
 
{
	"_id" : "NubiDo.NubiDo",
	"user" : "NubiDo",
	"db" : "NubiDo",
	"roles" : [
		{
			"role" : "readWrite",
			"db" : "NubiDo"
		}
	]
}
 
{
	"_id" : "User2.NubiDo",
	"user" : "NubiDo",
	"db" : "User2",
	"roles" : [
		{
			"role" : "readWrite",
			"db" : "User2"
		}
	]
}
 
{
	"_id" : "NubiDo2.NubiDo",
	"user" : "NubiDo",
	"db" : "NubiDo2",
	"roles" : [
		{
			"role" : "readWrite",
			"db" : "NubiDo2"
		}
	]
}

Comment by Jeffrey Yemin [ 28/Dec/16 ]

Hi Kamal,

Since MongoDB 2.6 it has been possible to create a single user with rights to multiple databases. There is no need any more to create more than one user for a single application. For this use case, you could create a single "NubiDo" user in the "admin" database with the "readWrite" role on the "User, "NubiDo", "User2", and NubiDo2" databases, or particular collections in those databases.

Regardless, you're right that having multiple users is still supported and should work. Unfortunately, I'm not able to reproduce the authorization error with the 3.3.0 driver, either with a single user or with multiple users.

One thing I noticed is that the exception from the stack trace does not match the mongod.log file output. In the exception, the error message from the server is:

not authorized on NubiDo2 to execute command { update: "Nodes", ...

while in the log file the message is:

not authorized on User2 to execute command { update: "NewUserRequests" ...

Notice that both the database and collection names are different.

Also, in the code sample four credentials are added for four different databases, while in the log file there are three successful authentications, but none for the "NubiDo2" database.

Please double check the log files and make sure that the provided logs are for the failed execution. Also, please investigate why only three

Also, in the shell please run the command

db.getUser("NubiDo")

in each of the four databases and attach the responses for all of them.

Comment by Kamal Kalra [ 27/Dec/16 ]

I'm a bit confused. I was under the impression that a user had to be created for a particular database. Where would I create a user that can span a specific set of databases? I have root users in the admin DB but those are specifically for administration purposes and not used by the app. The other users are specifically for the app and the appropriate DBs.

But why would having multiple users make a difference? Either case, it should work (and in fact does in the mongo shell). This seems to be a driver issue, not a multiple user issue, correct?

Also, when executing the command in the shell, I use db.auth(user, password) to authenticate or pass in -u and -p options to the command line when entering the shell, both of which work correctly. I don't recall if the logs I attached has that particular use case but I've tested this many times in the shell. I'll get you a new log file with that specific use case.

Comment by Jeffrey Yemin [ 27/Dec/16 ]

Hi Kamal,

No, we're not able to replicate this yet. One difference between your configuration and the ones that we regularly test with is the fact that you've defined multiple users, all with the same user name, in four different databases. Can you explain your rationale for that? Can you simplify your test case to a single user that has all necessary permissions? With role-based access control, that's certainly possible to do. For example, you can create a single user with the built-in readWriteAnyDatabase role.

Also, as requested earlier please provide the series of mongo shell commands that are successfully executed (including the steps to authenticate the user), as well as the mongod logs for that successful execution (or is that 'conn2' in the already-attached log file?).

Comment by Kamal Kalra [ 27/Dec/16 ]

Hi Jeff,
Here's the snippet that initializes the MongoClient. The host name, port, user and password are read from an encrypted property file (decrypted before use)...

                Properties props = decryptNubiDoProperties();
		List<ServerAddress> addresses = new ArrayList<>();
 
                //
		// populate addresses from property file ...
                // ex:)
                // addresses.add(new ServerAddress(host, port));
                //
 
		String userName = props.getProperty("MongoDB.User");
		String password = props.getProperty("MongoDB.Password");
		List<MongoCredential> credentials = new ArrayList<>();
 
		credentials.add(MongoCredential.createCredential(userName, "User", password.toCharArray()));
		credentials.add(MongoCredential.createCredential(userName, "NubiDo", password.toCharArray()));
		credentials.add(MongoCredential.createCredential(userName, "User2", password.toCharArray()));
		credentials.add(MongoCredential.createCredential(userName, "NubiDo2", password.toCharArray()));
 
		mMongoClient = new MongoClient(addresses, credentials);

Comment by Jeffrey Yemin [ 27/Dec/16 ]

Kamal, please share a snippet of Java code that shows how you specify the username/password for your MongoClient instance (of course, elide the real password), e.g.

     MongoClient client = new MongoClient(new ServerAddress("localhost"), Arrays.asList(MongoCredential#createCredential("NubiDo", "User2", "<password".toCharArray());

or

     MongoClient client = new MongoClient(new MongoClientURI("mongodb://NubiDo:<password>@localhost/?authSource=User2");

I'm curious what you're doing given that the mongod logs show three different authentications on three different databases, all successful, before the failed authorization to update a collection in the User2 database:

2016-11-14T17:11:09.498-0500 I ACCESS   [conn3] Successfully authenticated as principal NubiDo on User
2016-11-14T17:11:09.516-0500 I ACCESS   [conn3] Successfully authenticated as principal NubiDo on NubiDo
2016-11-14T17:11:09.534-0500 I ACCESS   [conn3] Successfully authenticated as principal NubiDo on User2
2016-11-14T17:12:20.474-0500 I ACCESS   [conn3] Unauthorized: not authorized on User2 to execute command { update: "NewUserRequests", ...}

Comment by Kamal Kalra [ 27/Dec/16 ]

One more update. I'm able to get this working with the 3.0.3 Java driver and MongodDB 3.2.10. According to the MongoDB documentation, this driver shouldn't even be compatible with MongoDB 3.2.X but it seems to work. Switching back to a more recent driver immediately causes the authorization exception. But since the 3.0.3 driver isn't listed to be compatible, I'm not sure whether it should be trusted in a production environment. Please advise.

Comment by Kamal Kalra [ 26/Dec/16 ]

Is there any update on this? Are you guys able to replicate this behavior? I've even tried a clean install of 3.2.10 and it exhibits the same problem where the query works correctly via the shell, but not the Java driver.

Comment by Kamal Kalra [ 15/Dec/16 ]

Hi Thomas,
The upgrade was a drop in binary replacement from 3.0 to 3.2. I shutdown the server, installed the new version and re-started the server with the new binary. Also, here's a stack trace from when the problem occurs. If I login to the mongo shell and execute a similar update command, it works fine.

com.mongodb.MongoCommandException: Command failed with error 13: 'not authorized on NubiDo2 to execute command { update: "Nodes", ordered: true, bypassDocumentValidation: true, updates: [ { q: { _id: "94ac4027163b4387a2c0273c2334840b" }, u: { $setOnInsert: { _id: "94ac4027163b4387a2c0273c2334840b", spaceId: "67cb1b63c3214bdea08a3d253ec34c5f", parentKeyPath: "c5ba2f746b3c4ea7aae0d87570d98164/lists/1811f8c83e06467890a67897ba0a9146/tasks", key: "47613cc4826f4791a7d5af005c28dc68", createdBy: "3f3288d0c03d416ea053bdac5759218a", operationCount: 0, createdOn: new Date(1481763976326), updatedOn: new Date(1481763976326), type: "Task", payload: { NubiDocumentType: "Task", NubiDocumentData: { name: "test task" } } } }, upsert: true } ] }' on server 127.0.0.1:27017. The full response is { "ok" : 0.0, "errmsg" : "not authorized on NubiDo2 to execute command { update: \"Nodes\", ordered: true, bypassDocumentValidation: true, updates: [ { q: { _id: \"94ac4027163b4387a2c0273c2334840b\" }, u: { $setOnInsert: { _id: \"94ac4027163b4387a2c0273c2334840b\", spaceId: \"67cb1b63c3214bdea08a3d253ec34c5f\", parentKeyPath: \"c5ba2f746b3c4ea7aae0d87570d98164/lists/1811f8c83e06467890a67897ba0a9146/tasks\", key: \"47613cc4826f4791a7d5af005c28dc68\", createdBy: \"3f3288d0c03d416ea053bdac5759218a\", operationCount: 0, createdOn: new Date(1481763976326), updatedOn: new Date(1481763976326), type: \"Task\", payload: { NubiDocumentType: \"Task\", NubiDocumentData: { name: \"test task\" } } } }, upsert: true } ] }", "code" : 13 }
	at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
	at com.mongodb.connection.WriteCommandProtocol.receiveMessage(WriteCommandProtocol.java:268)
	at com.mongodb.connection.WriteCommandProtocol.execute(WriteCommandProtocol.java:104)
	at com.mongodb.connection.UpdateCommandProtocol.execute(UpdateCommandProtocol.java:64)
	at com.mongodb.connection.UpdateCommandProtocol.execute(UpdateCommandProtocol.java:37)
	at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168)
	at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289)
	at com.mongodb.connection.DefaultServerConnection.updateCommand(DefaultServerConnection.java:143)
	at com.mongodb.operation.MixedBulkWriteOperation$Run$3.executeWriteCommandProtocol(MixedBulkWriteOperation.java:481)
	at com.mongodb.operation.MixedBulkWriteOperation$Run$RunExecutor.execute(MixedBulkWriteOperation.java:647)
	at com.mongodb.operation.MixedBulkWriteOperation$Run.execute(MixedBulkWriteOperation.java:400)
	at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:180)
	at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:169)
	at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:232)
	at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:223)
	at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:169)
	at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:75)
	at com.mongodb.Mongo.execute(Mongo.java:827)
	at com.mongodb.Mongo$2.execute(Mongo.java:810)
	at com.mongodb.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:515)
	at com.mongodb.MongoCollectionImpl.update(MongoCollectionImpl.java:508)
	at com.mongodb.MongoCollectionImpl.updateOne(MongoCollectionImpl.java:355)
	at com.gennubi.nubido.service.da.node.mongo.MongoNode.dbCreateNode(MongoNode.java:102)
	at com.gennubi.nubido.service.da.node.DBNode.putAndGetNode(DBNode.java:140)
	at com.gennubi.nubido.service.da.node.DBNode.putAndGetNode(DBNode.java:115)
	at com.gennubi.nubido.service.command.app.CreateTaskCommand.doCreateTask(CreateTaskCommand.java:179)
	at com.gennubi.nubido.service.command.app.CreateTaskCommand.execute(CreateTaskCommand.java:151)
	at com.gennubi.nubido.service.rest.internal.NubiDoAppServer.handleCreateTask(NubiDoAppServer.java:548)
	at com.gennubi.nubido.service.rest.internal.NubiDoAppServer$$Lambda$92/1520850897.handle(Unknown Source)
	at io.vertx.ext.web.impl.BlockingHandlerDecorator.lambda$handle$0(BlockingHandlerDecorator.java:48)
	at io.vertx.ext.web.impl.BlockingHandlerDecorator$$Lambda$124/996089465.handle(Unknown Source)
	at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:303)
	at io.vertx.core.impl.ContextImpl$$Lambda$126/2034093149.run(Unknown Source)
	at io.vertx.core.impl.OrderedExecutorFactory$OrderedExecutor.lambda$new$0(OrderedExecutorFactory.java:94)
	at io.vertx.core.impl.OrderedExecutorFactory$OrderedExecutor$$Lambda$23/835227336.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Comment by Kelsey Schubert [ 28/Nov/16 ]

Hi kamalkalra,

The logs indicate that there was an authorization error. So we can better understand what is going on would you please provide the series of mongo shell commands that are successfully executed (including the steps to authenticate the user)? And describe where the failure occurs using the JAVA driver?

Would you also please clarify how you upgraded to MongoDB 3.2? (e.g. did you perform an initial sync?)

Thank you,
Thomas

Comment by Kamal Kalra [ 14/Nov/16 ]

Hi Thomas,
I've uploaded the logs for when the failure occurs. I'm using driver version 3.3 for either MongoDB 3.0 or 3.2.10.

Thanks,
Kamal

Comment by Kelsey Schubert [ 14/Nov/16 ]

Hi kamalkalra,

Thank you for the report. I'd like to confirm a few details with you.

My understanding is that you upgraded only the mongod binaries from 3.0.0 to 3.2.10, and kept the Java Driver at version 3.2.1. Is this correct?

Additionally, would you please upload the logs of the mongod that you are unable to auth against?

Thank you for your help,
Thomas

Generated at Thu Feb 08 04:13:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.