[SERVER-57851] Optimize usersInfo calls from mongos to CSRS PRIMARY for Authz User Role resolution Created: 18/Jun/21  Updated: 23/Jun/21

Status: Investigating
Project: Core Server
Component/s: Sharding
Affects Version/s: 4.2.12
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Alex Bevilacqua Assignee: Salman Baset
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Sprint: Security 2021-06-28
Participants:
Case:

 Description   

In a sharded cluster for users with a large number of roles, generating the role graph can result in a BSONObjectTooLarge error when the mongos calls usersInfo.

For example:

# setup cluster
mlaunch init --replicaset --shards 1 --nodes 1 --csrs --binarypath $(m bin 4.2.12) --auth
 
# connect to the mongos
mongo --port 27017 -u user -p password
 
# once connected to the cluster setup as follows:
var array = []
for (var i = 1; i < 20000; i++) {  array.push({ "role" : "readWrite", "db" : "test_" + i }, { "role" : "read", "db" : "test_" + i }, { "role" : "dbAdmin", "db" : "test_" + i }, { "role" : "userAdmin", "db" : "test_" + i }); }
db.getSiblingDB("admin").grantRolesToUser( "user", array );

Running any operation that requires authentication will now fail. For example (using Java):

 
import com.mongodb.ConnectionString;
import com.mongodb.client.*;
import com.mongodb.client.result.InsertOneResult;
 
import org.bson.Document;
 
public class Test {
 
    public static void main(final String[] args) {
 
      ConnectionString mongoURI = new ConnectionString("mongodb://user:password@localhost:27017/admin");
      MongoClient mongoClient = MongoClients.create(mongoURI);
      MongoDatabase db = mongoClient.getDatabase("test");
      MongoCollection<Document> coll =  db.getCollection("foo");
 
      InsertOneResult res = coll.insertOne(new Document());
 
      System.out.println(res);
      mongoClient.close();
    }
}

This will fail with:

Jun. 16, 2021 11:54:23 A.M. com.mongodb.diagnostics.logging.Loggers shouldUseSLF4J
WARNING: SLF4J not found on the classpath.  Logging is disabled for the 'org.mongodb.driver' component
Exception in thread "main" com.mongodb.MongoCommandException: Command failed with error 202 (NetworkInterfaceExceededTimeLimit): 'Request 223 timed out, deadline was 2021-06-16T11:55:31.264-0400, op was RemoteCommand 223 -- target:[localhost:27019] db:admin expDate:2021-06-16T11:55:31.264-0400 cmd:{ usersInfo: [ { user: "user", db: "admin" } ], showPrivileges: true, showCredentials: true, showAuthenticationRestrictions: true, maxTimeMS: 30000 }' on server localhost:27017. The full response is {"ok": 0.0, "errmsg": "Request 223 timed out, deadline was 2021-06-16T11:55:31.264-0400, op was RemoteCommand 223 -- target:[localhost:27019] db:admin expDate:2021-06-16T11:55:31.264-0400 cmd:{ usersInfo: [ { user: \"user\", db: \"admin\" } ], showPrivileges: true, showCredentials: true, showAuthenticationRestrictions: true, maxTimeMS: 30000 }", "code": 202, "codeName": "NetworkInterfaceExceededTimeLimit", "operationTime": {"$timestamp": {"t": 1623858901, "i": 1}}, "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1623858921, "i": 1}}, "signature": {"hash": {"$binary": {"base64": "axnu+Ks4frPv1AukxC6/+5ArxVk=", "subType": "00"}}, "keyId": 6974419404234686470}}}
        at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:175)
        at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:359)
        at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:280)
        at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
        at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
        at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:107)
        at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:62)
        at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:144)
        at com.mongodb.internal.connection.UsageTrackingInternalConnection.open(UsageTrackingInternalConnection.java:51)
        at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.open(DefaultConnectionPool.java:431)
        at com.mongodb.internal.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:115)
        at com.mongodb.internal.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:100)
        at com.mongodb.internal.connection.DefaultServer.getConnection(DefaultServer.java:92)
        at com.mongodb.internal.binding.ClusterBinding$ClusterBindingConnectionSource.getConnection(ClusterBinding.java:119)
        at com.mongodb.client.internal.ClientSessionBinding$SessionBindingConnectionSource.getConnection(ClientSessionBinding.java:135)
        at com.mongodb.internal.operation.OperationHelper.withReleasableConnection(OperationHelper.java:620)
        at com.mongodb.internal.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:185)
        at com.mongodb.internal.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:76)
        at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:195)
        at com.mongodb.client.internal.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:1009)
        at com.mongodb.client.internal.MongoCollectionImpl.executeInsertOne(MongoCollectionImpl.java:470)
        at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:453)
        at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:447)

CSRS Primary Log:

1267:2021-06-16T11:56:33.547-0400 I  COMMAND  [conn27] command admin.system.users command: usersInfo { usersInfo: [ { user: "user", db: "admin" } ], showPrivileges: true, showCredentials: true, showAuthenticationRestrictions: true, maxTimeMS: 30000, $readPreference: { mode: "primaryPreferred" }, $replData: 1, $clusterTime: { clusterTime: Timestamp(1623858901, 1), signature: { hash: BinData(0, 9118C6B05E865794A76F66FD50AF96B414D67022), keyId: 6974419404234686470 } }, $client: { driver: { name: "mongo-java-driver|sync", version: "4.1.1" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "5.4.0-73-generic" }, platform: "Java/AdoptOpenJDK/11.0.1+13", mongos: { host: "alex-ThinkCentre-M70e:27017", client: "127.0.0.1:36816", version: "4.2.12" } }, $configServerState: { opTime: { ts: Timestamp(1623858891, 1), t: 2 } }, $db: "admin" } numYields:0 queryHash:0A298B98 planCacheKey:C2D1BA7E ok:0 errMsg:"BSONObj size: 40929769 (0x27089E9) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: \"admin.user\"" errName:BSONObjectTooLarge errCode:10334 reslen:682 locks:{ ReplicationStateTransition: { acquireCount: { w: 1 } }, Global: { acquireCount: { r: 1 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } }, Mutex: { acquireCount: { r: 1 } } } protocol:op_msg 92282ms


Generated at Thu Feb 08 05:42:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.