[CDRIVER-2549] (kerberos) failed to connect if the principle name has '@' character Created: 15/Mar/18  Updated: 17/Mar/19  Resolved: 08/Jun/18

Status: Closed
Project: C Driver
Component/s: libmongoc
Affects Version/s: 1.7.0
Fix Version/s: 1.11.0

Type: Bug Priority: Major - P3
Reporter: winnie_quest Assignee: Matt Broadstone
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File addUser.png     PNG File add_to_kdc.png     PNG File env_KRB5_TRACE.png     PNG File kdc.png     PNG File kinit.png     PNG File klist.png     PNG File log_shell.png     File mongod.conf     PNG File regedit.png     PNG File server_log.png     PNG File where.png    
Issue Links:
Depends
Related
related to SERVER-33993 (kerberos) failed to connect if the p... Closed

 Description   

nI added a user in KDC server: addprinc user1@a@KER.COM
note: user1@a is the username (I use \ character as the escape character), and KER.COM is the realm name.

then added the user in mongodb server:

db.getSiblingDB("$external").createUser(
{
user : "user1@a@KER.COM",
roles: [ { role: "userAdminAnyDatabase", db: "admin" }, { role: "readWrite", db: "quest_stage" } ]
}
)

through mongodb shell, I can connect with the following cmd successfully:

mongo --host mon.ker.com --authenticationMechanism=GSSAPI --authenticationDatabase='$external' --username "user1@a@KER.COM"

but with mongo c driver in my program:

		auto client = mongocxx::client{ mongocxx::uri{ "mongodb://user1%5C%40a%40KER.COM:123@mon.ker.com/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } }; //ensure to use FQDN instead of IP ADDRESS

or

auto client = mongocxx::client{ mongocxx::uri{ "mongodb://user1%5C%5C%40a%40KER.COM:123@mon.ker.com/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } }; //ensure to use FQDN instead of IP ADDRESS

can't connect, and get error in log:

[info@mongocxx] libmongoc logging callback enabled
[error@mongoc] SSPI: InitializeSecurityContext: The specified target is unknown
or unreachable
 
[debug@cluster] Authentication failed: Received invalid SSPI data.



 Comments   
Comment by A. Jesse Jiryu Davis [ 08/Jun/18 ]

Winnie, please accept our apologies, we're not investigating this further with you. Our own research has convinced us that the MongoDB C Driver does not have a bug related to Kerberos usernames with "@" in them, and we can't spend more time debugging the configuration of your other systems.

Comment by winnie_quest [ 18/May/18 ]

for your two question:

1) On my client machine, 

     before I connect from my program which uses mongodb cxx driver, I have executed a command: "ksetup /addkdc TICQUEST.CLOUDAPP.NET 10.101.xxx.xxx"  in my client windows machine,  ksetup.exe is in this path: C:\Windows\System32\ksetup.exe, I think it would make the driver know where the KDC is.

 2) I have linux machine,but I haven't done linux development before, and the driver I compiled on windows should not be able to be used on linux machine.

Comment by Benjamin Caimano (Inactive) [ 17/May/18 ]

Okay, so I have two questions:

  • On your client machine, what steps did you do to inform that machine of at what ip/hostname it could find your KDC? (These would be different for MIT Kerberos and ActiveDirectory.)
  • Do you have any linux nodes on which you can test your driver as well? (In a pinch, the node on which you run your mongod server will do.)

I'm going to set up a windows node so that I can verify the client program there as well. It may take me a bit of time.

(Re: "MIT Kerberos-based mongod server", I mean a mongod server that was built on linux with the default parameters.)

Comment by winnie_quest [ 17/May/18 ]

hi Ben, 

   for your question "Are your MIT kerberos-based mongod server and SSPI-based client program talking to the same KDC?"

   answer: we have only one KDC in our testing environment. what do you mean by "MIT kerberos-based mongod server"?

   my mongod server is installed on a ubuntu machine.  on that machine, I installed keberos client with "sudo apt-get install krb5-user libpam-krb5 libpam-ccreds auth-client-config libsasl2-modules-gssapi-mit" command , and I installed kerberos server on another ubuntu machine with "sudo apt-get install krb5-kdc krb5-admin-server" command.

    I don't think I have installed MIT keberos on mongod server.

   on the client machine which my program runs, I installed MIT kerberos, the client machine is a windows machine.

  

   

Comment by Benjamin Caimano (Inactive) [ 16/May/18 ]

The SSPI option is about what I'd expect to see.

I think I get what is going on here. The only situation in which you won't see any KRB5_TRACE output from an authentication attempt is when kerberos authentication is not available for the client. I can provoke this with my linux client by removing my ticket cache with the 'kdestroy' command. The result looks like this (without any KRB5_TRACE output):

$ kdestroy
$ env KRB5_TRACE=/tmp/client.trace ./quest-client test%5C%40 'PIZZABAGEL.LOC' localhost
[info@mongocxx] libmongoc logging callback enabled
Trying mongo URI: 'mongodb://test%5C%40%40PIZZABAGEL.LOC@localhost/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb'.
[debug@cluster] Authentication failed: SASL Failure: (-1): generic failure
connection failed: SASL Failure: (-1): generic failure: generic server error

This gets less obvious with SSPI because the kerberos TGT is acquired during runtime using your principal:password combo (see here). This code is a strict branch, so you will only use MIT Kerberos/Cyrus or ActiveDirectory/SSPI, not both. Which brings me to a silly question: Are your MIT kerberos-based mongod server and SSPI-based client program talking to the same KDC? This would be evident in the log files for your MIT Kerberos kdc if you don't have any lines that are relevant to the client principal. For instance, my '/var/log/krb5kdc.log' has lines like this:

May 16 14:57:06 pizzabagel krb5kdc[9575](info): AS_REQ (8 etypes {18 17 20 19 16 23 25 26}) ::1: ISSUE: authtime 1526497026, etypes {rep=18 tkt=18 ses=18}, test\@@PIZZABAGEL.LOC for krbtgt/PIZZABAGEL.LOC@PIZZABAGEL.LOC
May 16 14:57:11 pizzabagel krb5kdc[9575](info): TGS_REQ (8 etypes {18 17 20 19 16 23 25 26}) ::1: ISSUE: authtime 1526497026, etypes {rep=18 tkt=18 ses=18}, test\@@PIZZABAGEL.LOC for mongodb/pizzabagel@PIZZABAGEL.LOC

Comment by winnie_quest [ 16/May/18 ]

hi Ben,
you are right, I compiled mongo c driver with:
cmake -G "Visual Studio 14 2015 Win64" "-DENABLE_SSL=OPENSSL" "-DENABLE_SASL=SSPI" "-DCMAKE_INSTALL_PREFIX=C:\mongo-c-driver" "-DCMAKE_PREFIX_PATH=C:\mongo-c-driver" "-DCMAKE_BUILD_TYPE=Release". Well, I didn't do much investigation for the ENABLE_SASL option, just follow the mongodb c driver compile manual to set it to "SSPI", I can't find detailed explanation for the meaning of this value in manual.

for the KRB5_TRACE output, yes, now I can get it. but I can't get any log info if I tried to connect with the at-signed principle, I can only see the log info if I connect with no at-signed principle:
[27923] 1526421084.934542: Negotiated enctype based on authenticator: aes256-cts
[27923] 1526421084.934563: Authenticator contains subkey: aes256-cts/9D7D
[27923] 1526421084.934607: Creating AP-REP, time 1526421084.130, subkey aes256-cts/C1EE, seqnum 596294609
[27923] 1526434572.769450: Decrypted AP-REQ with server principal mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET: aes256-cts/00E2
[27923] 1526434572.769495:AP-REQ ticket: testkrb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET -> mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET, session key aes256-cts/AF5E
[27923] 1526434572.795100: Negotiated enctype based on authenticator: aes256-cts
[27923] 1526434572.795123: Authenticator contains subkey: aes256-cts/985F
[27923] 1526434572.795160: Creating AP-REP, time 1526434572.416, subkey aes256-cts/EFEE, seqnum 1030166959

Comment by Benjamin Caimano (Inactive) [ 15/May/18 ]

Welcome back, Winnie.

So I notice you have MIT Kerberos for Windows installed and preferred in your executable path. By default, on windows machines, the CXX driver will use SSPI/ActiveDirectory for its Kerberos implementation. This also leads me to a fact of which I was not aware: There is no kinit equivalent in SSPI. Your password in the URI is the correct way to acquire the kerberos ticket via SSPI. If you were to use the klist.exe in Windows/System32, it will probably show that your ticket cache is only for the principals associated with your user account. (For a linux client, the password method does not work because the implementation depends on active tickets in cache.) I apologize for not providing this information earlier. To understand these details, I had to confer with both Jesse within the driver team and Spencer Jackson within the server team.

All of that said, I am curious what principal your mongod server sees when you attempt client connection for your at-signed principal. Are you now able to acquire KRB5_TRACE output?

Comment by winnie_quest [ 15/May/18 ]

hi Ben,
sorry for my late reply, I just come back from business trip.

1) I noticed your "Modified Client Code" doesn't include the password for kerberos principle in your connection uri, which means you have to do "kinit principle_name" first to get the ticket before you tried to connect. but on windows, I think we don't to do this if we use mongodb cxx driver, we just put the password in the uri, the driver would help us to get the ticket if my understanding is correct.
2) well, anyway, I had a try to get ticket with "kinit" on windows. my my working environment is complex, I have many klist.exe, and kinit.exe. look at this: , and I tried with the username without '@' first, see , but even I have got the ticket , I can't connect successfully without the password in uri,
failed:

	auto client = mongocxx::client{ mongocxx::uri{ "mongodb://testkrb%2Fdb-mongo-kerberos.cloudapp.net%40TICQUEST.CLOUDAPP.NET@db-mongo-kerberos.cloudapp.net:27017/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } };

mongodb cxx driver log shows:
[info@mongocxx] libmongoc logging callback enabled
[error@mongoc] SSPI: InitializeSecurityContext: The logon attempt failed

[debug@cluster] Authentication failed: Received invalid SSPI data.

then I added password, it runs successful:

	auto client = mongocxx::client{ mongocxx::uri{ "mongodb://testkrb%2Fdb-mongo-kerberos.cloudapp.net%40TICQUEST.CLOUDAPP.NET:{color:red}123{color}@db-mongo-kerberos.cloudapp.net:27017/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } }; 

so I feel quite puzzlled, which ticket cache does mongodb cxx driver uses

by the way, in the krv5 log file, I can't see any connection info of this user if it connects failed :
dbadmin@db-mongo-kerberos:/lib/systemd/system$ sudo vi /var/log/mongodb/mongod.krb5.log

[27923] 1526368593.809740: Retrieving mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
[27923] 1526368593.810335: Retrieving mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
[27923] 1526368593.810845: Retrieving mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
[27923] 1526368593.811358: Retrieving mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
[27923] 1526368593.811862: Retrieving mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
[27923] 1526368593.812473: Retrieving mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
[27923] 1526368593.813568: Retrieving mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
[27923] 1526368593.816184: Retrieving mongodb/db-mongo-kerberos.cloudapp.net@TICQUEST.CLOUDAPP.NET from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success

Comment by Benjamin Caimano (Inactive) [ 14/May/18 ]

Re: Verifying client code. I've modified the code provided ever so slightly to take in principal/hostname components. It seems to work for me. I've attached the modified code, a snippet of it functioning as a client, and the accompanying server log.

Modified Client Code

Show all

#include <cstdlib>
#include <iostream>
#include <string>
 
#include <bsoncxx/builder/stream/document.hpp>
#include <bsoncxx/json.hpp>
 
#include <mongocxx/client.hpp>
#include <mongocxx/instance.hpp>
 
#include <bsoncxx/builder/stream/array.hpp>
#include <bsoncxx/stdx/make_unique.hpp>
#include <mongocxx/logger.hpp>
#include <mongocxx/options/client.hpp>
#include <mongocxx/options/ssl.hpp>
#include <mongocxx/pool.hpp>
 
using namespace mongocxx;
using bsoncxx::builder::stream::document;
using bsoncxx::builder::stream::finalize;
 
class mylogger final : public mongocxx::logger {
public:
  explicit mylogger(std::ostream *stream) : _stream(stream) {}
 
  void operator()(mongocxx::log_level level, mongocxx::stdx::string_view domain,
                  mongocxx::stdx::string_view message) noexcept override {
    if (level > mongocxx::log_level::k_trace)
      return;
    *_stream << '[' << mongocxx::to_string(level) << '@' << domain << "] "
             << message << '\n';
  }
 
private:
  std::ostream *const _stream;
};
 
int main(int argc, char **argv) {
  mongocxx::instance inst{bsoncxx::stdx::make_unique<mylogger>(&std::cout)};
  try {
    char * user = argv[1];
    char * domain = argv[2];
    char * host = argv[3];
    char uriBuff[1024]{};
    std::sprintf(uriBuff,
                 "mongodb://"
                 "%s%%40%s@%s"
                 "/?authMechanism=GSSAPI"
                 "&authMechanismProperties=SERVICE_NAME:mongodb",
                 user, domain, host);
    std::cerr << "Trying mongo URI: '" << uriBuff << "'.\n";
 
    auto client = mongocxx::client{mongocxx::uri{uriBuff}};
    mongocxx::database mongo_db = client["quest_stage"];
    std::string _mongoDBVersion = "";
 
    bsoncxx::document::value command = bsoncxx::builder::stream::document{}
                                       << "buildInfo" << 1
                                       << bsoncxx::builder::stream::finalize;
 
    auto response = mongo_db.run_command(command.view());
    _mongoDBVersion = response.view()["version"].get_utf8().value.to_string();
  } catch (const std::exception &xcp) {
    std::cerr << "connection failed: " << xcp.what() << "\n";
    return EXIT_FAILURE;
  }
 
  return 0;
}

Client Shell

$ klist
Ticket cache: KCM:1000:82148
Default principal: test\@@PIZZABAGEL.LOCValid starting       Expires              Service principal
05/14/2018 14:23:34  05/15/2018 14:23:31  mongodb/pizzabagel@
05/14/2018 14:23:31  05/15/2018 14:23:31  krbtgt/PIZZABAGEL.LOC@PIZZABAGEL.LOC
05/14/2018 14:23:34  05/15/2018 14:23:31  mongodb/pizzabagel@PIZZABAGEL.LOC
$ env KRB5_TRACE=/dev/stderr ./quest-client test%5C%40 PIZZABAGEL.LOC localhost
[info@mongocxx] libmongoc logging callback enabled
Trying mongo URI: 'mongodb://test%5C%40%40PIZZABAGEL.LOC@localhost/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb'.
[34431] 1526325577.409805: ccselect module realm chose cache KCM:1000:82148 with client principal test\@@PIZZABAGEL.LOC for server principal mongodb/pizzabagel@PIZZABAGEL.LOC
[34431] 1526325577.409806: Getting credentials test\@@PIZZABAGEL.LOC -> mongodb/pizzabagel@ using ccache KCM:1000:82148
[34431] 1526325577.409807: Retrieving test\@@PIZZABAGEL.LOC -> mongodb/pizzabagel@ from KCM:1000:82148 with result: 0/Success
[34431] 1526325577.409809: Creating authenticator for test\@@PIZZABAGEL.LOC -> mongodb/pizzabagel@, seqnum 463029457, subkey aes256-cts/63B4, session key aes256-cts/59E2
[34431] 1526325577.409814: Read AP-REP, time 1526325577.409810, subkey aes256-cts/0E81, seqnum 770275228
 

Server Shell/Log

$ env KRB5_TRACE=/dev/stderr ~/scripts/mongod-with-auth.sh 
2018-05-14T15:19:34.082-0400 I CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
[34402] 1526325574.92244: Retrieving mongodb/pizzabagel@PIZZABAGEL.LOC from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
[34402] 1526325574.92248: Retrieving mongodb/pizzabagel@PIZZABAGEL.LOC from FILE:/etc/krb5.keytab (vno 0, enctype 0) with result: 0/Success
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten] MongoDB starting : pid=34402 port=27017 dbpath=/data/db 64-bit host=pizzabagel
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten] db version v3.7.9-150-gf0f8b61516
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten] git version: f0f8b61516e8355df6907820dead4be0b26cf07f
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.0h-fips  27 Mar 2018
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten] allocator: tcmalloc
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten] modules: enterprise 
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten] build environment:
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten]     distarch: x86_64
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten]     target_arch: x86_64
2018-05-14T15:19:34.108-0400 I CONTROL  [initandlisten] options: { security: { authorization: "enabled" }, setParameter: { authenticationMechanisms: "GSSAPI", saslHostName: "pizzabagel" } }
2018-05-14T15:19:34.108-0400 I STORAGE  [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2018-05-14T15:19:34.108-0400 I STORAGE  [initandlisten] 
2018-05-14T15:19:34.108-0400 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2018-05-14T15:19:34.108-0400 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2018-05-14T15:19:34.108-0400 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=31530M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2018-05-14T15:19:34.623-0400 I STORAGE  [initandlisten] WiredTiger message [1526325574:622994][34402:0x7fd2d2329c80], txn-recover: Main recovery loop: starting at 648/7296
2018-05-14T15:19:34.710-0400 I STORAGE  [initandlisten] WiredTiger message [1526325574:710275][34402:0x7fd2d2329c80], txn-recover: Recovering log 648 through 649
2018-05-14T15:19:34.766-0400 I STORAGE  [initandlisten] WiredTiger message [1526325574:766812][34402:0x7fd2d2329c80], txn-recover: Recovering log 649 through 649
2018-05-14T15:19:34.811-0400 I STORAGE  [initandlisten] WiredTiger message [1526325574:811012][34402:0x7fd2d2329c80], txn-recover: Set global recovery timestamp: 0
2018-05-14T15:19:34.830-0400 I RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0)
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] 
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] ** NOTE: This is a development version (3.7.9-150-gf0f8b61516) of MongoDB.
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] **       Not recommended for production.
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] 
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] ** WARNING: This server is bound to localhost.
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] **          Remote systems will be unable to connect to this server. 
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] **          Start the server with --bind_ip <address> to specify which IP 
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] **          addresses it should serve responses from, or with --bind_ip_all to
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] **          bind to all interfaces. If this behavior is desired, start the
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] **          server with --bind_ip 127.0.0.1 to disable this warning.
2018-05-14T15:19:34.896-0400 I CONTROL  [initandlisten] 
2018-05-14T15:19:34.934-0400 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
2018-05-14T15:19:34.935-0400 I NETWORK  [initandlisten] waiting for connections on port 27017
2018-05-14T15:19:37.404-0400 I NETWORK  [listener] connection accepted from 127.0.0.1:44220 #1 (1 connection now open)
2018-05-14T15:19:37.404-0400 I NETWORK  [conn1] received client metadata from 127.0.0.1:44220 conn1: { driver: { name: "mongoc / mongocxx", version: "1.10.0-dev / 3.2.0" }, os: { type: "Linux", name: "Fedora", version: "27", architecture: "x86_64" }, platform: "cfg=0x215680e9 posix=200809 stdc=201112 CC=GCC 7.3.1 20180303 (Red Hat 7.3.1-5) CFLAGS="" LDFLAGS=""" }
[34402] 1526325577.496396: Decrypted AP-REQ with server principal mongodb/pizzabagel@PIZZABAGEL.LOC: aes256-cts/D81A
[34402] 1526325577.496397: AP-REQ ticket: test\@@PIZZABAGEL.LOC -> mongodb/pizzabagel@PIZZABAGEL.LOC, session key aes256-cts/59E2
[34402] 1526325577.496398: Negotiated enctype based on authenticator: aes256-cts
[34402] 1526325577.496399: Authenticator contains subkey: aes256-cts/63B4
[34402] 1526325577.496401: Creating AP-REP, time 1526325577.409810, subkey aes256-cts/0E81, seqnum 770275228
2018-05-14T15:19:37.555-0400 I ACCESS   [conn1] Successfully authenticated as principal test\@@PIZZABAGEL.LOC on $external
2018-05-14T15:19:37.562-0400 I NETWORK  [conn1] end connection 127.0.0.1:44220 (0 connections now open)

Comment by Benjamin Caimano (Inactive) [ 10/May/18 ]

Re: Your ubuntu server. The setup gets a bit more complex because it looks like you're running our provided systemd 'mongod.service' file. It won't pick up the KRB5_TRACE environment variable because it uses the environment from systemd instead of your terminal shell.

I believe that you'll want to modify the file at '/lib/systemd/system/mongod.service'. You'll need to add a line setting an environment variable after '[Service]'. Putting this in context, I've added the line starting with 'Environment=' here:

[Service]
User=mongodb
Group=mongodb
Environment=KRB5_TRACE=/var/log/mongodb/mongod.krb5.log
ExecStart=/usr/bin/mongod --config /etc/mongod.conf

After you do this, you'll need to reload the service file and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart mongod

Comment by Benjamin Caimano (Inactive) [ 10/May/18 ]

Re: Your windows client. It seems that the flag I had you set in the registry only logs errors. It does not log successful auth events. This means that it is successfully becoming whatever principal is your primary. The klist command I mentioned should hopefully show which principal is your primary and thus is used.

Comment by Benjamin Caimano (Inactive) [ 10/May/18 ]

Apologies on my slow speed here, I'm in the midst of a similar week-long commitment.

I'll break up my responses for clarity. To start with, I'll happily give your CXX driver code a test.

Comment by winnie_quest [ 08/May/18 ]

I am going to be on business trip from tomorrow, so I won't give feedback quickly in next 7 days.

Comment by winnie_quest [ 08/May/18 ]

well, I am wondering if you can reproduce my issue by your own testing program using CXX driver.
I would like to provide my c++ program for you to reference as well:

 
#include <cstdlib>
#include <iostream>
#include <string>
 
 
#include <bsoncxx/builder/stream/document.hpp>
#include <bsoncxx/json.hpp>
 
#include <mongocxx/client.hpp>
#include <mongocxx/instance.hpp>
 
#include <mongocxx/options/client.hpp>
#include <mongocxx/options/ssl.hpp>
#include <bsoncxx/builder/stream/array.hpp>
#include <mongocxx/pool.hpp>
#include <bsoncxx/stdx/make_unique.hpp>
#include <mongocxx/logger.hpp>
 
using namespace mongocxx;
using bsoncxx::builder::stream::document;
using bsoncxx::builder::stream::finalize;
 
 
 
 
	class mylogger final : public mongocxx::logger {
	public:
		explicit mylogger(std::ostream* stream) : _stream(stream) {
		}
 
		void operator()(mongocxx::log_level level, mongocxx::stdx::string_view domain,
			mongocxx::stdx::string_view message) noexcept override {
			if (level > mongocxx::log_level::k_trace) return;
			*_stream << '[' << mongocxx::to_string(level) << '@' << domain << "] " << message << '\n';
		}
 
	private:
		std::ostream* const _stream;
	};
 
 
 
int main()
{
	mongocxx::instance inst{ bsoncxx::stdx::make_unique<mylogger>(&std::cout) };
    try {
		auto client = mongocxx::client{ mongocxx::uri{ "mongodb://user1%5C%40a%40TICQUEST.CLOUDAPP.NET:123@db-mongo-kerberos.cloudapp.net:27017/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } }; //ensure to use FQDN instead of IP ADDRESS
        mongocxx::database mongo_db = client["quest_stage"];
        std::string _mongoDBVersion = "";
 
		bsoncxx::document::value command = bsoncxx::builder::stream::document{}
		<< "buildInfo" << 1 << bsoncxx::builder::stream::finalize;
 
		auto response = mongo_db.run_command(command.view());
		_mongoDBVersion = response.view()["version"].get_utf8().value.to_string();
		}
	catch (const std::exception& xcp) {
		std::cout << "connection failed: " << xcp.what() << "\n";
		return EXIT_FAILURE;
	}
 
	 
    return 0;
}
 

Comment by winnie_quest [ 08/May/18 ]

hi Ben,
1. about KRB5_TRACE:
I am not starting mongod by the command mongod --auth xx ....., instead, I just start mongod service (the corresponding command line is: /usr/bin/mongod --config /etc/mongod.conf, I have uploaded the configuration file , see mongod.conf .
I did this:

dbadmin@db-mongo-kerberos:/tmp$ sudo vi /etc/mongod.conf
dbadmin@db-mongo-kerberos:/tmp$ export KRB5_TRACE=/tmp/kinit_trace
dbadmin@db-mongo-kerberos:/tmp$ echo $KRB5_TRACE
/tmp/kinit_trace
dbadmin@db-mongo-kerberos:/tmp$ sudo service mongod restart
dbadmin@db-mongo-kerberos:/tmp$ ps -aux | grep mongod
mongodb   26774 12.1  2.0 1110136 80672 ?       SLsl 05:42   0:02 /usr/bin/mongod --config /etc/mongod.conf
dbadmin   26852  0.0  0.0  12944   936 pts/2    S+   05:43   0:00 grep --color=auto mongod
dbadmin@db-mongo-kerberos:/tmp$ ll kinit_trace
-rw------- 1 root dbadmin {color:red}0{color} May  8 05:27 kinit_trace

but nothing was written into kinit_trace file, what's wrong?

then I tried to connect to mongod server with other username without '@':

	auto client = mongocxx::client{ mongocxx::uri{ "mongodb://testkrb%2Fdb-mongo-kerberos.cloudapp.net%40TICQUEST.CLOUDAPP.NET:testkrb@db-mongo-kerberos.cloudapp.net:27017/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } }; 

I can connect successfully.

but if I connect to mongod server with the username with '@', I failed:

auto client = mongocxx::client{ mongocxx::uri{ "mongodb://user1%5C%40a%40TICQUEST.CLOUDAPP.NET:123@db-mongo-kerberos.cloudapp.net:27017/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } }; //ensure to use FQDN instead of IP ADDRESS

and I used mongodb driver's log mechanism, the log shows:

[info@mongocxx] libmongoc logging callback enabled
[error@mongoc] SSPI: InitializeSecurityContext: The specified target is unknown
or unreachable
 
[debug@cluster] Authentication failed: Received invalid SSPI data.

2. about kerberos event log:
I have added the regedit value according to here,see , and restarted my computer , but I after i have run my program I can't find any new error in Event viewer -> windows logs ->system. I did find some kerberos error, but all of them happens before I started my program, so I think it doesn't help, anyway, I just paste some of them:

A Kerberos Error Message was received:
 on logon session 
 Client Time: 
 Server Time: 6:2:25.0000 5/8/2018 Z
 Error Code: 0xe KDC_ERR_ETYPE_NOTSUPP
 Extended Error: 
 Client Realm: 
 Client Name: 
 Server Realm: PROD.QUEST.CORP
 Server Name: krbtgt/PROD.QUEST.CORP
 Target Name: krbtgt/PROD.QUEST.CORP@PROD.QUEST.CORP
 Error Text: 
 File: 9
 Line: fdb
 Error Data is in record data.

A Kerberos Error Message was received:
 on logon session 
 Client Time: 
 Server Time: 6:0:39.0000 5/8/2018 Z
 Error Code: 0x7  KDC_ERR_S_PRINCIPAL_UNKNOWN
 Extended Error: 
 Client Realm: 
 Client Name: 
 Server Realm: PROD.QUEST.CORP
 Server Name: HTTP/webmail.quest.com
 Target Name: HTTP/webmail.quest.com@PROD.QUEST.CORP
 Error Text: 
 File: 9
 Line: fdb
 Error Data is in record data.

Comment by Benjamin Caimano (Inactive) [ 07/May/18 ]

winnie_quest,

Thank you for providing those OS details.

The KRB5_TRACE variable will only be useful on your Ubuntu machine that runs your mongod. Generally, you can use it like this:

$ export KRB5_TRACE=/dev/stderr
$ mongod "${MY_MONGOD_ARGS[@]}" 

(The part in quotes should be replaced by whatever arguments you would normally use. I'm sure there is at least --auth and some other details.) The trace will tell us with which principals your server comes in contact.

I'd be more interested in the trace details for your client. However, getting trace details for Windows kerberos auth is trickier. If you have registry/event log access for your windows machine, you can have kerberos trace to the event log using the steps described here. The klist output would be useful for the client because it should tell us what your client thinks your default principal is. I believe you should have access to the form of klist described here. It seems to vary by Windows version, so let me know if you cannot find it.

Comment by winnie_quest [ 07/May/18 ]

hi Ben,
my program using mongo CXX driver is running on windows, mongo server(mongod) is running ubuntu, kerberos server is running on another ubuntu machine.
I didn't quite understand what you said, are you asking me to provide the "klist' output on the mongo server(mongod) machine? please take a look at
my mongo server(mongod) machine doesn't have KRB5_TRACE environment, see

Comment by Benjamin Caimano (Inactive) [ 03/May/18 ]

I've been able to reproduce my steps for SERVER-33993 with the C driver:

MongoDB Enterprise > use $external
MongoDB Enterprise > show users
...
{{ "_id" : "$external.test\\@@PIZZABAGEL.LOC", "user" : "test\\@@PIZZABAGEL.LOC", "db" : "$external", "roles" : [ { "role" : "read", "db" : "test" }, ... ], "mechanisms" : [ "external" ]}
...

$ env KRB5_TRACE=/dev/stderr ./example-client 'mongodb://test%5C%40%40PIZZABAGEL.LOC@localhost/?authMechanism=GSSAPI'
[10652] 1525368678.440222: ccselect module realm chose cache KCM:1000:82148 with client principal test\@@PIZZABAGEL.LOC for server principal mongodb/pizzabagel@PIZZABAGEL.LOC
[10652] 1525368678.440223: Getting credentials test\@@PIZZABAGEL.LOC -> mongodb/pizzabagel@ using ccache KCM:1000:82148
[10652] 1525368678.440224: Retrieving test\@@PIZZABAGEL.LOC -> mongodb/pizzabagel@ from KCM:1000:82148 with result: 0/Success
[10652] 1525368678.440226: Creating authenticator for test\@@PIZZABAGEL.LOC -> mongodb/pizzabagel@, seqnum 106307259, subkey aes256-cts/D2B2, session key aes256-cts/3221
[10652] 1525368678.440231: Read AP-REP, time 1525368678.440227, subkey aes256-cts/68A1, seqnum 532144668 

 

Hey winnie_quest, would you be willing to provide the output of klist via the shell you use to run your client and a similar use of the KRB5_TRACE environment variable to mine above? This feels like a subtle configuration issue. (The KRB5_TRACE from your mongod might be helpful as well.) 

Comment by winnie_quest [ 02/May/18 ]

this is my source code for connection

 auto client = mongocxx::client{ mongocxx::uri{ "mongodb://user1%5C%40a%40TICQUEST.CLOUDAPP.NET:123@db-mongo-kerberos.cloudapp.net:27017/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } };

note: I have changed to use another mongodb server, so the realm name in the connection string has been changed.
and I am sure this server could be connected in Kerberos mode, because I can connect to it successfully with other user name:

auto client = mongocxx::client{ mongocxx::uri{ "mongodb://testkrb%2Fdb-mongo-kerberos.cloudapp.net%40TICQUEST.CLOUDAPP.NET:testkrb@db-mongo-kerberos.cloudapp.net:27017/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb" } }; 

Comment by A. Jesse Jiryu Davis [ 02/May/18 ]

Just to be certain, could you please paste here the complete connection string that you use with the C++ driver to connect to MongoDB?

Comment by winnie_quest [ 02/May/18 ]

Hi Jesse, As I mentioned in https://jira.mongodb.org/browse/SERVER-33993?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=1878142#comment-1878142 , both methods(%5C%40 or %5C%5C%40) don't work

Comment by A. Jesse Jiryu Davis [ 01/May/18 ]

Thanks Winnie. I think I gave you incorrect instructions before. Here's the properly escaped version of the URI:

mongodb://user1%5C%40a%40KER.COM:123@mon.ker.com/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb

In the properly escaped URI there is only one "%5C" escape sequence, not two. This URI encodes the following exact username:

user1\@a@KER.COM

The username contains one backslash before the "@" sign.

Comment by winnie_quest [ 27/Apr/18 ]

hi, could you please take a look at my last comment for my ticket on server side, it looks like the driver still has an issue

Comment by winnie_quest [ 20/Mar/18 ]

ok, thanks.
I created a server ticket https://jira.mongodb.org/browse/SERVER-33993 for following this issue.
thanks for your help

Comment by A. Jesse Jiryu Davis [ 19/Mar/18 ]

Winnie, I don't think the C Driver has a bug related to encoding @-signs or backslashes in usernames or passwords.

The username "user1@a@KER.COM" should be encoded as "user1%40a%40KER.COM". Just replace the @-symbols with %40, don't include any backslashes in the username before encoding it, that's not necessary.

A good way to determine how to encode a username or password when you include it in a MongoDB connection string is with Python:

$ python
>>> import urllib
>>> urllib.quote('user1@a')
'user1%40a'

I have confirmed that the C Driver properly unquotes that string before authenticating to the server.

If "user1%40a%40KER.COM" doesn't work, either with the C Driver, or with the mongo shell, please open a SERVER ticket for further investigation.

Comment by winnie_quest [ 19/Mar/18 ]

hi Jesse,
why does this issue set closed?

Comment by winnie_quest [ 16/Mar/18 ]

@ is a reserved character in KDC principle name, if you want to use it , you have to escape it,
I have a case that I don't escape '@' , and got error, see

Comment by A. Jesse Jiryu Davis [ 16/Mar/18 ]

What if you don't put a backslash anywhere at all, does that work? Don't use a backslash in "addprinc", or in "kinit", or anywhere.

Comment by winnie_quest [ 16/Mar/18 ]

hi Jesse,
I made following test:
firstly, I added user1@a@KER.COM into KDC, see . (note: for escape purpose, we have to add a back-slash before @ if the principle name has @)
and then I added the user "user1@a@KER.COM" into mongodb server. see
then try to connect from mongo shell
I can't login successfully from mongo shell, and I have tried to login with driver in my program:
"mongodb://user1%40a%40KER.COM:123@mon.ker.com/?authMechanism=GSSAPI&authMechanismProperties=SERVICE_NAME:mongodb"
also get failed.

the mongodb server shows the log as:

Comment by A. Jesse Jiryu Davis [ 16/Mar/18 ]

Actually I don't think the C Driver has a bug related to encoding @-signs or backslashes in usernames or passwords.

The username "user1@a@KER.COM" should be encoded as "user1%40a%40KER.COM". Just replace the @-symbols with %40, don't include any backslashes in the username before encoding it, that's not necessary.

A good way to determine how to encode a username or password when you include it in a MongoDB connection string is with Python:

$ python
>>> import urllib
>>> urllib.quote('user1@a')
'user1%40a'

Comment by winnie_quest [ 16/Mar/18 ]

the username is "user1" followed by an "@" sign followed by the character "a"?
yes.

and we found for mongo c driver , if the username contains a backslash - '\' (for escaping purpose), the user can't login.
are you going to fix this issue?

Comment by A. Jesse Jiryu Davis [ 15/Mar/18 ]

So, to be very clear, the username is "user1" followed by an "@" sign followed by the character "a"?

As a workaround for this bug, perhaps you could choose a username that doesn't contain an "@" symbol?

Comment by winnie_quest [ 15/Mar/18 ]

it looks Jira would display double-backslash as '\n' .
acutally, the username in the command for creating a user in mongodb has double-backslash after "user1". the username in mongo shell connection has double-backslash after "user1" as well.

Generated at Wed Feb 07 21:15:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.