[CDRIVER-2000] GSSAPI Authentication failure on RHEL7.1 (ppc64le) Created: 25/Jan/17  Updated: 01/May/18  Resolved: 01/May/18

Status: Closed
Project: C Driver
Component/s: auth
Affects Version/s: None
Fix Version/s: Not Planned

Type: Bug Priority: Minor - P4
Reporter: Hannes Magnusson Assignee: Roberto Sanchez
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to CDRIVER-2133 Coverity analysis defect 100116: Out-... Closed
related to CDRIVER-2040 Run Cyrus SASL tests on Windows Closed

 Description   

When connecting to ldaptest from Solaris and the RHEL7.1 PPC64le evergreen instances, we hit strange problems:

2017/01/24 22:44:44.0494: [23523]:    DEBUG:      cluster: Authentication failed: SASL Failure: no payload provided from server.: SASL(-1): generic failure: Unable to find a callback: 32775

which is

include/sasl.h:#define SASL_CB_CANON_USER (0x8007)

Implementing the callback resolves the sasl error, but we still get no payload from the server after our initial saslStart command:

    --> { "saslStart" : 1, "mechanism" : "GSSAPI", "payload" :
    "YIICmAYJKoZIhvcSAQICAQBuggKHMIICg6ADAgEFoQMCAQ6iBwMFAAAAAACjggGTYYIBjzCCAYugAwIBBaETGxFMREFQVEVTVC4xMEdFTi5DQ6InMCWgAwIBA6EeMBwbB21vbmdvZGIbEWxkYX
    "autoAuthorize" : 1 }
    <-- { "conversationId" : 1, "done" : false, "payload" : "", "ok" : 1.0 }



 Comments   
Comment by A. Jesse Jiryu Davis [ 01/May/18 ]

After a long investigation we've decided not to fix this. Please reopen if you encounter this issue.

Comment by Kevin Albertson [ 01/Feb/18 ]

Note our evergreen config.yml has a comment for a test that should be included after this is resolved, here: https://github.com/mongodb/mongo-c-driver/blob/4d30eb7508f79fd334a04f26fa440870f9eeb2f2/.evergreen/config.yml#L8009

Comment by Roberto Sanchez [ 22/Dec/17 ]

jesse, Is there a way to reproduce the failure manually? I found a command in .evergreen/config.yml under run auth tests:

AUTH_HOST='${auth_host}' AUTH_PLAIN='${auth_plain}' AUTH_MONGODBCR='${auth_mongodbcr}' AUTH_GSSAPI='${auth_gssapi}' AUTH_CROSSREALM='${auth_crossrealm}' AUTH_GSSAPI_UTF8='${auth_gssapi_utf8}' ATLAS_FREE='${atlas_free}' ATLAS_REPLSET='${atlas_replset}' ATLAS_SHARD='${atlas_shard}' sh .evergreen/run-auth-tests.sh

I also found a prepare kerberos stanza that I think will allow me to create the necessary keytab. However, I am not sure where those variables come from. They are not reflected in any logs that I can see. I would like to be able to create a complete command line so that I can reproduce the problem by hand.

Additionally, the rhel71-power8-build distro is not available for spawning Evergreen hosts. Is this something for which I need to be granted permission? Or does this mean that I can only see the results of building on that distro by doing a patch build? If the latter, is it possible to make a patch build only execute on particular distros?

Comment by A. Jesse Jiryu Davis [ 21/Nov/17 ]

I can still reproduce this if I reenable the GSSAPI test on POWER8 little-endian in a patch build:

[2017/11/21 13:48:51.044] 2017/11/21 18:48:51.0042: [22818]:    DEBUG:      cluster: Authentication failed: SASL Failure: no payload provided from server: SASL(0): successful result:
[2017/11/21 13:48:51.045] Ping failure: SASL Failure: no payload provided from server: SASL(0): successful result:
[2017/11/21 13:48:51.051] Command failed: script finished with error: exit status 3

https://evergreen.mongodb.com/task/mongo_c_driver_power8_rhel71_authentication_tests_sasl_cyrus_patch_f81e63dd554946c5acc025ba4e9fffe90fbcfde9_5a146f8ee3c3311bbe00e369_17_11_21_18_25_18

Comment by Hannes Magnusson [ 07/Aug/17 ]

Since this is not a user discovered issue and therefore noone is running into this in the wild, I'm moving this back to Not Planned as I cannot imagine what the dealio is.

Comment by Hannes Magnusson [ 05/May/17 ]

I'm at a loss.

The gssapi sasl plugin keeps sending mongodb/ec2-54-225-237-121.compute-1.amazonaws.com@LDAPTEST.10GEN.CC instead of the correct mongodb/ldaptest.10gen.cc@LDAPTEST.10GEN.CC.
If I change /etc/krb5/krb5.conf and add rdns=false to the [libdefaults] it sends mongodb/ip-10-16-156-85.ec2.internal@LDAPTEST.10GEN.CC instead, but I cannot figureout why it doesn't send ldaptest.10gen.cc like we are passing in to sasl_client_new as service_host.

The installed kerberos on the solaris spawnhost is: Solaris Kerberos (based on MIT Kerberos 5 release 1.6.3) which I tried upgrading to Kerberos 5 release 1.14.5 without it seemingly making any difference (see also this note in the MIT docs)

Upgraded to Cyrus SASL 2.1.26 (needed slight mongoc header re-adjustment), and still nothing.

Comment by Githook User [ 10/Feb/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: CDRIVER-2040 CDRIVER-2000 Disable GSSAPI tests for now
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/e746b2a0723f94560800277036e0ef1404746dfb

Comment by A. Jesse Jiryu Davis [ 30/Jan/17 ]

Tentatively 1.7.0. Awaiting access to the ldaptest server so we can debug the server side.

Comment by Githook User [ 28/Jan/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: Merge branch 'master' of github.com:mongodb/mongo-c-driver

Comment by Githook User [ 27/Jan/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: CDRIVER-2000 Implement sasl logger and fail early on unsupported mechanism
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/113a29b2badeb86ece094aa7fa1e7ef703032c56

Comment by Githook User [ 26/Jan/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: Merge branch 'CDRIVER-1955-gssapi-tests-in-evergreen'

Comment by Githook User [ 26/Jan/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: Merge branch 'CDRIVER-1955-gssapi-tests-in-evergreen'

Comment by Githook User [ 26/Jan/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: Merge branch 'CDRIVER-1955-gssapi-tests-in-evergreen'

Comment by Githook User [ 26/Jan/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: CDRIVER-2000 Disable the sasl test for now
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/7ea1f91aefd60561e909c84b5f734e9c23e7ee75

Comment by Githook User [ 26/Jan/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: CDRIVER-2000 Skip Solaris and RHEL7.1 PPC64le

After implementing SASL_CB_CANON_USER the MongoDB on ldaptest is
replying with empty payload. Disable these until we can find out why

{
"saslStart" : 1,
"mechanism" : "GSSAPI",
"payload" :
"YIICmAYJKoZIhvcSAQICAQBuggKHMIICg6ADAgEFoQMCAQ6iBwMFAAAAAACjggGTYYIBjzCCAYugAwIBBaETGxFMREFQVEVTVC4xMEdFTi5DQ6InMCWgAwIBA6EeMBwbB21vbmdvZGIbEWxkYXB0ZXN0LjEwZ2VuLmNjo4IBRDCCAUCgAwIBEqEDAgEEooIBMgSCAS7
"autoAuthorize" : 1
}
returns:
{
"conversationId" : 1,
"done" : false,
"payload" : "",
"ok" : 1.0
}
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/2ae5f350a4943a87b9fe891f8a3f6f12ceca73b4

Comment by Githook User [ 26/Jan/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: CDRIVER-2000 Implement user canonicalization
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/426d233291b9fdea4b5af3417a8cc125003dc75c

Comment by Hannes Magnusson [ 25/Jan/17 ]

commit bef4fe73a58ab8abea14906784bc76c036bfa74e
Author: Hannes Magnusson <bjori@php.net>
Date:   Tue Jan 24 17:09:27 2017 -0800
 
    Implement user canonicalization
 
diff --git a/src/mongoc/mongoc-sasl.c b/src/mongoc/mongoc-sasl.c
index aa3beb7..ee48d56 100644
--- a/src/mongoc/mongoc-sasl.c
+++ b/src/mongoc/mongoc-sasl.c
@@ -23,6 +23,7 @@
 #include "mongoc-error.h"
 #include "mongoc-sasl-private.h"
 #include "mongoc-util-private.h"
+#include "mongoc-trace-private.h"
 
 
 #ifndef SASL_CALLBACK_FN
@@ -71,6 +72,23 @@ _mongoc_sasl_set_pass (mongoc_sasl_t *sasl, const char *pass)
 
 
 static int
+_mongoc_sasl_canon_user (sasl_conn_t *conn,
+                         void *context,
+                         const char *in,
+                         unsigned inlen,
+                         unsigned flags,
+                         const char *user_realm,
+                         char *out,
+                         unsigned out_max,
+                         unsigned *out_len)
+{
+   TRACE ("Canonicalizing %s (%" PRIu32 ")\n", in, inlen);
+   strcpy (out, in);
+   *out_len = inlen;
+   return SASL_OK;
+}
+
+static int
 _mongoc_sasl_get_user (mongoc_sasl_t *sasl,
                        int param_id,
                        const char **result,
@@ -185,6 +203,7 @@ _mongoc_sasl_init (mongoc_sasl_t *sasl)
       {SASL_CB_AUTHNAME, SASL_CALLBACK_FN (_mongoc_sasl_get_user), sasl},
       {SASL_CB_USER, SASL_CALLBACK_FN (_mongoc_sasl_get_user), sasl},
       {SASL_CB_PASS, SASL_CALLBACK_FN (_mongoc_sasl_get_pass), sasl},
+      {SASL_CB_CANON_USER, SASL_CALLBACK_FN (_mongoc_sasl_canon_user), sasl},
       {SASL_CB_LIST_END}};
 
    BSON_ASSERT (sasl);
@@ -340,6 +359,7 @@ _mongoc_sasl_step (mongoc_sasl_t *sasl,
    BSON_ASSERT (outbuf);
    BSON_ASSERT (outbuflen);
 
+   TRACE ("Running %d, inbuflen: %" PRIu32, sasl->step, inbuflen);
    sasl->step++;
 
    if (sasl->step == 1) {
@@ -352,11 +372,13 @@ _mongoc_sasl_step (mongoc_sasl_t *sasl,
       return false;
    }
 
+   TRACE ("Running %d, inbuflen: %" PRIu32, sasl->step, inbuflen);
    if (!inbuflen) {
       bson_set_error (error,
                       MONGOC_ERROR_SASL,
                       MONGOC_ERROR_CLIENT_AUTHENTICATE,
-                      "SASL Failure: no payload provided from server.");
+                      "SASL Failure: no payload provided from server: %s",
+                      sasl_errdetail (sasl->conn));
       return false;
    }
 

Generated at Wed Feb 07 21:13:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.