[GODRIVER-2147] Automatic FLE decryption does not work with session Created: 03/Sep/21  Updated: 28/Oct/23  Resolved: 04/Oct/21

Status: Closed
Project: Go Driver
Component/s: None
Affects Version/s: None
Fix Version/s: 1.7.3

Type: Bug Priority: Major - P3
Reporter: Elena Flat Assignee: Kevin Albertson
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
is caused by GODRIVER-1832 Sharing a MongoClient for metadata lo... Closed
Related
related to DRIVERS-1937 Support explicit sessions in CSFLE au... Backlog

 Description   

We are using manual mongo field level encryption with automatic decryption.

Starting in go mongo driver 1.5.1 we get this error on decryption: "key vault communication error: session was not created by this client"

To reproduce, I have taken the go mongo driver example 

Test_Example_explictEncryptionWithAutomaticDecryption and modified it to use a session (instead of empty context):

 

// code placeholder
//taken from mongo_driver 1.5.1
func Test_Example_explictEncryptionWithAutomaticDecryption(t *testing.T) {
   // Automatic encryption requires MongoDB 4.2 enterprise, but automatic decryption is supported for all users.
 
   //var localMasterKey []byte // This must be the same master key that was used to create the encryption key.
   kmsProviders := map[string]map[string]interface{}{
      "local": {
         "key": fakeKey,
      },
   }
 
   // The MongoDB namespace (db.collection) used to store the encryption data keys.
   keyVaultDBName, keyVaultCollName := "encryption", "testKeyVault"
   keyVaultNamespace := keyVaultDBName + "." + keyVaultCollName
 
   // Create the Client for reading/writing application data. Configure it with BypassAutoEncryption=true to disable
   // automatic encryption but keep automatic decryption. Setting BypassAutoEncryption will also bypass spawning
   // mongocryptd in the driver.
   autoEncryptionOpts := options.AutoEncryption().
      SetKmsProviders(kmsProviders).
      SetKeyVaultNamespace(keyVaultNamespace).
      SetBypassAutoEncryption(true)
   clientOpts := options.Client().
      ApplyURI("mongodb://localhost:27017").
      SetAutoEncryptionOptions(autoEncryptionOpts)
   client, err := mongo.Connect(context.TODO(), clientOpts)
   if err != nil {
      panic(err)
   }
   defer func() { _ = client.Disconnect(context.TODO()) }()
 
   // Get a handle to the application collection and clear existing data.
   coll := client.Database("test").Collection("coll")
   _ = coll.Drop(context.TODO())
 
   // Set up the key vault for this example.
   keyVaultColl := client.Database(keyVaultDBName).Collection(keyVaultCollName)
   _ = keyVaultColl.Drop(context.TODO())
   // Ensure that two data keys cannot share the same keyAltName.
   keyVaultIndex := mongo.IndexModel{
      Keys: bson.D{{"keyAltNames", 1}},
      Options: options.Index().
         SetUnique(true).
         SetPartialFilterExpression(bson.D{
            {"keyAltNames", bson.D{
               {"$exists", true},
            }},
         }),
   }
   if _, err = keyVaultColl.Indexes().CreateOne(context.TODO(), keyVaultIndex); err != nil {
      panic(err)
   }
 
   // Create the ClientEncryption object to use for explicit encryption/decryption. The Client passed to
   // NewClientEncryption is used to read/write to the key vault. This can be the same Client used by the main
   // application.
   clientEncryptionOpts := options.ClientEncryption().
      SetKmsProviders(kmsProviders).
      SetKeyVaultNamespace(keyVaultNamespace)
   clientEncryption, err := mongo.NewClientEncryption(client, clientEncryptionOpts)
   if err != nil {
      panic(err)
   }
   defer func() { _ = clientEncryption.Close(context.TODO()) }()
 
   // Create a new data key for the encrypted field.
   dataKeyOpts := options.DataKey().SetKeyAltNames([]string{"go_encryption_example"})
   dataKeyID, err := clientEncryption.CreateDataKey(context.TODO(), "local", dataKeyOpts)
   if err != nil {
      panic(err)
   }
 
   // Create a bson.RawValue to encrypt and encrypt it using the key that was just created.
   rawValueType, rawValueData, err := bson.MarshalValue("123456789")
   if err != nil {
      panic(err)
   }
   rawValue := bson.RawValue{Type: rawValueType, Value: rawValueData}
   encryptionOpts := options.Encrypt().
      SetAlgorithm("AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic").
      SetKeyID(dataKeyID)
   encryptedField, err := clientEncryption.Encrypt(context.TODO(), rawValue, encryptionOpts)
   if err != nil {
      panic(err)
   }
 
   // Insert a document with the encrypted field and then find it. The FindOne call will automatically decrypt the
   // field in the document.
   if _, err = coll.InsertOne(context.TODO(), bson.D{{"encryptedField", encryptedField}}); err != nil {
      panic(err)
   }
   var foundDoc bson.M
   //if err = coll.FindOne(context.TODO(), bson.D{}).Decode(&foundDoc); err != nil {
   // panic(err)
   //}
   //fmt.Printf("Decrypted document: %v\n", foundDoc)
   ctx := context.Background()
   err = client.UseSessionWithOptions(ctx, nil, func(sessionContext mongo.SessionContext) error { //sess
      if err = coll.FindOne(sessionContext, bson.D{}).Decode(&foundDoc); err != nil {
         panic(err)
      }
      fmt.Printf("Decrypted document: %v\n", foundDoc)
      return nil
   })
}

This throws error "key vault communication error: session was not created by this client".

When going back to driver 1.5.0-beta1, things work because if 

SetKeyVaultClientOptions are not provided as part of AutoEncryption options on client creation, client is re-used for key vault communication.

When going to driver 1.5.1 (also checked 1.7.2), we get this error.

 

 



 Comments   
Comment by Githook User [ 04/Oct/21 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: GODRIVER-2147 remove session from context in internal CSFLE operations (#762)
Branch: release/1.7
https://github.com/mongodb/mongo-go-driver/commit/b178b283a50471691a58688a4c36bad2689516ef

Comment by Githook User [ 04/Oct/21 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: GODRIVER-2147 remove session from context in internal CSFLE operations (#762)
Branch: master
https://github.com/mongodb/mongo-go-driver/commit/0bef69c1036c6502848a55684093fb38cd66a33a

Comment by Kevin Albertson [ 01/Oct/21 ]

PR: https://github.com/mongodb/mongo-go-driver/pull/762

Comment by Kevin Albertson [ 16/Sep/21 ]

Hi eflat@league.com, I was able to reproduce the error using the example provided. This is a regression caused by the changes in GODRIVER-1832.

The cause of the regression is that a separate internal mongo.Client is created for key vault operations when there is a limited connection pool size on the mongo.Client configured for automatic encryption.

The mongo.Client configured with automatic encryption is reused when there is no maximum connection pool size.

We should have fix for this as soon as possible. A possible workaround is to override the default maximum connection pool size (100) to be unlimited:

	clientOpts := options.Client().
		ApplyURI("mongodb://localhost:27017").
		SetAutoEncryptionOptions(autoEncryptionOpts).
		SetMaxPoolSize(0)

That may be undesirable depending on your use case. But we will have a fix applied as soon as possible.

Comment by Kevin Albertson [ 07/Sep/21 ]

Hi eflat@league.com, thank you for the detailed report and code snippet. We will look into this soon.

Generated at Thu Feb 08 08:37:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.