[SERVER-68371] Enabling CSFLE in your MongoClient causes Atlas Search to fail Created: 27/Jul/22  Updated: 29/Oct/23  Resolved: 27/Sep/22

Status: Closed
Project: Core Server
Component/s: Field Level Encryption
Affects Version/s: None
Fix Version/s: 6.0.3, 6.1.0-rc4, 6.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: James Kovacs Assignee: Jacob Evans
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Documented
Problem/Incident
Related
related to SERVER-79169 Support queryable encryption for $vec... Closed
related to SERVER-50092 [FLE] with encryption on collection a... Backlog
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.1, v6.0
Steps To Reproduce:

1. Create a MongoDB Atlas cluster. (M0 is fine.)
2. Click ... and select Load Sample Dataset.
3. Select the Search tab.
4. Click Create Index and create an Atlas Search index on the sample_mflix.movies namespace with default options.
5. Once the index is done building, run the following C# sample. Note that the line settings.AutoEncryptionOptions = autoEncryptionOptions; is intentionally commented out.
6. You should see 47 movies returned.
7. Uncomment settings.AutoEncryptionOptions = autoEncryptionOptions; and run the sample again.
8. You should see the following exception thrown:
Command aggregate failed: Unrecognized pipeline stage name: '$search'.

using System;
using System.Collections.Generic;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Encryption;
 
var uri = "<<YOUR_MONGODB_ATLAS_URI>>";
var settings = MongoClientSettings.FromConnectionString(uri);
 
var localMasterKey = Convert.FromBase64String("Mng0NCt4ZHVUYUJCa1kxNkVyNUR1QURhZ2h2UzR2d2RrZzh0cFBwM3R6NmdWMDFBMUN3YkQ5aXRRMkhGRGdQV09wOGVNYUMxT2k3NjZKelhaQmRCZGJkTXVyZG9uSjFk");
var kmsProviders = new Dictionary<string, IReadOnlyDictionary<string, object>>();
var localKey = new Dictionary<string, object>
{
    { "key", localMasterKey }
};
kmsProviders.Add("local", localKey);
var keyVaultNamespace = CollectionNamespace.FromFullName("encryption.__keyVault");
var autoEncryptionOptions = new AutoEncryptionOptions(keyVaultNamespace, kmsProviders);
 
// THIS LINE INTENTIONALLY COMMENTED OUT
// settings.AutoEncryptionOptions = autoEncryptionOptions;
 
var client = new MongoClient(settings);
var db = client.GetDatabase("test");
var collection = db.GetCollection<BsonDocument>("sample_mflix");
 
// Simple search example
var searchStage = @"
{
    $search : {
        text : {
            query : 'baseball',
            path : 'plot'
        }
    }
}";
var projection = @"
{
    _id : 0,
    title : 1,
    plot : 1
}";
var pipeline = new EmptyPipelineDefinition<BsonDocument>()
    .AppendStage<BsonDocument, BsonDocument, BsonDocument>(searchStage)
    .Project(projection);
var moviesAboutBaseball = collection.Aggregate(pipeline).ToList();
 
Console.WriteLine($"{moviesAboutBaseball.Count} movies about baseball:");
foreach (var movie in moviesAboutBaseball)
{
    Console.WriteLine($"  {movie}");
}
Console.WriteLine();

Sprint: QO 2022-08-22, QO 2022-09-05, QO 2022-09-19, QO 2022-10-03
Participants:
Case:
Linked BF Score: 167

 Description   

When CSFLE is enabled, all commands are sent to the mongocryptd (or the shared library) for processing since drivers do not know which fields must be encrypted. mongocryptd/shared library is not aware of Atlas Search's $search aggregation pipeline stage and errs with:

Unrecognized pipeline stage name: '$search'.

Because all commands must be sent to mongocryptd/shared library for processing prior to dispatch to a MongoDB Atlas cluster node, Atlas Search ($search) stops working as soon as you enable CSFLE in your MongoClient.



 Comments   
Comment by Githook User [ 03/Oct/22 ]

Author:

{'name': 'Jacob Evans', 'email': 'jacob.evans@10gen.com'}

Message: SERVER-68371 Allow search queries to pass through query analysis
Branch: v6.1
https://github.com/mongodb/mongo/commit/1d5760aab2bd1053fe73a2d610bdd801cae86f7f

Comment by Githook User [ 03/Oct/22 ]

Author:

{'name': 'Jacob Evans', 'email': 'jacob.evans@10gen.com'}

Message: SERVER-68371 Allow search queries to pass through query analysis
Branch: v6.1
https://github.com/10gen/mongo-enterprise-modules/commit/7511e996410f718fc60a935122ab34e6f6e92f1d

Comment by Githook User [ 03/Oct/22 ]

Author:

{'name': 'Jacob Evans', 'email': 'jacob.evans@10gen.com'}

Message: SERVER-68371 Allow search queries to pass through query analysis
Branch: v6.0
https://github.com/mongodb/mongo/commit/62271b8469b68d654282acc7e288d29f2446d012

Comment by Githook User [ 03/Oct/22 ]

Author:

{'name': 'Jacob Evans', 'email': 'jacob.evans@10gen.com'}

Message: SERVER-68371 Allow search queries to pass through query analysis
Branch: v6.0
https://github.com/10gen/mongo-enterprise-modules/commit/625a261aa2bf66503f85e05f64a2802c62d0316e

Comment by Githook User [ 26/Sep/22 ]

Author:

{'name': 'Jacob Evans', 'email': 'jacob.evans@10gen.com'}

Message: SERVER-68371 Allow search queries to pass through query analysis
Branch: master
https://github.com/mongodb/mongo/commit/3e4634f54f17b661165423f344311fba520d4008

Comment by Githook User [ 26/Sep/22 ]

Author:

{'name': 'Jacob Evans', 'email': 'jacob.evans@10gen.com'}

Message: SERVER-68371 Allow search queries to pass through query analysis
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/e9726d1dbd6a2f942d90dcae9627e8d5a5574c9e

Comment by Githook User [ 22/Sep/22 ]

Author:

{'name': 'Uladzimir Makouski', 'email': 'uladzimir.makouski@mongodb.com', 'username': 'umakouski'}

Message: Revert "SERVER-68371 Allow search queries to pass through query analysis"

This reverts commit 1399e9cb228f5360bf0950933616d548f19cc730.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/bafc7891069c2268045c4b2057a0b665c185dc25

Comment by Githook User [ 21/Sep/22 ]

Author:

{'name': 'Jacob Evans', 'email': 'jacob.evans@10gen.com'}

Message: SERVER-68371 Allow search queries to pass through query analysis
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/1399e9cb228f5360bf0950933616d548f19cc730

Comment by James Kovacs [ 02/Aug/22 ]

I ran the repro on my Intel-based MBP, though OS and CPU shouldn't make a difference. mongocryptd version is 6.0.0.

> mongocryptd --version
mongos version v6.0.0
Build Info: {
    "version": "6.0.0",
    "gitVersion": "e61bf27c2f6a83fed36e5a13c008a32d563babe2",
    "modules": [
        "enterprise"
    ],
    "allocator": "system",
    "environment": {
        "distarch": "x86_64",
        "target_arch": "x86_64"
    }
}

Comment by James Kovacs [ 02/Aug/22 ]

elizabeth.roytburd@mongodb.com: My investigation started with a Community Forums question. Based on that report, they were using MongoDB .NET/C# Driver 2.16 (which was latest at the time). I debugged through the issue on our main branch. It will present itself on 2.17.1 (latest) as well.

If you have .NET 6 installed on any OS, you can create a new console app and add the latest .NET/C# Driver to it:

dotnet new console
dotnet add package MongoDB.Driver

Then replace Program.cs with the above repro being sure to replace <<YOUR_MONGODB_ATLAS_URI>> appropriately.

Comment by Chris Kelly [ 29/Jul/22 ]

Assigning to security team because they have ownership of file level encryption.

Comment by James Kovacs [ 27/Jul/22 ]

One possible workaround is to instantiate two MongoClient instances with the same connection string, but one configured for CSFLE and the other not. You must then run all Atlas Search queries using the MongoClient without CSFLE enabled.

Generated at Thu Feb 08 06:10:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.