[SERVER-69946] QE upsert excludes encrypted field in filter Created: 23/Sep/22 Updated: 05/Dec/22 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | Queryable Encryption |
| Affects Version/s: | 6.0.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Kevin Albertson | Assignee: | Backlog - Security Team |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | buildfest-2022 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Assigned Teams: |
Server Security
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Steps To Reproduce: | A full repro is included here: https://github.com/kevinAlbs/go-bootstrap/blob/418081b2e33688c8d88f47a1fecb75beceefc042/csfle/qe_updateOne/main.go It runs an UpdateOne operation to upsert a new document. A Queryable Encryption (QE) field "encryptedIndexed" is included in the filter.
Results in an upsert of the document:
The expectation is for the document to include "encryptedIndexed". The full update command sent is the following:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | Security 2022-10-03, Security 2022-10-17 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Participants: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Upserting a document to a collection with Queryable Encryption excludes the encrypted field from the filter. For example:
Results in an upsert of the document:
|
| Comments |
| Comment by Sergey Galtsev (Inactive) [ 07/Oct/22 ] | |||||||||||||||||||||||||||||||||||||
|
Upsert behavior can be logically represented as a transaction:
Expected behavior with upsert, where fields from filter are added to the "$set" is implemented on the server by taking a filter expression and transforming it into an insert. Notably, fields from filter are only applied when upsert results in an insert, not update. With FLE2, the fields in the filter are not transferred verbatim. Instead, they are converted to crypto material, which could be used to find required record. This material can not be used to gerenate an insert. There are two ways to implement the expected behavior: 1. have query analysis transform
into
before involving FLE2 logic. That means however, that id would be always modified, even when upsert results in an update instead of an insert, which could cause a leakage 2. a protocol change could be made to send additional element (something like: $set-on-insert) in update protocol. The data sent would become (conceptually):
At this time, my recommendation would be to document behavior as a known limitation of FLE2. If required, customers could emulate this behavior either by splitting upsert transaction into update/insert in their code (e.g.: manually implementing approach #2), or supply the search field twice - in filter and in update (e.g: implementing approach #1) | |||||||||||||||||||||||||||||||||||||
| Comment by Sergey Galtsev (Inactive) [ 27/Sep/22 ] | |||||||||||||||||||||||||||||||||||||
|
To reproduce this behavior in a shell, use following code:
Upserted document will look as follows:
According to the following link:
In other words, because encryptedIndexed: "bar" is in filter, it should also be upserted, but it is not |