[SERVER-69626] FLE2 Compact - delete from ESC not in a transaction Created: 13/Sep/22 Updated: 29/Mar/23 Resolved: 29/Mar/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Queryable Encryption |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Mark Benvenuto | Assignee: | Backlog - Security Team |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Server Security
|
||||||||
| Sprint: | Security 2022-10-17 | ||||||||
| Participants: | |||||||||
| Description |
|
If the ESC table has N rows, the N-1 rows should be deleted with a delete that is not in a MongoDB user transaction. Normally, all of the ESC operations run in SyncTransactionWithRetries::runNoThrow` which means they run in a TXN. Due to the large number of ESC rows that must be deleted this could create a very large txn, the deletes should instead happen with a delete that does not run as part of the transaction. 1. Extend FLEQueryInterfaceImpl with a new delete method Note: ECC deletes remain under the txn. Also, the ESC null document update remains under the TXN. |
| Comments |
| Comment by Erwin Pe [ 29/Mar/23 ] |
|
This work has been superseded by the v2 compaction algorithm in OST-1 v6, which was implemented in |
| Comment by Mark Benvenuto [ 12/Oct/22 ] |
|
After some more consideration, we need to be a little careful such the changes to ESC are sequenced correctly in relation to the EmuBinary. The new proposed algorithm with ESC non-transactional deletes is below. The algorithm is the same as the originally but the order of operations is sequenced differently. Within ESC and ECC, the order of how the null record and non-null records.
|