[DRIVERS-2524] Drivers should not create the ECC collection in v2 of queryable encryption Created: 13/Jan/23  Updated: 06/Jun/23  Resolved: 06/Jun/23

Status: Closed
Project: Drivers
Component/s: Client Side Encryption
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Erwin Pe Assignee: Kevin Albertson
Resolution: Done Votes: 0
Labels: equality-ga
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Initiative
Issue split
split to MOTOR-1089 Drivers should not create the ECC col... Closed
split to CDRIVER-4563 Drivers should not create the ECC col... Closed
split to CSHARP-4492 Drivers should not create the ECC col... Closed
split to CXX-2642 Drivers should not create the ECC col... Closed
split to GODRIVER-2741 Drivers should not create the ECC col... Closed
split to JAVA-4858 Drivers should not create the ECC col... Closed
split to NODE-5004 Drivers should not create the ECC col... Closed
split to PHPLIB-1071 Drivers should not create the ECC col... Closed
split to PYTHON-3583 Drivers should not create the ECC col... Closed
split to RUBY-3211 Drivers should not create the ECC col... Closed
split to RUST-1581 Drivers should not create the ECC col... Closed
Related
related to DRIVERS-2435 Update libmongocrypt payloads to new ... Closed
related to DRIVERS-2586 Create and drop collection helpers sh... Closed
is related to DRIVERS-2619 fle2v2-CreateCollection tests expect ... Implementing
is related to SERVER-75683 Return error if encryptedFields conta... Closed
Driver Changes: Needed
Server Compat: 7.0, 7.1
Quarter: FY24Q1
Downstream Changes Summary:

See https://github.com/mongodb/mongo-c-driver/pull/1232 for an example of changes.

Further changes:
Friday, April 21, 2023: https://github.com/mongodb/specifications/commit/eb3d882bb8c08d0f25f54709abcd876caeccba7f fixes commandStarted events with encryptionInformation

Engineering Lead: Kevin Albertson Kevin Albertson
Program Manager: Esha Bhargava Esha Bhargava
Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4563 Fixed 1.24.0
CXX-2642 Fixed 3.8.0
CSHARP-4492 Fixed 2.20.0
GODRIVER-2741 Fixed 1.12.0
JAVA-4858 Done 4.10.0
NODE-5004 Fixed 5.5.0
MOTOR-1089 Fixed 3.2
PYTHON-3583 Duplicate
PHPLIB-1071 Fixed 1.16.0
RUBY-3211 Fixed 2.19.0
RUST-1581 Fixed 2.6.0
SWIFT-1696 Won't Do

 Description   

Summary

Once the protocol changes in version 2 of queryable encryption have been made in the server, the ECC collection is no longer required when creating a new QE-encrypted collection. Thus, the drivers must update their QE helpers to no longer create the ECC state collection when creating an encrypted collection.

Motivation

Who is the affected end user?

Users of the new QE v2 wire protocol.

How does this affect the end user?

Since the ECC collection is obsoleted in v2 of QE, this will reduce the number of state collections needed for QE to work.

How likely is it that this problem or use case will occur?

This occurs every time the user creates a QE-encrypted collection.

If the problem does occur, what are the consequences and how severe are they?

If the server is using v2 of the QE protocol, and the driver is still creating ECC collections, then those ECC collections are never going to be used, and will only serve to clutter the database.

Is this issue urgent?

This should be implemented once PM-2972 is complete.

Is this ticket required by a downstream team?

n/a

Is this ticket only for tests?

no



 Comments   
Comment by Githook User [ 01/Jun/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2524 assert collections are not created on unsupported server (#1428)
Branch: master
https://github.com/mongodb/specifications/commit/9e770b54bbb6315aa82a162c42c577a5b3f8037a

Comment by Githook User [ 21/Apr/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2524 fix commandStarted events with encryptionInformation (#1405)

  • add escCollection and ecocCollection to commandStarted events

escCollection and ecocCollection are appended by libmongocrypt

  • fix indent of encryptionInformation
  • add missing `ordered: true`
Comment by Githook User [ 12/Apr/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2524 do not create or drop `eccCollection` (#1396)

  • do not create or drop the eccCollection
  • DRIVERS-2586 do not document `escCollection` and `ecocCollection` options
  • remove eccCollection from fle2v2-CreateCollection
  • remove `eccCollection` from `encryptedFields` data files
  • regenerate fle2v2 tests
  • remove eccCollection from fle2v2-EncryptedFields-vs-EncryptedFieldsMap
  • remove eccCollection from fle2v2-Range-WrongType
  • assert eccCollection is not created
  • remove incorrect comment
  • add wire version check and test
  • remove `escCollection` and `ecocCollection` from test data
  • regenerate tests
  • add $$exists to legacy format
  • add $$exists to tests

Assert that state collections names are not sent to server

  • test encryptedFields is consulted for metadata collection names
  • change SHOULD to MUST
  • use YAML anchors for collection names
  • add comment that ecc collection is no longer created for QEv2
  • remove `encryptedFieldsMap with cyclic entries does not loop`
  • use YAML anchors for encryptedFields
  • use `null`, not $$exists
  • Revert "add $$exists to legacy format"

This reverts commit 72280f9050997e4ea7e49b1e707b699015f6cdd8.

  • swap order of `base64` and `subType`
  • remove unnecessary anchor
Comment by Kevin Albertson [ 21/Mar/23 ]

Also consider removing driver API and documentation of options for escCollection and ecocCollection. There does not seem to be a use case for a user to set these options. The name requirements of DRIVERS-2565 likely prevent a user from setting the collection names to any non-default value.

Generated at Thu Feb 08 08:25:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.