[DRIVERS-2586] Create and drop collection helpers should ignore custom names for Queryable Encryption metadata collections Created: 28/Mar/23  Updated: 14/Sep/23  Resolved: 12/Apr/23

Status: Closed
Project: Drivers
Component/s: Client Side Encryption
Fix Version/s: None

Type: Task Priority: Unknown
Reporter: Jeremy Mikola Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Issue split
split to CDRIVER-4606 Create and drop collection helpers sh... Blocked
split to CSHARP-4602 Create and drop collection helpers sh... Blocked
split to CXX-2667 Create and drop collection helpers sh... Blocked
split to GODRIVER-2799 Create and drop collection helpers sh... Blocked
split to JAVA-4924 Create and drop collection helpers sh... Blocked
split to MOTOR-1111 Create and drop collection helpers sh... Blocked
split to NODE-5175 Create and drop collection helpers sh... Blocked
split to PYTHON-3655 Create and drop collection helpers sh... Blocked
split to RUBY-3234 Create and drop collection helpers sh... Blocked
split to RUST-1624 Create and drop collection helpers sh... Blocked
split to PHPLIB-1102 Create and drop collection helpers sh... Closed
Related
is related to DRIVERS-2565 Update fle2-* tests to match name req... Closed
is related to SERVER-74069 Validate QE state collections match n... Closed
is related to DRIVERS-2524 Drivers should not create the ECC col... Closed
Driver Changes: Needed
Engineering Lead: Kevin Albertson Kevin Albertson
Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4606 Blocked
CXX-2667 Blocked
CSHARP-4602 Blocked
GODRIVER-2799 Blocked
JAVA-4924 Blocked
NODE-5175 Blocked
MOTOR-1111 Blocked
PYTHON-3655 Blocked
PHPLIB-1102 Duplicate
RUBY-3234 Blocked
RUST-1624 Blocked

 Description   

Summary

DRIVERS-2284 originally instructed for drivers to allow custom names for Queryable Encryption Metadata Collections via the eccCollection, ecocCollection, and escCollection fields within the encryptedFields option for the createCollection() helper.

SERVER-74069 has since added server-side validation for those fields and no longer allows names to deviate from the following:

  • enxcol_.<collectionName>.esc
  • enxcol_.<collectionName>.ecc
  • enxcol_.<collectionName>.ecoc

These custom options within "encryptedFields" were never documented (see: create). The Queryable Encryption API is also documented as unstable so there should be no concern with changing this behavior.

Drivers should remove any modeling of these options for "encryptedFields". The createCollection() and dropCollection() helpers should ignore any user-supplied values within the encryptedFields option and use generated names with the proper format. Any values within encryptedFields can be left as-is and passed on to the server (if that leads to an error, so be it).

Additional Context

Quoting discussion from an internal Slack thread in #drivers-fle:

Looking at the create command docs, there's no mention of escCollection, eccCollection, or ecocCollection in the encryptedFields option docs. Were those fields always option and the server would just default to the generated names that are now enforced (as of DRIVERS-2565)?

Yes. Manually creating a collection with the following "encryptedFields" option will apply the default names for escCollection, eccCollection, ecocCollection:

db.runCommand({create: "foo", encryptedFields: {fields: []}})

Presumably there is no reason for a user to ever define these in encryptedFields. Does it make sense to still display these fields in the encryptedFields example within the spec (they were left in place in 8f6f4a3 for DRIVERS-2565)? If they do remain, should there be language talking about how the format is enforced? Perhaps a suggestion that if drivers haven't already modeled these options, they shouldn't even give users the choice to set them?

I am in favor of removing driver API and documentation of options for escCollection, eccCollection, and ecocCollection. I see no use case for a user to pass these options. Also, eccCollection is being removed with the new perf and on-disk format changes (DRIVERS-2524).

Motivation

Is this issue urgent?

No, but should be done while the Queryable Encryption API is still unstable.

Is this ticket required by a downstream team?

No.

Is this ticket only for tests?

No.



 Comments   
Comment by Kevin Albertson [ 12/Apr/23 ]

Addressed in DRIVERS-2524

Comment by Githook User [ 12/Apr/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2524 do not create or drop `eccCollection` (#1396)

  • do not create or drop the eccCollection
  • DRIVERS-2586 do not document `escCollection` and `ecocCollection` options
  • remove eccCollection from fle2v2-CreateCollection
  • remove `eccCollection` from `encryptedFields` data files
  • regenerate fle2v2 tests
  • remove eccCollection from fle2v2-EncryptedFields-vs-EncryptedFieldsMap
  • remove eccCollection from fle2v2-Range-WrongType
  • assert eccCollection is not created
  • remove incorrect comment
  • add wire version check and test
  • remove `escCollection` and `ecocCollection` from test data
  • regenerate tests
  • add $$exists to legacy format
  • add $$exists to tests

Assert that state collections names are not sent to server

  • test encryptedFields is consulted for metadata collection names
  • change SHOULD to MUST
  • use YAML anchors for collection names
  • add comment that ecc collection is no longer created for QEv2
  • remove `encryptedFieldsMap with cyclic entries does not loop`
  • use YAML anchors for encryptedFields
  • use `null`, not $$exists
  • Revert "add $$exists to legacy format"

This reverts commit 72280f9050997e4ea7e49b1e707b699015f6cdd8.

  • swap order of `base64` and `subType`
  • remove unnecessary anchor
Comment by Kevin Albertson [ 29/Mar/23 ]

I agree with the rationale. I want to do this with DRIVERS-2524. DRIVERS-2524 will require drivers modify the behavior of create/drop for Queryable Encryption. I expect to start DRIVERS-2524 next week. I suggest moving this to Scheduled.

Generated at Thu Feb 08 08:25:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.