[DRIVERS-2435] Update libmongocrypt payloads to new QE protocol Created: 11/Sep/22  Updated: 28/Oct/23  Resolved: 30/May/23

Status: Closed
Project: Drivers
Component/s: Client Side Encryption
Fix Version/s: None

Type: Task Priority: Unknown
Reporter: Kevin Albertson Assignee: Kevin Albertson
Resolution: Fixed Votes: 0
Labels: equality-ga
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on MONGOCRYPT-561 Enable QE version 2 by default in lib... Closed
Gantt Dependency
has to be done before DRIVERS-2590 Remove workaround to enable featureFl... Closed
has to be done before DRIVERS-2589 Enable QEv2 tests on Serverless Implementing
Initiative
Issue split
split to PHPC-2203 Support Queryable Encryption v2 proto... Closed
split to CDRIVER-4584 Update libmongocrypt payloads to new ... Closed
split to CSHARP-4541 Update libmongocrypt payloads to new ... Closed
split to CXX-2656 Update libmongocrypt payloads to new ... Closed
split to GODRIVER-2761 Update libmongocrypt payloads to new ... Closed
split to MOTOR-1100 Update libmongocrypt payloads to new ... Closed
split to NODE-5074 Update libmongocrypt payloads to new ... Closed
split to PHPLIB-1088 Update Queryable Encryption tests for... Closed
split to PYTHON-3614 Update libmongocrypt payloads to new ... Closed
split to RUBY-3226 Update libmongocrypt payloads to new ... Closed
split to RUST-1605 Update libmongocrypt payloads to new ... Closed
split to JAVA-4891 Update libmongocrypt payloads to new ... Closed
Related
is related to DRIVERS-2524 Drivers should not create the ECC col... Closed
Driver Changes: Needed
Server Compat: 7.0, 7.1
Quarter: FY24Q1, FY24Q2
Upstream Changes Summary:

QE will introduce breaking changes to the protocol. After this project is complete, the server will incompatible with older drivers and tools.

Downstream Changes Summary:

Drivers are expected to update the Queryable Encryption (QE) tests. See https://github.com/mongodb/specifications/commit/64deb2837a2355f6002775c49b9b6c50c9dc560f for changes. Here is a summary:

See https://github.com/mongodb/mongo-go-driver/pull/1213 for the example Go driver implementation.

Also consider: To fix test failures without enabling QEv2: sync spec tests to https://github.com/mongodb/specifications/commit/baf3724155500acbb557e158b8a5d4334e7f3512 and skip to QE prose tests on server versions > 6.2.99. That may be helpful to backport to release branches so the tests on release branches do not fail against latest servers.

Changelog:
2023-04-26: libmongocrypt 1.8.0-alpha1 was released and removes the `mongocrypt_setopt_fle2v2` symbol. QEv2 is enabled by default.

Engineering Lead: Kevin Albertson Kevin Albertson
Program Manager: Esha Bhargava Esha Bhargava
Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4584 Fixed 1.24.0
CXX-2656 Fixed 3.8.0
CSHARP-4541 Fixed 2.20.0
GODRIVER-2761 Fixed 1.12.0
JAVA-4891 Fixed 4.10.0
NODE-5074 Fixed 5.3.0
MOTOR-1100 Fixed 3.2
PYTHON-3614 Fixed 4.4
PHPLIB-1088 Fixed 1.16.0
RUBY-3226 Fixed 2.19.0
RUST-1605 Fixed 2.6.0
PHPC-2203 Fixed 1.16.0

 Description   

Summary

PM-2972 proposes updating the Queryable Encryption protocol. This will require updating the payloads produced in libmongocrypt. Drivers will need to upgrade libmongocrypt and resync tests.

Motivation

Who is the affected end user?

Users of Queryable Encryption. This does not impact Client-Side Field Level Encryption.

How does this affect the end user?

Users will need updated libmongocrypt to use Queryable Encryption with the new protocol. The protocol change is backwards breaking.

Is this issue urgent?

Must be completed before PM-2972 is released. The protocol changes are backwards breaking.

Is this ticket required by a downstream team?

Yes. Compass and mongosh.

Is this ticket only for tests?

No



 Comments   
Comment by Githook User [ 12/Jul/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4632 Add tasks to test MongoDB Server 7.0 release (#1334)

  • add 7.0 to config_generator/components
  • add comment noting 6.0+ is required for QE
  • add 7.0 to legacy_config_generator
  • make API version task names unique
  • add missing 6.0 tasks
  • test Queryable Encryption on 7.0+ replica set (not 6.0+)

As of DRIVERS-2435, Queryable Encryption tests no longer run on 6.0.

  • add more missing 6.0 tasks
  • schedule missing AWS tasks
  • add `test-aws` tag and use all AWS tasks

Co-authored-by: Ezra Chung <88335979+eramongodb@users.noreply.github.com>
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/d896d3d4311cee4463d62bcc73dd86a22af0953a

Comment by Jeremy Mikola [ 17/Apr/23 ]

Regarding the following note in downstream changes:

Also consider: To fix test failures without enabling QEv2: sync spec tests to mongodb/specifications@baf3724 and skip to QE prose tests on server versions > 6.2.99. That may be helpful to backport to release branches so the tests on release branches do not fail against latest servers.

In addition to spec and prose tests, drivers will probably also want to update the version checks in any documentation examples added for DRIVERS-2311.

Comment by Githook User [ 31/Mar/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2435 replace `fle2-` tests with `fle2v2-` tests (#1391)

  • add `serverless: forbid` to fle2v2 tests

This will be reverted in DRIVERS-2589

  • add fle2v2 tests

The fle2v2 tests are copies of fle2-* tests
The expected payloads have been updated.
deleteTokens have been removed from the expectations.

  • bump minServerVersion to 7.0.0
  • note that libmongocrypt 1.8.0 is required for fle2v2 tests
  • remove fle2-* tests
  • require server 7.0 for QE prose tests
Comment by Githook User [ 31/Mar/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2435 add option to enable `featureFlagFLE2ProtocolVersion2` (#285)

  • remove unnecessary import
  • add setfle2parameter.py
  • add ENABLE_featureFlagFLE2ProtocolVersion2 option
  • only enable feature flag for "latest"

Older servers will error with `Unknown --setParameter`

  • use bash

find-python3.sh requires bash

  • remove requirement of python3
  • add SERVER ticket to emitted message
  • check for version 7.0
  • fix path to mongod
Comment by Githook User [ 27/Mar/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: DRIVERS-2435 add maxServerVersion 6.2.99 to fle2v1 tests (#1388)
Branch: master
https://github.com/mongodb/specifications/commit/baf3724155500acbb557e158b8a5d4334e7f3512

Comment by PM Bot [ 06/Oct/22 ]

Moved to Needs Triage because a linked PM issue PM-2972 was moved to Ready for Work.

Generated at Thu Feb 08 08:25:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.