[JAVA-3792] ChangeStreams are not triggered anymore when AutoEncryptionSettings are used Created: 10/Jul/20  Updated: 17/Aug/20  Resolved: 17/Aug/20

Status: Closed
Project: Java Driver
Component/s: Reactive Streams, Scala
Affects Version/s: 4.0.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Martin Thomalla Assignee: Ross Lawley
Resolution: Cannot Reproduce Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

windows 7, 10, 2012
jdk 11
atlas 4.2.8
scala 2.13.1
"org.mongodb.scala" %% "mongo-scala-driver" % "4.0.5"


Case:

 Description   

We activated client side field encryption following the relevant guides.

  private val client = {
    val jKmsProviders = kmsProviders.map { case (k, v) => (k, v.asJava) }.asJava
    val autoEncryptionSettings = AutoEncryptionSettings
      .builder()
      .keyVaultNamespace("encryption.__keyvault")
      .kmsProviders(jKmsProviders)
      .build()
 
    MongoClient(
      MongoClientSettings
        .builder()
        .applyConnectionString(new ConnectionString(mongoConfig.connectionString))
        .autoEncryptionSettings(autoEncryptionSettings)
        .codecRegistry(MongoClient.DEFAULT_CODEC_REGISTRY)
        .uuidRepresentation(UuidRepresentation.JAVA_LEGACY)
        .build()
    )
  }

We use the AutoEncryptionSettings to get transparent En-/Decrypt for one of our fields.
This works fine.

Unfortunately this stops all our watches from being triggered on any change.
It doesn't matter if the watched colletion uses encrypted fields or not.

  override def watch(callback: (A, Action) => Unit): Unit =
    coll
      .watch()
      .subscribe((csd: ChangeStreamDocument[DTO]) => {
        val (c, op) = (csd.getFullDocument, csd.getOperationType)
        logger.debug("Found {} with {}", ct, c)
        fromDto(c) match {
          case Right(a) =>
            op match {
              case OperationType.INSERT                         => callback(a, Action.Insert)
              case OperationType.UPDATE | OperationType.REPLACE => callback(a, Action.Update)
              case OperationType.DELETE                         => callback(a, Action.Delete)
              case _                                            => logger.warn("Operation on {} of type {} not supported", ct, op)
            }
          case Left(t) =>
            logger.warn("Could not transform {} with {} (reason: {})", ct, c, t)
        }
      })

If we remove the AutoEncryptionSettings from our connection all watches work again.



 Comments   
Comment by Ross Lawley [ 17/Aug/20 ]

Hi martin.thomalla@rwe.com,

No worries, glad you were able to resolve the issue.

Ross

Comment by Martin Thomalla [ 17/Aug/20 ]

Hi @Ross,

You are correct, onError wasn't implemented on our site and all errors were just not logged.

I did not see that while working with the code which was written by someone else.

Sorry for the confusion.

Martin

Comment by Ross Lawley [ 20/Jul/20 ]

Hi martin.thomalla@rwe.com,

When running:

    collection
      .watch()
      .subscribe(new Observer[ChangeStreamDocument[Document]] {
 
        override def onNext(result: ChangeStreamDocument[Document]): Unit = println(s"Result: $result")
 
        override def onError(e: Throwable): Unit = println(s">> Error: ${e.getMessage}")
 
        override def onComplete(): Unit = println("COMPLETED")
      })

I do see the error passed to the onError method and it prints it:

>> Error: Exception in encryption library: Command failed with error 31011 (Location31011): 'Aggregation stage $changeStream is not allowed or supported with automatic encryption.' on server localhost:27020. The full response is

Unknown macro: {"ok"}

I also tested in the sync and reactive streams driver and got the same result.

Ross

Comment by Ross Lawley [ 17/Jul/20 ]

Hi martin.thomalla@rwe.com,

Thank you for reporting the issue. Agreed there should be a client error passed here to the onError method. This ticket has now been scheduled for fixing.

Kind Regards,

Ross

Comment by Martin Thomalla [ 17/Jul/20 ]

https://support.mongodb.com/case/00671738 explains that CSFLE and watches do not work together.

We should get a client error though.

Comment by Ross Lawley [ 17/Jul/20 ]

Moved to the JAVA project as the driver version is 4.0.5 which is co hosted with the Java driver.

Generated at Thu Feb 08 09:00:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.