createSearchIndexes operation is not releasing connections correctly

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 5.5.2, 5.6.1
    • Affects Version/s: 5.4.0, 5.5.1
    • Component/s: Index Management
    • None
    • šŸ”µ Done
    • Fully Compatible
    • Java Drivers
    • Not Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      Using MongoCollection.createSearchIndexes of the reactivestreams driver causes Connections to be consumed without releasing them.

      Consider the following code-snippet

      @Slf4j
      @RestController
      @RequiredArgsConstructor
      public class IndexSearchTest {
          private final ReactiveMongoTemplate mongoTemplate;
      
          @RequestMapping(
              method = {RequestMethod.GET},
              value = {"/indextest"},
              produces = {"application/json"}
          )
          Mono<ResponseEntity<String>> indexTest() {
      //this is our atlas search definition
              var searchIndexDefinition = """
                  {
                      "mappings": {
                          "dynamic": false,
                          "fields": {
                              "nameDE": {
                                  "analyzer": "lucene.german",
                                  "store": false,
                                  "type": "string"
                              }
                          }
                      }
                    }
                  """;
              var model = new SearchIndexModel("article", Document.parse(searchIndexDefinition));
      //list of collections is very long in our case (> 100)
              return Flux.fromIterable(List.of("article.090009"))
                  .concatMap(collectionName -> mongoTemplate.getCollection(collectionName))
                  .concatMap(coll -> Mono.from(coll.createSearchIndexes(List.of(model))))
                  .collectList()
                  .flatMap(result -> Mono.just(ResponseEntity.ok(result.stream().collect(Collectors.joining(",")))));
          }
      }
      

      When calling the endpoint above I would expect that the checkedout connection counter goes to back to the original value after the Restservice returns. However, the counter continously goes up until the limit is reached. Afterwards no new connections are accepted. The connections are also never released. Also after waiting for hours.

      I'm checking the checkedout connection counter through the prometheus endpoint like this:

      curl http://localhost:8081/actuator/prometheus | grep checkedout
      

      Background: We are using this functionality within a daily scheduler to ensure that every index is set appropriatly in the db. Since the createSearchIndexes is called per collection the amount of consumed connections equals to every collection where the method is called.

      We testet with Ā org.mongodb:mongodb-driver-reactivestreams library in versions 5.4.0 and 5.5.1 All with the same result

      Ā 

      I investigated a little bit in the code and found out, that the com.mongodb.client.internal.ClientSessionBinding.release() is only releasing the connection, if theĀ Ā referenceCount is 0. However this is not the case here. com.mongodb.internal.operation.AbstractWriteSearchIndexOperation seems to not count down the counter correctly. The reason might be that the whenComplete lambda of the following snippet isn't invoked correctly

      //com.mongodb.internal.operation.AsyncOperationHelper#withAsyncSourceAndConnection
      curriedFunction.whenComplete(resource::release).get(errorHandlingCallback); 

        1. curl-result.txt
          0.6 kB
        2. IndexSearchTest.java
          3 kB

            Assignee:
            Ross Lawley
            Reporter:
            Marko Modsching
            Nabil Hachicha
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: