Uploaded image for project: 'Go Driver'
  1. Go Driver
  2. GODRIVER-1613

fix how Pool handles MaxPoolSize

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 1.3.2
    • Fix Version/s: 1.3.5
    • Component/s: None
    • Labels:
      None
    • Environment:
      MacBook Pro with macOs Catalina. Version 10.15.4 (19E287)
      processor: 2,2 GHz Quad-Core Intel Core i7
      memory: 16 GB 1600 MHz DDR3
      Golang version: go1.13.5 darwin/amd64
      Database version: 3.6.17 runnning on docker for tests

      Description

      Under heavy load, using a context with timeout in a find operation, if timeout is reached, derives in a weird behavior of driver connection pool.
      Connections starts to be discarded by timeout and pool fall in a snow ball of discard/reconnect attemps until driver panics. 
      In tests running about 2 minutes, we observe, using a pool monitor, about 27k opened  and 26k closed connections, with a minPoolSize of 30 and a maxPoolSize of 100.

      To be clear, this is a sniped of code tested:

      filter := bson.M{"store_id": bson.M{"$in": idsList}}
      ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond)
      defer cancel()
       
      fOptions := options.Find().
         SetMaxTime(500*time.Millisecond).
         SetBatchSize(int32(len(idsList)))
       
      records, err := m.storesCollection.Find(ctx, filter, fOptions)
      

      where:
      idList is a variable length slice of ids to search (indexed field, unique)
      ctx: Problematic context with timeout
      fOptions–>MaxTime: works ok

      Two files attached:
      1- Test result with metrics and error messages
      2- console output with traces

        Attachments

        1. console.zip
          25 kB
        2. Test_result.zip
          12 kB

          Issue Links

            Activity

              People

              Assignee:
              isabella.siu Isabella Siu (Inactive)
              Reporter:
              gaston.acevedo@rappi.com Gaston Gregorio Blas Acevedo
              Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: