[DRIVERS-2774] SDAM heartbeats timeout spec test expects closure of connections before they attempt an operation Created: 13/Nov/23  Updated: 03/Jan/24

Status: Implementing
Project: Drivers
Component/s: CMAP
Fix Version/s: None

Type: Spec Change Priority: Major - P3
Reporter: Noah Stapp Assignee: Valentin Kavalenka
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Issue split
split to CDRIVER-4804 SDAM heartbeats timeout spec test exp... Backlog
split to CSHARP-4898 SDAM heartbeats timeout spec test exp... Backlog
split to CXX-2806 SDAM heartbeats timeout spec test exp... Backlog
split to JAVA-5282 SDAM heartbeats timeout spec test exp... Backlog
split to GODRIVER-3084 SDAM heartbeats timeout spec test exp... Closed
split to MOTOR-1229 SDAM heartbeats timeout spec test exp... Closed
split to PHPLIB-1329 SDAM heartbeats timeout spec test exp... Closed
split to PYTHON-4116 SDAM heartbeats timeout spec test exp... Closed
split to RUBY-3365 SDAM heartbeats timeout spec test exp... Closed
split to RUST-1817 SDAM heartbeats timeout spec test exp... Closed
split to NODE-5807 SDAM heartbeats timeout spec test exp... Scheduled
Problem/Incident
Driver Changes: Needed
Quarter: FY23Q4
Downstream Changes Summary:

Remove pool-clear-interrupt-immediately.json/yml as per https://github.com/mongodb/specifications/pull/1487.

Start date:
Driver Compliance:
Key Status/Resolution FixVersion
CDRIVER-4804 Backlog
CXX-2806 Backlog
CSHARP-4898 Backlog
GODRIVER-3084 Fixed 1.14.0
JAVA-5282 Backlog
NODE-5807 Scheduled
MOTOR-1229 Duplicate
PYTHON-4116 Duplicate
PHPLIB-1329 Works as Designed
RUBY-3365 Fixed 2.20.0
RUST-1817 Works as Designed

 Description   

shane.harvey@mongodb.com and I discovered an unexpected behavior of the spec test Connections MUST be interrupted as soon as possible. As currently written, the test checks out two connections and then clears the connection pool with the interrupt flag set. According to the spec, this should interrupt any operations the active connections are performing as soon as possible. However, the two connections are checked out but idle at the time of interruption, so the test does not interrupt any operations. As a result, they aren't checked in or closed: Python doesn't allow the pool-managing thread to forcibly signal the threads that own the connections. If either connection attempted to perform an operation after the pool clear, they would be correctly checked in and closed. This behavior fails the test, but does it actually differ from the expected behavior laid out in the spec?

Changing this test to attempt an operation after clearing the pool with the interrupt flag set could look something like this:

operations:
  - name: ready
  - name: start
    target: thread1
  - name: start
    target: thread2
  - name: checkOutAndRunCommand  # <------ new operation
    thread: thread1
  - name: checkOutAndRunCommand  # <------ new operation
    thread: thread2
  - name: clear
    interruptInUseConnections: true
  - name: waitForEvent
    event: ConnectionPoolCleared
    count: 1
    timeout: 1000
  - name: waitForEvent
    event: ConnectionClosed
    count: 2
    timeout: 1000
  - name: close 

  or this:

operations:
  - name: ready
  - name: checkOut
    label: conn1
  - name: checkOut
    label: conn2
  - name: clear
    interruptInUseConnections: true
  - name: waitForEvent
    event: ConnectionPoolCleared
    count: 1
    timeout: 1000
  - name: runCommand # <------ new operation
    label: conn1
    result: error
  - name: runCommand # <------ new operation
    label: conn2
    result: error
  - name: checkIn
    connection: conn1
  - name: checkIn
    connection: conn2
  - name: waitForEvent
    event: ConnectionClosed
    count: 2
    timeout: 1000
  - name: close 

 

 

 



 Comments   
Comment by Githook User [ 03/Jan/24 ]

Author:

{'name': 'Valentin Kovalenko', 'email': 'valentin.kovalenko@mongodb.com', 'username': 'stIncMale'}

Message: DRIVERS-2774 Delete `pool-clear-interrupt-immediately` (#1487)

SDAM already have tests covering the functionality, for example
https://github.com/mongodb/specifications/blob/fe95775ad9f4d0599c4b5e02516181d963bc2626/source/server-discovery-and-monitoring/tests/unified/interruptInUse-pool-clear.yml#L23.
Given that currently pool-clear-interrupt-immediately cannot possibly work in Python and requires modifying the test runner, it seems better to remove the test.
Branch: master
https://github.com/mongodb/specifications/commit/9c122c8660ed112d67eaf49fe7cfbb65dd60823a

Comment by Shane Harvey [ 05/Dec/23 ]

In Python there is no race. The connection will be closed and returned to the pool as soon as the driver attempts to send the next command. It's not possible to do it any other way in Python as reading/writing to the socket is the only interrupt point.

Generated at Thu Feb 08 08:26:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.