Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-60764

Change stream getMore still reports non-zero cursor ID in response including invalidate event

    • Type: Icon: Bug Bug
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • ALL

      While testing the PHP driver against a "latest" server build (5.1.0-alpha-1523-g565f818), I noticed a change in how mongos responds to a getMore on a change stream cursor.

      The test in question does the following:

      • creates a collection (not sharded)
      • opens a change stream on that collection
      • drops the collection
      • advances the cursor until an operation is encountered

      On 5.0.x and earlier, the final getMore observed in this test reports zero for the cursor ID; however, 5.1.0 still reports a non-zero value. Is that an intentional change? Bypassing mongos to the same test against the replica set shard directly still reports the cursor ID as zero, so this appears to only happen with mongos.

      Note: in the logs below, I've omitted some extra fields (e.g. lsid, $clusterTime) to save space.

      5.0.2 command log

      The following was observed using a sharded cluster (backed by replica sets) running mongodb-linux-x86_64-enterprise-ubuntu2004-5.0.2. On 5.0.2 and earlier server versions, the getMore response including the "drop" and "invalidate" events reports zero as the cursor ID, which signals to the driver that the cursor has been closed on the server side and that no more iteration is possible.

      command: {
          "getMore": 5141813473153987695,
          "collection": "WatchFunctionalTest.4e48a302",
          "maxTimeMS": 500,
          "$db": "phplib_test"
      }
      
      reply: {
          "cursor": {
              "id": 5141813473153987695,
              "ns": "phplib_test.WatchFunctionalTest.4e48a302",
              "nextBatch": [],
              "postBatchResumeToken": {
                  "_data": "82616A00BD000000062B0229296E04"
              }
          },
          "ok": 1
      }
      
      command: {
          "getMore": 5141813473153987695,
          "collection": "WatchFunctionalTest.4e48a302",
          "maxTimeMS": 500,
          "$db": "phplib_test"
      }
      
      reply: {
          "cursor": {
              "id": 0,
              "ns": "phplib_test.WatchFunctionalTest.4e48a302",
              "nextBatch": [
                  {
                      "_id": {
                          "_data": "82616A00BD0000000F2B022C0100296E5A10048510AB446C2F471FBF1AF36623B03F1F04"
                      },
                      "operationType": "drop",
                      "clusterTime": {
                          "$timestamp": {
                              "t": 1634336957,
                              "i": 15
                          }
                      },
                      "ns": {
                          "db": "phplib_test",
                          "coll": "WatchFunctionalTest.4e48a302"
                      }
                  },
                  {
                      "_id": {
                          "_data": "82616A00BD0000000F2B022C0100296F5A10048510AB446C2F471FBF1AF36623B03F1F04"
                      },
                      "operationType": "invalidate",
                      "clusterTime": {
                          "$timestamp": {
                              "t": 1634336957,
                              "i": 15
                          }
                      }
                  }
              ],
              "postBatchResumeToken": {
                  "_data": "82616A00BD000000112B0229296E04"
              }
          },
          "ok": 1
      }
      

      5.1.0 command log

      The following was observed using a sharded cluster (backed by replica sets) running mongodb-linux-x86_64-enterprise-ubuntu2004-5.1.0-alpha-1523-g565f818. Here, the second response with a "drop" and "invalidate" events still includes a cursor ID. This suggests that the cursor is still alive on the server side, despite the "invalidate" event (presumably, there's no reason for the driver or application to iterate beyond an "invalidate" event).

      command: {
          "getMore": 3138377033263876049,
          "collection": "WatchFunctionalTest.4e48a302",
          "maxTimeMS": 500,
          "$db": "phplib_test"
      }
      
      reply: {
          "cursor": {
              "id": 3138377033263876049,
              "ns": "phplib_test.WatchFunctionalTest.4e48a302",
              "nextBatch": [],
              "postBatchResumeToken": {
                  "_data": "826169FFC1000000112B0229296E04"
              }
          },
          "ok": 1
      }
      
      command: {
          "getMore": 3138377033263876049,
          "collection": "WatchFunctionalTest.4e48a302",
          "maxTimeMS": 500,
          "$db": "phplib_test"
      }
      
      reply: {
          "cursor": {
              "id": 3138377033263876049,
              "ns": "phplib_test.WatchFunctionalTest.4e48a302",
              "nextBatch": [
                  {
                      "_id": {
                          "_data": "826169FFC1000000192B022C0100296E5A1004F465544733ED4D40BF461DA4C87B53E504"
                      },
                      "operationType": "drop",
                      "clusterTime": {
                          "$timestamp": {
                              "t": 1634336705,
                              "i": 25
                          }
                      },
                      "ns": {
                          "db": "phplib_test",
                          "coll": "WatchFunctionalTest.4e48a302"
                      }
                  },
                  {
                      "_id": {
                          "_data": "826169FFC1000000192B022C0100296F5A1004F465544733ED4D40BF461DA4C87B53E504"
                      },
                      "operationType": "invalidate",
                      "clusterTime": {
                          "$timestamp": {
                              "t": 1634336705,
                              "i": 25
                          }
                      }
                  }
              ],
              "postBatchResumeToken": {
                  "_data": "826169FFC1000000192B022C0100296F5A1004F465544733ED4D40BF461DA4C87B53E504"
              }
          },
          "ok": 1
      }
      

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            jmikola@mongodb.com Jeremy Mikola
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: