[SERVER-37200] $match stage following $listSessions not working against mongos Created: 19/Sep/18  Updated: 29/Oct/23  Resolved: 19/Nov/18

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: 3.6.7, 4.0.3
Fix Version/s: 3.6.10, 4.0.5, 4.1.6

Type: Bug Priority: Major - P3
Reporter: Josef Ahmad Assignee: Jacob Evans
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.0, v3.6
Steps To Reproduce:

The following is a modified jstests/core/list_all_sessions.js test which demonstrates this issue. The following will pass when run against the core suite but will fail against the sharding_jscore_passthrough suite:

 

// Sessions are asynchronously flushed to disk, so a stepdown immediately after calling
// startSession may cause this test to fail to find the returned sessionId.
// @tags: [does_not_support_stepdowns]
 
 
// Basic tests for the $listSessions {allUsers:true} aggregation stage.
 
 
(function() {
    'use strict';
    load('jstests/aggregation/extras/utils.js');
 
 
    const admin = db.getSiblingDB("admin");
    const config = db.getSiblingDB("config");
    const pipeline = [{'$listSessions': {allUsers: true}}];
    function listSessions() {
        return config.system.sessions.aggregate(pipeline);
    }
    function listSessionsWithFilter(filter) {
        return config.system.sessions.aggregate([{'$listSessions': {allUsers: true}}, {$match: filter}]);
    }
 
 
    // Get current log level.
    let originalLogLevel = assert.commandWorked(admin.setLogLevel(1)).was.verbosity;
 
 
    try {
        // Start a new session and capture its sessionId.
        const myid = assert.commandWorked(admin.runCommand({startSession: 1})).id.id;
        assert(myid !== undefined);
        assert.commandWorked(admin.runCommand({refreshLogicalSessionCacheNow: 1}));
 
 
        // Ensure that the cache now contains the session and is visible by admin.
        assert.soon(function() {
            const resultArray = listSessions().toArray();
            if (resultArray.length < 1) {
                return false;
            }
            const resultArrayMine = resultArray
                                        .map(function(sess) {
                                            return sess._id.id;
                                        })
                                        .filter(function(id) {
                                            return 0 == bsonWoCompare({x: id}, {x: myid});
                                        });
            return resultArrayMine.length == 1;
        }, "Failed to locate session in collection");
 
 
        const sessionList = listSessionsWithFilter({_id: "non_existent"}).toArray();
        assert.eq(0, sessionList.length, tojson(sessionList));
 
 
        // Make sure pipelining other collections fail.
        assertErrorCode(admin.system.collections, pipeline, ErrorCodes.InvalidNamespace);
    } finally {
        admin.setLogLevel(originalLogLevel);
    }
})(); 

Sprint: Query 2018-11-19, Query 2018-12-03
Participants:

 Description   

Observed on 3.6.7. I haven't checked if previous/later versions are also affected.

It seems that a $match stage following $listSessions is ignored.

 Without $match stage:

Josefs-MacBook-Pro(mongos-3.6.7)[mongos] config> db.system.sessions.aggregate( [  { $listSessions: { allUsers: true } } ], {cursor:{}} )
{
  "cursor": {
    "id": NumberLong("0"),
    "ns": "config.system.sessions",
    "firstBatch": [
      {
        "_id": {
          "id": UUID("fade4449-fde7-482f-8885-eadba6cdfdc0")),
          "uid": BinData(0, "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
        },
        "lastUse": ISODate("2018-09-19T11:19:14.542Z")
      },
      {
        "_id": {
          "id": UUID("e048de11-40bd-4c9a-9d49-57312a7038a3")),
          "uid": BinData(0, "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
        },
        "lastUse": ISODate("2018-09-19T11:24:07.797Z")
      },
      {
        "_id": {
          "id": UUID("282ffe3c-d93a-414b-9a4a-69169689fdee")),
          "uid": BinData(0, "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
        },
        "lastUse": ISODate("2018-09-19T11:24:07.797Z")
      }
    ]
  },
  "ok": 1,
  "operationTime": Timestamp(1537356383, 1),
  "$clusterTime": {
    "clusterTime": Timestamp(1537356388, 1),
    "signature": {
      "hash": BinData(0, "AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId": NumberLong("0")
    }
  }
}

With $match stage:

Josefs-MacBook-Pro(mongos-3.6.7)[mongos] config> db.system.sessions.aggregate( [  { $listSessions: { allUsers: true } }, {"$match":{ "_id.id":UUID("e048de11-40bd-4c9a-9d49-57313") }} ], {cursor:{}} )
{
  "cursor": {
    "id": NumberLong("0"),
    "ns": "config.system.sessions",
    "firstBatch": [
      {
        "_id": {
          "id": UUID("fade4449-fde7-482f-8885-eadba6cdfdc0")),
          "uid": BinData(0, "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
        },
        "lastUse": ISODate("2018-09-19T11:19:14.542Z")
      },
      {
        "_id": {
          "id": UUID("e048de11-40bd-4c9a-9d49-57312a7038a3")),
          "uid": BinData(0, "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
        },
        "lastUse": ISODate("2018-09-19T11:24:07.797Z")
      },
      {
        "_id": {
          "id": UUID("282ffe3c-d93a-414b-9a4a-69169689fdee")),
          "uid": BinData(0, "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
        },
        "lastUse": ISODate("2018-09-19T11:24:07.797Z")
      }
    ]
  },
  "ok": 1,
  "operationTime": Timestamp(1537356383, 1),
  "$clusterTime": {
    "clusterTime": Timestamp(1537356383, 1),
    "signature": {
      "hash": BinData(0, "AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId": NumberLong("0")
    }
  }
}



 Comments   
Comment by Githook User [ 07/Dec/18 ]

Author:

{'name': 'Charlie Swanson', 'email': 'charlie.swanson@mongodb.com', 'username': 'cswanson310'}

Message: SERVER-37200 Avoid incorrect $listSessions + $match optimization

(cherry picked from commit d3c939f0fa71118cb7d65296bd449265c52f67eb)
Branch: v3.6
https://github.com/mongodb/mongo/commit/4b967bc68b65afd145b8b0a98c4868fd217bd14e

Comment by Githook User [ 07/Dec/18 ]

Author:

{'name': 'Charlie Swanson', 'email': 'charlie.swanson@mongodb.com', 'username': 'cswanson310'}

Message: SERVER-37200 Avoid incorrect $listSessions + $match optimization
Branch: v4.0
https://github.com/mongodb/mongo/commit/32ca7344593a6c8e283f74295f36200fff889484

Comment by Githook User [ 19/Nov/18 ]

Author:

{'name': 'Jacob Evans', 'email': 'jacob.evans@10gen.com'}

Message: SERVER-37200 Communicate optimized $listSessions correctly from mongos to mongod
Branch: master
https://github.com/mongodb/mongo/commit/c3c6d2b2ba473caa78841b37743a747e46a18d9d

Comment by James Wahlin [ 08/Oct/18 ]

asya- injecting a stage in-between the 2 won't work unless it can trick our optimizer into not combining match stages. I think the best way to work around this would be to query the config.system.sessions collection directly, without using $listSessions.

Comment by Asya Kamsky [ 08/Oct/18 ]

james.wahlin is there a workaround? Would injecting a stage between $listSessions and $match negate what's happening?

Comment by James Wahlin [ 05/Oct/18 ]

This issue arises because DocumentSourceListSessions is derived from DocumentSourceMatch and does not account for pipeline optimization which will merge a subsequent DocumentSourceMatch into a DocumentSourceListSessions stage. While the optimization itself works correctly, the merged result is written to the DocumentSourceMatch::_predicate. When this optimization occurs on mongos, DocumentSourceListSessions serialization then ignores the merged DocumentSourceMatch::_predicate field and sends a $listSessions command to mongod without the $match predicates.

Comment by James Wahlin [ 05/Oct/18 ]

I can reproduce this against 4.0.3 as well. This is an issue for aggregations run against mongos only. When run on mongod, the $match stage filters as expected.

Generated at Thu Feb 08 04:45:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.