Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-93773

Ensure running aggregation on an existing database in write_change_stream_pit_preimage test

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 8.1.0-rc0
    • Affects Version/s: 8.1.0-rc0
    • Component/s: Change streams
    • None
    • Query Execution
    • Fully Compatible
    • ALL
    • Hide

      Run

      buildscripts/resmoke.py run --suites=replica_sets_jscore_passthrough_with_replica_set_endpoint --excludeWithAnyTags=incompatible_aubsan --jobs=8 --shuffle --runAllFeatureFlagTests '--mongodSetParameters={internalQueryEnableAggressiveSpillsInGroup: true}' --storageEngineCacheSizeGB=1 --mongodSetParameters='{internalQueryEnableAggressiveSpillsInGroup: True}' --mongosSetParameters='{}' jstests/core/write/write_change_stream_pit_preimage.js
      Show
      Run buildscripts/resmoke.py run --suites=replica_sets_jscore_passthrough_with_replica_set_endpoint --excludeWithAnyTags=incompatible_aubsan --jobs=8 --shuffle --runAllFeatureFlagTests '--mongodSetParameters={internalQueryEnableAggressiveSpillsInGroup: true}' --storageEngineCacheSizeGB=1 --mongodSetParameters='{internalQueryEnableAggressiveSpillsInGroup: True}' --mongosSetParameters='{}' jstests/core/write/write_change_stream_pit_preimage.js
    • QE 2024-09-02
    • 200

      The bugfix PR for https://jira.mongodb.org/browse/SERVER-93058 (https://github.com/10gen/mongo/pull/25784) caused some testsuite to fail.

      https://jira.mongodb.org/browse/BF-34620 was opened to track these test failures.

      The test that fails is

              uncaught exception: Error: command failed: {
                      "ok" : 0,
                      "errmsg" : "pipeline is required to run on mongoS, but cannot :: caused by :: $merge must write to disk",
                      "code" : 20,
                      "codeName" : "IllegalOperation",
                      "$clusterTime" : {
                              "clusterTime" : Timestamp(1724143331, 19),
                              "signature" : {
                                      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                      "keyId" : NumberLong(0)

      The aggregate command that triggers the failure is the following:

              db.aggregate([
                  {$documents: [updatedDoc]},
                  {
                      $merge: {
                          into: {db: coll.getDB().getName(), coll: coll.getName()},
                          on: "_id",
                          whenMatched: "merge",
                          whenNotMatched: "fail"
                      }
                  }
              ]);

      It is an aggregate which does not run on a particular collection, but runs on the database with a $documents input.

      The database `db` at the point of execution of the aggregate command is a database named `test`, which does not exist.

      When executing the aggregation command on the replica set, there is no collection routing information (CRI) for the db `test`, so the following code decides that the merge must run on mongos:

      https://github.com/10gen/mongo/blob/c7df2842470f7ef497684f781a4203d69bc7aac6/src/mongo/s/query/planner/cluster_aggregation_planner.cpp#L773

      But as the merge uses the `$out` operator, it cannot be executed on mongos, and the command fails.

      Changing the aggregate command in the test so that it uses `testDB` instead of `db` fixes the issue. The reason is that `testDB` points to an existing database when the test executes.

      It is current unclear if the test is supposed to work when running the aggregate on a db that does not exist.

      The manual (https://www.mongodb.com/docs/manual/reference/operator/aggregation/merge/) states that

      • If the output collection does not exist, $merge creates the collection:
        • For a replica set or a standalone, if the output database does not exist, $merge also creates the database.
        • For a sharded cluster, the specified output database must already exist.

      But here the problem is not the output collection, but rather the input.

            Assignee:
            jan.steemann@mongodb.com Jan Steemann
            Reporter:
            jan.steemann@mongodb.com Jan Steemann
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: