Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-39166

$graphLookup should force a pipeline to split in sharded cluster

    XMLWordPrintable

Details

    • Fully Compatible
    • ALL
    • v4.0, v3.6, v3.4
    • Query 2019-02-11

    Description

      The $graphLookup stage does not inherit from NeedsMergerDocumentSource, which means that it is eligible to execute in parallel on all shards. This doesn't work because it will only ever do the searching locally. Instead, it should force itself to split the pipeline and execute in the merging half of the pipeline so that it can always run on the primary shard for the database.

      The $graphLookup StageConstraints are correct in saying that it must run on the primary shard, but this would only apply if some other stage before $graphLookup forced the pipeline to split.

      For example, see the following running against a mongos:

      mongos> db.adminCommand({enableSharding: "test"})
      ...
      mongos> db.adminCommand({shardCollection: "test.foo", key: {_id: "hashed"}})
      ...
      mongos> db.foo.insert([{}, {}, {}, {}])
      ...
      mongos> db.bar.insert({_id: 1, x: 1})
      WriteResult({ "nInserted" : 1 })
      mongos> db.foo.aggregate([{$project: {l_x: {$literal: 1}}}, {$graphLookup: {from: "bar", startWith: "$l_x", connectFromField: "x", connectToField: "_id", as: "res"}}])
      { "_id" : ObjectId("5c48f159a3c842122a8dec28"), "l_x" : 1, "res" : [ ] }  // Uh oh, these should have the same results as below.
      { "_id" : ObjectId("5c48f159a3c842122a8dec29"), "l_x" : 1, "res" : [ ] }
      { "_id" : ObjectId("5c48f159a3c842122a8dec26"), "l_x" : 1, "res" : [ { "_id" : 1, "x" : 1 } ] }
      { "_id" : ObjectId("5c48f159a3c842122a8dec27"), "l_x" : 1, "res" : [ { "_id" : 1, "x" : 1 } ] }
      

      Attachments

        Activity

          People

            martin.neupauer@mongodb.com Martin Neupauer
            charlie.swanson@mongodb.com Charlie Swanson
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: