$lookup on sharded view with concurrent drops and create matches no documents

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Execution
    • ALL
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Quick summary:  $lookup doesn't handle multiple view kickbacks correctly. The repro explains one situation where multiple view kickbacks are possible.

      Repro steps (also can be found in the comments):

      1. Run a $lookup on a sharded view as the foreign collection (view A backed by collection B).
      2. After the 'CommandOnShardedViewNotSupportedOnMongod' is raised and while mongos is resolving the view, we drop collection B and recreate it as view B on collection C. This can also happen between getMores (the collection is remade between getMores).
      3. We will finish the query and the new batches will return no matched results.

      This was found because of the viewless timeseries upgrade/downgrade effort. SERVER-120567 implemented a fix specific to timeseries, since this can happen during a FCV downgrade turning viewless timeseries to viewful timeseries, and this downgrade cannot cause concurrent queries to return incorrect results.

      This ticket should be used to determine the correct behavior outside of the viewless timeseries upgrade/downgrade path. We have 2 options

      1. Fail the query. If the foreign collection is dropped and remade between getMores, we should fail the query since the collection has changed.
      2. Apply the fix from SERVER-120567 to all collections. We decide it is OK for the collection to change mid-aggregation and $lookup should return accurate results based on what documents the foreign collection has at that time. I had a PR up for this before deciding to make the change specific to timeseries and give this ticket to QE.

            Assignee:
            Foteini Alvanaki
            Reporter:
            Gil Alon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: