LINQ `Join`/`GroupJoin` with an uncorrelated `OrderBy(...).Take(n)`/`Skip(n)` inner subquery returns wrong results

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Unknown
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • None
    • Dotnet Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      This is an intentional overlap with CSHARP-6118 - that one will be used to track supporting sub-queries properly in a later release. This one should be used to throw ExpressionNotSupportedException as before.

      When the inner sequence of a Queryable.Join/GroupJoin is an uncorrelated subquery containing order-dependent operators (OrderBy/Take/Skip), 3.10 translates the subquery's stages into the correlated $lookup sub-pipeline. Because that sub-pipeline runs against the foreign documents already correlated to each outer document, the $sort+$limit (or $skip) execute per outer document rather than once over the whole subquery. The query silently returns far more rows than the LINQ expresses.

      In 3.9 this shape was rejected with ExpressionNotSupportedException; 3.10 accepts it and produces incorrect data (no error), which is the more dangerous failure mode.

      This was introduced by CSHARP-6017 (#2017, commit 5475e008f5) — the LeftJoin work reworked the shared JoinMethodToPipelineTranslator (which also serves plain Join) to translate a non-collection inner into a $lookup carrying both localField/foreignField and the inner sub-pipeline. It does not distinguish correlation-invariant stages ($match on a constant predicate — safe) from order-dependent stages ($sort/$limit/$skip — not safe to run correlated).

      Steps to reproduce

      LINQ (Northwind-style), inner is the global first N orders:

      var query =
          from c in db.Customers
          join o in db.Orders.OrderBy(o => o.OrderId).Take(4)
              on c.CustomerId equals o.CustomerId
          select new { c.ContactName, o.OrderId };
      var result = query.ToList();
      

      Actual (3.10.0)

      Emitted MQL folds $sort+$limit into the correlated $lookup:

      Customers.
      { "$project": { "_outer": "$$ROOT", "_id": 0 } },
      { "$lookup": {
          "from": "Orders",
          "localField": "_outer._id",
          "foreignField": "CustomerID",
          "pipeline": [ { "$sort": { "_id": 1 } }, { "$limit": 4 } ],
          "as": "_inner" } },
      { "$unwind": "$_inner" }, ...
      

      Take(4) becomes "up to 4 orders per customer" -> 342 rows.

      A predicate + take variant (Orders.Where(o => o.OrderId > 0).OrderBy(...).Take(5)) behaves the same -> 5 rows instead of 1, with $match$sort$limit:5 all inside the correlated $lookup.

      Fix (this ticket): restore the 3.9 behaviour — throw

      The scope of this ticket is only to stop returning wrong data. Restore the pre-3.10 behaviour: a Join/GroupJoin whose inner is a subquery (anything other than a bare collection) must again be rejected with ExpressionNotSupportedException, exactly as 3.9 did. It is always safe to throw for an unsupported shape; it is never acceptable to return incorrect results.

      Correctly supporting inner subqueries (folding correlation-invariant $match into the $lookup while evaluating order-dependent operators — OrderBy/Take/Skip — once before the join) is deliberately out of scope here and will be tracked in a separate follow-up ticket.

      Where to fix

      src/MongoDB.Driver/Linq/Linq3Implementation/Translators/ExpressionToPipelineTranslators/JoinMethodToPipelineTranslator.cs — the else branch added for non-ConstantExpression inner sequences (which builds innerFilterPipeline and emits AstStage.Lookup(innerCollectionName, localField, foreignField, [], innerFilterPipeline, "_inner")). Remove/guard that branch so a non-collection inner falls through to ExpressionNotSupportedException as it did in 3.9, i.e. keep the pipeline-carrying $lookup form off the plain Join/GroupJoin path until proper support lands.

            Assignee:
            Damien Guard
            Reporter:
            Damien Guard
            None
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: