ExportXMLWordPrintableJSON

    • Type: Task
    • Resolution: Fix Not Needed
    • Priority: Unknown
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Dotnet Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      On the unmerged native-query branch (NativeQueryOngoing, epic EF-322), a single-level reference Include such as Orders.Include(o => o.Customer) is served by the driver-LINQ fallback, not by the native translator. This is the first slice of the joins work stream and the next piece in the recorded native cutover order.

      The native machinery to emit the pipeline already exists and is dormant: a read-only spike drove the lowerer at IR level and it produced exactly the right pipeline.

      [{ "$lookup": { "from": "Customers", "localField": "CustomerId",
                      "foreignField": "_id", "as": "_lookup_Customer" } },
       { "$unwind": { "path": "$_lookup_Customer", "preserveNullAndEmptyArrays": true } }]
      
      

      The lowerer is never reached. Rejection happens earlier, at translation time, at two sites that both drive the route to Fallback: the native slot populator's catch-all does not whitelist Join/GroupJoin/LeftJoin, and the trailing projection selector that nav-expansion always emits matches none of the existing pass-through predicates. Each is a small change.

      What nav-expansion actually produces

      Measured identically on EF8, EF9 and EF10:

      DbSet(Order)
          .Join(inner: DbSet(Customer),
                outerKeySelector: o => EF.Property(o, "CustomerId"),
                innerKeySelector: c => EF.Property(c, "Id"),
                resultSelector: (o, i) => TransparentIdentifier(Outer = o, Inner = i))
          .Select(u => Include(Entity: u.Outer, Navigation: Customer, u.Inner))
      
      

      A required FK nav-expands to Queryable.Join; an optional FK to LeftJoin. This corrects the Query area docs, which say reference Include uniformly expands to a LeftJoin, and the distinction is not cosmetic (see the hazard below). Filters and sorts compose ahead of the join, which already matches the lowerer's stage order, so no reordering work is needed.

      Correctness hazard that must ship with the recognizer

      The native lowerer's reference arm emits preserveNullAndEmptyArrays: true unconditionally. The driver-LINQ path it replaces emits a bare $unwind for a required FK, which is an INNER join, and preserveNullAndEmptyArrays: true only for an optional FK.

      So for the required-FK case, going native as-is would change the result set: an order whose FK matches no customer document is dropped today and would be returned with a null navigation natively. MongoDB has no referential integrity, so a dangling FK is an ordinary data state, not a contrived one. That is a Native-versus-DriverLinq row-count divergence on the default query mode, which the versioning rubric's not-a-break carve-out for changed MQL does not cover.

      The fix is cheap: pass the correct preserveNullAndEmptyArrays value through, keyed on requiredness. It must land in the same task as the recognizer, gated on a differential test over a dangling-FK seed asserting Native equals DriverLinq. No existing fixture has such a seed, so one must be added.

      Central design question

      The emit side and the DOM read side are currently guaranteed to disagree for this shape. Lookup-alias synthesis fires only when the query expression reports that it uses driver join field names, and the DOM shaper reads the driver's _outer/_inner fields under exactly the same condition. For a lone reference Include that flag is always set the driver way.

      The reference read-back is therefore not missing from the DOM shaper, it is on the wrong side of a flag. The clean fix is to make emit and read flip together at translation time, by registering a real reference lookup so both agree on the lookup alias. But that same state also selects the driver-LINQ bridge's flat-lookup path, which today is only ever exercised for multi-join queries, so the fallback's emitted MQL changes too.

      Whether that flat fallback produces correct results for a lone reference Include is the one blocking unknown; a prototype is settling it before the design is written. The alternative, making the shaper alias-aware only on the native path, is ruled out: the shaper is built at translation time and native-versus-fallback is decided later, so it would be handed driver-shaped documents on any late fallback. That is a governing hazard already recorded in the Query area docs.

      Measured scope

      A read-only spike measured the spec delta on both axes against branch tip.

      • Axis 1, spec cases moving from failing to passing under NativeOnly: 56. 7 methods across the 4 Northwind Include suites, times 2 async values.
      • Axis 2, MQL baselines needing a rewrite: 56. 100 percent of the axis-1 movers carry a non-empty baseline containing the driver's _outer field, so the two axes coincide for this slice.

      Three of the seven methods compose an extra operator over the Include (Distinct, SingleOrDefault, First), so a deliberately minimal slice might land 32 rather than 56.

      Two corrections to the recorded estimates. The status report attributes a 54-test bucket to reference-nav lookups; the count is right but the label is wrong. All 54 are multi-level (48) or filtered (6) Includes, so this slice moves 0 of them. Separately, the same guard that rejects those multi-level cases is a single throw, and 212 of the 498 Include-suite failures are multi-level, so generalizing the lookup lowerer to chained paths would reach roughly 268 cases rather than 56. That is the adjacent prize, deliberately out of scope here, and it is the honest reason the 54 framing misled in both directions.

      Scope

      In scope: single-level, single-navigation reference Include, both required and optional FK, tracked and no-tracking, composing with filter/sort/paging ahead of the Include.

      Explicit declines, each with a tripwire test, and each keeping its current disposition unchanged: ThenInclude and transitive reference hops; two reference Includes on one query; composite FK or composite PK; reference Include composed after a terminal operator; reference plus collection Include on the same query; filtered Include.

      Out of scope, tracked separately: the streaming (one-pass) materializer path. Its reference plumbing is built, but the streaming eligibility gate rejects any root whose reference target carries a non-owned collection navigation, which is the ordinary bidirectional case, so streaming stays dormant for realistic models until that gate changes. That is a separate change with its own correctness story, not an activation of a dormant path.

      To decide explicitly: on EF8 and EF9 the optional-FK case uses an EF-internal LeftJoin that never reaches the provider's translation at all, so optional-FK reference Include hard-fails in every mode there today. A recognizer keyed on the Include shape rather than on the LINQ operator would turn that hard failure into a working query. A genuine improvement, but real added scope, so it should be a decision rather than a side effect.

      Non-goals

      • General join support. The recognizer is provably disjoint from a user-authored join: EF never wraps a user result selector in an Include node, so a user join still falls through both gates and still routes to the fallback.
      • Retiring the driver-LINQ bridge. That file is also used by the EF9-plus bulk update and delete plan builders.

            Assignee:
            Unassigned
            Reporter:
            Arthur Vickers
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: