ISSUE DESCRIPTION AND IMPACT

Matching by $type on a looked up field after a $lookup stage in an aggregation operation leads to incorrect results, because the $match on $type is pushed into the $lookup without removing the reference to the new field created by $lookup. This failure is unique to the $type operator.

For example, the following operation matches on "lookedup.name" after a $lookup into the "lookedup" field:

[
  {
    $lookup: {
      from: "second",
      localField: "field",
      foreignField: "foreignField",
      as: "lookedup",
    },
  },
  { $unwind: "$lookedup" },
  { $match: { "lookedup.name": { $type: "string" } } },
]

MongoDB incorrectly optimizes the aggregation pipeline by attempting to match on "lookedup.name" in the "second" collection, where "lookedup.name" does not exist.

The correct behavior is for the optimized lookup stage to match on “name” instead.

DIAGNOSIS AND AFFECTED VERSIONS

All versions since 3.4.0 are affected by this issue.

Running the .explain() method will help identify the incorrect matching stage.

db.first.explain().aggregate([
  {
    $lookup: {
      from: "second",
      localField: "field",
      foreignField: "foreignField",
      as: "lookedup",
    },
  },
  { $unwind: "$lookedup" },
  { $match: { "lookedup.name": { $type: "string" } } },
]);

If the query optimizer moves the $match stage into the $lookup stage while still referencing the field created by $lookup, the operation is impacted. For example:

{
  $lookup: {
    from: "second",
    as: "lookedup",
    localField: "field",
    foreignField: "foreignField",
    unwinding: { preserveNullAndEmptyArrays: false },
    matching: { "lookedup.name": { $type: [2] } },
  },
};

where

matching: {“lookedup.name”: { $type: [2] } }

should be

matching: {“name”: { $type: [2] } }

REMEDIATION AND WORKAROUNDS
This fix will be included in 4.4.2, 4.2.11, 4.0.22, 3.6.21. To work around this issue, you can add a new field "newField" that represents "lookedup.name", run the match on "newField", then unset "newField".

db.first.aggregate([
  {
    $lookup: {
      from: "second",
      localField: "field",
      foreignField: "foreignField",
      as: "lookedup",
    },
  },
  { $unwind: "$lookedup" },
  { $addFields: { newField: "$lookedup.name" }},
  { $match: { "newField": { $type: "string" } } },
  { $unset: "newField" }
]);