[CSHARP-2071] Using Linq Aggregation to project the root document Created: 23/Oct/17  Updated: 28/Oct/23  Resolved: 25/Oct/21

Status: Closed
Project: C# Driver
Component/s: Linq
Affects Version/s: 2.4.4
Fix Version/s: 2.14.0

Type: Bug Priority: Major - P3
Reporter: Stephen Bond Assignee: Robert Stam
Resolution: Fixed Votes: 6
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by CSHARP-2104 $project or $group does not support <... Closed
Related
is related to CSHARP-1772 Quering for first item of GroupBy agg... Closed
is related to CSHARP-2104 $project or $group does not support <... Closed

 Description   

Expected Behaviour

Using the aggregation pipeline I can perform the following:

  • project an entire document using the $$ROOT variable
  • perform some filtering
  • then using ReplaceRoot return the same structure

Example below:

db.test.aggregate(
	[
		{
			$project: {
			    "doc": '$$ROOT', "dif":{"$setDifference":["$X",["tag1","tag2"]]},"_id":0
			}
		},
		{
			$match: {
				"dif":{"$size":0}
			}
		},
		{
			$replaceRoot: {
			    newRoot: '$doc'
			}
		},
	]
);

Using Linq, I believe this would be represented using the following Linq query:

collection.AsQueryable()
		.Select(c => new { doc = c, dif = c.X.Except(tags), Id = c.Id })
		.Where(c => c.dif.Count() == 0)
		.Select(c => c);

Actual Behaviour

Returns the following error when executed:

$project or $group does not support {document}.

Stack trace:

   at MongoDB.Driver.Linq.Translators.AggregateLanguageTranslator.TranslateValue(Expression node)
   at MongoDB.Driver.Linq.Translators.AggregateLanguageTranslator.TranslateMapping(ProjectionMapping mapping)
   at MongoDB.Driver.Linq.Translators.AggregateLanguageTranslator.TranslateValue(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslateProjectValue(Expression selector)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslateSelect(SelectExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslateWhere(WhereExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslatePipeline(PipelineExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node, IBsonSerializerRegistry serializerRegistry, ExpressionTranslationOptions translationOptions)
   at MongoDB.Driver.Linq.MongoQueryProviderImpl`1.Translate(Expression expression)
   at MongoDB.Driver.Linq.MongoQueryProviderImpl`1.Execute(Expression expression)
   at MongoDB.Driver.Linq.MongoQueryableImpl`2.GetEnumerator()
   at MongoDB.Driver.Linq.MongoQueryableImpl`2.System.Collections.IEnumerable.GetEnumerator()



 Comments   
Comment by Richard Collette [ 29/Oct/21 ]

 Hours spent on this today.  

Hopefully the new version will also address a related typing issue.

At design time, the IGrouping<> is seen as an IEnumerable, but at runtime, it wan't an array constructor.   To get things to compile and get to the point where the error mentioned in this ticket occurs, I have to add an unused constructor to my Group class like:

private record DataDeliveryGroup
{
    public DataDeliveryGroup(IEnumerable<DataDelivery> deliveries)
    {
        Deliveries = deliveries.ToArray();
    }
 
    public DataDeliveryGroup(DataDelivery[] deliveries)
    {
        Deliveries = deliveries;
    }
 
    public DataDelivery[] Deliveries { get; } = null!;
} 

WhereAs I could have just done something simpler like

private record DataDeliveryGroup(DataDelivery[] Deliveries);

or

private record DataDeliveryGroup(IEnumerable<DataDeliveries> Deliveries);

If the types were in alignment

In the mean time, I fell back to using BsonDocument (As is frequently necessary)

     
private static readonly BsonDocument s_perpetualLicenseGroupProjection = BsonDocument.Parse(@"
{ _id:{
      'releaseFriendlyName': '$releaseFriendlyName',
      'vintageFriendlyName': '$releaseInfo.vintageFriendlyName',
  },
  'deliveries': { $push : '$$ROOT' }
}");
...
...
 
.Group<DataDeliveryGroup>(s_perpetualLicenseGrouping)
 
...
...
 
 
 
private record DataDeliveryGroup
{ 
// Don't. use a record constructor here, mapping gets confused.
 public DataDelivery[] Deliveries { get; set; } = null!;
}

Comment by Robert Stam [ 25/Oct/21 ]

This issue has been fixed in the new LINQ provider (known as LINQ3) which will be included in the upcoming 2.14 release.

Configure your MongoClientSettings to use LinqProvider.V3 if you want to use this functionality.

To configure a client to use the LINQ3 provider use code like the following

var connectionString = "mongodb://localhost";
var clientSettings = MongoClientSettings.FromConnectionString(connectionString);
clientSettings.LinqProvider = LinqProvider.V3;
var client = new MongoClient(clientSettings);

Comment by Githook User [ 25/Oct/21 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-2071: Verify that the queries reported in this ticket work in LINQ3.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/16de11523f1fd4b25bc0bfd64c84f4b4a1b894da

Comment by Dmitry Lukyanov (Inactive) [ 20/Jan/21 ]

Unfortunately, with the suggested changes we can generate unsupported queries in some cases. Since this logic is already covered by LINQ3, I think we can postpone working on $$ROOT until releasing it

Comment by James Kovacs [ 03/Dec/20 ]

Thank you for expressing your interest in seeing this issue fixed. Although the PR looks straightforward, modifying LINQ AST parsing can often have unforeseen consequences - fixing some usages while breaking others - especially in our current implementation.

We are currently working on a new LINQ translator to address shortcomings of our current LINQ implementation. We are prioritizing work on this new implementation over fixing unsupported use cases in the current one. We will take this use case into consideration for the new LINQ translator.

Comment by Kirill Rakhman [ 27/Nov/20 ]

Just stumbled open this. Please fix, there's a PR open since 2018.

Comment by Shiya Kohn [ 20/Feb/20 ]

Why is this issue this old and has not been handled, from my dive into the source code this looks like a fairly simple implementation. This is one of many known serialization/translation bugs, that the MongoDB .NET team has been neglecting. There seems to be little updates happening in the practical side of the code base, and it's kinda making hard to maintain a clean code base while using this driver

Comment by Marko Bini? [ 05/Apr/18 ]

I've opened GitHub pull request #319, with a small change that helps resolve this. Please see my comments over there.

Comment by Marko Bini? [ 05/Apr/18 ]

Here's a test that shows it failing (for AggregateGroupTranslatorTests):

[Fact]
public void Should_translate_push_with_root()
{
    var result = Group(x => x.A, g => new { Result = g.Select(x => x) });
 
    result.Projection.Should().Be("{ _id: \"$A\", Result: { \"$push\": \"$$ROOT\" } }");
}

Comment by Stephen Bond [ 23/Oct/17 ]

Sorry forgot to mention the Exception type is: NotSupportedException

Generated at Wed Feb 07 21:41:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.