[CSHARP-3965] OrderBy with an expression that doesn't resolve to a field is not supported Created: 11/Nov/21  Updated: 01/Nov/22  Resolved: 01/Nov/22

Status: Closed
Project: C# Driver
Component/s: LINQ3
Affects Version/s: 2.13.2
Fix Version/s: 2.19.0

Type: New Feature Priority: Major - P3
Reporter: Rick van Lieshout Assignee: Robert Stam
Resolution: Done Votes: 8
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File exception.json     PNG File image-2021-11-11-10-14-04-081.png    
Issue Links:
Duplicate
is duplicated by CSHARP-2711 only fields are allowed in a $sort wh... Closed
is duplicated by CSHARP-3911 LINQ3: OrderBy/OrderByDescending with... Closed
Backwards Compatibility: Fully Compatible

 Description   

When compiling a Queryable with an order-by expression that uses null-coalescing to default(T) errors are thrown during execution.

I've attached a picture with one of our queries that results in an error.
I've also attached our problem details error json.

2022-11-12: See comment below for a minimal reproduction.



 Comments   
Comment by Githook User [ 01/Nov/22 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-3965: Support OrderBy with an expression that does not resolve to a field.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/64518d94a51a4d7c08063b60d409c0085e5a321c

Comment by James Kovacs [ 26/Jan/22 ]

Hi, Rick,

Thanks for reaching out again. This enhancement has not been prioritized yet compared to other work in our backlog. Please feel free to upvote the issue as we do take votes into account when prioritizing new feature requests.

James

Comment by Rick van Lieshout [ 26/Jan/22 ]

Any updates perhaps?

Comment by Rick van Lieshout [ 12/Nov/21 ]

Hey Robert,

I also feel that the driver should support the use-case
I agreed with the move from bug -> task (or feature) because I understand that it "works as originally intended" but the (newer) LINQ syntax support would be cool.

Comment by Robert Stam [ 12/Nov/21 ]

Reopening this ticket because the workaround proposed could actually be implemented internally by the LINQ provider itself.

For example:

.OrderBy(x => f(x))
 
=>
 
{ $project : { _sortedDocument : "$$ROOT", _key1 : <f(x)>, _id : 0 } }
{ $sort : { _key1 : 1 } }
{ $replaceRoot : { newRoot : "$_sortedDocument" } }

Using `_key1` as the name of the key to sort by allows for adding additional sort keys (`_key2` etc...) if `ThenBy` methods are part of the LINQ expression.
 

Comment by Robert Stam [ 12/Nov/21 ]

This is due to how the server $sort stage works. The $sort stage can only sort documents by fields, not by arbitrary computed expressions.

See: https://docs.mongodb.com/manual/reference/operator/aggregation/sort

I used the following class to reproduce this exception as simply as possible:

private class C
{
    public int Id { get; set; }
    public int? X { get; set; }
}

The following test reproduces what you are seeing:

[Fact]
public void Linq_throws_when_OrderBy_key_is_not_a_field()
{
    var client = DriverTestConfiguration.Client;
    var database = client.GetDatabase("test");
    var collection = database.GetCollection<C>("test");
 
    var queryable = collection.AsQueryable().OrderBy(x => x.X ?? 1);
    var exception = Record.Exception(() => queryable.ToList());
 
    exception.Should().BeOfType<NotSupportedException>();
}

You can work around this server limitation by creating a temporary field set to the value you want to sort by. NOTE: because of limitations in the current LINQ implementation (known as LINQ2) this workaround only works with the new LINQ implementation (known as LINQ3). LINQ3 is available in beta form in the 2.14.0-beta1 version of the driver.

For example:

[Fact]
public void Linq3_workaround()
{
    var client = DriverTestConfiguration.Linq3Client;
    var database = client.GetDatabase("test");
    var collection = database.GetCollection<C>("test");
 
    collection.Database.DropCollection("test");
    collection.InsertMany(new[]
    {
        new C { Id = 1, X = 3 },
        new C { Id = 2, X = 2 },
        new C { Id = 3, X = null },
    });
 
    var queryable = collection.AsQueryable()
        .Select(x => new { Key = x.X ?? 1, Root = x }) // Key is a temporary field to sort by
        .OrderBy(x => x.Key)
        .Select(x => x.Root);
    var result = queryable.ToList();
 
    result.Select(x => x.Id).Should().Equal(3, 2, 1);
}

Generated at Wed Feb 07 21:46:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.