[CSHARP-1315] Refactor LINQ infrastructure to enable support of new aggregation operators in server 3.2. Created: 14/Jun/15  Updated: 27/May/22  Resolved: 02/Sep/15

Status: Closed
Project: C# Driver
Component/s: Linq
Affects Version/s: 2.0.1
Fix Version/s: 2.1

Type: New Feature Priority: Minor - P4
Reporter: Kieren Johnstone Assignee: Craig Wilson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-17943 add $filter expression to $project to... Closed
Duplicate
is duplicated by CSHARP-1405 add $filter expression to $project to... Closed
Server Compat: 3.1
Backwards Compatibility: Minor Change

 Description   

3.1.4 of the server brings the new $project-stage aggregate expression "$filter", allowing for filtering out elements of arrays which are sub-elements of documents.

This works much like $elemMatch in find() - but in an aggregation pipeline.



 Comments   
Comment by Githook User [ 03/Sep/15 ]

Author:

{u'username': u'rstam', u'name': u'rstam', u'email': u'robert@robertstam.org'}

Message: CSHARP-1315: Replace int with object in calls to GetMethodInfo when the type really can be anything, not just int.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/95a34cb2c70481dddfbe363193bde63c2c0edb15

Comment by Githook User [ 02/Sep/15 ]

Author:

{u'username': u'craiggwilson', u'name': u'Craig Wilson', u'email': u'craiggwilson@gmail.com'}

Message: CSHARP-1315: refactor linq to use infrastructure supporting future 3.2
operators.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/347e05ffae6d6e9806603a6504034129d056dd65

Comment by Craig Wilson [ 25/Jun/15 ]

After looking at the Select().Where() problem, I think I'm going to have to do something drastic. It's just not something I thought of initially. I don't believe it will be difficult, just some refactoring.

Comment by Kieren Johnstone [ 25/Jun/15 ]

Ah great! I haven't had a chance to do anything since putting the first PR together. Will be interesting to see what you did to implement.

Comment by Craig Wilson [ 23/Jun/15 ]

k@river.red,

So, I got the embedded chaining working. Not a lot of modification, just the understanding that previously chained methods that don't change the shape can simply use the same serialization info. So, Where().Select() works, and Where().Where().Select(). However, Select().Where() does not. I'll have to figure out what to do there, but for now, this is a good start.

https://github.com/craiggwilson/mongo-csharp-driver/tree/csharp1315

Craig

Comment by Kieren Johnstone [ 16/Jun/15 ]

Friday would be great, I'm in the UK so fairly early would be great if possible (3pm Friday here? = 9am CST). Perhaps you could pass me a Skype name/other details to os at river.red ? Really appreciate it, thanks.

Comment by Craig Wilson [ 15/Jun/15 ]

Ok. Give me some time to think about it. Thursday or Friday? I'm in CST (Chicago, USA time zone). Let me know what times work for you or if another day is better.

Comment by Kieren Johnstone [ 15/Jun/15 ]

Ah excellent, thanks. Basically this works fine with what I have:

        [Test]
        [RequiresServer(MinimumVersion = "3.1.3")]
        public async Task Should_translate_where_to_filter()
        {
            var result = await Project(x => new { Result = x.G.Where(c => c.E.F == 33) });
 
            result.Projection.Should().Be("{ Result: { \"$filter\": { \"input\": \"$G\", \"as\": \"c\", \"cond\": { \"$eq\": [\"$$c.E.F\", 33] } } }, _id: 0 }");
 
            result.Value.Result.Should().HaveCount(1);
            result.Value.Result.Single().D.Should().Be("Don't");
        }

This does not:

        [Test]
        [RequiresServer(MinimumVersion = "3.1.3")]
        public async Task Should_translate_where_and_select_to_map_and_filter()
        {
            var result = await Project(x => new { Result = x.G.Where(c => c.E.F == 33).Select(y => new { result = y.D }) });
 
            result.Projection.Should().Be("TBD - should be a $map with a $filter as the input");
 
            result.Value.Result.Should().HaveCount(1);
            result.Value.Result.Single().result.Should().Be("Don't");
        }

It's the "y.D" part in the Select not being 'covered' by existing code - the serialization info binder doesn't dig into it, and the projection transformer can't understand things like that without serialization info.

A Skype or similar would be great, which timezone/times do you work in?

Thanks again
Kieren

Comment by Craig Wilson [ 15/Jun/15 ]

No quick pointers. LINQ is just hard and MongoDB LINQ is different than most examples out there because we support very different sets of expressions. Could you provide an example of what exactly you are trying to do with the Select and maybe I can help. We could even jump on a video call if necessary. I'm just not envisioning exactly where the Select fits in.

Craig

Comment by Kieren Johnstone [ 15/Jun/15 ]

I'm fairly confident I have this completed as an isolated feature - in a projection you can do a Where() and have it map to a $filter. However I was hoping to support a following Select() too. I'm still circling in on the solution - but it just doesn't look like the SerializationInfoBinder / AggregateLanguageTransformer are set up for chaining calls. There seem to be a few approaches possible:

  • With no changes to SerializationInfoBinder, it won't bind the contents of the chained Select()'s lambda. This means in the TryBindMap stage, the BuildValue on this lambda fails. Solution - implement/get binding information at this stage? But seems like we should have preprocessed this?
  • Having SerializationInfoBinder make the Where() a SerializationExpression means this information is available. It kind of glean that this is not correct, that an ISerializationExpression should basically be just a field or chain of field accessors in Mongo. To have the parameter replacement work in TryBindMap, we'd need to give this SerExpr an element name - but then BuildValue() on the Where() yields this name and no longer the generated $filter expression. Is this a viable way forward?

It would seem that to 'flow' serialization info from the base SerializationExpression through the Where() ready for the Select() needs a change in approach.

Still getting to grips with the role/persona of each class. Is there a quick pointer to give so I could run with this one?

If not, I can prep a PR with just the $filter work.

Comment by Kieren Johnstone [ 14/Jun/15 ]

I could really use this - I've started putting together some code to implement this which I hope will be suitable for a PR.

I have it working successfully, with some tests too - except when chained with a Select(). I'm working on this actively, hope the code is written in the correct style - will keep this ticket updated.

Generated at Wed Feb 07 21:39:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.