[CSHARP-3988] Filtering does not work properly on Time-Series enabled collections Created: 08/Dec/21 Updated: 27/Oct/23 Resolved: 24/Dec/21 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Unknown |
| Reporter: | Dennis Stuhr | Assignee: | Robert Stam |
| Resolution: | Gone away | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Description |
|
I'm using the C# driver version 2.14.0 and I am unable to make filtering work properly on time-series collections. A little about my setup: I'm using .NET 6.0, Visual Studio 2022, my app is running via docker-compose where MongoDB 5.0.4 is used as a datastore via an image definition in docker-compose. It works. I have created several collections that all have the time-series feature enabled in the same way:
This works. The time series collection is created and I can verify that the collections are created as time series collection in the MongoDb container. All my other methods in my generic repositories also works as expected. I can create, delete and find single records denoted by their id, using Linq. However, I've created some integration tests that I cannot get to work properly when using date filtering:
This does not work. I have tried a myriad of variations without success. The only way I have gotten it to work, is to create a new repository method to replace the generic Find method (which just takes a Linq expression) and practically spell out the "native MongoDB" command to get the results:
This works! But, it would be really nice if I could just use regular Linq expressions like I normally do to create basic filters for my data, instead of having to create specialized methods that incorporates native MongoDb commands to get it to work.
|
| Comments |
| Comment by PM Bot [ 24/Dec/21 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Robert Stam [ 09/Dec/21 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Glad you figured out what was different between your code and my attempt to reproduce. BSON DateTime values stored in the database don't specify which time zone they are in. Originally the server specified that all BSON DateTime values were in UTC (though recent versions of the server have some support for timezones). Passing DateTimeKind.Utc to new DateTimeSerializer(DateTimeKind.Utc) only affects deserialization of BSON DateTime values. When serializing .NET DateTime values to BSON DateTime values (to send to the server) they are always converted to UTC. The recommended best practice is to always use UTC with MongoDB. That means to also use UTC in your data model classes. If you consistently use UTC everywhere that interacts with MongoDB there won't be any local/UTC mismatches.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Dennis Stuhr [ 09/Dec/21 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thanks a lot for the quick feedback Robert! After looking at your code, it dawned on me that you utilize a different method of setting your time values with. So I tried to replicate it, and lo and behold, it worked!
I then inspected the difference in values between "first", "last", "minuteToGetDataFrom" and "minuteToGetDataTo": As you can see, the variables "minuteToGetDataFrom" and "minuteToGetDataTo" are UTC values, whereas "first" and "last" are local time values (UTC + 1). But, using UTC values should work, as I have used theBsonClassMap.RegisterClassMap method to configure the mapping of the properties in the class. And unless I'm misunderstanding something, the following piece of code should denote that MongoDb should interpret the "Timestamp" property (which is a DateTime? type) as being UTC:
I have confirmed that the configuration is being used at the time of extracting the data, as you can see in this screenshot:
When I look in the MongoDb collection after the items have been stored into the database, the ISODate values are all in UTC. I would expect that the above mapping should tell MongoDb, that the "Timestamp" property passed to it, is also using UTC, but it doesn't seem to recognize it. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Robert Stam [ 09/Dec/21 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
I am unable to reproduce the behavior you are seeing. In order to have the simplest possible scenario I have modified your partial code slightly:
This is the test code I used:
The test passes, so it doesn't look like there is a problem in the driver. If you see how my test does not correctly reproduce the problem you are seeing please follow up.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Dennis Stuhr [ 08/Dec/21 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
I forgot to add the Find function in my repository method for reference:
I've tested this with other Linq expressions and they all seem to work. Just not for filtering on the timestamp / datetime constraint on time-series enabled collections... |