Uploaded image for project: 'C# Driver'
  1. C# Driver
  2. CSHARP-3988

Filtering does not work properly on Time-Series enabled collections

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Gone away
    • Icon: Unknown Unknown
    • None
    • None
    • None
    • None

    Description

      I'm using the C# driver version 2.14.0 and I am unable to make filtering work properly on time-series collections.

      A little about my setup: I'm using .NET 6.0, Visual Studio 2022, my app is running via docker-compose where MongoDB 5.0.4 is used as a datastore via an image definition in docker-compose. It works.

      I have created several collections that all have the time-series feature enabled in the same way:

       

      var collectionName = Context.GetCollectionName<ARandomClass>(); // gets collection name from attribute defined for class
      var filter = new BsonDocument("name", collectionName);
      var collectionCursor = Database.ListCollections(new ListCollectionsOptions { Filter = filter });
      if (collectionCursor.Any()) return;
      Database.CreateCollection(collectionName, new CreateCollectionOptions { TimeSeriesOptions = new TimeSeriesOptions(nameof(ARandomClass.Timestamp), nameof(ARandomClass.SessionId), Optional.Create<TimeSeriesGranularity?>(TimeSeriesGranularity.Minutes)
       });
      Collection = Database.GetCollection<ARandomClass>(collectionName);
      

       

      This works. The time series collection is created and I can verify that the collections are created as time series collection in the MongoDb container. All my other methods in my generic repositories also works as expected. I can create, delete and find single records denoted by their id, using Linq.

      However, I've created some integration tests that I cannot get to work properly when using date filtering:

       

       

      [Fact]
      public async void Data_Is_Stored_As_TimeSeries_Data_And_Queryable_In_Time_Buckets()
      {
          var ARandomClassRepository = _unitOfWorkFactory.CreateUnitOfWork().Repository<IARandomClassRepository>();
          await ARandomClassRepository.DeleteAll();
          var sessionIds = new [] { 111, 222, 333, 444 };
          var timestampStart = DateTime.UtcNow;
          var dataList = new List<ARandomClass>();
          for (var i = 0; i < 1000; i++)
          {
              dataList.Add(new ARandomClass
              {
                  SessionId = sessionIds[RandomNo.Between(0, sessionIds.Length)],
                  Timestamp = timestampStart.AddSeconds(i),
                  ActivityLevel = RandomNo.Between(1, 100),
                  Amplitude = RandomNo.Between(1, 1000),
                  RrInterval = RandomNo.Between(1, 10000)
              });
          }
          await ARandomClassRepository.CreateMany(dataList);
          var minuteToGetDataFrom = timestampStart.WithoutSeconds().AddMinutes(1);
          var minuteToGetDataTo = minuteToGetDataFrom.AddMinutes(1);
          var randomClasses = await ARandomClassRepository.Find(x => x.Timestamp >= minuteToGetDataFrom && x.Timestamp <= minuteToGetDataTo).ToList();
          randomClasses.ShouldNotBeNull();
          randomClasses.Count.ShouldBeGreaterThan(0);
          await lifetouchHeartBeatRepository.DeleteAll();
       }
      

       

      This does not work. I have tried a myriad of variations without success. The only way I have gotten it to work, is to create a new repository method to replace the generic Find method (which just takes a Linq expression) and practically spell out the  "native MongoDB" command to get the results:

       

       

      public async Task<List<ARandomClass>> GetBetweenDateRanges(DateTime startDate, DateTime endDate)
      {
          var filterStage = new StringBuilder()
              .Append("{$match:{$and:[ {$expr: {$gte:")
              .Append("[ \"$Timestamp\",")
              .Append($"ISODate('{startDate.ToString("yyyy-MM-ddTHH:mm:ss.fffZ")}'),")
              .Append("]}}, {$expr:{ $lte:")
              .Append("[ \"$Timestamp\",")
              .Append($"ISODate('{endDate.ToString("yyyy-MM-ddTHH:mm:ss.fffZ")}')")
              .Append("]}} ]}}");
          return await Collection.Aggregate().AppendStage<LifetouchHeartBeat>(filterStage.ToString()).ToListAsync();
      }
      

       

      This works!

      But, it would be really nice if I could just use regular Linq expressions like I normally do to create basic filters for my data, instead of having to create specialized methods that incorporates native MongoDb commands to get it to work.

       

       

       

      Attachments

        1. image-2021-12-09-14-06-27-296.png
          image-2021-12-09-14-06-27-296.png
          35 kB
        2. image-2021-12-09-14-08-14-604.png
          image-2021-12-09-14-08-14-604.png
          19 kB
        3. image-2021-12-09-14-21-52-328.png
          image-2021-12-09-14-21-52-328.png
          213 kB

        Activity

          People

            robert@mongodb.com Robert Stam
            dennisstuhr@gmail.com Dennis Stuhr
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: