[CSHARP-4445] Support 64-bit values for Skip and Limit/Take Created: 08/Dec/22  Updated: 23/Mar/23  Resolved: 14/Dec/22

Status: Closed
Project: C# Driver
Component/s: Aggregation, Builders
Affects Version/s: 2.15.1, 2.18.0
Fix Version/s: 2.19.0

Type: Improvement Priority: Unknown
Reporter: Justin Chase Assignee: Robert Stam
Resolution: Done Votes: 0
Labels: Bug
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Summary

The pipeline builder has an offset argument of type `int`:
https://github.com/mongodb/mongo-csharp-driver/blob/master/src/MongoDB.Driver/PipelineStageDefinitionBuilder.cs#L1582

Which in C# is a 32-bit signed integer.

But the specification for the $skip function is actually a 64-bit positive integer:

https://www.mongodb.com/docs/rapid/reference/operator/aggregation/skip/

The documentation doesn't make it clear if its supposed to be a signed or unsigned 64 bit integer but given that it says "positive" I'm assuming its a signed integer equal to or greater than 0.

Please provide the version of the driver. If applicable, please provide the MongoDB server version and topology (standalone, replica set, or sharded cluster).

I'm using 1.15.1, but the code is still in master so at this time 1.18.0, probably all recent versions.

All topologies.

How to Reproduce

Steps to reproduce. If possible, please include a Short, Self Contained, Correct (Compilable), Example.

PipelineStageDefinitionBuilder.Skip<Example>(long.MaxValue)

Additional Background

Please provide any additional background information that may be helpful in diagnosing the bug.



 Comments   
Comment by Githook User [ 14/Dec/22 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-4445: Support 64-bit values for Skip and Limit/Take.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/701dc20d527d20e936ab8f33e8112f2595388a85

Comment by James Kovacs [ 12/Dec/22 ]

Skip and Take appear to have changed in MongoDB 5.0 to 64-bit integers. We should test whether changing these to long breaks pre-5.0 servers. Ensure that this change is made in both builders and LINQ.

Comment by Robert Stam [ 09/Dec/22 ]

Fair points. We will discuss this at our next triage meeting.

Thanks again for bringing this up.

Comment by Justin Chase [ 09/Dec/22 ]

Why not I guess? It seems inaccurate to have it be an int. If you have an int it should cast up but if you need to skip 3 billion rows you're screwed. Unless there is a work around I'm unaware of? I'm guessing it will end up casting it to a long somewhere, since that is the actual data type so why not just use the correct data type?

Comment by Robert Stam [ 08/Dec/22 ]

Thanks for creating this ticket.

I think one reason we haven't used `long` as the parameter type for `skip` is that it's hard to imagine skipping more than `int.MaxValue` documents. Just think about how long the server would have to work to skip millions of documents.

Given that, do you still think we should have an overload of `Skip` that takes a `long skip`?

Generated at Wed Feb 07 21:48:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.