[SERVER-18604] inconsistent data on read/write worload due to DateTime precision difference between MongoDB and C# Created: 21/May/15 Updated: 26/Apr/19 Resolved: 18/Jun/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | WiredTiger |
| Affects Version/s: | 3.0.3 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Sverdel [X] | Assignee: | Ramon Fernandez Marina |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Operating System: | ALL |
| Participants: |
| Description |
|
I'm use mongodb 3.0.3 on Windows with wiredTiger storage engine. When a write some data into db and then try to read it without any timeout, i can't get those document. When i add some timeout (about 10-100 ms) all is fine I'm use C# driver version 1.10. With driver v.2.0 i had same result |
| Comments |
| Comment by Ian Whalen (Inactive) [ 26/Apr/19 ] |
|
Switching "Drivers Changes Needed" from "Maybe" to "Not Needed" since this was closed as something other than Fixed. |
| Comment by Ramon Fernandez Marina [ 18/Jun/15 ] |
|
Troop1, a colleague pointed me to this blog post about handling DateTime precision with MongoDB and C#, which contains a detailed explanation of the issue – so no need for you to post any further details if you choose not to. Thanks, |
| Comment by Ramon Fernandez Marina [ 18/Jun/15 ] |
|
Thanks for the detailed update Troop1, and glad to hear the mystery is solved. It would be helpful if you could share the few lines of code you use showing the use of datetime in C# and how you're inserting and querying for those documents, so other users can benefit from your expertise and avoid this issue in the future. Thanks, |
| Comment by Sverdel [X] [ 05/Jun/15 ] |
|
Hello, When I insert document into moingoDB, I set current datetime into one of document field. In some case mongoDB round datetime value up and store datetime that larger than current date for a few milliseconds. So, the “future” time is stored into database. When I used MMAPV1 storage engine, I didn’t get this situation, because insert duration was larger than mongodb datetime rounding lag. I was very surprised, when I found that mongoDB has worse datetime precision than C#. In future I will be more accurate with datetime using in mongoDB query. But I’m glad that WiredTiger engine has so good write performance. Nonetheless, thanks a lot for help. Task may be closed |
| Comment by Ramon Fernandez Marina [ 01/Jun/15 ] |
|
Troop1, apologies for the confusion, I've re-opened this ticket to better understand the issue. Can you please provide us with one of those tests that fail, as well as with details of the deployment you're using for your testing? Thanks, |
| Comment by Sverdel [X] [ 01/Jun/15 ] |
|
No, I've never said that I'm not waiting for acknowledgement. I understand, that with Unacknowledged write concern I can't expect that read immediately after write operation will return correct data. In my situation I write data to mongodb and wait for acknowledgement and write into Journal. After that I try to get data from database. And, as I said earlier, in some tests I can't get data from database. |
| Comment by Ramon Fernandez Marina [ 01/Jun/15 ] |
|
If I understood things correctly, you're relying on undefined behavior (namely writing without waiting for acknowledgement and expecting the data to be written by the time you read it) and obtaining two different behaviors depending on which storage engine is used. This is essentially a race condition in your design, where you pit readers against writers and if the writers "win" then you get the data you expect. The fact that with MMAPv1 you can immediately read the data is just luck: perhaps MMAPv1 is faster for writes in this particular scenario, or the different threads in the storage engine are scheduled in a particular way on your machine, etc. In short, one can't expect to immediately read data that was written with Unacknowledged write concern – this is by design – and if in your application this happens to work with MMAPv1 it's just by sheer chance. If you really need this behavior for your application I'd strongly recommend you use Acknowledged write concern. Regards, |
| Comment by Sverdel [X] [ 30/May/15 ] |
|
Ok, “It is not a bug, it’s feature” – is very useful answer. But you did not answer on my question: why with mmapv1 storage engine everything works fine? And why WiredTiger storage engine has different behavior in this situation? By the way, did I correctly understand your comment: "acknowledged" write concern did not guarantee, that data was saved into mongodb and can be read in other application at once they was written? And it's only because of other application use other connection? For experiment I decrease connection pool size down to only one connection. So in single test I use only one connection for write and read operations. But the situation has not changed – some part of tests is still failed. And so, even if I use single connection, I can’t read data, which was written right now? What about "journaled" write concern? I have the same result whe I use it. I could not believe that bug is in Mongodb driver, because with the same driver I have no problems, when I use mmapv1 storage engine. May be you need some additional information? MongoDB logs, for example? Which verbosity level should I use? I expect for reopening this bug |
| Comment by Sam Kleinman (Inactive) [ 29/May/15 ] |
|
Hello, If you're using "unacknowledged" writes, then we would not necessarily expect to be able to read back the write operation after the write returns, however, we would expect this using the "acknowledged" write concern, assuming that you were reading from the primary. These acknowledged writes do take longer to return than unacknowledged writes, but allow a client on a single connection to read its own writes, which is a useful property. Removing write concern could definitely explain the behavior that you're seeing. Having said that, compliant MongoDB drivers, including the 2.0 .NET driver do *not* use a single connection per thread. Rather drivers maintain a pool of connections and may use different connections for different operations, so it's possible that a read operation could return data using a different connection that could reflect a point in time before the write completed. In both of these cases, the behavior you see is a property of the read isolation semantics of the database, and the relationships between write operations, write concern, and queries. In light of this, I'm going to close this ticket as "works as designed," as I do not believe that there is an underlying MongoDB bug here, and the behavior we're seeing here is consistent with the design of the system. Sorry for the confusion and frustration. Regards, |
| Comment by Sverdel [X] [ 28/May/15 ] |
|
Yes, there is some misunderstanding. I know, that ReplicaAcknowledged decreases write performance. I don't use it generally in my application. Main problem is not in write concern. I've referred to ReplicaAcknowledged just for example. |
| Comment by Sam Kleinman (Inactive) [ 27/May/15 ] |
|
By increasing the conditions for success that must be satisfied for a write concern, in many cases you will decrease the overall write throughput of the operations. In this respect, the fact that ReplicaAcknowledged decreases write performance is entirely expected. If you're doing synchronous write operations and checking the write result before attempting to query the collection (with Primary read preference,) you will be able to read the documents that you wrote to the collection. I'm going to go ahead and close this issue. The SERVER project is for bug reports and feature requests against the core MongoDB engine itself: it seems like a question like this would be best addressed to one of our support channels. If I have misunderstood the nature of your problem, please feel free to re-open this ticket. Cheers, |
| Comment by Sverdel [X] [ 22/May/15 ] |
|
I've attach some part of source code and document example. Methods add and get are called sequential. |
| Comment by Ramon Fernandez Marina [ 21/May/15 ] |
|
Can you please share with us the code you're using to write and read your data? Thanks, |