[SERVER-18604] inconsistent data on read/write worload due to DateTime precision difference between MongoDB and C# Created: 21/May/15  Updated: 26/Apr/19  Resolved: 18/Jun/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.0.3
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Sverdel [X] Assignee: Ramon Fernandez Marina
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File add.cs     File config.cfg     File document.json     Text File get.cs    
Operating System: ALL
Participants:

 Description   

I'm use mongodb 3.0.3 on Windows with wiredTiger storage engine. When a write some data into db and then try to read it without any timeout, i can't get those document. When i add some timeout (about 10-100 ms) all is fine
MongoDB configuration in attachment

I'm use C# driver version 1.10. With driver v.2.0 i had same result



 Comments   
Comment by Ian Whalen (Inactive) [ 26/Apr/19 ]

Switching "Drivers Changes Needed" from "Maybe" to "Not Needed" since this was closed as something other than Fixed.

Comment by Ramon Fernandez Marina [ 18/Jun/15 ]

Troop1, a colleague pointed me to this blog post about handling DateTime precision with MongoDB and C#, which contains a detailed explanation of the issue – so no need for you to post any further details if you choose not to.

Thanks,
Ramón.

Comment by Ramon Fernandez Marina [ 18/Jun/15 ]

Thanks for the detailed update Troop1, and glad to hear the mystery is solved. It would be helpful if you could share the few lines of code you use showing the use of datetime in C# and how you're inserting and querying for those documents, so other users can benefit from your expertise and avoid this issue in the future.

Thanks,
Ramón.

Comment by Sverdel [X] [ 05/Jun/15 ]

Hello,
I found the reason of tests instability. The main problem was in datetime precision in mongoDB. (C# have a little bit better datetime precision).

When I insert document into moingoDB, I set current datetime into one of document field. In some case mongoDB round datetime value up and store datetime that larger than current date for a few milliseconds. So, the “future” time is stored into database.
When in test I call “get” operation (when date in document are still little bit larger than current datetime), I’ll get nothing from database, because I use current datetime when I query documents from database. (As you can see in attached source code)

When I used MMAPV1 storage engine, I didn’t get this situation, because insert duration was larger than mongodb datetime rounding lag.

I was very surprised, when I found that mongoDB has worse datetime precision than C#. In future I will be more accurate with datetime using in mongoDB query. But I’m glad that WiredTiger engine has so good write performance.

Nonetheless, thanks a lot for help. Task may be closed

Comment by Ramon Fernandez Marina [ 01/Jun/15 ]

Troop1, apologies for the confusion, I've re-opened this ticket to better understand the issue. Can you please provide us with one of those tests that fail, as well as with details of the deployment you're using for your testing?

Thanks,
Ramón.

Comment by Sverdel [X] [ 01/Jun/15 ]

No, I've never said that I'm not waiting for acknowledgement. I understand, that with Unacknowledged write concern I can't expect that read immediately after write operation will return correct data.
I’ve never said that I'm using unacknowledged write concern.

In my situation I write data to mongodb and wait for acknowledgement and write into Journal. After that I try to get data from database. And, as I said earlier, in some tests I can't get data from database.
In some experiments I add some delay (about 100 ms) after MongoDB write into Journal. With this delay all tests are passed. But I can’t add this delay into production, because I will not get enough performance.
I don’t think, that MMAPV1 faster than WiredTiger for write operation. Even my tests run 4-5 times faster with WiredTiger.

Comment by Ramon Fernandez Marina [ 01/Jun/15 ]

If I understood things correctly, you're relying on undefined behavior (namely writing without waiting for acknowledgement and expecting the data to be written by the time you read it) and obtaining two different behaviors depending on which storage engine is used. This is essentially a race condition in your design, where you pit readers against writers and if the writers "win" then you get the data you expect. The fact that with MMAPv1 you can immediately read the data is just luck: perhaps MMAPv1 is faster for writes in this particular scenario, or the different threads in the storage engine are scheduled in a particular way on your machine, etc.

In short, one can't expect to immediately read data that was written with Unacknowledged write concern – this is by design – and if in your application this happens to work with MMAPv1 it's just by sheer chance. If you really need this behavior for your application I'd strongly recommend you use Acknowledged write concern.

Regards,
Ramón.

Comment by Sverdel [X] [ 30/May/15 ]

Ok, “It is not a bug, it’s feature” – is very useful answer. But you did not answer on my question: why with mmapv1 storage engine everything works fine? And why WiredTiger storage engine has different behavior in this situation?

By the way, did I correctly understand your comment: "acknowledged" write concern did not guarantee, that data was saved into mongodb and can be read in other application at once they was written? And it's only because of other application use other connection?

For experiment I decrease connection pool size down to only one connection. So in single test I use only one connection for write and read operations. But the situation has not changed – some part of tests is still failed. And so, even if I use single connection, I can’t read data, which was written right now? What about "journaled" write concern? I have the same result whe I use it.

I could not believe that bug is in Mongodb driver, because with the same driver I have no problems, when I use mmapv1 storage engine.

May be you need some additional information? MongoDB logs, for example? Which verbosity level should I use?

I expect for reopening this bug

Comment by Sam Kleinman (Inactive) [ 29/May/15 ]

Hello,

If you're using "unacknowledged" writes, then we would not necessarily expect to be able to read back the write operation after the write returns, however, we would expect this using the "acknowledged" write concern, assuming that you were reading from the primary. These acknowledged writes do take longer to return than unacknowledged writes, but allow a client on a single connection to read its own writes, which is a useful property. Removing write concern could definitely explain the behavior that you're seeing.

Having said that, compliant MongoDB drivers, including the 2.0 .NET driver do *not* use a single connection per thread. Rather drivers maintain a pool of connections and may use different connections for different operations, so it's possible that a read operation could return data using a different connection that could reflect a point in time before the write completed.

In both of these cases, the behavior you see is a property of the read isolation semantics of the database, and the relationships between write operations, write concern, and queries. In light of this, I'm going to close this ticket as "works as designed," as I do not believe that there is an underlying MongoDB bug here, and the behavior we're seeing here is consistent with the design of the system.

Sorry for the confusion and frustration.

Regards,
sam

Comment by Sverdel [X] [ 28/May/15 ]

Yes, there is some misunderstanding. I know, that ReplicaAcknowledged decreases write performance. I don't use it generally in my application. Main problem is not in write concern. I've referred to ReplicaAcknowledged just for example.
The main problem: when i use mmapv1 storage engine with Acknowledged write concern, everything is fine - all tests are passed, and I can get data from database after i've add it (without any simulated delay). But, when I use wiredTiger engine with the same mongoDB configuration, including write concern, about 4-5% of my tests are failed, because of data loss. I didn't get data after I've add it. This situation occures only if delay between insert and read are less then 100ms.
If it is not bug, why different storage engines have different behavior with same other configurations including write concern?

Comment by Sam Kleinman (Inactive) [ 27/May/15 ]

By increasing the conditions for success that must be satisfied for a write concern, in many cases you will decrease the overall write throughput of the operations. In this respect, the fact that ReplicaAcknowledged decreases write performance is entirely expected. If you're doing synchronous write operations and checking the write result before attempting to query the collection (with Primary read preference,) you will be able to read the documents that you wrote to the collection.

I'm going to go ahead and close this issue. The SERVER project is for bug reports and feature requests against the core MongoDB engine itself: it seems like a question like this would be best addressed to one of our support channels. If I have misunderstood the nature of your problem, please feel free to re-open this ticket.

Cheers,
sam

Comment by Sverdel [X] [ 22/May/15 ]

I've attach some part of source code and document example. Methods add and get are called sequential.
In my unit test this situation occures in only 4-5% of tests (Different tests on different test runs). I tried to use different write concern - all tests are passed only with Replica Acknowledged, but with this write concern i didn't have enough write performance.

Comment by Ramon Fernandez Marina [ 21/May/15 ]

Can you please share with us the code you're using to write and read your data?

Thanks,
Ramón.

Generated at Thu Feb 08 03:48:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.