[JAVA-448] Tailable Cursor and awaitdata Created: 13/Oct/11 Updated: 11/Sep/19 Resolved: 19/Oct/11 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | API |
| Affects Version/s: | 2.6.5 |
| Fix Version/s: | 2.7 |
| Type: | Task | Priority: | Minor - P4 |
| Reporter: | Guy Boertje | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | cursor, driver, tailable | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Linux 64 bit Ubuntu, Java 7 Jruby 1.6.4 with 2.6.5 of the java driver on Mongo 1.8.3 |
||
| Backwards Compatibility: | Major Change |
| Description |
|
In tests, using a tailable cursor and a capped collection, I can fill the collection with say 10 docs and read them back. If I then ask for next doc, my test never ends. In the test setup if I set a thread to wait for 6 seconds before adding a doc to the collection the test does end. I did a thread dump while the test blocked. See further down Please confirm that the java driver should eventually return a nil doc assuming the await data option is set. Or confirm that this behaviour is for a future version. – Dump (some jruby lines removed) – "ScriptThreadProcess: ./test/collection_test.rb" daemon prio=10 tid=0x0000000002959000 nid=0x55b9 runnable [0x00007f2b2b7c1000]
JRUBY 10.times do Thread.new(@capped) do |col| assert true, tail.alive?
assert tail.next_document |
| Comments |
| Comment by Guy Boertje [ 19/Oct/11 ] |
|
Much appreciated. I have a modified fork of the driver from commit tagged 2.6.5, which commits will I have to cherry pick to have this behavour in my fork? I know its a long shot. I modded to include the change for Symbol in BasicBSONCallback as per |
| Comment by Antoine Girbal [ 19/Oct/11 ] |
|
The fix here keeps the looping for the AWAIT case as fixed in |
| Comment by auto [ 19/Oct/11 ] |
|
Author: {u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}Message: |
| Comment by Brendan W. McAdams [ 19/Oct/11 ] |
|
Tailable cursors w/ no data but not in await mode will no longer block and sleep but instead return null. |
| Comment by auto [ 19/Oct/11 ] |
|
Author: {u'login': u'bwmcadams', u'name': u'Brendan W. McAdams', u'email': u'brendan@10gen.com'}Message:
|
| Comment by Antoine Girbal [ 18/Oct/11 ] |
|
when QUERYOPTION_AWAITDATA is on, it behaves properly: blocks in the hasNext() loop. |
| Comment by Brendan W. McAdams [ 18/Oct/11 ] |
|
There's one relevant bit above that try{Thread.sleep however though: // have a tailable cursor if ( ( _flags & Bytes.RESULTFLAG_AWAITCAPABLE ) > 0 && ( queryOptions & Bytes.QUERYOPTION_AWAITDATA ) > 0 ) Which should behave correctly there, and not block. Basically it checks that both the server is await capable and AWAIT was set, and returns immediately. OTHERWISE it blocks... |
| Comment by Antoine Girbal [ 18/Oct/11 ] |
|
I think there is some bad code in the driver, this loops: public boolean hasNext(){ } public boolean hasGetMore( int queryOptions ){ catch ( Exception e ){} return true; This result in hasNext() being stuck in a loop even without AWAIT. |
| Comment by Antoine Girbal [ 18/Oct/11 ] |
|
I did a quick test and indeed the request blocks even without AWAIT_DATA. |
| Comment by Guy Boertje [ 18/Oct/11 ] |
|
I will double check. I recall that the original test did not set AWAITDATA but still blocked. This is the test adds 10 docs in setup should "find using a tailable cursor" do |
| Comment by Antoine Girbal [ 18/Oct/11 ] |
|
I am not sure I fully understand the need. |
| Comment by Guy Boertje [ 16/Oct/11 ] |
|
This presents a bit of a problem because the ruby driver returns a nil document immediately. I have looked at the RabbitMq Java driver - it has a QueuingConsumer class that allows for a timeout when waiting for a delivery. A tailable cursor is similar and I suggest that this driver should provide similar semantics. I encourage you to look at the class I mentioned. Using a tailable cursor in an event driven way needs a loop with a callback and the loop needs to exit occasionally (to shutdown perhaps). I think this the loop, timeout and callback invocation are driver side duties. For now I am going be invasive and, in another thread after a timeout period, put a 'poison' doc into the collection from the cursor and return nil from the next method if it receives a 'poison' doc. |
| Comment by Antoine Girbal [ 14/Oct/11 ] |
|
I have not tested the behavior, but it seems this is independent of the driver and it's a server feature. |