Don Anderson Would you mind owning this issue?

Hi Denis Shkirya, thanks for the question. The documentation for WT_SESSION::join (http://source.wiredtiger.com/2.7.0/struct_w_t___s_e_s_s_i_o_n.html#ae0ab118df83d173c6a20eb1ea3f3fd84) says that a cursor should be positioned before it can be added to a join cursor. This can be done using a search(), search_near() or next() call on the cursor - and these should return without an error. When you have a search that fails, the API may leave the cursor in a state that should not be usable for a join. The fact that the current implementation of join uses stale information in the cursor (even if it sometimes gives sensible results) is really a bug. I'll push a fix for that soon.

Thanks!

Thanks for the detailed answer!

Author:

Message: WT-2331 Require that reference cursors that are joined be positioned.

Additionally add tests to verify that cursors that merely have a key set, or
that have failed a positioning API (search), not be usable in a join. Give
a better error message in all these cases.

Add a check in __wt_struct_repack so that an empty or shorter than expected
input array give an error, and not be interpreted as unbounded size.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/b8f3effa90017b60f8509b0b50aeba3a073f1767

Author:

Message: Merge pull request #2435 from wiredtiger/WT-2331-join-search

WT-2331 Require that reference cursors that are joined be positioned
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/dd7811875ee5f6154ee8eb035ae6f8d24f58f59f

Author:

Message: Merge pull request #2435 from wiredtiger/WT-2331-join-search

WT-2331 Require that reference cursors that are joined be positioned
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/dd7811875ee5f6154ee8eb035ae6f8d24f58f59f

Author:

Message: Import wiredtiger-wiredtiger-2.7.0-505-g7fea169.tar.gz from wiredtiger branch mongodb-3.4

ref: 44463c5..7fea169

WT-2355 Fix minor scratch buffer usage in logging.
WT-2348 xargs -P isn't portable
WT-2347 Java: schema format edge cases
WT-2344 OS X compiler warning
WT-2342 Enhance wtperf to support background create and drop operations
WT-2340 Add logging guarantee assertions, whitespace
WT-2339 format post-rebalance verify failure (stress run #11586)
WT-2338 Disable using pre-allocated log files when backup cursor is open
WT-2335 NULL pointer crash in config_check_search with invalid configuration string
WT-2333 Add a flag so drop doesn't block
WT-2332 Bug in logging write-no-sync mode
WT-2331 Checking of search() result for reference cursors before join()
WT-2328 schema drop does direct unlink, it should use a block manager interface.
WT-2326 Change WTPERF to use new memory allocation functions instead of the standard
WT-2321 WT-2321: race between eviction and worker threads on the eviction queue
WT-2320 Only check copyright when cutting releases
WT-2316 stress test failure: WT_CURSOR.prev out-of-order returns
WT-2314 page-swap error handling is inconsistent
WT-2313 sweep-server: conn_dhandle.c, 610: dhandle != conn->cache->evict_file_next
WT-2312 re-creating a deleted column-store page can corrupt the in-memory tree
WT-2308 custom extractor for ref_cursors in join cursor
WT-2305 Fix coverity scan issues on 23/12/2015
WT-2296 New log algorithm needs improving for sync/flush settings
WT-2295 WT_SESSION.create does a full-scan of the main table
WT-2287 WT_SESSION.rebalance
WT-2275 broken DB after application crash
WT-2267 Improve wtperf throttling implementation to provide steady load
WT-2247 variable-length column-store in-memory page splits
WT-2242 WiredTiger treats dead trees the same as other trees in eviction
WT-2142 Connection cleanup in Python tests
WT-2073 metadata cleanups
WT-1801 Add a directory sync after rollback of a WT_SESSION::rename operation
WT-1517 schema format edge cases
SERVER-22064 Coverity analysis defect 77699: Unchecked return value
SERVER-21619 sys-perf: WT crash during core_workloads_WT execution
Branch: master
https://github.com/mongodb/mongo/commit/90118b147a6943b19dc929862a11071538db1438

Author:

Message: Import wiredtiger-wiredtiger-2.7.0-559-g07966a4.tar.gz from wiredtiger branch mongodb-3.2

ref: 3c2ad56..07966a4

WT-1517 schema format edge cases
WT-1801 Add a directory sync after rollback of a WT_SESSION::rename operation
WT-2060 Simplify aggregation of statistics
WT-2073 metadata cleanups
WT-2099 Seeing memory underflow messages
WT-2113 truncate01 sometimes fails
WT-2142 Connection cleanup in Python tests
WT-2177 Add an optional per-thread seed to random number generator
WT-2198 bulk load and column store appends
WT-2216 simplify row-store search loop slightly
WT-2225 New split code performance impact
WT-2231 pinned page cursor searches could check parent keys
WT-2235 wt printlog option without unicode
WT-2242 WiredTiger treats dead trees the same as other trees in eviction
WT-2244 Trigger in-memory splits sooner
WT-2245 WTPERF Truncate has no ability to catch up when it falls behind
WT-2246 column-store append searches the leaf page; the maximum record number fails CRUD operations
WT-2247 variable-length column-store in-memory page splits
WT-2256 WTPERFs throttle option fires in bursts
WT-2257 wtperf doesn't handle overriding workload config
WT-2258 WiredTiger preloads pages even when direct-IO is configured.
WT-2259 __wt_evict_file_exclusive_on() should clear WT_BTREE_NO_EVICTION on error
WT-2260 Workloads evict internal pages unexpectedly
WT-2262 Random sampling is skewed by tree shape
WT-2265 Wiredtiger related change in ppc64le specific code block in gcc.h
WT-2266 Add wtperf config to set if perf thresholds are fatal
WT-2267 Improve wtperf throttling implementation to provide steady load
WT-2269 wtperf should dump its config everytime it runs
WT-2272 Stress test assertion in the sweep server
WT-2275 broken DB after application crash
WT-2276 tool to decode checkpoint addr
WT-2277 Remove WT check against big-endian systems
WT-2279 Define WT_PAUSE(), WT_FULL_BARRIER(), etc when s390x is defined
WT-2281 wtperf smoke.sh fails on ppc64le
WT-2282 error in wt_txn_update_oldest verbose message test
WT-2283 retry in txn_update_oldest results in a hang
WT-2284 Repeated macro definition
WT-2285 configure should set BUFFER_ALIGNMENT_DEFAULT to 4kb on linux
WT-2287 WT_SESSION.rebalance
WT-2289 failure in fast key check
WT-2290 WT_SESSION.compact could be more effective.
WT-2291 Random cursor walk inefficient in skip list only trees
WT-2295 WT_SESSION.create does a full-scan of the main table
WT-2296 New log algorithm needs improving for sync/flush settings
WT-2297 Fix off-by-one error in Huffman config file parsing
WT-2299 upper-level WiredTiger code is reaching into the block manager
WT-2301 Add reading a range to wtperf
WT-2303 Build warning in wtperf
WT-2304 wtperf crash dumping config
WT-2305 Fix coverity scan issues on 23/12/2015
WT-2307 Internal page splits can corrupt cursor iteration
WT-2308 custom extractor for ref_cursors in join cursor
WT-2311 Support Sparc
WT-2312 re-creating a deleted column-store page can corrupt the in-memory tree
WT-2313 sweep-server: conn_dhandle.c, 610: dhandle != conn->cache->evict_file_next
WT-2314 page-swap error handling is inconsistent
WT-2316 stress test failure: WT_CURSOR.prev out-of-order returns
WT-2320 Only check copyright when cutting releases
WT-2321 WT-2321: race between eviction and worker threads on the eviction queue
WT-2326 Change WTPERF to use new memory allocation functions instead of the standard
WT-2328 schema drop does direct unlink, it should use a block manager interface.
WT-2331 Checking of search() result for reference cursors before join()
WT-2332 Bug in logging write-no-sync mode
WT-2333 Add a flag so drop doesn't block
WT-2335 NULL pointer crash in config_check_search with invalid configuration string
WT-2338 Disable using pre-allocated log files when backup cursor is open
WT-2339 format post-rebalance verify failure (stress run #11586)
WT-2340 Add logging guarantee assertions, whitespace
WT-2342 Enhance wtperf to support background create and drop operations
WT-2344 OS X compiler warning
WT-2347 Java: schema format edge cases
WT-2348 xargs -P isn't portable
WT-2355 Fix minor scratch buffer usage in logging
SERVER-21833 Compact does not release space to the system with WiredTiger
SERVER-21887 $sample takes disproportionately long time on newly created collection
SERVER-22064 Coverity analysis defect 77699: Unchecked return value
SERVER-21944 WiredTiger changes for 3.2.2
Branch: v3.2
https://github.com/mongodb/mongo/commit/5d6532f3d5227ff76f62c4810c98a4ef4d0c8c56