[WT-2332] Bug in logging write-no-sync mode Created: 14/Jan/16  Updated: 12/Oct/17  Resolved: 25/Jan/16

Status: Closed
Project: WiredTiger
Component/s: None
Affects Version/s: WT2.6.1
Fix Version/s: WT2.7.0

Type: Bug Priority: Critical - P2
Reporter: Sue LoVerso Assignee: Sue LoVerso
Resolution: Fixed Votes: 0
Labels: None

Attachments: File wt-2332.diff     File wt-2332.revised.diff    

 Description   

I discovered a bug/path where a thread could return from writing to the log requesting write-no-sync mode without waiting for the write LSN to properly advance. Affects all versions 2.6.1 and earlier but NOT 2.7.0.



 Comments   
Comment by Sue LoVerso [ 15/Jan/16 ]

Dmitri Shubin I know you use write-no-sync mode. I discovered a bug with that mode and have a fix. The bug is not in our current release, 2.7.0, but is in 2.6.1 and earlier. It is a small patch. Would you like me to send you the patch or would you prefer if we commit it and cut a 2.6.2 release with this fix?

Comment by Dmitri Shubin [ 16/Jan/16 ]

Hi Sue LoVerso, patch is ok for us.
Thank you!

Comment by Sue LoVerso [ 19/Jan/16 ]

The attached patch is based off the 2.6.1 release. Let me know if you have any problems with it.

Comment by Githook User [ 20/Jan/16 ]

Author:

{u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@wiredtiger.com'}

Message: WT-2332 Add logging guarantee assertions.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/6ba817a87a6853c885c7fd0ddc943e9f002c465f

Comment by Githook User [ 20/Jan/16 ]

Author:

{u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@wiredtiger.com'}

Message: WT-2332 Remove if statement and merge it into the assert.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/87c3b645bab28e015cf7fbc7e2ba31f3e050c457

Comment by Githook User [ 20/Jan/16 ]

Author:

{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

Message: Merge pull request #2443 from wiredtiger/WT-2332

WT-2332 Add logging guarantee assertions.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/6874ec478cfc059bb49febad0697af3ce0a220ec

Comment by Githook User [ 20/Jan/16 ]

Author:

{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

Message: Merge pull request #2443 from wiredtiger/WT-2332

WT-2332 Add logging guarantee assertions.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/6874ec478cfc059bb49febad0697af3ce0a220ec

Comment by Sue LoVerso [ 25/Jan/16 ]

I have uploaded a revised diff to eliminate an empty conditional when running in production mode.

Comment by Sue LoVerso [ 25/Jan/16 ]

Dmitri Shubin Have you taken the patch? I made a slight modification and attached a revised one.

Comment by Dmitri Shubin [ 26/Jan/16 ]

Hi Sue LoVerso,

I missed your original patch (seems no notification for file attach in JIRA?).
Will apply revised one.

Thank you!

Comment by Githook User [ 27/Jan/16 ]

Author:

{u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

Message: Import wiredtiger-wiredtiger-2.7.0-505-g7fea169.tar.gz from wiredtiger branch mongodb-3.4

ref: 44463c5..7fea169

WT-2355 Fix minor scratch buffer usage in logging.
WT-2348 xargs -P isn't portable
WT-2347 Java: schema format edge cases
WT-2344 OS X compiler warning
WT-2342 Enhance wtperf to support background create and drop operations
WT-2340 Add logging guarantee assertions, whitespace
WT-2339 format post-rebalance verify failure (stress run #11586)
WT-2338 Disable using pre-allocated log files when backup cursor is open
WT-2335 NULL pointer crash in config_check_search with invalid configuration string
WT-2333 Add a flag so drop doesn't block
WT-2332 Bug in logging write-no-sync mode
WT-2331 Checking of search() result for reference cursors before join()
WT-2328 schema drop does direct unlink, it should use a block manager interface.
WT-2326 Change WTPERF to use new memory allocation functions instead of the standard
WT-2321 WT-2321: race between eviction and worker threads on the eviction queue
WT-2320 Only check copyright when cutting releases
WT-2316 stress test failure: WT_CURSOR.prev out-of-order returns
WT-2314 page-swap error handling is inconsistent
WT-2313 sweep-server: conn_dhandle.c, 610: dhandle != conn->cache->evict_file_next
WT-2312 re-creating a deleted column-store page can corrupt the in-memory tree
WT-2308 custom extractor for ref_cursors in join cursor
WT-2305 Fix coverity scan issues on 23/12/2015
WT-2296 New log algorithm needs improving for sync/flush settings
WT-2295 WT_SESSION.create does a full-scan of the main table
WT-2287 WT_SESSION.rebalance
WT-2275 broken DB after application crash
WT-2267 Improve wtperf throttling implementation to provide steady load
WT-2247 variable-length column-store in-memory page splits
WT-2242 WiredTiger treats dead trees the same as other trees in eviction
WT-2142 Connection cleanup in Python tests
WT-2073 metadata cleanups
WT-1801 Add a directory sync after rollback of a WT_SESSION::rename operation
WT-1517 schema format edge cases
SERVER-22064 Coverity analysis defect 77699: Unchecked return value
SERVER-21619 sys-perf: WT crash during core_workloads_WT execution
Branch: master
https://github.com/mongodb/mongo/commit/90118b147a6943b19dc929862a11071538db1438

Comment by Githook User [ 29/Jan/16 ]

Author:

{u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

Message: Import wiredtiger-wiredtiger-2.7.0-559-g07966a4.tar.gz from wiredtiger branch mongodb-3.2

ref: 3c2ad56..07966a4

WT-1517 schema format edge cases
WT-1801 Add a directory sync after rollback of a WT_SESSION::rename operation
WT-2060 Simplify aggregation of statistics
WT-2073 metadata cleanups
WT-2099 Seeing memory underflow messages
WT-2113 truncate01 sometimes fails
WT-2142 Connection cleanup in Python tests
WT-2177 Add an optional per-thread seed to random number generator
WT-2198 bulk load and column store appends
WT-2216 simplify row-store search loop slightly
WT-2225 New split code performance impact
WT-2231 pinned page cursor searches could check parent keys
WT-2235 wt printlog option without unicode
WT-2242 WiredTiger treats dead trees the same as other trees in eviction
WT-2244 Trigger in-memory splits sooner
WT-2245 WTPERF Truncate has no ability to catch up when it falls behind
WT-2246 column-store append searches the leaf page; the maximum record number fails CRUD operations
WT-2247 variable-length column-store in-memory page splits
WT-2256 WTPERFs throttle option fires in bursts
WT-2257 wtperf doesn't handle overriding workload config
WT-2258 WiredTiger preloads pages even when direct-IO is configured.
WT-2259 __wt_evict_file_exclusive_on() should clear WT_BTREE_NO_EVICTION on error
WT-2260 Workloads evict internal pages unexpectedly
WT-2262 Random sampling is skewed by tree shape
WT-2265 Wiredtiger related change in ppc64le specific code block in gcc.h
WT-2266 Add wtperf config to set if perf thresholds are fatal
WT-2267 Improve wtperf throttling implementation to provide steady load
WT-2269 wtperf should dump its config everytime it runs
WT-2272 Stress test assertion in the sweep server
WT-2275 broken DB after application crash
WT-2276 tool to decode checkpoint addr
WT-2277 Remove WT check against big-endian systems
WT-2279 Define WT_PAUSE(), WT_FULL_BARRIER(), etc when s390x is defined
WT-2281 wtperf smoke.sh fails on ppc64le
WT-2282 error in wt_txn_update_oldest verbose message test
WT-2283 retry in txn_update_oldest results in a hang
WT-2284 Repeated macro definition
WT-2285 configure should set BUFFER_ALIGNMENT_DEFAULT to 4kb on linux
WT-2287 WT_SESSION.rebalance
WT-2289 failure in fast key check
WT-2290 WT_SESSION.compact could be more effective.
WT-2291 Random cursor walk inefficient in skip list only trees
WT-2295 WT_SESSION.create does a full-scan of the main table
WT-2296 New log algorithm needs improving for sync/flush settings
WT-2297 Fix off-by-one error in Huffman config file parsing
WT-2299 upper-level WiredTiger code is reaching into the block manager
WT-2301 Add reading a range to wtperf
WT-2303 Build warning in wtperf
WT-2304 wtperf crash dumping config
WT-2305 Fix coverity scan issues on 23/12/2015
WT-2307 Internal page splits can corrupt cursor iteration
WT-2308 custom extractor for ref_cursors in join cursor
WT-2311 Support Sparc
WT-2312 re-creating a deleted column-store page can corrupt the in-memory tree
WT-2313 sweep-server: conn_dhandle.c, 610: dhandle != conn->cache->evict_file_next
WT-2314 page-swap error handling is inconsistent
WT-2316 stress test failure: WT_CURSOR.prev out-of-order returns
WT-2320 Only check copyright when cutting releases
WT-2321 WT-2321: race between eviction and worker threads on the eviction queue
WT-2326 Change WTPERF to use new memory allocation functions instead of the standard
WT-2328 schema drop does direct unlink, it should use a block manager interface.
WT-2331 Checking of search() result for reference cursors before join()
WT-2332 Bug in logging write-no-sync mode
WT-2333 Add a flag so drop doesn't block
WT-2335 NULL pointer crash in config_check_search with invalid configuration string
WT-2338 Disable using pre-allocated log files when backup cursor is open
WT-2339 format post-rebalance verify failure (stress run #11586)
WT-2340 Add logging guarantee assertions, whitespace
WT-2342 Enhance wtperf to support background create and drop operations
WT-2344 OS X compiler warning
WT-2347 Java: schema format edge cases
WT-2348 xargs -P isn't portable
WT-2355 Fix minor scratch buffer usage in logging
SERVER-21833 Compact does not release space to the system with WiredTiger
SERVER-21887 $sample takes disproportionately long time on newly created collection
SERVER-22064 Coverity analysis defect 77699: Unchecked return value
SERVER-21944 WiredTiger changes for 3.2.2
Branch: v3.2
https://github.com/mongodb/mongo/commit/5d6532f3d5227ff76f62c4810c98a4ef4d0c8c56

Generated at Fri May 25 07:24:59 UTC 2018 using JIRA 7.8.2#78002-sha1:944b71ecbe2e09c23503821098ef280c785b44a8.