[SERVER-17017] WT_SESSION::begin_transaction and WT_CURSOR::reset can return WT_ROLLBACK Created: 23/Jan/15  Updated: 18/Sep/15  Resolved: 25/Mar/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: Michael Cahill (Inactive)
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

Index: src/third_party/wiredtiger/src/evict/evict_lru.c
diff --git a/src/third_party/wiredtiger/src/evict/evict_lru.c b/src/third_party/wiredtiger/src/evict/evict_lru.c
index a4ae0aaf55b5301b5236120b8d4fd55eecfdb115..1f4fe14e117179d533a762b6c1e0c4a73106e1c0 100644
--- a/src/third_party/wiredtiger/src/evict/evict_lru.c
+++ b/src/third_party/wiredtiger/src/evict/evict_lru.c
@@ -1223,6 +1223,11 @@ __evict_get_ref(
 		__wt_yield();
 	}
 
+    // 50% of the time return WT_ROLLBACK
+    if ((rand() % 100) < 50) {
+        return (WT_ROLLBACK);
+    }
+
 	/*
 	 * The eviction server only tries to evict half of the pages before
 	 * looking for more.

Participants:

 Description   

When the patch pasted in the repro steps section is applied to simulate running out of buffer space during eviction, WT_ROLLBACK failures start popping in unexpected places such as WT_SESSION::begin_transaction and WT_CURSOR::reset.

I am not sure whether this is a 100% realistic scenario, because presumably new transactions and closed cursors should not be selected as victims, but I am filing this bug to investigate the possibility.

Here are the stack traces:

00000049`9a9584e0 00007ff7`9e358ff7 mongod!mongo::fassertFailed+0x99 [e:\workspace\mongo\src\mongo\util\assert_util.cpp @ 166]
00000049`9a958620 00007ff7`9ef14255 mongod!mongo::fassert+0x47 [e:\workspace\mongo\src\mongo\util\assert_util.h @ 214]
00000049`9a958660 00007ff7`9eee40ca mongod!mongo::wtRCToStatus_slow+0xe5 [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp @ 64]
00000049`9a9587b0 00007ff7`9eef7d5f mongod!mongo::wtRCToStatus+0x7a [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_util.h @ 61]
00000049`9a9587f0 00007ff7`9eef7458 mongod!mongo::WiredTigerRecoveryUnit::_txnOpen+0x16f [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_recovery_unit.cpp @ 226]
00000049`9a958930 00007ff7`9eef81e9 mongod!mongo::WiredTigerRecoveryUnit::getSession+0x58 [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_recovery_unit.cpp @ 187]
00000049`9a958960 00007ff7`9eef7f26 mongod!mongo::WiredTigerCursor::_init+0x89 [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_recovery_unit.cpp @ 267]
00000049`9a958a30 00007ff7`9eeec7f9 mongod!mongo::WiredTigerCursor::WiredTigerCursor+0x56 [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_recovery_unit.cpp @ 259]
00000049`9a958a70 00007ff7`9edc4635 mongod!mongo::WiredTigerRecordStore::findRecord+0x79 [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_record_store.cpp @ 336]
00000049`9a958b60 00007ff7`9edc20e7 mongod!mongo::KVCatalog::_findEntry+0x405 [e:\workspace\mongo\src\mongo\db\storage\kv\kv_catalog.cpp @ 241]
00000049`9a958e10 00007ff7`9edd1f7c mongod!mongo::KVCatalog::getMetaData+0x87 [e:\workspace\mongo\src\mongo\db\storage\kv\kv_catalog.cpp @ 256]
00000049`9a959110 00007ff7`9ed8b7e2 mongod!mongo::KVCollectionCatalogEntry::_getMetaData+0x6c [e:\workspace\mongo\src\mongo\db\storage\kv\kv_collection_catalog_entry.cpp @ 175]
00000049`9a959160 00007ff7`9e57a6c0 mongod!mongo::BSONCollectionCatalogEntry::isIndexReady+0x72 [e:\workspace\mongo\src\mongo\db\storage\bson_collection_catalog_entry.cpp @ 100]
00000049`9a959280 00007ff7`9e57a606 mongod!mongo::IndexCatalogEntry::_catalogIsReady+0x60 [e:\workspace\mongo\src\mongo\db\catalog\index_catalog_entry.cpp @ 221]
00000049`9a9592c0 00007ff7`9e563973 mongod!mongo::IndexCatalogEntry::isReady+0x46 [e:\workspace\mongo\src\mongo\db\catalog\index_catalog_entry.cpp @ 110]
00000049`9a959300 00007ff7`9e5636b5 mongod!mongo::IndexCatalog::IndexIterator::_advance+0xd3 [e:\workspace\mongo\src\mongo\db\catalog\index_catalog.cpp @ 946]
00000049`9a959370 00007ff7`9e5628bd mongod!mongo::IndexCatalog::IndexIterator::more+0x35 [e:\workspace\mongo\src\mongo\db\catalog\index_catalog.cpp @ 920]
00000049`9a9593b0 00007ff7`9e516f13 mongod!mongo::IndexCatalog::findIdIndex+0x5d [e:\workspace\mongo\src\mongo\db\catalog\index_catalog.cpp @ 957]
00000049`9a959470 00007ff7`9e689a75 mongod!mongo::Collection::insertDocument+0xb3 [e:\workspace\mongo\src\mongo\db\catalog\collection.cpp @ 205]
00000049`9a959590 00007ff7`9e68c683 mongod!mongo::singleInsert+0x135 [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 1136]
00000049`9a959670 00007ff7`9e686e4b mongod!mongo::insertOne+0x2b3 [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 1071]
00000049`9a959850 00007ff7`9e686c79 mongod!mongo::WriteBatchExecutor::execOneInsert+0x14b [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 1113]
00000049`9a95adf0 00007ff7`9e68671d mongod!mongo::WriteBatchExecutor::execInserts+0x169 [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 883]
00000049`9a95af00 00007ff7`9e685be4 mongod!mongo::WriteBatchExecutor::bulkExecute+0x6d [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 764]
00000049`9a95b060 00007ff7`9e697c23 mongod!mongo::WriteBatchExecutor::executeBatch+0x544 [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 279]
00000049`9a95b5d0 00007ff7`9e6db690 mongod!mongo::WriteCmd::run+0x263

and

00000049`98988680 00007ff7`9e358ff7 mongod!mongo::fassertFailed+0x99 [e:\workspace\mongo\src\mongo\util\assert_util.cpp @ 166]
00000049`989887c0 00007ff7`9ef14255 mongod!mongo::fassert+0x47 [e:\workspace\mongo\src\mongo\util\assert_util.h @ 214]
00000049`98988800 00007ff7`9eee40ca mongod!mongo::wtRCToStatus_slow+0xe5 [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp @ 64]
00000049`98988950 00007ff7`9eefa1b9 mongod!mongo::wtRCToStatus+0x7a [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_util.h @ 61]
00000049`98988990 00007ff7`9eef803a mongod!mongo::WiredTigerSession::releaseCursor+0x1d9 [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_session_cache.cpp @ 93]
00000049`98988a20 00007ff7`9eeed1e2 mongod!mongo::WiredTigerCursor::~WiredTigerCursor+0x8a [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_recovery_unit.cpp @ 277]
00000049`98988a60 00007ff7`9eeed32a mongod!mongo::WiredTigerRecordStore::insertRecord+0x4e2 [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_record_store.cpp @ 559]
00000049`98988c50 00007ff7`9e5172f0 mongod!mongo::WiredTigerRecordStore::insertRecord+0x10a [e:\workspace\mongo\src\mongo\db\storage\wiredtiger\wiredtiger_record_store.cpp @ 587]
00000049`98988cf0 00007ff7`9ec7fa9f mongod!mongo::Collection::insertDocument+0x1b0 [e:\workspace\mongo\src\mongo\db\catalog\collection.cpp @ 192]
00000049`98988da0 00007ff7`9ec7d0df mongod!mongo::repl::`anonymous namespace'::_logOpRS+0x65f [e:\workspace\mongo\src\mongo\db\repl\oplog.cpp @ 268]
00000049`989892f0 00007ff7`9e689af1 mongod!mongo::repl::logOp+0xcf [e:\workspace\mongo\src\mongo\db\repl\oplog.cpp @ 378]
00000049`989895a0 00007ff7`9e68c683 mongod!mongo::singleInsert+0x1b1 [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 1143]
00000049`98989680 00007ff7`9e686e4b mongod!mongo::insertOne+0x2b3 [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 1071]
00000049`98989860 00007ff7`9e686c79 mongod!mongo::WriteBatchExecutor::execOneInsert+0x14b [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 1113]
00000049`9898ae00 00007ff7`9e68671d mongod!mongo::WriteBatchExecutor::execInserts+0x169 [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 883]
00000049`9898af10 00007ff7`9e685be4 mongod!mongo::WriteBatchExecutor::bulkExecute+0x6d [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 764]
00000049`9898b070 00007ff7`9e697c23 mongod!mongo::WriteBatchExecutor::executeBatch+0x544 [e:\workspace\mongo\src\mongo\db\commands\write_commands\batch_executor.cpp @ 279]
00000049`9898b5e0 00007ff7`9e6db690 mongod!mongo::WriteCmd::run+0x263 [e:\workspace\mongo\src\mongo\db\commands\write_commands\write_commands.cpp @ 149]



 Comments   
Comment by Michael Cahill (Inactive) [ 23/Mar/15 ]

We reviewed this on the WT side in https://github.com/wiredtiger/wiredtiger/issues/1600 and determined that neither WT_CURSOR::reset or WT_SESSION::begin_transaction could in fact return WT_ROLLBACK. So there was no code change, but I don't believe this issue needs to stay open.

Comment by Daniel Pasette (Inactive) [ 23/Mar/15 ]

ramon.fernandez, where do you this as addressed in next drop?

Comment by Daniel Pasette (Inactive) [ 05/Feb/15 ]

This has not been seen in the wild, it was only forced by adding a failpoint in the WT code. Marking for backport.

Generated at Thu Feb 08 03:43:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.