[SERVER-17562] Invariant failure: s->commit_transaction(s, NULL) resulted in status BadValue 22 Created: 12/Mar/15  Updated: 19/Sep/15  Resolved: 27/Mar/15

Status: Closed
Project: Core Server
Component/s: Storage, WiredTiger
Affects Version/s: 3.1.0
Fix Version/s: 3.0.2, 3.1.1

Type: Bug Priority: Major - P3
Reporter: Kaloian Manassiev Assignee: Geert Bosch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File full_log.txt    
Issue Links:
Duplicate
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Sprint: Quint Iteration 3.1.1
Participants:
Linked BF Score: 0

 Description   

One of the MCI parallel suites failed with the invariant violation below (the call stack has been demangled). The full logs are attached.

2015-03-12T00:00:30.596+0000 I COMMAND  [conn82] dropDatabase jstests_dropdb starting
2015-03-12T00:00:30.598+0000 I COMMAND  [conn82] dropDatabase jstests_dropdb finished
2015-03-12T00:00:30.674+0000 E STORAGE  [conn82] WiredTiger (0) [1426118430:674072][29115:0x2ba4b76cc940], session.commit_transaction: failed transaction requires rollback
2015-03-12T00:00:30.674+0000 I -        [conn82] Invariant failure: s->commit_transaction(s, NULL) resulted in status BadValue 22: Invalid argument at src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp 306
2015-03-12T00:00:30.680+0000 I CONTROL  [conn82] 
 0xef4db9 0xea3721 0xe8823a 0xd256b5 0x93a43f 0x9a29e4 0x9a38f2 0x9a45de 0xb63aa0 0xa7d573 0x7eb15f 0xeb772b 0x2ba4ae44b83d 0x2ba4ae736fcd
----- BEGIN BACKTRACE -----
mongod(mongo::printStackTrace(std::ostream&) 0x29) [0xef4db9]
 mongod(mongo::logContext(char const*) 0xE1) [0xea3721]
 mongod(mongo::invariantOKFailed(char const*, mongo::Status const&, char const*, unsigned int) 0xDA) [0xe8823a]
 mongod(mongo::WiredTigerRecoveryUnit::_txnClose(bool) 0x335) [0xd256b5]
 mongod(mongo::CmdListDatabases::run(mongo::OperationContext*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&, bool) 0x8CF) [0x93a43f]
 mongod(mongo::_execCommand(mongo::OperationContext*, mongo::Command*, std::string const&, mongo::BSONObj&, int, std::string&, mongo::BSONObjBuilder&, bool) 0x34) [0x9a29e4]
 mongod(mongo::Command::execCommand(mongo::OperationContext*, mongo::Command*, int, char const*, mongo::BSONObj&, mongo::BSONObjBuilder&, bool) 0xBC2) [0x9a38f2]
 mongod(mongo::_runCommands(mongo::OperationContext*, char const*, mongo::BSONObj&, mongo::_BufBuilder<mongo::TrivialAllocator>&, mongo::BSONObjBuilder&, bool, int) 0x27E) [0x9a45de]
 mongod(mongo::runQuery(mongo::OperationContext*, mongo::Message&, mongo::QueryMessage&, mongo::NamespaceString const&, mongo::CurOp&, mongo::Message&) 0x1450) [0xb63aa0]
 mongod(mongo::assembleResponse(mongo::OperationContext*, mongo::Message&, mongo::DbResponse&, mongo::HostAndPort const&) 0xAF3) [0xa7d573]
 mongod(mongo::MyMessageHandler::process(mongo::Message&, mongo::AbstractMessagingPort*, mongo::LastError*) 0xDF) [0x7eb15f]
 mongod(mongo::PortMessageServer::handleIncomingMsg(void*) 0x34B) [0xeb772b]
 libpthread.so.0( 0x683D) [0x2ba4ae44b83d]
 libc.so.6(clone 0x6D) [0x2ba4ae736fcd]
-----  END BACKTRACE  -----
2015-03-12T00:00:30.680+0000 I -        [conn82] 
***aborting after invariant() failure



 Comments   
Comment by Geert Bosch [ 02/Apr/15 ]

Yes, that would be good, even though we've worked around this issue in the integration layer now.

-Geert

Comment by Michael Cahill (Inactive) [ 02/Apr/15 ]

This is arguably a case of WiredTiger being too strict. We track whether any operation has "failed" during a transaction, with a fairly broad definition of failure. If an operation fails, any attempt to call WT_SESSION::commit_transaction will cause a rollback and return an error.

In the case of read-only transactions, this is a non-issue, because rollback and commit are equivalent. I'll open an issue to review this, it seems like we should be able to relax the restriction and allow commit after failures in read-only transactions.

Comment by Geert Bosch [ 01/Apr/15 ]

Nothing, as there shouldn't be any writes.

There will never be an active write transaction. Essentially, the commitAndRestart function is about the read side of the transaction. It is OK for the storage engine at this point to drop any snapshots or other resources associated with the given recoveryUnit and have the next read reflect new updates etc.

-Geert

Comment by Igor Canadi [ 01/Apr/15 ]

So mongo-rocks should also rollback transaction in this method? What can go wrong if we try to commit?

From my understanding, commitAndRestart() should never be called with active write transaction (because _depth == 0), right? Under which conditions can there be active write transaction during commitAndRestart()?

Comment by Geert Bosch [ 31/Mar/15 ]

Yes, sure. The answer is that the method is badly named. We'll be renaming it.

-Geert

Comment by Igor Canadi [ 31/Mar/15 ]

Geert can you please provide more context here? Method name is commitAndRestart(), so it's a bit counter-intuitive that correct behavior is to rollback the transaction.

Comment by Githook User [ 30/Mar/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-17562: Rollback transaction in WT commitAndRestart

(cherry picked from commit bbe95e94bc7231a0b06c395a52c6575d7d23e03e)
Branch: v3.0
https://github.com/mongodb/mongo/commit/6d12fa62f9604a985b48da19bf52ebe2d683864b

Comment by Githook User [ 25/Mar/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-17562: Rollback transaction in WT commitAndRestart
Branch: master
https://github.com/mongodb/mongo/commit/bbe95e94bc7231a0b06c395a52c6575d7d23e03e

Generated at Thu Feb 08 03:44:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.