[SERVER-33367] Enforce a cursor is iterated in a transaction/session iff it was created in that transaction/session Created: 16/Feb/18  Updated: 29/Oct/23  Resolved: 06/Mar/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.7.3

Type: Task Priority: Major - P3
Reporter: James Wahlin Assignee: Tess Avitabile (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-33368 Add snapshot read testing for txnNumb... Closed
Documented
is documented by DOCS-11425 Update getMore for Transactions (SERV... Closed
Related
related to SERVER-34204 Tailable cursor fails on getMore agai... Closed
Backwards Compatibility: Fully Compatible
Sprint: Storage NYC 2018-03-12
Participants:

 Description   

If a cursor was created in a transaction (or snapshot read), it should not be allowed to getMore it outside of that transaction. If a cursor was not created in a transaction (or snapshot read), it should not be allowed to getMore it inside a transaction.

Additionally, a cursor should be iterated in a session iff it was created in that session.



 Comments   
Comment by Tess Avitabile (Inactive) [ 11/Oct/18 ]

This might be risky because not all versions of 3.6 have the fix inĀ SERVER-34204 that ensured mongos always attaches the lsid for tailable cursors. So there could be an issue with mixed minor versions, if the mongos does not attach the lsid but the mongod attempts to check the lsid.

Comment by Blake Oler [ 11/Oct/18 ]

Do we want to backport the getMore LSID cursor verification to 3.6? I can backport the "guarantee full LSID" code from SERVER-35323, but without the getMore verification, there's not a solid way to verify the behavior via jsTest.

tess.avitabile misha.tyulenev

Comment by Tess Avitabile (Inactive) [ 06/Mar/18 ]

I do not expect there to be Drivers changes, since the Drivers should always attach the lsid/txnNumber of the originating command to the getMore, but I want to inform the Drivers that this is now enforced on the Server.

Comment by Githook User [ 06/Mar/18 ]

Author:

{'email': 'tess.avitabile@mongodb.com', 'name': 'Tess Avitabile', 'username': 'tessavitabile'}

Message: SERVER-33367 Enforce a cursor is iterated in a transaction/session iff it was created in that transaction/session
Branch: master
https://github.com/mongodb/mongo/commit/965439580c811393eb87581f21af8684bfe41d45

Comment by Tess Avitabile (Inactive) [ 27/Feb/18 ]

Per in-person discussion, the work done for txnNumber integrity will also cover local snapshot reads.

This ticket should cover getMores of transaction cursors from outside the transaction and getMores non-transaction cursors from inside a transaction in a general way that covers both local snapshot reads and multi-statement transactions.

Comment by Tess Avitabile (Inactive) [ 27/Feb/18 ]

spencer, just to check, will the work you do for ensuring txnNumber does not move backward and statement IDs are not repeated also cover local snapshot reads, or only multistatement transactions?

Comment by Tess Avitabile (Inactive) [ 27/Feb/18 ]

In the case where we create a snapshot cursor then exhaust it outside of the transaction, I think the only problem is that we don't clean up transaction state. Maybe we don't care. In the case where we create a normal cursor then iterate it inside a transaction (and we have stashed transaction state), we hit the following invariant:

__unknown_name__:PRIMARY> sessionDB.runCommand({find: "c", batchSize: 2, readConcern: {level: "snapshot"}, txnNumber: NumberLong(0)})
{
	"cursor" : {
		"firstBatch" : [
			{
				"_id" : 0
			},
			{
				"_id" : 1
			}
		],
		"id" : NumberLong("56605089430"),
		"ns" : "test.c"
	},
	"ok" : 1,
	"operationTime" : Timestamp(1519748065, 5),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1519748065, 5),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
__unknown_name__:PRIMARY> var res = sessionDB.runCommand({find: "c", batchSize: 2})
__unknown_name__:PRIMARY> sessionDB.runCommand({getMore: res.cursor.id, collection: "c", txnNumber: NumberLong(0)})

d20000| ----- BEGIN BACKTRACE -----
d20000| {"backtrace":[{"b":"55BA3A3E0000","o":"221A691","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55BA3A3E0000","o":"22198A9"},{"b":"55BA3A3E0000","o":"2219D8D"},{"b":"7F7DC7842000","o":"10D10"},{"b":"7F7DC7478000","o":"35267","s":"gsignal"},{"b":"7F7DC7478000","o":"36ECA","s":"abort"},{"b":"55BA3A3E0000","o":"983B60","s":"_ZN5mongo22invariantFailedWithMsgEPKcS1_S1_j"},{"b":"55BA3A3E0000","o":"10D13C0","s":"_ZN5mongo15PlanYieldPolicy11shouldYieldEv"},{"b":"55BA3A3E0000","o":"10C9635","s":"_ZN5mongo12PlanExecutor11getNextImplEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE"},{"b":"55BA3A3E0000","o":"10C9C6B","s":"_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE"},{"b":"55BA3A3E0000","o":"E21735"},{"b":"55BA3A3E0000","o":"1B42636","s":"_ZN5mongo12BasicCommand11enhancedRunEPNS_16OperationContextERKNS_12OpMsgRequestERNS_14BSONObjBuilderE"},{"b":"55BA3A3E0000","o":"1B3DA8E","s":"_ZN5mongo7Command9publicRunEPNS_16OperationContextERKNS_12OpMsgRequestERNS_14BSONObjBuilderE"},{"b":"55BA3A3E0000","o":"A2489C"},{"b":"55BA3A3E0000","o":"A26DFA"},{"b":"55BA3A3E0000","o":"A27C5D","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE"},{"b":"55BA3A3E0000","o":"A1728A","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE"},{"b":"55BA3A3E0000","o":"A209BA","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE"},{"b":"55BA3A3E0000","o":"A1BEA7","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"55BA3A3E0000","o":"A1F731"},{"b":"55BA3A3E0000","o":"18E79E2","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE"},{"b":"55BA3A3E0000","o":"A1AC1F","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE"},{"b":"55BA3A3E0000","o":"A1D255","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE"},{"b":"55BA3A3E0000","o":"A1DB0B","s":"_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE"},{"b":"55BA3A3E0000","o":"A1BF2D","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"55BA3A3E0000","o":"A1F731"},{"b":"55BA3A3E0000","o":"18E7F45"},{"b":"55BA3A3E0000","o":"20C4084"},{"b":"7F7DC7842000","o":"76AA"},{"b":"7F7DC7478000","o":"106EED","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.7.2-193-ge3f3617", "gitVersion" : "e3f361769cd13ba88aa24c1c0a71c76b187f64dd", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.2.0-16-generic", "version" : "#19-Ubuntu SMP Thu Oct 8 15:35:06 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "b" : "55BA3A3E0000", "elfType" : 3, "buildId" : "56F6F27B68A0C2A013B282C8BA9C7C5A9E76354C" }, { "b" : "7FFCD5BF3000", "elfType" : 3, "buildId" : "BC784BEAA199A23C0D47D6AC888C3DAEA5EEAA0C" }, { "b" : "7F7DC838B000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "AEFCB8C813AA8B1836E584EA5152317DA6C7A25F" }, { "b" : "7F7DC8183000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "CB26CB6169CBAA9DA2E38A70BCF7D57A8047CCF3" }, { "b" : "7F7DC7F7F000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "F974B99C0C327670EF882BA13912E995A12C6402" }, { "b" : "7F7DC7C77000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "372ADD7EB11B9D51E634BC91DAAA58DF90567FD1" }, { "b" : "7F7DC7A60000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "0C3C07EE15CFA81346847A679E8444B876D9CC58" }, { "b" : "7F7DC7842000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "A37A144BCBEE86A9E02DFF5021A111EDE6A1F212" }, { "b" : "7F7DC7478000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "8ACD43CF74A9756CD727B8516B08679EE071A92D" }, { "b" : "7F7DC85A7000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "903BB7A6DEEFD966DCEEC4566C70444C727ED294" } ] }}
d20000|  mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55ba3c5fa691]
d20000|  mongod(+0x22198A9) [0x55ba3c5f98a9]
d20000|  mongod(+0x2219D8D) [0x55ba3c5f9d8d]
d20000|  libpthread.so.0(+0x10D10) [0x7f7dc7852d10]
d20000|  libc.so.6(gsignal+0x37) [0x7f7dc74ad267]
d20000|  libc.so.6(abort+0x16A) [0x7f7dc74aeeca]
d20000|  mongod(_ZN5mongo22invariantFailedWithMsgEPKcS1_S1_j+0x0) [0x55ba3ad63b60]
d20000|  mongod(_ZN5mongo15PlanYieldPolicy11shouldYieldEv+0x80) [0x55ba3b4b13c0]
d20000|  mongod(_ZN5mongo12PlanExecutor11getNextImplEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE+0x555) [0x55ba3b4a9635]
d20000|  mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE+0x4B) [0x55ba3b4a9c6b]
d20000|  mongod(+0xE21735) [0x55ba3b201735]
d20000|  mongod(_ZN5mongo12BasicCommand11enhancedRunEPNS_16OperationContextERKNS_12OpMsgRequestERNS_14BSONObjBuilderE+0x76) [0x55ba3bf22636]
d20000|  mongod(_ZN5mongo7Command9publicRunEPNS_16OperationContextERKNS_12OpMsgRequestERNS_14BSONObjBuilderE+0x2E) [0x55ba3bf1da8e]
d20000|  mongod(+0xA2489C) [0x55ba3ae0489c]
d20000|  mongod(+0xA26DFA) [0x55ba3ae06dfa]
d20000|  mongod(_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE+0x2CD) [0x55ba3ae07c5d]
d20000|  mongod(_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE+0x3A) [0x55ba3adf728a]
d20000|  mongod(_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE+0xBA) [0x55ba3ae009ba]
d20000|  mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x97) [0x55ba3adfbea7]
d20000|  mongod(+0xA1F731) [0x55ba3adff731]
d20000|  mongod(_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE+0x1A2) [0x55ba3bcc79e2]
d20000|  mongod(_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE+0x15F) [0x55ba3adfac1f]
d20000|  mongod(_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE+0xAF5) [0x55ba3adfd255]
d20000|  mongod(_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE+0x1FB) [0x55ba3adfdb0b]
d20000|  mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x11D) [0x55ba3adfbf2d]
d20000|  mongod(+0xA1F731) [0x55ba3adff731]
d20000|  mongod(+0x18E7F45) [0x55ba3bcc7f45]
d20000|  mongod(+0x20C4084) [0x55ba3c4a4084]
d20000|  libpthread.so.0(+0x76AA) [0x7f7dc78496aa]
d20000|  libc.so.6(clone+0x6D) [0x7f7dc757eeed]
d20000| -----  END BACKTRACE  -----

At the very least we will need to address this issue.

Comment by Spencer Brody (Inactive) [ 26/Feb/18 ]

We plan on erroring if txnNumber goes backwards, or if we see a stmtId we've already seen for the current txnNumber. We don't currently have any plans to do any cursor-specific checks.

Comment by Tess Avitabile (Inactive) [ 26/Feb/18 ]

spencer, do you have any similar work planned for multi statement transactions? For example, erroring if a getMore is received with the txnNumber of the currently open multi-statement transaction, but with a cursor ID of a cursor not created as part of the transaction?

Comment by Tess Avitabile (Inactive) [ 16/Feb/18 ]

We should also ban running a getMore outside of a transaction that is on a snapshot cursor.

Generated at Thu Feb 08 04:33:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.