[SERVER-68556] Allow transactions on system.buckets collections Created: 04/Aug/22  Updated: 29/Oct/23  Resolved: 31/Aug/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.2.0-rc0, 6.0.9

Type: Task Priority: Major - P3
Reporter: Michael Gargiulo Assignee: Matthew Saltz (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Backport Requested:
v6.1, v6.0
Sprint: Execution Team 2022-09-05
Participants:

 Description   

Currently we prohibit operations inside of a multi-document transaction from being performed on all system collections. In order for time-series collections to integrate with the C2C replicator, the ability to perform operations within multi-doc transactions on the system.buckets collection is needed.



 Comments   
Comment by Githook User [ 12/Jul/23 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-68556 SERVER-78576 add TimeseriesTest.getBucketsCollName()

(cherry picked from commit 135eefc1bd00a5ab5c97ba679805dc150c9429ec)
Branch: v6.0
https://github.com/mongodb/mongo/commit/fb5d4765400b1292d68e73c276fe9ec673bab6eb

Comment by Githook User [ 30/Aug/22 ]

Author:

{'name': 'Matthew Saltz', 'email': 'matthew.saltz@mongodb.com', 'username': 'saltzm'}

Message: SERVER-68556 Allow transactions on timeseries buckets collections
Branch: master
https://github.com/mongodb/mongo/commit/135eefc1bd00a5ab5c97ba679805dc150c9429ec

Comment by Matthew Saltz (Inactive) [ 09/Aug/22 ]

Okay, so I modified my test to actually run in a transaction (I had to use session.getDatabase("test").getCollection(...)) to get the buckets collection), and it turned out that it still passes when I run an insert. However, when I run findAndModify (for example), it fails with

[js_test:server65886] uncaught exception: Error: findAndModifyFailed failed: {
[js_test:server65886]   "ok" : 0,
[js_test:server65886]   "errmsg" : "Cannot write to system collection test.system.buckets.timeseries_bucket_limit_time_range_1 within a transaction.",
[js_test:server65886]   "code" : 50781,
[js_test:server65886]   "codeName" : "Location50781",
[js_test:server65886]   "$clusterTime" : {
[js_test:server65886]           "clusterTime" : Timestamp(1660072361, 14),
[js_test:server65886]           "signature" : {
[js_test:server65886]                   "hash" : BinData(0,"7xKFIqYC1M3Rkf4yhrhOTR4stDQ="),
[js_test:server65886]                   "keyId" : NumberLong("7129956452243865607")
[js_test:server65886]           }
[js_test:server65886]   },
[js_test:server65886]   "operationTime" : Timestamp(1660072361, 13)
[js_test:server65886] } :

This is because we explicitly allowed inserts to timeseries buckets collections as part of SERVER-59164, but we didn't make the same allowance for findAndModify.

So we'll probably still have to do some work for this ticket, and maybe should unify those transaction checks somehow.

Comment by Matthew Saltz (Inactive) [ 09/Aug/22 ]

Ah - I think my test is wrong and may not actually be running a transaction at the moment... Stay tuned

Comment by Matthew Saltz (Inactive) [ 09/Aug/22 ]

I wrote a test to run a transaction on a system.buckets.<collection> collection to see if it failed, but it actually succeeded. (See the bottom of this comment for the full test). I'm wondering if the idea that we don't support running transactions on "system collections" comes from the fact that we used to ban transactions on the config database. (These conditions are now more relaxed after SERVER-42114 - in certain cases it is allowed to run transactions on certain collections in the config database.) Relevant code.

Unless someone else knows of a different prevention mechanism in place for collections with the prefix system. or can point out an issue in my test below (which is very possible), I think we can either close this ticket or try to commit a test proving that it works.

/**
 * Tests maximum time-range of measurements held in each bucket in a time-series buckets collection.
 * @tags: [
 *   does_not_support_stepdowns,
 *   does_not_support_transactions,
 * ]
 */
(function() {
"use strict";
 
load("jstests/core/timeseries/libs/timeseries.js");  // For 'TimeseriesTest'.
 
TimeseriesTest.run((insert) => {
    const collNamePrefix = 'timeseries_bucket_limit_time_range_';
    const timeFieldName = 'time';
 
    // Assumes the measurements in each bucket span at most one hour (based on the time field).
    // Make sure we have three measurements to trigger compression if enabled. The data types in
    // this test are so small so two measurements may not yield a smaller compressed object
    const docTimes = [
        ISODate("2020-11-13T01:00:00Z"),
        ISODate("2020-11-13T01:00:01Z"),
        ISODate("2020-11-13T01:00:02Z"),
        ISODate("2020-11-13T03:00:00Z")
    ];
    const numDocs = 4;
 
    const runTest = function(numDocsPerInsert) {
        const coll = db.getCollection(collNamePrefix + numDocsPerInsert);
        const bucketsColl = db.getCollection('system.buckets.' + coll.getName());
        coll.drop();
 
        assert.commandWorked(
            db.createCollection(coll.getName(), {timeseries: {timeField: timeFieldName}}));
        assert.contains(bucketsColl.getName(), db.getCollectionNames());
 
        let docs = [];
        for (let i = 0; i < numDocs; i++) {
            docs.push({_id: i, [timeFieldName]: docTimes[i], x: i});
            if ((i + 1) % numDocsPerInsert === 0) {
                assert.commandWorked(insert(coll, docs), 'failed to insert docs: ' + tojson(docs));
                docs = [];
            }
        }
 
        // Check view.
        const viewDocs = coll.find().sort({_id: 1}).toArray();
        assert.eq(numDocs, viewDocs.length, viewDocs);
        for (let i = 0; i < numDocs; i++) {
            const viewDoc = viewDocs[i];
            assert.eq(i, viewDoc._id, 'unexpected _id in doc: ' + i + ': ' + tojson(viewDoc));
            assert.eq(i, viewDoc.x, 'unexpected field x in doc: ' + i + ': ' + tojson(viewDoc));
            assert.eq(docTimes[i],
                      viewDoc[timeFieldName],
                      'unexpected time in doc: ' + i + ': ' + tojson(viewDoc));
        }
 
        // Check bucket collection.
        const bucketDocs = bucketsColl.find().sort({'control.min._id': 1}).toArray();
        assert.eq(2, bucketDocs.length, bucketDocs);
 
        const session = db.getMongo().startSession({causalConsistency: false});
        const txnOptions = {readConcern: {level: "snapshot"}};
        session.startTransaction(txnOptions);
 
        const testDoc = {
            "_id": ObjectId("5fadda90b08faa80f76078c1"),
            "control": {
                "version": 1,
                "min": {"_id": 0, "time": ISODate("2020-11-13T01:00:00Z"), "x": 0},
                "max": {"_id": 0, "time": ISODate("2020-11-13T01:00:00Z"), "x": 0}
            },
            "data": {"_id": {"0": 0}, "x": {"0": 0}, "time": {"0": ISODate("2020-11-13T01:00:00Z")}}
        };
        const testDoc2 = {
            "_id": ObjectId("5fadda90b08faa80f76078c2"),
            "control": {
                "version": 1,
                "min": {"_id": 0, "time": ISODate("2020-11-13T01:00:00Z"), "x": 0},
                "max": {"_id": 0, "time": ISODate("2020-11-13T01:00:00Z"), "x": 0}
            },
            "data": {"_id": {"0": 0}, "x": {"0": 0}, "time": {"0": ISODate("2020-11-13T01:00:00Z")}}
        };
 
        assert.commandWorked(bucketsColl.insert(testDoc, {writeConcern: {w: "majority"}}));
        assert.commandWorked(bucketsColl.insert(testDoc2, {writeConcern: {w: "majority"}}));
 
        assert.commandWorked(session.commitTransaction_forTesting());
    };
 
    runTest(1);
    runTest(numDocs);
});
})();

Generated at Thu Feb 08 06:11:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.