[SERVER-12326] Drop operations for temporary aggregation (tmp.agg_out.n) collections can be interrupted Created: 10/Jan/14  Updated: 10/Dec/14  Resolved: 16/Jan/14

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kamran K. Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: 26qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Operating System: ALL
Steps To Reproduce:

var mongo = MongoRunner.runMongod();
var testDB = mongo.getDB('interrupts');
 
testDB.dropDatabase();
var coll = testDB.aggInterrupt;
 
// insert dummy data
var bigStr = Array(1024*1024 + 1).toString(); // 1MB of ','
for (var i = 0; i < 1000; i++) {
    coll.insert({_id: i, bigStr: String(i) + bigStr, random: Math.random()});
}
 
// set up failpoint
var conn = testDB.currentOp(true).inprog.filter(function(element){return element.connectionId;})[0].connectionId;
testDB.getSiblingDB('admin').runCommand({configureFailPoint: 'checkForInterruptFail', mode: 'alwaysOn', data: {chance: 0.05, conn: conn, allowNested: true}});
 
// loop until the drop of a temp table is interrupted
for (var i = 0; i < 1000; i++) {
	print(i);
 
    try {
        var pipeline = [{$sort: {random:1}}, {$group: {_id: '$_id', bigStr: {$first: '$bigStr'}}} , {$out: 'outputColl'}];
        coll.aggregate(pipeline, {allowDiskUsage: true});
    } catch (e) {
        print(e);
    }
 
    var res = {};
    do {
        try {
            print('attempting to drop output collection');
            res = testDB.outputColl.drop();
        } catch (e) {
            print('failed to drop output collection -- retrying');
            res.ok = 0;
        }
    } while(res.ok === 0);
 
    var names;
    do {
        try {
            print('attempting to get collection names');
            names = testDB.getCollectionNames();
        } catch (e) {
            print('failed to get collection names -- retrying');
            names = [];
        }
    } while(names.length === 0);
 
    assert.eq(names, ['aggInterrupt', 'system.indexes']);
}

Participants:

 Description   

Using $out in an aggregation pipeline creates a temporary collection of the form 'tmp.agg.out.n'. The drop operations for these temporary collections can be interrupted, which results in the collections being stored on disk until they're manually cleaned up or until the server is restarted.

The included JS test reproduces the problem by using the checkForInterruptFail failpoint to simulate maxTimeMS timeouts (SERVER-11695).

Edit: The temporary collections are cleaned up if you restart the server, so I corrected the initial description.


Version: 5af9a7433876664ac75bc0345f5903e04b5e1e7f


Generated at Thu Feb 08 03:28:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.