[SERVER-58943] Add more test coverage for unique indexes Created: 29/Jul/21  Updated: 29/Oct/23  Resolved: 03/Sep/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Task Priority: Critical - P2
Reporter: Henrik Edin Assignee: Haley Connelly
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-58936 Unique index constraints may not be e... Closed
is related to SERVER-56509 Wrap unique index insertion _keyExist... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2021-08-09, Execution Team 2021-08-23, Execution Team 2021-09-06
Participants:

 Description   

SERVER-56509 introduced a bug where the unique index constraint was not enforced in some cases. We should add more test coverage for this.

The issue could be reproduced with the following script that we can base additional testing on:

function repro1() {
 
    db.c.drop()
    db.c.createIndex({t: 1, i: 1}, {unique: true})
 
    const n = 10000
    const t = 0
    for (var i = 0; i < n; i++)
        db.c.insert({t, i})
    for (iter = 0; ; iter++) {
        print("iter", iter)
        for (var i = 0; i < n; i++) {
            r = db.c.insert({t, i, x: 'x'})
            if (r.nInserted != 0)
                print("dup!", t, i, "iter", iter, "count", db.c.count({t, i}))
        }
    }
}

Which reproduces the issue after a couple of iterations:

iter 0
iter 1
dup! 0 9999 iter 1 count 2

A multithreaded version also reproduces:

function repro() {
 
    db.c.drop()
    db.c.createIndex({t: 1, i: 1}, {unique: true})
 
    nthreads = 100
    threads = []
    for (var t=0; t<nthreads; t++) {
        thread = new Thread(function(t) {
            for (var i = 0; i < 100; i++)
                db.c.insert({t, i, x: 'x'})
            while (true) {
                for (var i = 0; i < 100; i++) {
                    r = db.c.insert({t, i, x: 'x'})
                    if (r.nInserted != 0)
                        print("dup!", t, i, "count", db.c.count({t, i}))
                }
            }
        }, t)
        threads.push(thread)
        thread.start()
    }
    for (var t = 0; t < nthreads; t++)
        threads[t].join()
}



 Comments   
Comment by Githook User [ 02/Sep/21 ]

Author:

{'name': 'Haley Connelly', 'email': 'haley.connelly@mongodb.com', 'username': 'haleyConnelly'}

Message: SERVER-58943 Add more test coverage for unique indexes
Branch: master
https://github.com/mongodb/mongo/commit/ed4da207dcd5b8b6c0006853a58cf6ed421f13fd

Comment by Bruce Lucas (Inactive) [ 29/Jul/21 ]

Agree, there should also be additional more targeted WT tests.

Comment by Eric Milkie [ 29/Jul/21 ]

This is the kind of test that would be more effective in the WiredTiger storage engine code itself, since we can more easily manipulate and find b-tree-format specific problems.
Putting these tests in the MongoDB layer is hit-or-miss at best, since we are trying to fuzz the storage engine and hope that we can generate interesting storage engine behavior, both now and in the future.

Comment by Bruce Lucas (Inactive) [ 29/Jul/21 ]

The multi-threaded version reproduces the issue much more quickly and in more of the index entries, so I suspect it generates a variety of btree conditions that might more readily find future issues.

Generated at Thu Feb 08 05:45:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.