Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-39323

Don't consider in-progress indexes when deciding whether or not to build an index under an intent lock

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.1.12
    • Affects Version/s: 4.1.8
    • Component/s: None
    • Labels:
      None
    • Fully Compatible
    • ALL
    • Storage NYC 2019-04-08, Storage NYC 2019-04-22, Storage NYC 2019-05-06

      I started seeing two Pymongo test failures on 4.2-latest that I think are related to the hybrid index build project. Both tests pass on 4.1.7 but fail on v4.1.7-176-gd807556. They are listed in PYTHON-1734 but I will put them here too along with how to run each case individually:

      $ python3.7 setup.py test -s test.test_legacy_api.TestLegacy.test_ensure_index_threaded
      running test
      ...
      test_ensure_index_threaded (test.test_legacy_api.TestLegacy) ... FAIL
      
      ======================================================================
      FAIL: test_ensure_index_threaded (test.test_legacy_api.TestLegacy)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/Users/shane/git/mongo-python-driver/test/test_legacy_api.py", line 1296, in test_ensure_index_threaded
          self.assertEqual(index_doc, first)
      AssertionError: {'_id[250 chars]ion'}, 'foo2_1': {'v': 2, 'key': [('foo2', 1)][43 chars]on'}} != {'_id[250 chars]ion'}}
        {'_id_': {'key': [('_id', 1)],
                  'ns': 'pymongo_test.threaded_index_creation',
                  'v': 2},
         'foo0_1': {'key': [('foo0', 1)],
                    'ns': 'pymongo_test.threaded_index_creation',
                    'v': 2},
         'foo1_1': {'key': [('foo1', 1)],
                    'ns': 'pymongo_test.threaded_index_creation',
      -             'v': 2},
      -  'foo2_1': {'key': [('foo2', 1)],
      -             'ns': 'pymongo_test.threaded_index_creation',
                    'v': 2}}
      
      ----------------------------------------------------------------------
      Ran 1 test in 0.334s
      
      FAILED (failures=1)
      

      The source for this test is here: https://github.com/mongodb/mongo-python-driver/blob/3.7.2/test/test_legacy_api.py#L1266-L1293
      From the output, it looks like the test expects that an index created with createIndex should show up immediately in the listIndexes output.

      And the second failing test:

      python3.7 setup.py test -s test.test_legacy_api.TestLegacy.test_ensure_unique_index_threaded
      running test
      ...
      test_ensure_unique_index_threaded (test.test_legacy_api.TestLegacy) ... FAIL
      
      ======================================================================
      FAIL: test_ensure_unique_index_threaded (test.test_legacy_api.TestLegacy)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/Users/shane/git/mongo-python-driver/test/test_legacy_api.py", line 1355, in test_ensure_unique_index_threaded
          self.assertEqual(10001, coll.count())
      AssertionError: 10001 != 10018
      
      ----------------------------------------------------------------------
      Ran 1 test in 0.493s
      
      FAILED (failures=1)
      

      The source for this test is here: https://github.com/mongodb/mongo-python-driver/blob/3.7.2/test/test_legacy_api.py#L1328-L1354
      This test creates the same unique index in multiple threads and then every thread attempts to insert a document with the same value for the unique field (and only one insert should succeed). It looks even though the createIndex succeeds (in some threads) the index is never actually created.

      These failures are reproducible on standalones, replica sets, and sharded clusters.

            Assignee:
            dianna.hohensee@mongodb.com Dianna Hohensee (Inactive)
            Reporter:
            shane.harvey@mongodb.com Shane Harvey
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: