Details

      Description

      WiredTiger currently (2.5) has a schema lock to single-thread schema-modifying operations combined with moderately complex locking of data handles to provide shared or exclusive access.

      In addition, there is a "table lock" to single-thread schema-modifying operations on the same table that may not exclusively lock the same file (e.g., adding or dropping independent indices). Further, there is a "data handle list list" lock to protect the list of data handles.

      This is complicated, fragile and hard to maintain.

      Instead, we should move to a situation where:

      • all data sources in the system that require locking (including at least tables, indices and LSM trees) become WT_DATA_HANDLEs.
        _ all operations start by locking the data handles they need. This involves searching the session for cached data handles, then _protected by the data handle mutex* searching and optionally creating the data handles in the connection cache.
      • once a required handle is found, it is cached in the session cache, and only at this stage is there any blocking on the data handle lock (after the handle list mutex is dropped).
      • blocking should be made configurable (default blocking=true), rather than the current behavior where read-only operations block but exclusive operations sometimes fail with EBUSY.
      • since each operation only operates on a single high-level handle (e.g., a single table, even if that includes multiple btrees or LSM trees underneath), and exclusive locks are never held for longer than a single operation, we can continue to avoid data handle deadlocks.

      These changes would mean:

      • we can eliminate the schema lock, the table lock and LSM locks: the operations they currently protect would be protected by handle locks on tables, indices or LSM trees.
      • no operation ever blocks while holding the data handle list lock: it is purely used to protect the shared handle list
      • more concurrency: independent schema-modifying operations can complete independently. This requires some care with regard to metadata changes, particularly with regard to checkpoints and when newly-created data sources become durable.

        Issue Links

          Activity

          Hide
          michael.cahill Michael Cahill added a comment -

          In case it isn't clear: this is a medium-term project with a number of sub-tasks. It isn't going to be slotted into a scheduled release.

          Show
          michael.cahill Michael Cahill added a comment - In case it isn't clear: this is a medium-term project with a number of sub-tasks. It isn't going to be slotted into a scheduled release.
          Hide
          ramon.fernandez Ramon Fernandez added a comment -

          Additional ticket information from GitHub

          This ticket was referenced in the following commits:
          Show
          ramon.fernandez Ramon Fernandez added a comment - Additional ticket information from GitHub This ticket was referenced in the following commits: 1ce3b94d6e40d37a77e62eda500f286bd3816eb9
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Restructure handle locking so it all happens at the session level, avoid entanglements between handle locks, the schema lock and the handle list lock.

          refs WT-1598
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/4ca25b3678b98822dddde86ffd6b693a78c20dd2

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Restructure handle locking so it all happens at the session level, avoid entanglements between handle locks, the schema lock and the handle list lock. refs WT-1598 Branch: develop https://github.com/wiredtiger/wiredtiger/commit/4ca25b3678b98822dddde86ffd6b693a78c20dd2
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: If we get part way through a btree open then fail, make sure the checkpoint is unloaded. Hopefully this will fix a hard-to-reproduce leak found recently by valgrind.

          refs WT-1598
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/1b790113a6f80aa1345c1eee84f9fbfa13a689ce

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: If we get part way through a btree open then fail, make sure the checkpoint is unloaded. Hopefully this will fix a hard-to-reproduce leak found recently by valgrind. refs WT-1598 Branch: develop https://github.com/wiredtiger/wiredtiger/commit/1b790113a6f80aa1345c1eee84f9fbfa13a689ce
          Hide
          michael.cahill Michael Cahill added a comment -

          In working on WT-2396, I realized that checkpoint is not gathering all of its handles at the beginning of the operation. It currently walks the trees to be checkpointed and gathers the handle of each current (live) tree in shared mode. Then the schema lock is dropped. Later, as each tree is processed, it acquires exclusive locks on the checkpoint handles to be dropped.

          It would be better to gather all of these handles while holding the schema lock. This would involve storing the list of WT_CKPT structs for each handle being checkpointed. It would further involve care around hot backups, which is currently checked separately for each handle being checkpointed.

          Show
          michael.cahill Michael Cahill added a comment - In working on WT-2396 , I realized that checkpoint is not gathering all of its handles at the beginning of the operation. It currently walks the trees to be checkpointed and gathers the handle of each current (live) tree in shared mode. Then the schema lock is dropped. Later, as each tree is processed, it acquires exclusive locks on the checkpoint handles to be dropped. It would be better to gather all of these handles while holding the schema lock. This would involve storing the list of WT_CKPT structs for each handle being checkpointed. It would further involve care around hot backups, which is currently checked separately for each handle being checkpointed.

            People

            • Votes:
              4 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Days since reply:
                1 year, 14 weeks, 6 days ago
                Date of 1st Reply: