Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-321

bulk-load blocks checkpoint

    • Type: Icon: Task Task
    • Resolution: Done
    • WT1.3.1
    • Affects Version/s: None
    • Component/s: None
    • Labels:

      Bulk-load blocks checkpoint operations (bulk-load has an exclusive lock on a handle, and checkpoint will wait to acquire a lock on the handle). This is a bug, we can't block checkpoints for a multi-hour bulk load.

      We should fix this with two changes:

      1. Change attempts to open handles that are exclusive (bulk-load, verify, salvage, upgrade) to return EBUSY. This is already the case in the other direction, that is, attempts to bulk-load, verify, salvage, or upgrade an object that is open in another cursor already returns EBUSY. While it is possible an application might want cursor-open to wait on a bulk-load operation, it's just as likely an application doesn't want to wait for a few hours while a bulk-load finishes, and (1) it's simpler for us to return EBUSY consistently, and (2) it's possible for an application to implement "loop until open succeeds" code, but it isn't possible for an application to implement "don't wait if it's going to take a long time code", and if we don't make this change, we're going to end up driving a O_NONBLOCK flag down through the API a couple of years from now.

      2. Change checkpoint to simply create empty-file checkpoints for files which return EBUSY on the open. (We can't just skip checkpoints for bulk-loaded files – if the application creates a named checkpoint, a checkpoint for the name has to exist.) This means a checkpoint of a partially-loaded bulk-file will look empty, but I don't see any problem with that.

            Assignee:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Reporter:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: