Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-3034

Add support for single-writer named snapshots.

    • Type: Icon: New Feature New Feature
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • WT2.9.1, 3.2.12, 3.4.2, 3.5.1
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Storage 2016-12-12

      Message #1:

      Our application is designed to use many threads to coordinate the work done for a transaction, and we want to use snapshot isolation for that transaction. We are able to do this for read-only operations by having multiple Sessions which share a named snapshot.

      Further, we designate one of those Session objects as the "writer". Any mutations contend for that writer. This works fine. However, once we have applied a mutation to the "writer" Session, that mutation is not visible in other Sessions objects for the same named snapshot. It would be nice to have such mutations become visible among the Session objects that we use to coordinate the work for a transaction (e.g., same named snapshot), but to not become visible outside of that scope until a commit.

      I am trying to use multiple threads doing work in parallel as part of the same transaction. This works fine if there are no mutations. All I need to do is have each thread use a Session object which is sharing the same named snapshot. They all see the same data.

      Ideally I would like to find a way to have those threads see the updates as well. That is, I would like them to be part of the same transaction.

      I was hoping that WT named snapshots would have these semantics. So I could designate one Session as the "writer" and the other Sessions sharing the same named snapshot would see the same data.

      But this does not appear to be the case. The named snapshot let's the different Session objects see the same starting state, but updates do not become visible. Thus if there is a mutation applied, I need to single thread all subsequent operations through the same Session in order to see those updates.

      Basically, WT Session = Transaction, which implies that a multiple threads can not coordinate to service the transaction since a Session can only be used by a single thread at a time.

      Message #2:

      WiredTiger's named snapshots were designed to allow multiple transactions to read from the same snapshot. The requirement in MongoDB is that many of those transactions reading from the same snapshot may go on to do (independent) updates. For that reason, they don't see each other's writes.

      We could support what you're asking for relatively easily, if the "writer" can create the named snapshot and "readers" are forced to be read-only. I'd suggest adding a new configuration to WT_SESSION::snapshot to support this mode (called something like "include_updates"). Transactions started from such a snapshot would see the updates of the writer but would be forced to be read-only. There are data structures inside the session that cache the transaction's write set and log records that are not thread safe, so it would be a very different change to support multiple writer sessions for a single transaction.

            michael.cahill@mongodb.com Michael Cahill (Inactive)
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            0 Vote for this issue
            3 Start watching this issue