|
Currently the decision of which writer commits a bucket is made when an individual writer makes the first uncommitted insert to a bucket. In cases with large batches and many writers, this can be highly problematic. A single writer W1 may have committer status on multiple buckets, e.g. B1 and B2. Many writers may be waiting on B2 while W1 is busy committing B1. One of these writers, say W2, could be committing B2, increasing throughput.
Shifting the committer assignment to commit time rather than insert time should allow us to minimize wait time.
At the same time, we will also simplify the state management a bit. Instead of keeping a promise per measurement insert, we will maintain likely a single state variable (promise or condition variable, etc.) at a time, or maybe a few, but definitely fewer than we currently maintain.
|