sueloverso This one deserves an issue, so we remember why we made the decision.

I'd like opinions on what the new conn->async_flush call should do when there are racing calls.

Currently if two threads call conn->async_flush, the first one sets a flag, and waits for the worker threads to complete the flush. Then it clears the flag and returns. The second one sees the flag set and returns (with 0). Other threads can continue to enqueue ops, but the workers will not process them until the flush completes. So async_flush is a barrier point.

Should that be changed so that the 2nd thread returns an error? Or maybe it waits until the in-progress flush completes before returning? A competing flush call returning 0 implies to the application that the flush is actually complete when it is not.

Or should it convoy on the in-progress flush? Consider this scenario:

• threads enqueue several async ops. Call these the A1 set of ops.
• T1 calls async_flush and waits for the workers to complete it.
• threads enqueue several more async ops (enqueues, but not processed until flush completes). Call these the A2 ops.
• T2 calls async_flush, wanting to wait for the A2 ops to finish.
• T2 finds T1's flush in progress and (currently) returns 0 immediately.
• T2 fails when it looks for the A2 ops.

Even if T2 waited until the in-progress flush completed, the A2 ops are not processed.

I think I just convinced myself that we must convoy racing flush operations.