ExportXMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.1.0-rc0, 9.1.0-rc1025
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • Fully Compatible
    • Storage Execution 2026-09-28, Storage Execution 2026-10-12, Storage Execution 2026-10-26, Storage Execution 2026-11-09, Storage Execution 2026-11-23
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Each POA worker is currently backed by a single-threaded ThreadPool, and every enqueued work item is dispatched via ThreadPool::schedule(), which allocates a unique_function on the heap and locks the ThreadPool's internal queue mutex per item. In A/B test patch builds, flame graphs show the threadpool per-item dispatch overhead consumes ~92% of worker CPU, leaving only ~8% for actual oplog application. This eliminates the replication-lag reduction win POA is designed to deliver. Replacing each worker's ThreadPool with a raw stdx::thread backed by a mutex + condition_variable + deque (as in the original POA POC) gives the same public API (enqueue, shutdownAndJoin, queueDepth, report) with no caller or test changes, and restores the ~98% apply efficiency measured with the original POC's raw-thread worker loop.

      The new plan for this ticket is to build a new self-contained executor primitive that fits our use case. We should ensure this is header-only and add explicit unit tests. The requirements at a glance are:

      • single-worker executor (this can be expanded later)
      • reusable queue storage, so you can queue work items directly by std::move
      • use a persistent worker thread

       Further requirements ("processor" refers to the primitive, "consumer" refers to the worker thread):

      • templated, so that it can be used to process other item types
      • supports multiple producers (although this will not be part of our use case)
      • only two states: RUNNING and SHUTDOWN
      • consumer uses only one callback to process items, and it cannot be changed at runtime
      • consumer has a way to inform main thread of an error that occurred during its processing that does not involve throwing exceptions
      • shutdown is idempotent and irreversible
      • errors reported by the consumer trigger shutdown of the processor
      • errors reported by the consumer can be retrieved
      • calls to push an item while the processor is shut(ting) down fail
      • a way to "drain" the queue without triggering a shutdown
      • worker finishes processing its current item, and ignores all other items in the queue, if a shutdown is triggered before joining

       

            Assignee:
            Joseph Obaraye
            Reporter:
            Ali Mir
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: