Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-71946

Async RPC expects all executor shutdown errors to be in the "cancellation" category, but those from ScopedTaskExecutor may not be

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 6.3.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Service Arch
    • Fully Compatible
    • ALL
    • Service Arch 2022-12-26, Service Arch 2023-01-09, Service Arch 2023-01-23, Service Arch 2023-02-06
    • 10

      The async_rpc library should not expose any errors other than RemoteCommandExecutionError, with additional information contained in the ErrorExtraInfo for that type. However, executor shutdown can prevent earlier continuations that should correctly process the error from running, so at the API boundary we re-write such executor shutdown errors before exiting the library. 

      Because we rely on the internals of the library to append correct information to the EEI for non-executor-shutdown errors, we try to expect (in debug mode) that all errors recieved at this boundary are either RemoteCommandExecutionError, or executor shutdown errors. We best-effort check if an error is an executor-shutdown error by checking if it's in the Cancellation error category. 

      However, ScopedTaskExecutor can use arbitrary error codes to reject work at shutdown, so it violates the assumption that "executor shutdown errors are in the Cancellation category." One is example is it uses the (non-cancellation) InterruptedDueToReplStateChange when it is used in PrimaryOnlyService. 

       

      To fix this, we should consider either relaxing or removing the async_rpc expectation on executor-shutdown errors, or add InterruptedDueToReplStateChange to the "cancellation" error category and ensure future error codes used for ScopedTaskExecutorShutdown are in this category as well. 

            Assignee:
            alex.li@mongodb.com Alex Li
            Reporter:
            george.wangensteen@mongodb.com George Wangensteen
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: