When we schedule/submit a task via ScheduledExecutorService/ExecutorService, and do not properly consume the returned Future, the uncaught task failures end up wrapped in that Future and GCed without ever being handled. That is especially bad when they are Error​s. JAVA-6240 introduced MongoThreadPoolExecutor, MongoScheduledThreadPoolExecutor to deal with this problem (see the API documentation on these classes for all the details, including the information on why "hiding" Error​s this way is problematic).

We should:

1. Use MongoThreadPoolExecutor/MongoScheduledThreadPoolExecutor instead of ThreadPoolExecutor/ScheduledThreadPoolExecutor (created either by calling the constructors explicitly, or by calling the java.util.concurrent.Executors methods) everywhere we can (whether we should add more modifications to the vendored tlschannel library code is a question).
   • We should replace the threads we create directly with single-thread MongoThreadPoolExecutor/MongoScheduledThreadPoolExecutor instances. This way the code will be more homogeneous, and locating such places will be easier.
2. Make sure no tasks we schedule/submit, or even execute (this method does not return Future) leave Exception​s uncaught, as MongoThreadPoolExecutor/MongoScheduledThreadPoolExecutor treat those as bugs and propagate as AssertionFailure​s to the UncaughtExceptionHandler. If we have nothing better to do with an Exception, we should log it instead of letting it being thrown from a task.
   • This includes the tasks we submit to the threads we create directly, without using ExecutorService.

This, combined with JAVA-5907 (we replaced submit invocations with execute) and JAVA-6110, ensures that the UncaughtExceptionHandler configured by an application will have a chance to handle all Errors that originate from our code. And if no UncaughtExceptionHandler, the Error​s at least will be handled the same way the Java SE handles all uncaught exceptions by default: by printing them to the stderr.

This ticket was created based on this PR comment, and then updated based on this other PR comment.