Spawned from the TSAN project, we identified that connection->reconfigure create a ton of potential data races. The code currently has conn->reconfig lock but is never used. Instead we rely on the concurrency between reconfiguring and the related threads to just work. This should not be recommended practice. This ticket aims to investigate the correct expectations around the conn->reconfigure and implement a consistent view of the system. There are been some ideas thrown around:

• Use a read/write lock and make sure that the system sees all the reconfigured values at once.
• If we don't care about concurrency control, use relax atomics throughout the project.
• Close all background threads and re-open all threads again.