We should force-reload the config data the same way we do for queries after multiple retries with stale config exceptions.