We have never tested what it means to configure LSM and cache_resident at the same time.

Configuring both is likely to stop the process of moving the in-memory chunks to disk - limiting the number of chunks to how much fits into cache.

That may be expected behavior - if so we should consider documenting and close this issue.