[SERVER-60510] Changing zstdCompressionLevel does not affect storageSize Created: 06/Oct/21  Updated: 27/Oct/23  Resolved: 19/Sep/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.3
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Platon workaccount Assignee: Yuan Fang
Resolution: Works as Designed Votes: 0
Labels: Bug
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to WT-6887 Storage size incorrectly reported wit... Closed
Operating System: ALL
Steps To Reproduce:

1.

sudo nano /etc/mongod.conf

storage:
  wiredTiger:
    engineConfig:
      zstdCompressionLevel: 10

2. DB creation.
3.

use db_name


4.

db.stats()

storageSize: 9838592

5.

sudo nano /etc/mongod.conf

storage:
  wiredTiger:
    engineConfig:
      zstdCompressionLevel: 22

6. Creating DB with the same data.
7.

use db_name


8.

db.stats()

storageSize: 9838592

Participants:

 Description   

As I understood from the documentation, storageSize depends on compression. But in practice it turned out that changing the compression level does not alter this value. By the way, indexSize slightly varies.



 Comments   
Comment by Platon workaccount [ 22/Sep/22 ]

I measured storageSize again only after re-creating DB from scratch. I prefer to reboot the whole OS, not just the mongod.

Even if the compression level limit is reached, it is very strange that the storageSize remains unchanged to the exact byte.

I recommend to involve test team in a thorough investigation of the issue.

Comment by Yuan Fang [ 20/Sep/22 ]

Hi platon.work@gmail.com,

Did you recreate the collection after you restart the mongod process? Note that changing the settings of compression level only affects new collections, and existing collections stay with the old compression schema.

If you did recreate the collection after starting mongod with the new "zstdCompressionLevel" and noticed the value of "storageSize" remains the same, it might be the case where the lower levels of compression have reached the lower limit and higher levels of compression cannot compress much further beyond that. The amount of compression generally depends on the type of data as well. I found a post at MongoDB Community that asked a similar question.

Regards,

Yuan

Comment by Platon workaccount [ 20/Sep/22 ]

Hi, @Yuan Fang,

I reported a problem with the collections themselves, not the journaled data.

I didn't just change zstdCompressionLevel, I also rebuilt DB. The database created before the compression level was changed and the database created afterwards have the same size. Obviously, if compression works, the size of two DB versions cannot be equal.

Comment by Yuan Fang [ 19/Sep/22 ]

Hi platon.work@gmail.com ,

Thank you for your report. If I understand correctly, you observed that the value of "storageSize" output from db.collection.stat() is not as expected after changing the "zstdCompressionLevel" when using the WiredTiger storage engine. This seems to line up with the WiredTiger 60 seconds checkpoint interval (See details in Snapshots and Checkpoints). Journaled data is written to disk with each write, but it's not written to the collection file, so we wouldn't expect those initial writes to be counted in "db.collection.stats().storageSize". We would expect to see accurate "storageSize" after the journaled operations are fully applied and a new checkpoint is created. You may find it helpful to check db.collection.stats() Accuracy after Unexpected Shutdown.

For similar questions, we'd like to encourage you to start by asking our community for help by posting on the MongoDB Developer Community Forums. If the discussion there leads you to suspect a bug in the MongoDB server, then we'd want to investigate it as a possible bug here in the SERVER project.

Regards,

Yuan

Generated at Thu Feb 08 05:49:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.