[SERVER-60510] Changing zstdCompressionLevel does not affect storageSize Created: 06/Oct/21 Updated: 27/Oct/23 Resolved: 19/Sep/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 5.0.3 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor - P4 |
| Reporter: | Platon workaccount | Assignee: | Yuan Fang |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | Bug | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Steps To Reproduce: | 1.
2. DB creation.
4.
5.
6. Creating DB with the same data.
8.
|
||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
As I understood from the documentation, storageSize depends on compression. But in practice it turned out that changing the compression level does not alter this value. By the way, indexSize slightly varies. |
| Comments |
| Comment by Platon workaccount [ 22/Sep/22 ] |
|
I measured storageSize again only after re-creating DB from scratch. I prefer to reboot the whole OS, not just the mongod. Even if the compression level limit is reached, it is very strange that the storageSize remains unchanged to the exact byte. I recommend to involve test team in a thorough investigation of the issue. |
| Comment by Yuan Fang [ 20/Sep/22 ] |
|
Did you recreate the collection after you restart the mongod process? Note that changing the settings of compression level only affects new collections, and existing collections stay with the old compression schema. If you did recreate the collection after starting mongod with the new "zstdCompressionLevel" and noticed the value of "storageSize" remains the same, it might be the case where the lower levels of compression have reached the lower limit and higher levels of compression cannot compress much further beyond that. The amount of compression generally depends on the type of data as well. I found a post at MongoDB Community that asked a similar question. Regards, Yuan |
| Comment by Platon workaccount [ 20/Sep/22 ] |
|
Hi, @Yuan Fang, I reported a problem with the collections themselves, not the journaled data. I didn't just change zstdCompressionLevel, I also rebuilt DB. The database created before the compression level was changed and the database created afterwards have the same size. Obviously, if compression works, the size of two DB versions cannot be equal. |
| Comment by Yuan Fang [ 19/Sep/22 ] |
|
Thank you for your report. If I understand correctly, you observed that the value of "storageSize" output from db.collection.stat() is not as expected after changing the "zstdCompressionLevel" when using the WiredTiger storage engine. This seems to line up with the WiredTiger 60 seconds checkpoint interval (See details in Snapshots and Checkpoints). Journaled data is written to disk with each write, but it's not written to the collection file, so we wouldn't expect those initial writes to be counted in "db.collection.stats().storageSize". We would expect to see accurate "storageSize" after the journaled operations are fully applied and a new checkpoint is created. You may find it helpful to check db.collection.stats() Accuracy after Unexpected Shutdown. For similar questions, we'd like to encourage you to start by asking our community for help by posting on the MongoDB Developer Community Forums. If the discussion there leads you to suspect a bug in the MongoDB server, then we'd want to investigate it as a possible bug here in the SERVER project. Regards, Yuan |