[SERVER-37807] startup warning if oplog maxSize exceeds disk size Created: 29/Oct/18 Updated: 25/Jun/19 Resolved: 25/Jun/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Kevin Pulo | Assignee: | Zach Yam (Inactive) |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | neweng | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Sprint: | Execution Team 2019-06-17, Execution Team 2019-07-01 | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
It's possible to create an oplog (whether by --oplogSize, replSetResizeOplog, or manually) where the capped maxSize is larger than the total size of the storage volume which holds the oplog. In this case, the oplog is basically guaranteed to (eventually) fill that disk, ie. without intervention the mongod can be expected to fail at some unknown time in the future. mongod should check the maxSize of the local.oplog.rs collection, and issue a startup warning if it exceeds the total physical size of the storage volume which that collection is stored on. This will alert users to this situation, so that it can be corrected before it becomes a problem. The check should be done after any new oplog has been created, as well as after the size has been changed with replSetResizeOplog (or perhaps check during this command, and refuse to run (or require force: true in that case). |
| Comments |
| Comment by Zach Yam (Inactive) [ 25/Jun/19 ] |
|
After discussing with geert.bosch, milkie, and brian.lane, we have decided to close this issue and leave it unresolved. As commented by Geert above, the oplog maxSize warning is not an accurate description of the amount of space left. Due to the fact that the compressibility of the oplog entries is variable, it is possible that the amount of data that can be stored on a volume may be several times the raw storage capacity of the disk. |
| Comment by Geert Bosch [ 14/Jun/19 ] |
|
OK, here is a straw man proposal:
|
| Comment by Geert Bosch [ 15/Nov/18 ] |
|
Note that the capped max size relates to data size, while the disk size is compressed size. Depending on compressibility of the oplog entries, and the selected compression algorithm, the amount of data that can be stored on a volume may be several times the raw storage capacity of the disk. At the same time, if the oplog volume is also used for data and journal files, there would be no warning even if the specified oplog size is many times larger than there is space on the volume. Would a warning still be helpful given these caveats, or would it cause confusion in cases that are actually fine and give a false sense of security in cases that aren't? |