[SERVER-57638] Benchmark Compression of Individual Audit Logs Created: 11/Jun/21 Updated: 29/Jun/21 Resolved: 29/Jun/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Shreyas Kalyan | Assignee: | Adrian Gonzalez Montemayor |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Sprint: | Security 2021-06-28, Security 2021-07-12 |
| Participants: |
| Description |
|
For PM-2126, we would like to offer customers log compression to reduce the overall size of their logs. However, because of constraints from cloud and concerns regarding data loss, we are unable to compress the entire audit log file. Instead, we want to see if compressing individual audit logs is a viable solution. In order to determine if it is a viable solution, we would like to see if compression of individual logs provides any considerable benefit for customers. We should write a script that ingests some audit log data, compresses the logs, and compares the sizes of compressed and uncompressed data. Using this tool, we will be able to make a better decision on whether to pursue allowing users to compress individual audit logs. |
| Comments |
| Comment by Adrian Gonzalez Montemayor [ 29/Jun/21 ] |
|
After investigation we deemed that compression is a worthwhile feature to add to PM-2126. From investigating user audit log compression we found that compressing JSON audit logs resulted in a ~50% size decrease and BSON audit logs into a ~30% size decrease. I have attached my script as a source to run these benchmarks in the future. |