[SERVER-79842] Differentiate serverStatus.network.compression by internal and external sources Created: 08/Aug/23  Updated: 18/Aug/23  Resolved: 18/Aug/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Alex Bevilacqua Assignee: Backlog - Service Architecture
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to DRIVERS-1335 Improve Awareness of Network Compress... Backlog
is related to SERVER-80150 Log negotiated network compressor wit... Closed
Assigned Teams:
Service Arch
Participants:

 Description   

serverStatus performance counters for network.compression can't currently be used to identify to what degree application traffic is compressed as (compressed) replication traffic (SERVER-3018) would also be incrementing these counters.

For example:

Atlas atlas-7cbfmf-shard-0 [primary] test> db.serverStatus().network
{
  [...]
  compression: {
    snappy: {
      compressor: { bytesIn: Long("774633295"), bytesOut: Long("544876919") },
      decompressor: { bytesIn: Long("475303790"), bytesOut: Long("730245460") }
    },
    zstd: {
      compressor: { bytesIn: Long("0"), bytesOut: Long("0") },
      decompressor: { bytesIn: Long("0"), bytesOut: Long("0") }
    },
    zlib: {
      compressor: { bytesIn: Long("0"), bytesOut: Long("0") },
      decompressor: { bytesIn: Long("0"), bytesOut: Long("0") }
    }
  },
  [...]
}

If the snappy network compressor were being used by one or more applications there would be no way to identify how much compressed application traffic was being generated (as snappy is the current default for networkMessageCompressors) .



 Comments   
Comment by Alex Bevilacqua [ 16/Aug/23 ]

Hey patrick.freed@mongodb.com, I've opened SERVER-80150 now as well which may be a more appropriate way to capture this information. Feel free to close this ticket.

Comment by Patrick Freed [ 16/Aug/23 ]

I imagine it would be an easier lift to expand the server's metadata logging to include the list of compressors advertised by the client rather than the original ask, would that be sufficient here? Or is the bytes breakdown a requirement?

Comment by Alex Bevilacqua [ 15/Aug/23 ]

patrick.freed@mongodb.com,

Could you elaborate a bit on how these metrics might be used, just so we can rule out other ways of gleaning similar information?

This would allow us to better understand how frequently network compression from client side applications is being used and (potentially) to what effect.

For example, if we're trying to figure out whether customers are using compression at all, we could check the contents of the client handshake, since they're required to specify their supported compressors there.

Unfortunately the compressors are specified via the connection string. Connection URI options aren't logged server-side as they're not part of the metadata transmitted with the initial handshake.

Comment by Patrick Freed [ 15/Aug/23 ]

Introducing this would involve a schema change to the serverStatus command response, which has the potential to break downstream users of it, so any change here will need some careful consideration. Could you elaborate a bit on how these metrics might be used, just so we can rule out other ways of gleaning similar information? For example, if we're trying to figure out whether customers are using compression at all, we could check the contents of the client handshake, since they're required to specify their supported compressors there.

Comment by Jason Chan [ 15/Aug/23 ]

Oops, sorry yeah that makes sense. For some reason I was thinking it was a 'minus' instead of a hyphen. Thank you for clarifying.

Comment by Alex Bevilacqua [ 15/Aug/23 ]

jason.chan@mongodb.com no strict urgency as this was raised as a blind spot while researching DRIVERS-1335. As for "what is 'n' here", I just wanted to indicate "1 or more applications" (the "n" wasn't significant, and likely more confusing than it needed to be)

I've updated the description accordingly

Comment by Jason Chan [ 15/Aug/23 ]

alex.bevilacqua@mongodb.com, I see this is linked to DRIVERS-1335. What's the timeline/urgency on this?

If the snappy network compressor were being used by 1-n applications there would be no way to identify how much compressed traffic was being generated (as snappy is the current default for networkMessageCompressors) .

To clarify, what is "n" here?

Generated at Thu Feb 08 06:42:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.