[SERVER-52919] Wire compression not enabled for initial sync Created: 17/Nov/20  Updated: 29/Oct/23  Resolved: 19/Dec/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.4.1, 4.2.10
Fix Version/s: 4.9.0, 4.2.12, 4.4.4

Type: Bug Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Amirsaman Memaripour
Resolution: Fixed Votes: 2
Labels: sa-groomed
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File initsync.png    
Issue Links:
Backports
Depends
Problem/Incident
Related
related to SERVER-53473 Move compression out of processMessag... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4, v4.2
Sprint: Service arch 2020-12-28
Participants:
Linked BF Score: 0

 Description   

Initial sync for identical highly compressible data sets on 4.0, 4.2, 4.4:

Total bytes transferred ("InOctets") is much higher for 4.4 and 4.2 than 4.0, and snappy decompressor metrics show that compression is not being used.

Impact on performance is not certain as I'm not sure whether this test is limited by network bandwidth, but if it is impacting performance, then

  • Initial sync time for 4.2 is longer than 4.0 because of this
  • Initial sync time for 4.4 is a little less than than 4.0 in spite of this


 Comments   
Comment by Ian Whalen (Inactive) [ 04/Jan/21 ]

Author:

{'username': u'evrg-bot-webhook', 'name': u'Amirsaman Memaripour', 'email': u'amirsaman.memaripour@mongodb.com'}

Message:SERVER-52919 Wire compression not enabled for initial sync
Branch:master
https://github.com/mongodb/mongo/commit/762eb95cd160aab9bec295b1e46d3c9a8ab505e2

Comment by Ian Whalen (Inactive) [ 04/Jan/21 ]

Author:

{'username': u'evrg-bot-webhook', 'name': u'Amirsaman Memaripour', 'email': u'amirsaman.memaripour@mongodb.com'}

Message:SERVER-52919 Wire compression not enabled for initial sync

(cherry picked from commit 762eb95cd160aab9bec295b1e46d3c9a8ab505e2)
Branch:v4.4
https://github.com/mongodb/mongo/commit/fee7b1b5b7849f0a137de86c8134e95134390509

Comment by Ian Whalen (Inactive) [ 04/Jan/21 ]

Author:

{'username': u'evrg-bot-webhook', 'name': u'Amirsaman Memaripour', 'email': u'amirsaman.memaripour@mongodb.com'}

Message:SERVER-52919 Wire compression not enabled for initial sync

(cherry picked from commit 762eb95cd160aab9bec295b1e46d3c9a8ab505e2)
Branch:v4.2
https://github.com/mongodb/mongo/commit/37e81074de8577873b515f7e7ba9ef1fbc40a93a

Comment by Suganthi Mani [ 19/Nov/20 ]

I think I know the reason why we are not compressing the initial sync data on 4.2+. It seems like a side-effect of initial sync using exhaust cursor.
Currently, we compress the response data only if the request is compressed data (see here). For exhaust cursor, the sync source creates a dummy exhaust request which isn't compressed and the dummy request is just a copy of the original decompressed getMore request. That’s the reason, the sync source isn’t compressing the initial sync response data.

Since this a bug in exhaust cursor, even oplog fetcher (PM-1232 which landed in 4.4) is also affected.

Generated at Thu Feb 08 05:29:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.