[SERVER-26573] Poor compression of diagnostic data during chunk migrations Created: 11/Oct/16 Updated: 19/Nov/16 Resolved: 17/Oct/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Diagnostics |
| Affects Version/s: | 3.4.0-rc0 |
| Fix Version/s: | 3.4.0-rc1 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Bruce Lucas (Inactive) | Assignee: | Mark Benvenuto |
| Resolution: | Done | Votes: | 0 |
| Labels: | DF | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Sprint: | Platforms 2016-10-31 | ||||||||
| Participants: | |||||||||
| Description |
|
During chunk migrations there are frequent changes in the serverStatus schema which results in poor compression and much reduced retention period. This is because the sharding.migrations sub-document is only present when a migration is active. |
| Comments |
| Comment by Githook User [ 17/Oct/16 ] | |||
|
Author: {u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}Message: | |||
| Comment by Bruce Lucas (Inactive) [ 12/Oct/16 ] | |||
|
After discussion with dianna.hohensee and mark.benvenuto we decided to exclude the "sharding" serverStatus section from FTDC data by specifying sharding:0 on the serverStatus command, as the section doesn't currently have anything useful for FTDC. | |||
| Comment by Dianna Hohensee (Inactive) [ 11/Oct/16 ] | |||
|
Actually, after looking at the original ticket that put migrations in serverStatus, Leaning towards making the sharding section not included by default, since the understanding over here is that it isn't used by FTDC. I'll ask cloud whether they depend on the sharding section being included. | |||
| Comment by Dianna Hohensee (Inactive) [ 11/Oct/16 ] | |||
|
Another option would be for the application using serverStatus to check for a migrations field and, if it doesn't exist, put in its own dummy data to make compression nicer. No idea if that's easy to do or not. There are several things that are possible to do on the server end, but I'm not sure which would be most ideal.
| |||
| Comment by Bruce Lucas (Inactive) [ 11/Oct/16 ] | |||
|
That's consistent with what I'm seeing. The issue for ftdc arises because the migration information comes and goes with some frequency (if a lot of balancing is happening), i.e. the serverStatus schema changes, and that requires starting a new ftdc chunk, so compression and therefore retention suffer quite a bit. TBD what to do about it. | |||
| Comment by Dianna Hohensee (Inactive) [ 11/Oct/16 ] | |||
|
So the migration information is added only if a migration is running. Migration information is included in the Sharding section of serverStatus' response. And the Sharding section is included by default. | |||
| Comment by Kaloian Manassiev [ 11/Oct/16 ] | |||
|
bruce.lucas, the sharding migrations information should not be reported by default in serverStatus. Did you find it to actually be reported all the time? cc dianna.hohensee. |