[SERVER-26573] Poor compression of diagnostic data during chunk migrations Created: 11/Oct/16  Updated: 19/Nov/16  Resolved: 17/Oct/16

Status: Closed
Project: Core Server
Component/s: Diagnostics
Affects Version/s: 3.4.0-rc0
Fix Version/s: 3.4.0-rc1

Type: Bug Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Mark Benvenuto
Resolution: Done Votes: 0
Labels: DF
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-22671 Implement serverStatus section with a... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Platforms 2016-10-31
Participants:

 Description   

During chunk migrations there are frequent changes in the serverStatus schema which results in poor compression and much reduced retention period. This is because the sharding.migrations sub-document is only present when a migration is active.



 Comments   
Comment by Githook User [ 17/Oct/16 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-26573 Poor compression of diagnostic data during chunk migrations
Branch: master
https://github.com/mongodb/mongo/commit/030fa6ba25c329f8ff5f29b3bbb5a95c6556115d

Comment by Bruce Lucas (Inactive) [ 12/Oct/16 ]

After discussion with dianna.hohensee and mark.benvenuto we decided to exclude the "sharding" serverStatus section from FTDC data by specifying sharding:0 on the serverStatus command, as the section doesn't currently have anything useful for FTDC.

Comment by Dianna Hohensee (Inactive) [ 11/Oct/16 ]

Actually, after looking at the original ticket that put migrations in serverStatus, SERVER-22671, it doesn't seem like this is wanted for anything specific. Migrations were added to serverStatus partly because it was originally intended to be used by balancer recovery, for which it didn't end up being used, and partly for a user to diagnose whether a chunk migration is stuck somehow.

Leaning towards making the sharding section not included by default, since the understanding over here is that it isn't used by FTDC. I'll ask cloud whether they depend on the sharding section being included.

Comment by Dianna Hohensee (Inactive) [ 11/Oct/16 ]

Another option would be for the application using serverStatus to check for a migrations field and, if it doesn't exist, put in its own dummy data to make compression nicer. No idea if that's easy to do or not.

There are several things that are possible to do on the server end, but I'm not sure which would be most ideal.

  • don't include the sharding section by default (or is this always wanted in FTDC reporting?)
  • move the migration subsection of sharding to be its own section, entirely separate from sharding, and make is not included by default (this seems perhaps less than ideal because it's confusing to have part of sharding not under the sharding section?)
  • what kind of dummy data would be needed, specifically, for nice data compression? Could the field and value just be

    "migrations: {}"

    , or maybe something like

    "migrations: { source: "", destination: "", isDonorShard: "", chunk: "", collection: ""}"

    , or does it need the complete dummy data like

    "migrations: {source: "NoShard", destination: "NoShard", isDonorShard: false, chunk: {min: 0, max: 0}, collection: "NoDB.NoCollection"}"

Comment by Bruce Lucas (Inactive) [ 11/Oct/16 ]

That's consistent with what I'm seeing. The issue for ftdc arises because the migration information comes and goes with some frequency (if a lot of balancing is happening), i.e. the serverStatus schema changes, and that requires starting a new ftdc chunk, so compression and therefore retention suffer quite a bit. TBD what to do about it.

Comment by Dianna Hohensee (Inactive) [ 11/Oct/16 ]

So the migration information is added only if a migration is running. Migration information is included in the Sharding section of serverStatus' response. And the Sharding section is included by default.

Comment by Kaloian Manassiev [ 11/Oct/16 ]

bruce.lucas, the sharding migrations information should not be reported by default in serverStatus. Did you find it to actually be reported all the time? cc dianna.hohensee.

Generated at Thu Feb 08 04:12:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.