[SERVER-9526] Log messages regarding chunks not very informative when the shard key is of type BinData Created: 01/May/13  Updated: 11/Jul/16  Resolved: 23/May/14

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.2.4, 2.4.3
Fix Version/s: 2.6.4, 2.7.2

Type: Bug Priority: Minor - P4
Reporter: Johan Hedin Assignee: Greg Studer
Resolution: Done Votes: 0
Labels: logging, sharding
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Duplicate
is duplicated by SERVER-9236 Wrong binary serialization when Dupli... Closed
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Participants:

 Description   

When the data type of the shard key is BinData, log messages regarding chunk limits and split keys are not very informative since BinData keys are printed in abbreviated form like:

{ _id: BinData } --> { _id: BinData }

If, on the other hand, the type of the shard key is say a number, the log messages are useful:

{ _id: 103 } --> { _id: 206 }

This truncation of BinData in the log messages makes it difficult to track sharding problems when BinData is used as a shard key.



 Comments   
Comment by Githook User [ 16/Jul/14 ]

Author:

{u'username': u'johanhedin', u'name': u'Johan Hedin', u'email': u'johan.o.hedin@gmail.com'}

Message: SERVER-9526 a more elegant solution to printout of BinData in log messages

This commit changes the behaviour of BSONElement::toString() to print BinData in the same way as the other element types. It puts a cap on the lenght of the binary data in the same way as for string. Using full == true overrides the cap in the same way.

Signed-off-by: Greg Studer <greg@10gen.com>
(cherry picked from commit 54845ac08d80b9556cbcbea711ad0529feb09c0f)
Branch: v2.6
https://github.com/mongodb/mongo/commit/5bd773e65e130fc46d6969356a94400d5a37b1f6

Comment by Githook User [ 23/May/14 ]

Author:

{u'username': u'johanhedin', u'name': u'Johan Hedin', u'email': u'johan.o.hedin@gmail.com'}

Message: SERVER-9526 a more elegant solution to printout of BinData in log messages

This commit changes the behaviour of BSONElement::toString() to print BinData in the same way as the other element types. It puts a cap on the lenght of the binary data in the same way as for string. Using full == true overrides the cap in the same way.

Signed-off-by: Greg Studer <greg@10gen.com>
Branch: master
https://github.com/mongodb/mongo/commit/54845ac08d80b9556cbcbea711ad0529feb09c0f

Comment by Johan Hedin [ 03/May/13 ]

I'm running the second patch my self right now on 2.4.3. My fist concern with that patch was that other pieces of code relied on BSONObject::operator<< but the comment for that function mention it's usage to be for things like log printouts. And when you use operator<< today for strings that are longer than 160 characters they will get truncated so the usage of operator<< can't be in any critical parts of the code.

A quick look among the tools (like mongodump etc) show that they use there own conversion of BinData to JSON. And the shell does not use BSONObject::operator<< ether.

So, based on this quick scan of the code, the second patch looks like a good solution

Comment by Eliot Horowitz (Inactive) [ 03/May/13 ]

Not 100% sure what the right solution is, but agree it needs improvement.

Comment by Johan Hedin [ 01/May/13 ]

A more elegant fix: https://github.com/johanhedin/mongo/commit/a551787f799a38f9bfdbdbaec9996d71079bfac1

This commit fixes BSONElement::toString so that it prints out BinData "correctly". It contain a cap in the same way as for strings to prevent overflow the log in cases where the bindata contain a lot of bytes.

This fix should probably be moved to its own ticket for clarity, but I'll keep it here until I get some feedback. I can create a pull request for any of the solutions as well if this is something that you would like to consider.

Comment by Johan Hedin [ 01/May/13 ]

This is one way of fixing it: https://github.com/johanhedin/mongo/commit/f97b2ce741f71420b346ed7d04f7edb09b42fa51

The commit adds .toString(false, true) to a lot of places where chunk max/min:s are printed to the log. This triggers full printout of BinData in the logs.

I'll admit that this is a bit ugly and error prone since it is easy to forget to write .toString(false, true) everywhere chunk limits and shard keys should be printed to the log. I'm not sure that I have traced down all places ether, but it is a start.

Generated at Thu Feb 08 03:20:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.