[SERVER-13242] Remove tab characters from ProgressMeter log messages Created: 17/Mar/14  Updated: 05/Apr/17  Resolved: 23/Feb/17

Status: Closed
Project: Core Server
Component/s: Logging
Affects Version/s: 2.4.9
Fix Version/s: 3.0.0

Type: Bug Priority: Trivial - P5
Reporter: David Barshow Assignee: Benety Goh
Resolution: Done Votes: 1
Labels: logging, pull-request
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

launch a mongod instance with --syslog and build a index.

Participants:

 Description   

In the syslog you will see formatting problems with #011 instead of spaces.

Mar 17 23:45:22 mongo-1-d mongod.27017[15304]: Mon Mar 17 23:45:22.582 [rsSync] #011#011Index: (1/3) External Sort Progress: 60000000/95818859#01162%



 Comments   
Comment by Benety Goh [ 23/Feb/17 ]

Title reworded to reflect work done in pull request. Original ticket description was referring to the tabs in the ProgressMeter output. Since those log messages have been fixed, this ticket can be resolved.

Future work to remove embedded and literal tabs from messages can be ticketed separately.

Comment by Benety Goh [ 23/Feb/17 ]

I think the original intent of this ticket was to remove the tabs from the ProgressMeter messages. This was done in the merged pull request.

Comment by Benety Goh [ 23/Feb/17 ]

Here are the rest of the strings containing tabs as of commit ab0951. Not sure if all of these should be changes:

> find src/mongo/ -name gotools -prune -o -type f -name "*.[ch]*" -exec grep '\\t' {} /dev/null \;
src/mongo//bson/bsonobj.cpp:        builder << i << '\t' << (0xff & ((unsigned)*p));
src/mongo//bson/bsonobj.cpp:            builder << '\t' << *p;
src/mongo//bson/json.cpp:#define CONTROL "\a\b\f\n\r\t\v"
src/mongo//bson/json.cpp:                    result->push_back('\t');
src/mongo//bson/json.h:     *   | \t
src/mongo//bson/json.h:     *   | \t
src/mongo//crypto/tom/tomcrypt_macros.h:        "bswapl %0     \n\t"           \
src/mongo//crypto/tom/tomcrypt_macros.h:        "movl   %0,(%1)\n\t"           \
src/mongo//crypto/tom/tomcrypt_macros.h:        "bswapl %0     \n\t" ::"r"(x), \
src/mongo//crypto/tom/tomcrypt_macros.h:        "movl (%1),%0\n\t" \
src/mongo//crypto/tom/tomcrypt_macros.h:        "bswapl %0\n\t"    \
src/mongo//crypto/tom/tomcrypt_macros.h:        "bswapq %0     \n\t"           \
src/mongo//crypto/tom/tomcrypt_macros.h:        "movq   %0,(%1)\n\t"           \
src/mongo//crypto/tom/tomcrypt_macros.h:        "bswapq %0     \n\t" ::"r"(x), \
src/mongo//crypto/tom/tomcrypt_macros.h:        "movq (%1),%0\n\t" \
src/mongo//crypto/tom/tomcrypt_macros.h:        "bswapq %0\n\t"    \
src/mongo//db/catalog/database.cpp:    LOG(1) << "\t dropIndexes done";
src/mongo//db/catalog/index_catalog.cpp:        LOG(1) << "\t dropAllIndexes dropping: " << desc->toString();
src/mongo//db/catalog/index_create.cpp:            log() << "\t building index using bulk method; build may temporarily use up to "
src/mongo//db/catalog/index_create.cpp:        LOG(1) << "\t bulk commit starting for index: "
src/mongo//db/cloner.cpp:    LOG(2) << "\t\tcloning collection " << from_collection << " to " << to_collection << " on "
src/mongo//db/cloner.cpp:    LOG(2) << "\t\t copyIndexes " << from_collection << " to " << to_collection << " on "
src/mongo//db/cloner.cpp:        LOG(2) << "\t cloner got " << collection;
src/mongo//db/cloner.cpp:                LOG(2) << "\t\t not cloning because system collection";
src/mongo//db/cloner.cpp:            LOG(2) << "\t\t not cloning because has $ ";
src/mongo//db/cloner.cpp:            LOG(2) << "\t\t ignoring collection " << ns;
src/mongo//db/cloner.cpp:            LOG(2) << "\t\t not ignoring collection " << ns;
src/mongo//db/cloner.cpp:            LOG(1) << "\t\t cloning " << from_name << " -> " << to_name;
src/mongo//db/concurrency/lock_manager.cpp:            sb << '\t' << "LockRequest " << iter->locker->getId() << " @ " << iter->locker << ": "
src/mongo//db/concurrency/lock_manager.cpp:            sb << '\t' << "LockRequest " << iter->locker->getId() << " @ " << iter->locker << ": "
src/mongo//db/db.cpp:        log() << "\t To fix this, you need to create a unique index on _id."
src/mongo//db/db.cpp:                log() << "\t To fix this, you need to rebuild this index."
src/mongo//db/fts/tokenizer.cpp:        case '\t':
src/mongo//db/index/index_access_method.cpp:    LOG(timer.seconds() > 10 ? 0 : 1) << "\t done building bottom layer, going to commit";
src/mongo//db/query/index_bounds_builder_test.cpp:        "\\Af \t\vo\n\ro  \\ \\# #comment", "mx", testIndex, &tightness);
src/mongo//db/query/index_bounds_builder_test.cpp:        "\\Af \t\vo\n\ro  \\ \\# #comment", "msx", testIndex, &tightness);
src/mongo//db/query/plan_enumerator.cpp:        ss << "\tfirst indices: [";
src/mongo//db/query/plan_enumerator.cpp:        ss << "\tpred: " << pred->expr->toString();
src/mongo//db/query/plan_enumerator.cpp:        ss << "\tindexToAssign: " << pred->indexToAssign;
src/mongo//db/query/plan_enumerator.cpp:            ss << "\n\tchoice " << i << ":\n";
src/mongo//db/query/plan_enumerator.cpp:            ss << "\t\tsubnodes: ";
src/mongo//db/query/plan_enumerator.cpp:                ss << "\t\tidx[" << oie.index << "]\n";
src/mongo//db/query/plan_enumerator.cpp:                    ss << "\t\t\tpos " << oie.positions[k] << " pred " << oie.preds[k]->toString();
src/mongo//db/storage/key_string_test.cpp:                        log() << "\to1: " << o1;
src/mongo//db/storage/key_string_test.cpp:                            log() << "\t\t o2: " << o2;
src/mongo//db/storage/key_string_test.cpp:    log() << "\t\t\tskipping permutation testing on non-optimized build";
src/mongo//db/storage/mmap_v1/dur.cpp:                ss << "mismatch ofs:" << hex << i << "\tfilemap:" << setw(2) << (unsigned)w[i]
src/mongo//db/storage/mmap_v1/dur.cpp:                   << "\tprivmap:" << setw(2) << (unsigned)p[i];
src/mongo//db/storage/mmap_v1/dur.cpp:                    ss << '\t' << p[i];
src/mongo//db/storage/mmap_v1/dur.cpp:    return "cmts\t jrnMB\t wrDFMB\t cIWLk\t early\t prpLgB\t wrToJ\t wrToDF\t rmpPrVw";
src/mongo//db/storage/mmap_v1/dur.cpp:    ss << setprecision(2) << _commits << '\t' << _journaledBytes / 1000000.0 << '\t'
src/mongo//db/storage/mmap_v1/dur.cpp:       << _writeToDataFilesBytes / 1000000.0 << '\t' << _commitsInWriteLock << '\t' << 0 << '\t'
src/mongo//db/storage/mmap_v1/dur.cpp:       << (unsigned)(_prepLogBufferMicros / 1000) << '\t'
src/mongo//db/storage/mmap_v1/dur.cpp:       << (unsigned)(_writeToJournalMicros / 1000) << '\t'
src/mongo//db/storage/mmap_v1/dur.cpp:       << (unsigned)(_writeToDataFilesMicros / 1000) << '\t'
src/mongo//db/storage/mmap_v1/dur.cpp:       << (unsigned)(_remapPrivateViewMicros / 1000) << '\t' << (unsigned)(_commitsMicros / 1000)
src/mongo//db/storage/mmap_v1/dur.cpp:       << '\t' << (unsigned)(_commitsInWriteLockMicros / 1000) << '\t';
src/mongo//db/storage/mmap_v1/record_store_v1_capped.cpp:        DDD("\t" << i);
src/mongo//db/storage/mmap_v1/record_store_v1_capped.cpp:    DDD("\t drecs.size(): " << drecs.size());
src/mongo//db/storage/mmap_v1/record_store_v1_capped.cpp:            DDD("\t compact adddelrec");
src/mongo//db/storage/mmap_v1/record_store_v1_capped.cpp:                DDD("\t compact adddelrec2");
src/mongo//db/storage/mmap_v1/record_store_v1_capped.cpp:        DDD("\t compact adddelrec3");
src/mongo//db/storage/paths.cpp:                log() << "\tWARNING: This file system is not supported. For further information"
src/mongo//db/storage/paths.cpp:                log() << "\t\t\thttp://dochub.mongodb.org/core/unsupported-filesystems"
src/mongo//db/storage/paths.cpp:                log() << "\t\tPlease notify MongoDB, Inc. if an unlisted filesystem generated "
src/mongo//db/storage/wiredtiger/wiredtiger_index.cpp:        TRACE_INDEX << "\t" << kv->key << ' ' << kv->loc;
src/mongo//db/storage/wiredtiger/wiredtiger_index.cpp:            TRACE_CURSOR << "\t not found";
src/mongo//db/storage/wiredtiger/wiredtiger_index.cpp:        TRACE_CURSOR << "\t cmp: " << cmp;
src/mongo//dbtests/jsontests.cpp:        b.append("a", "\" \\ / \b \f \n \r \t");
src/mongo//dbtests/jsontests.cpp:        ASSERT_EQUALS("{ \"a\" : \"\\\" \\\\ / \\b \\f \\n \\r \\t\" }",
src/mongo//dbtests/jsontests.cpp:        b.append("\t", "b");
src/mongo//dbtests/jsontests.cpp:        ASSERT_EQUALS("{ \"\\t\" : \"b\" }", b.done().jsonString(Strict));
src/mongo//dbtests/jsontests.cpp:        b.append("a", "\" \\ / \b \f \n \r \t \v");
src/mongo//dbtests/jsontests.cpp:        return "{ \"a\" : \"\\\" \\\\ \\/ \\b \\f \\n \\r \\t \\v\" }";
src/mongo//dbtests/jsontests.cpp:        b.appendRegex("a", "\t", "i");
src/mongo//dbtests/jsontests.cpp:        return "{ \"a\" : { \"$regex\" : \"\\t\", \"$options\" : \"i\" } }";
src/mongo//dbtests/jsontests.cpp:        return "{ t\test : 1 }";
src/mongo//dbtests/jstests.cpp:        b.append("a", "\" \\ / \b \f \n \r \t \v");
src/mongo//dbtests/jstests.cpp:        return "{ \"a\" : \"\\\" \\\\ \\/ \\b \\f \\n \\r \\t \\v\" }";
src/mongo//dbtests/jstests.cpp:        cout << s << ":" << e.binDataType() << "\t" << len << endl;
src/mongo//dbtests/jstests.cpp:        cout << "\t";
src/mongo//executor/network_interface_mock.cpp:            log() << "\t\t " << item.getDiagnosticString();
src/mongo//s/chunk_manager.cpp:        sb << "\t" << i->second->toString() << '\n';
src/mongo//shell/dbshell.cpp:                   "For more comprehensive documentation, see\n\thttp://docs.mongodb.org/\n"
src/mongo//shell/dbshell.cpp:                   "Questions? Try the support group\n\thttp://groups.google.com/group/mongodb-user"
src/mongo//shell/linenoise.cpp:        if ('\n' == c || '\t' == c) {  // note newline or tab
src/mongo//unittest/unittest.cpp:            ss << "\t" << *i << '\n';
src/mongo//unittest/unittest.cpp:    LOG(1) << "\t about to setupTests" << std::endl;
src/mongo//unittest/unittest.cpp:    LOG(1) << "\t done setupTests" << std::endl;
src/mongo//unittest/unittest.cpp:            LOG(1) << "\t skipping test: " << tc->getName() << " because doesn't match filter"
src/mongo//unittest/unittest.cpp:        log() << "\t going to run test: " << tc->getName() << std::endl;
src/mongo//unittest/unittest.cpp:        err << tc->getName() << "\t";
src/mongo//unittest/unittest.cpp:    log() << "\t DONE running tests" << std::endl;
src/mongo//unittest/unittest.cpp:            log() << "\t " << s << " Failed";
src/mongo//util/debugger.cpp:        msg, sizeof(msg), "\n\n\t**** Launching gdbserver (use lsof to find port) ****\n\n");
src/mongo//util/options_parser/options_parser_test.cpp:                     "\t this = false \n#that = true\n #another = whocares"
src/mongo//util/processinfo_linux.cpp:            if (strncmp(fstr, "processor ", 10) == 0 || strncmp(fstr, "processor\t:", 11) == 0)
src/mongo//util/processinfo_linux.cpp:            if (strncmp(fstr, "cpu MHz\t\t:", 10) == 0)
src/mongo//util/processinfo_linux.cpp:            if (strncmp(fstr, "flags\t\t:", 8) == 0)
src/mongo//util/scopeguard.h:/// \typedef typedef const ScopeGuardImplBase& ScopeGuard
src/mongo//util/stacktrace_windows.cpp:        start = filename.find("\\src\\third_party\\");
src/mongo//util/stringutils.cpp:            case '\t':
src/mongo//util/stringutils.cpp:                ret << "\\t";
src/mongo//util/stringutils.h: * A method to escape whitespace and control characters in strings. For example, the string "\t"
src/mongo//util/stringutils.h: * goes to "\\t". If `escape_slash` is true, then "/" goes to "\\/".
src/mongo//util/text.cpp:    } else if (arg.find_first_of(" \t\"") == std::string::npos) {

Comment by Benety Goh [ 23/Feb/17 ]

spoke too soon. reopening.

Comment by Eric Milkie [ 23/Feb/17 ]

Just do this: ```git grep \\\\t```

Comment by Eric Milkie [ 23/Feb/17 ]

Are you sure your regexp worked? I find lots more:

src/mongo/db/storage/paths.cpp:                log() << "\tWARNING: This file system is not supported. For further information"
src/mongo/db/storage/paths.cpp:                log() << "\t\t\thttp://dochub.mongodb.org/core/unsupported-filesystems"
src/mongo/db/storage/paths.cpp:                log() << "\t\tPlease notify MongoDB, Inc. if an unlisted filesystem generated "

Comment by Benety Goh [ 23/Feb/17 ]

I think we can resolve this. The only tabs remaining in our codebase (excluding vendor code) are:

> find src/mongo/ -name gotools -prune -o -type f -name "*.[ch]*" -exec grep $'\t' {} /dev/null \;
src/mongo//base/parse_number_test.cpp:    ASSERT_EQUALS(ErrorCodes::FailedToParse, parseNumberFromString<double>("1e6	", &d));
src/mongo//base/parse_number_test.cpp:    ASSERT_EQUALS(ErrorCodes::FailedToParse, parseNumberFromString<double>("	1e6", &d));
src/mongo//util/md5main.cpp:    "    md5main --test		# run the self-test (A.5 of RFC 1321)\n     "
src/mongo//util/md5main.cpp:    "    md5main --t-values		# print the T values for the library\n   "
src/mongo//util/md5main.cpp:    "    md5main --version		# print the version of the package\n     ";

Comment by Githook User [ 01/Oct/14 ]

Author:

{u'username': u'justincase', u'name': u'Justin Case', u'email': u'hello@justincase.cx'}

Message: SERVER-13242 replace tabs in Progress Meter log messages with spaces

Closes #799

Signed-off-by: Benety Goh <benety@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/6150677b5cc6d933fd9e26d2efadafb3686033fa

Comment by Eric Milkie [ 29/Jul/14 ]

Yes, replacing the tabs with some spaces in progress_meter.cpp is acceptable.

Comment by Justin Case [ 29/Jul/14 ]

rsyslog escapes control characters by default. You can turn it off in /etc/rsyslog.conf. Is it acceptable though if I replace the tabs with spaces instead? (in progress_meter.cpp)

Comment by Eric Milkie [ 18/Mar/14 ]

Those are actually tab characters. We should probably not embed literal tabs in log messages.

Generated at Thu Feb 08 03:31:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.