[SERVER-22974] writing to stdout via C++ stream sometimes fails on OS X 10.10 Created: 04/Mar/16  Updated: 19/Nov/16  Resolved: 17/Mar/16

Status: Closed
Project: Core Server
Component/s: Internal Code, Logging
Affects Version/s: None
Fix Version/s: 3.2.5, 3.3.4

Type: Bug Priority: Major - P3
Reporter: Eric Milkie Assignee: Andrew Morrow (Inactive)
Resolution: Done Votes: 1
Labels: code-only
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Sprint: Platforms 12 (04/01/16)
Participants:

 Description   

When the system log appender writes to the stdout stream, sometimes the write fails and the fail bit gets set on the stream. I can reproduce this just by starting mongod – the first write to stdout that typically fails is the one from the HostnameCanonicalization thread saying that it is starting up.
The problem occurs even with binaries built on 10.8 and run on 10.10. The errno set after the stream goes bad seems to be either "0" or "2".



 Comments   
Comment by Githook User [ 04/Apr/16 ]

Author:

{u'username': u'acmorrow', u'name': u'Andrew Morrow', u'email': u'acm@mongodb.com'}

Message: SERVER-22974 Ensure single threaded initialization of std::cout

(cherry picked from commit 08f686baf91c9bf201d5a98ab7a1d93c1697a690)
Branch: v3.2
https://github.com/mongodb/mongo/commit/059ce80468b460c9b79d93a5d289f317675c0cba

Comment by Andrew Morrow (Inactive) [ 17/Mar/16 ]

Marking backport requested for v3.2 since the same unsafe ios_base::Init::Init call exists in the Console class there. I didn't see it on v3.0 or v2.6.

Comment by Githook User [ 17/Mar/16 ]

Author:

{u'username': u'acmorrow', u'name': u'Andrew Morrow', u'email': u'acm@mongodb.com'}

Message: SERVER-22974 Ensure single threaded initialization of std::cout
Branch: master
https://github.com/mongodb/mongo/commit/08f686baf91c9bf201d5a98ab7a1d93c1697a690

Comment by Brian Samek [ 09/Mar/16 ]

acm mpobrien I linked TOOLS-1100 to BUILD-893 for the Go/OpenSSL question.

Comment by Andrew Morrow (Inactive) [ 08/Mar/16 ]

Some interesting reading:

It appears that the inclusion of an ios_base::Init object in Console::Console is intended exactly as comment #4 in the GCC bug suggests. Interestingly, it appears that the GCC maintainers did ultimately add some form of synchronization, but libc++ appears to have not.

Comment by Andy Schwerin [ 07/Mar/16 ]

Is this only for the "console appender"? It would not be a ton of work to write a new implementation of the Console type for OS X that used write(2) or fwrite.

Comment by Eric Milkie [ 07/Mar/16 ]

One thing we didn't totally prove is that this problem doesn't exist on 10.8. I am only going on the evidence that the auditing tests have not failed on our 10.8 builders (this is how we first detected this problem on the new 10.10 instances.)

Comment by Eric Milkie [ 07/Mar/16 ]

Possible next steps:
1. Hack the log system to write directly to stdout using Unix IO (fwrite) or System IO (write), in order to eliminate the C++ streams library, and also to possibly obtain a more reliable error code.
2. Reduce the size and complexity of the reproducer by shrinking mongod, or attempt to write a very small reproducer from scratch.

Generated at Thu Feb 08 04:01:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.