<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:47:21 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-18339] Log only warning, error or fatal messages</title>
                <link>https://jira.mongodb.org/browse/SERVER-18339</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;We have lately started using syslog to hold the mongodb logs. However, it is a little overwhelming within the amount of informational logs we receive.&lt;/p&gt;

&lt;p&gt;Also, since enabling the --syslog flag the system has decreased in performance&lt;/p&gt;

&lt;p&gt;The below links describe, 0 is the default verbosity level and includes informational logs. How 1 - 5 only increases the verbosity level, is there a way to only log warning, error or fatal messages?&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://docs.mongodb.org/manual/reference/method/db.setLogLevel/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://docs.mongodb.org/manual/reference/method/db.setLogLevel/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;http://docs.mongodb.org/manual/reference/log-messages/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://docs.mongodb.org/manual/reference/log-messages/&lt;/a&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="202412">SERVER-18339</key>
            <summary>Log only warning, error or fatal messages</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="mira.carey@mongodb.com">Mira Carey</assignee>
                                    <reporter username="andy@ditno.com">Andy Walker</reporter>
                        <labels>
                            <label>log-verbosity</label>
                    </labels>
                <created>Wed, 6 May 2015 14:52:18 +0000</created>
                <updated>Mon, 8 Jan 2024 15:23:06 +0000</updated>
                            <resolved>Fri, 18 Mar 2016 20:10:11 +0000</resolved>
                                                                    <component>Admin</component>
                    <component>Logging</component>
                                        <votes>1</votes>
                                    <watches>18</watches>
                                                                                                                <comments>
                            <comment id="1976229" author="p_hollas@msn.com" created="Wed, 15 Aug 2018 12:20:27 +0000"  >&lt;p&gt;Hi, please could you consider looking at re-opening this. I have cases where sometimes we need to shut down secondary nodes for say 48-72 hours for a planned maintenance activity and the logs on the surviving nodes are spammed by network errors and there is no current way to suppress them. Even with log rotate and compression the logs can easily fill a file system and affect availability of surviving data nodes. Yes we could reconfigure a replica set and remove the nodes to stop this and indeed if&#160;an outage was going to be prolonged ie generally longer than the oplog window size, we would take this route and opt for a node initial sync. &#160;However, it would be very useful to be able to radically supress logging dynamically without having to do a replica set reconfig and of course support teams would have to live with the consequences of no logging. Have to add I&#160;work with many other database technologies and I laud to stake holders all the time about how good MongoDBs reliability is.&#160;&#160;&lt;/p&gt;

&lt;p&gt;Many thanks&lt;/p&gt;

&lt;p&gt;Paul&lt;/p&gt;</comment>
                            <comment id="1408575" author="judah.schvimer" created="Fri, 14 Oct 2016 14:35:18 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lpolonen&quot; class=&quot;user-hover&quot; rel=&quot;lpolonen&quot;&gt;lpolonen&lt;/a&gt;, &lt;/p&gt;

&lt;p&gt;On further review and testing, it looks like providing --quiet will not suppress replication log messages unless you are using master/slave replication: &lt;a href=&quot;https://docs.mongodb.com/v3.2/core/master-slave/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.mongodb.com/v3.2/core/master-slave/&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="1408340" author="lpolonen" created="Fri, 14 Oct 2016 08:08:30 +0000"  >&lt;p&gt;Any chance this could be reopened? With for example PHP applications it&apos;s next to impossible to have persistent long lived connections as it&apos;s very common to limit the amount of requests served by the workers for good reasons so with a lot of clients there are always going to be a lot of new connections. So there are cases where re-use is not possible to the same extent. Quiet mode just can&apos;t go to production due to lack of replication logging.&lt;/p&gt;</comment>
                            <comment id="1282266" author="lpolonen" created="Thu, 2 Jun 2016 13:22:19 +0000"  >&lt;p&gt;Thought about it. Thought the replication logging activities are important on especially resync when there are the highest amount of connection attempts. Those are the moments when the amount of logging is the biggest issue.&lt;/p&gt;</comment>
                            <comment id="1282198" author="jason.carey" created="Thu, 2 Jun 2016 11:58:32 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lpolonen&quot; class=&quot;user-hover&quot; rel=&quot;lpolonen&quot;&gt;lpolonen&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;A colleague reminded me about &lt;a href=&quot;https://docs.mongodb.com/manual/reference/parameters/#param.quiet&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;--quiet&lt;/a&gt;.  While it&apos;s generally not recommended for production systems (because connection information is very helpful for debugging), it does suppress connection logging.&lt;/p&gt;

&lt;p&gt;The full list of events --quiet suppresses:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;connection events;&lt;/li&gt;
	&lt;li&gt;the drop command, the dropIndexes command, the diagLogging command, the validate command, and the clean command; and&lt;/li&gt;
	&lt;li&gt;replication synchronization activities.&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="1282176" author="jason.carey" created="Thu, 2 Jun 2016 11:04:41 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lpolonen&quot; class=&quot;user-hover&quot; rel=&quot;lpolonen&quot;&gt;lpolonen&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;To clarify, you&apos;re generating 1k of logs per second entirely or mostly due to clients connecting and disconnecting?  We haven&apos;t considered connections and disconnections to be a frequent enough event to warrant hiding behind a higher verbosity setting because connections are generally expected to be long lived or infrequent.  If these log lines are occurring very often for you, it would indicate that you either have a very large number of clients (so large that even though connections are long lived, a substantial number are connecting and disconnecting at any one time) or that you&apos;re getting relatively little connection re-use (your client code may be connecting, performing a small number of operations and immediately disconnecting).  If the latter case is true, you may want to look into connection re-use in your application, as you may be experiencing more latency per operation than you really need to.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Jason&lt;/p&gt;</comment>
                            <comment id="1281882" author="lpolonen" created="Wed, 1 Jun 2016 23:13:31 +0000"  >&lt;p&gt;Would be much appreciated if the logging of connections could be reduced. Getting about 1k per second of:&lt;/p&gt;

&lt;p&gt;2016-06-01T23:09:38.703+0000 I NETWORK  &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from ...&lt;br/&gt;
2016-06-01T23:09:38.775+0000 I NETWORK  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn1199019493&amp;#93;&lt;/span&gt; end connection ..&lt;/p&gt;

&lt;p&gt;Sure there are many clients connecting, but those messages mostly fill disk and don&apos;t provide much useful information.&lt;/p&gt;</comment>
                            <comment id="1196013" author="hydrapolic" created="Tue, 8 Mar 2016 06:45:33 +0000"  >&lt;p&gt;Yes Jason, after setting slowOpThresholdMs, the logs are ok.&lt;/p&gt;</comment>
                            <comment id="1195632" author="jason.carey" created="Mon, 7 Mar 2016 21:21:05 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=hydrapolic&quot; class=&quot;user-hover&quot; rel=&quot;hydrapolic&quot;&gt;hydrapolic&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Are your logs still too large after tuning up the slowOpThresholdMs?  If not, I&apos;d like to close this out as works as designed.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Jason&lt;/p&gt;</comment>
                            <comment id="990567" author="hydrapolic" created="Tue, 28 Jul 2015 13:07:04 +0000"  >&lt;p&gt;Yes you are right, they all took more than 100ms. After adjusting that value the messages are gone.&lt;/p&gt;</comment>
                            <comment id="990516" author="dan@10gen.com" created="Tue, 28 Jul 2015 11:43:30 +0000"  >&lt;p&gt;I can&apos;t be sure unless you include full log lines, but I&apos;m assuming those operations are exceeding the operation time limit threshold. It is 100ms by default.  It can be adjusted by setting this configuration: &lt;a href=&quot;http://docs.mongodb.org/manual/reference/configuration-options/#operationProfiling.slowOpThresholdMs&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://docs.mongodb.org/manual/reference/configuration-options/#operationProfiling.slowOpThresholdMs&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="990361" author="hydrapolic" created="Tue, 28 Jul 2015 04:49:46 +0000"  >&lt;p&gt;The logging is set like:&lt;/p&gt;

&lt;p&gt;systemLog:&lt;br/&gt;
    destination: file&lt;br/&gt;
    path: &quot;/var/log/mongodb/mongodb.log&quot;&lt;br/&gt;
    quiet: true&lt;br/&gt;
    logAppend: true&lt;/p&gt;</comment>
                            <comment id="990359" author="hydrapolic" created="Tue, 28 Jul 2015 04:49:16 +0000"  >&lt;p&gt;We use mapReduce a lot so the lines look like:&lt;br/&gt;
2015-07-28T03:10:27.204+0000 I COMMAND  &lt;span class=&quot;error&quot;&gt;&amp;#91;conn1727&amp;#93;&lt;/span&gt; command db.$cmd command: mapReduce ...&lt;/p&gt;

&lt;p&gt;Or information about writes&lt;br/&gt;
2015-07-28T03:11:17.350+0000 I WRITE    &lt;span class=&quot;error&quot;&gt;&amp;#91;conn18080&amp;#93;&lt;/span&gt; insert ...&lt;br/&gt;
2015-07-28T03:13:08.019+0000 I WRITE    &lt;span class=&quot;error&quot;&gt;&amp;#91;conn57&amp;#93;&lt;/span&gt; update ...&lt;/p&gt;</comment>
                            <comment id="989632" author="dan@10gen.com" created="Mon, 27 Jul 2015 15:19:00 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=hydrapolic&quot; class=&quot;user-hover&quot; rel=&quot;hydrapolic&quot;&gt;hydrapolic&lt;/a&gt;, can you tell me if there is a particular (or set of) informational log message that is filling up your log files?&lt;/p&gt;

&lt;p&gt;The problem with disabling informational messages is that debugging problems with replication and sharding become very difficult without those log entries.  Under normal circumstances, the volume of messages should not be overwhelming, and thus it may be a symptom of an underlying problem.&lt;/p&gt;</comment>
                            <comment id="989407" author="hydrapolic" created="Mon, 27 Jul 2015 11:13:39 +0000"  >&lt;p&gt;Please reconsider this task. We are running a MongoDB 2.4 cluster that only has a few MB of logs during the day and a MongoDB 3.0 cluster with GBs of logs each day that are pretty useless. Please allow us to disable logging of informational log messages so we are not lost when looking for warnings/errors.&lt;/p&gt;</comment>
                            <comment id="920721" author="ramon.fernandez" created="Thu, 21 May 2015 18:25:50 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=andy%40ditno.com&quot; class=&quot;user-hover&quot; rel=&quot;andy@ditno.com&quot;&gt;andy@ditno.com&lt;/a&gt;, I forgot to mention that you may want to consider the user of &lt;a href=&quot;http://docs.mongodb.org/manual/reference/command/logRotate/#logrotate&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;logrotate&lt;/a&gt; to periodically rotate and compress logs &amp;#8211; hope this helps with the size of the logs you need to keep.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="910742" author="andy@ditno.com" created="Tue, 12 May 2015 00:02:09 +0000"  >&lt;p&gt;Hi Ramon&lt;/p&gt;

&lt;p&gt;we will be interested if the logging is reduced as we are curenttly not using syslog and dev/null&lt;/p&gt;

&lt;p&gt;I have raised another ticket regarding performance, as this is beginning to be a huge problem - &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18425&quot; title=&quot;High Memory usage on Primary and secondary&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18425&quot;&gt;&lt;del&gt;SERVER-18425&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Thanks&lt;br/&gt;
Andy&lt;/p&gt;</comment>
                            <comment id="907388" author="ramon.fernandez" created="Thu, 7 May 2015 10:31:23 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=andy%40ditno.com&quot; class=&quot;user-hover&quot; rel=&quot;andy@ditno.com&quot;&gt;andy@ditno.com&lt;/a&gt;, currently the lowest verbosity level is 0, and you&apos;re correct  that the amount of logs only goes up from that. Two things about your report:&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;I&apos;m re-purposing your ticket as an improvement suggestion to have a less verbose log. While it sounds like an attractive idea, we have to take into account the impact that lowering the amount of logs may have on diagnostics and troubleshooting.&lt;/li&gt;
	&lt;li&gt;If you&apos;re using a modern syslog you may want to investigate &lt;a href=&quot;https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/reference-filters.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;using filters&lt;/a&gt; for the logs generated by MongoDB. Assuming you&apos;re using 3.0, you can use the &lt;a href=&quot;http://docs.mongodb.org/manual/reference/log-messages/#severity-levels&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;different severity levels&lt;/a&gt; to decide what to filter in/out.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;Finally, can you please elaborate on the decrease in performance you mention? If there&apos;s a significant performance impact from using syslog we&apos;d like to understand what&apos;s happening.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;
</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="1027637">SERVER-44866</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="545197">SERVER-35035</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>18.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 7 May 2015 10:31:23 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 26 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 26 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andy@ditno.com</customfieldvalue>
            <customfieldvalue>dan@mongodb.com</customfieldvalue>
            <customfieldvalue>hydrapolic</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>lpolonen</customfieldvalue>
            <customfieldvalue>mira.carey@mongodb.com</customfieldvalue>
            <customfieldvalue>p_hollas@msn.com</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrl6pb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrdis7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsg24v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>