<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:45:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-37356] Unable to start mongo server due to corrupted admin database</title>
                <link>https://jira.mongodb.org/browse/SERVER-37356</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;We have an instance of mongo which does not start up anymore due to corrupted data, the root cause seems to be unclean shutdown. I tried using the --repair option but it did not help. Here is the error I see in the log files:&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;cite&gt;Fatal assertion 28548 NoSuchKey Unable to find metadata for table:admin/collection-184-1679902527159703400&lt;/cite&gt;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I did find a related issue &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-28364&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-28364&lt;/a&gt;. The final comment seem to indicate there is nothing that could be done. But I just wanted to see if anything has changed since then. Is there anything that could done to recover this database (admin). Our other databases contain sensitive data, but since this is just the admin database, I was wondering if I upload this securely so that somebody from mongo can take a look (don&apos;t want to just attach it here)?&lt;/p&gt;

&lt;p&gt;Please let me know your thoughts!&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="610976">SERVER-37356</key>
            <summary>Unable to start mongo server due to corrupted admin database</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="backlog-server-triage">Backlog - Triage Team</assignee>
                                    <reporter username="kanant">Karthik Anant</reporter>
                        <labels>
                            <label>nrp</label>
                            <label>wtc</label>
                    </labels>
                <created>Thu, 27 Sep 2018 18:23:56 +0000</created>
                <updated>Tue, 6 Dec 2022 03:17:10 +0000</updated>
                            <resolved>Thu, 25 Oct 2018 19:47:30 +0000</resolved>
                                    <version>3.2.1</version>
                                                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="2022152" author="nick.brewer" created="Wed, 3 Oct 2018 13:59:42 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kanant&quot; class=&quot;user-hover&quot; rel=&quot;kanant&quot;&gt;kanant&lt;/a&gt; Happy to help. If you could confirm for us the underlying platform of this machine (virtual machine, container, native hardware, etc) fit would be useful for us in tracking this issue.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
-Nick&lt;/p&gt;</comment>
                            <comment id="2021364" author="kanant" created="Tue, 2 Oct 2018 21:43:03 +0000"  >&lt;p&gt;Hello Nick,&lt;/p&gt;

&lt;p&gt;Unfortunately I cannot since it contains sensitive data. We will try to see if there is a newer backup (we do have an old one). Thanks for all your help with this issue!&lt;/p&gt;</comment>
                            <comment id="2021246" author="nick.brewer" created="Tue, 2 Oct 2018 20:36:20 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kanant&quot; class=&quot;user-hover&quot; rel=&quot;kanant&quot;&gt;kanant&lt;/a&gt; I suspect the error may be related to the difference in versions. If you&apos;d like, I can try performing a manual repair - I&apos;ll need the &lt;tt&gt;WiredTiger.wt&lt;/tt&gt; and &lt;tt&gt;WiredTiger.turtle&lt;/tt&gt; files, from a copy of the &lt;tt&gt;dbpath&lt;/tt&gt; &lt;b&gt;before&lt;/b&gt; the repair was performed.&lt;/p&gt;

&lt;p&gt;Thanks, &lt;br/&gt;
-Nick&lt;/p&gt;</comment>
                            <comment id="2020667" author="kanant" created="Tue, 2 Oct 2018 15:25:38 +0000"  >&lt;p&gt;Sorry, my mistake - I did try to start the server even though there were errors during repair. It did not start though. I have attached the log file - mongod.log. Please let me know if you have any questions.&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/197632/197632_mongod.log&quot; title=&quot;mongod.log attached to SERVER-37356&quot;&gt;mongod.log&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;</comment>
                            <comment id="2020643" author="nick.brewer" created="Tue, 2 Oct 2018 15:16:02 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kanant&quot; class=&quot;user-hover&quot; rel=&quot;kanant&quot;&gt;kanant&lt;/a&gt; Sorry if my instructions were unclear - please boot your 3.2.1 &lt;tt&gt;mongod&lt;/tt&gt; installation using the &lt;tt&gt;dbpath&lt;/tt&gt; that you repaired with the 4.0 build. If you encounter any errors, please include them here.&lt;/p&gt;

&lt;p&gt;Thanks, &lt;br/&gt;
-Nick&lt;/p&gt;</comment>
                            <comment id="2020291" author="kanant" created="Tue, 2 Oct 2018 12:59:01 +0000"  >&lt;p&gt;Hello Nick,&lt;/p&gt;

&lt;p&gt;Sorry for the late response. Lost track of this. Is that not what I am supposed to use? I used it since your initial reply indicated 4.0 nightly. If you could point me to the right one I can try with that.&lt;/p&gt;

&lt;p&gt;I have already attached the log file - Please check the mongod-cleanedup.txt attachment. This line caught my attention, got me wondering if a repair is going to be possible since our version of mongo is 3.2.1&lt;/p&gt;

&lt;p&gt;&lt;font color=&quot;#172b4d&quot;&gt;2018-09-27T16:07:38.691-0500 F CONTROL&#160;&#160;&lt;/font&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt;&lt;font color=&quot;#172b4d&quot;&gt;&#160;** IMPORTANT: UPGRADE PROBLEM: The data files need to be fully upgraded to version 3.6 before attempting an upgrade to 4.0; see&#160;&lt;/font&gt;&lt;a href=&quot;http://dochub.mongodb.org/core/4.0-upgrade-fcv&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://dochub.mongodb.org/core/4.0-upgrade-fcv&lt;/a&gt;&lt;font color=&quot;#172b4d&quot;&gt;&#160;for more details.&lt;/font&gt;&lt;/p&gt;

&lt;p&gt;Thank you!&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="2017545" author="nick.brewer" created="Fri, 28 Sep 2018 15:00:22 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kanant&quot; class=&quot;user-hover&quot; rel=&quot;kanant&quot;&gt;kanant&lt;/a&gt; The attached file appears to be from the 4.0 nightly - do you have the startup logs from the 3.2 &lt;tt&gt;mongod&lt;/tt&gt; when it was booted with the repaired &lt;tt&gt;dbpath&lt;/tt&gt;?&lt;/p&gt;

&lt;p&gt;-Nick&lt;/p&gt;</comment>
                            <comment id="2017543" author="kanant" created="Fri, 28 Sep 2018 14:58:19 +0000"  >&lt;p&gt;Hello Nick,&lt;/p&gt;

&lt;p&gt;Yes, that&apos;s what I eventually did, but did not help. The file I attached in my previous comment, that has the details. Please let me know if you have any thoughts on whether anything could be done. Thanks!&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="2017505" author="nick.brewer" created="Fri, 28 Sep 2018 14:43:13 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kanant&quot; class=&quot;user-hover&quot; rel=&quot;kanant&quot;&gt;kanant&lt;/a&gt; Sorry for the confusion - I meant that you should start our 3.2 installation using the same &lt;tt&gt;dbpath&lt;/tt&gt; on which the 4.0 nightly &lt;tt&gt;--repair&lt;/tt&gt; was run. &lt;/p&gt;

&lt;p&gt;-Nick&lt;/p&gt;</comment>
                            <comment id="2016980" author="kanant" created="Thu, 27 Sep 2018 21:32:15 +0000"  >&lt;p&gt;Hello Nick,&lt;/p&gt;

&lt;p&gt;Thank you for the quick response. I backed up the files in my db path and tried repairing with your instructions (using the 4.0 nightly build -&#160;&lt;a href=&quot;http://downloads.mongodb.org/win32/mongodb-win32-x86_64-2008plus-ssl-v4.0-latest.zip).&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://downloads.mongodb.org/win32/mongodb-win32-x86_64-2008plus-ssl-v4.0-latest.zip).&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;No luck. I just see errors in the log file -&#160;I have attached the log file (removed rows containing our product&apos;s collection names etc).&#160;One thing I wanted to reiterate is our mongo version is 3.2.1 (your last point refers to 3.4).&lt;/p&gt;

&lt;p&gt;Especially this line kind of supports my point - am I out of luck since the data files were created using 3.2.1? Or am I missing something?&lt;/p&gt;

&lt;p&gt;&lt;cite&gt;2018-09-27T16:07:38.691-0500 F CONTROL&#160; &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; ** IMPORTANT: UPGRADE PROBLEM: The data files need to be fully upgraded to version 3.6 before attempting an upgrade to 4.0; see &lt;a href=&quot;http://dochub.mongodb.org/core/4.0-upgrade-fcv&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://dochub.mongodb.org/core/4.0-upgrade-fcv&lt;/a&gt; for more details.&lt;/cite&gt;&lt;/p&gt;

&lt;p&gt;Please let me know if you have any questions.&lt;/p&gt;

&lt;p&gt;Thanks!&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/197356/197356_mongod-cleanedup.txt&quot; title=&quot;mongod-cleanedup.txt attached to SERVER-37356&quot;&gt;mongod-cleanedup.txt&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="2016855" author="nick.brewer" created="Thu, 27 Sep 2018 20:01:42 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kanant&quot; class=&quot;user-hover&quot; rel=&quot;kanant&quot;&gt;kanant&lt;/a&gt; There&apos;s a repair process you can try that makes use of some recently introduced updates to the &lt;tt&gt;mongod --repair&lt;/tt&gt; option:&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Important&lt;/b&gt;: Before you start this process, take a backup of your &lt;tt&gt;dbpath&lt;/tt&gt; via a manual copy of the directory, or a filesystem snapshot. &lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Download and untar the latest &lt;em&gt;4.0 Nightly&lt;/em&gt; release from the &lt;a href=&quot;https://www.mongodb.com/download-center#development&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;MongoDB Download Center&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;From the unpacked directory, run &lt;tt&gt;mongod --repair&lt;/tt&gt;, and point it to your &lt;tt&gt;dbpath&lt;/tt&gt;:
  &lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  bin/mongod --repair --dbpath /path/to/corrupted/dbpath&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;&lt;/li&gt;
	&lt;li&gt;Once the repair completes, stop the 4.0 nightly &lt;tt&gt;mongod&lt;/tt&gt;, and then start your regular 3.4  installation as you normally would, using the same &lt;tt&gt;dbpath&lt;/tt&gt; that you previously ran the &lt;tt&gt;--repair&lt;/tt&gt; on.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Please let us know how it works for you - we&apos;d be happy to help if you run into any issues. &lt;/p&gt;

&lt;p&gt;-Nick&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="197356" name="mongod-cleanedup.txt" size="8861" author="kanant" created="Thu, 27 Sep 2018 21:31:08 +0000"/>
                            <attachment id="197632" name="mongod.log" size="3933" author="kanant" created="Tue, 2 Oct 2018 15:25:14 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25137"><![CDATA[Server Triage]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 27 Sep 2018 20:01:42 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 19 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 19 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-triage</customfieldvalue>
            <customfieldvalue>kanant</customfieldvalue>
            <customfieldvalue>nick.brewer</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu91gv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htzfxb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu8nq7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>