<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:41:41 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-57379] &quot;WiredTiger error when building index.</title>
                <link>https://jira.mongodb.org/browse/SERVER-57379</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;h4&gt;&lt;a name=&quot;Wehaveacollection%22A%22.&quot;&gt;&lt;/a&gt;We have a collection &quot;A&quot;.&lt;/h4&gt;
&lt;p&gt; Collection &quot;A&quot; stats.&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;count : 3852675627&lt;/li&gt;
	&lt;li&gt;size : 3.31TB&lt;/li&gt;
	&lt;li&gt;storageSize : 481.60 GB&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;When we were building a compound index, &lt;br/&gt;
we got &quot;WiredTiger error ... read checksum error for 16384B block at offset 39527702528: calculated block checksum doesn&apos;t match expected checksum Raw:&quot;.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;We experienced similar error, at that time we changed the physical server to avoid error.(We already issued that situation here.)&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;So when we faced this error again, we changed physical server.&lt;br/&gt;
But in this time we could not make it.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;What we did.&lt;br/&gt;
1. we prepared new physical server. &lt;br/&gt;
 =&amp;gt; we got &quot;WiredTiger error&quot; again.&lt;br/&gt;
2. we changed ssd.&lt;br/&gt;
 =&amp;gt; we got &quot;WiredTiger error&quot; again.&lt;br/&gt;
3. We considered disk problem where collection &quot;A&quot; was written, we renamed collection &quot;A&quot; to &quot;A_old&quot; and made collection &quot;A&quot; again newly.&lt;br/&gt;
 =&amp;gt; we got &quot;WiredTiger error&quot; again.&lt;br/&gt;
4. We tried build index with some options ( -sparse, -name, - split index key... )&lt;br/&gt;
 =&amp;gt; we got &quot;WiredTiger error&quot; again.&lt;/p&gt;


&lt;p&gt;Environment.&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Server&lt;/li&gt;
&lt;/ul&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;OS : Ubuntu 18.04.5 LTS&lt;/li&gt;
	&lt;li&gt;Memory : 64Gb&lt;/li&gt;
	&lt;li&gt;Disk : SSD 7.3T (4T + 4T with lvm)&lt;/li&gt;
&lt;/ul&gt;
&lt;ul&gt;
	&lt;li&gt;Mongodb :&lt;br/&gt;
db version v4.2.8&lt;br/&gt;
git version: 43d25964249164d76d5e04dd6cf38f6111e21f5f&lt;br/&gt;
OpenSSL version: OpenSSL 1.1.1 11 Sep 2018&lt;br/&gt;
allocator: tcmalloc&lt;br/&gt;
modules: none&lt;br/&gt;
build environment:&lt;br/&gt;
 distmod: ubuntu1804&lt;br/&gt;
 distarch: x86_64&lt;br/&gt;
 target_arch: x86_64&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Do you have any advice?&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="1767140">SERVER-57379</key>
            <summary>&quot;WiredTiger error when building index.</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="edwin.zhou@mongodb.com">Edwin Zhou</assignee>
                                    <reporter username="jjongei0@gmail.com">jongsun moon</reporter>
                        <labels>
                            <label>build</label>
                            <label>index</label>
                    </labels>
                <created>Thu, 3 Jun 2021 09:18:59 +0000</created>
                <updated>Thu, 17 Jun 2021 01:13:52 +0000</updated>
                            <resolved>Tue, 15 Jun 2021 21:03:06 +0000</resolved>
                                    <version>4.2.8</version>
                                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="3883843" author="jjongei0@gmail.com" created="Thu, 17 Jun 2021 01:13:52 +0000"  >&lt;p&gt;Hi&#160;Edwin Zhou,&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Thank you very much for&#160; your help.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Regards,&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="3879703" author="JIRAUSER1257066" created="Tue, 15 Jun 2021 21:03:06 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jjongei0%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;jjongei0@gmail.com&quot;&gt;jjongei0@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Unfortunately we aren&apos;t able to repair this level of corruption.&lt;/p&gt;

&lt;p&gt;To avoid a problem like this in the future, it is our strong recommendation to:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Use a &lt;a href=&quot;https://docs.mongodb.com/manual/replication/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;replica set&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;Use the &lt;a href=&quot;https://docs.mongodb.com/manual/release-notes/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;most recent version&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;Keep up to date &lt;a href=&quot;https://docs.mongodb.com/manual/core/backups/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;backups of your databases&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;Have a process in place to restore backups.&lt;/li&gt;
	&lt;li&gt;Follow all &lt;a href=&quot;https://docs.mongodb.com/manual/administration/production-notes/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;production notes&lt;/a&gt;, especially those for &lt;a href=&quot;https://docs.mongodb.com/manual/administration/production-notes/#disk-and-storage-systems&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;underlying storage systems&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;Schedule and perform regular checks of the integrity of your filesystems and disks.&lt;/li&gt;
	&lt;li&gt;Never manipulate the underlying database files in any way.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Best,&lt;br/&gt;
Edwin&lt;/p&gt;</comment>
                            <comment id="3877852" author="jjongei0@gmail.com" created="Tue, 15 Jun 2021 08:27:52 +0000"  >&lt;p&gt;Hi&#160;Edwin Zhou,&lt;/p&gt;

&lt;p&gt;We have only one mongod server which is set as replica set 0(One mongos + one config server + one mongod server).&lt;/p&gt;

&lt;p&gt;So we can not do a clean resync.&lt;/p&gt;

&lt;p&gt;Is there any other thing to do for it?&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="3873721" author="JIRAUSER1257066" created="Fri, 11 Jun 2021 17:41:57 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jjongei0%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;jjongei0@gmail.com&quot;&gt;jjongei0@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Given that your topology appears to be a replica set, we may be able to resolve this corruption by doing a clean &lt;a href=&quot;https://docs.mongodb.org/manual/tutorial/resync-replica-set-member/#automatically-sync-a-member&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;resync&lt;/a&gt; from an unaffected node. &lt;/p&gt;

&lt;p&gt;We look forward to hearing if you&apos;re able to find success with this method.&lt;/p&gt;

&lt;p&gt;Best,&lt;br/&gt;
Edwin&lt;/p&gt;</comment>
                            <comment id="3872296" author="jjongei0@gmail.com" created="Fri, 11 Jun 2021 02:03:04 +0000"  >&lt;p&gt;Hi&#160;Edwin Zhou,&lt;/p&gt;

&lt;p&gt;Unfortunately, --repair operation was failed.&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;The logs leading up to the first occurrence of any issue
	&lt;ul&gt;
		&lt;li&gt;We got config server error, so mongod had connection error&#160; :&#160;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/319824/319824_shard_cn_1.log.2021-05-25T05-38-59&quot; title=&quot;shard_cn_1.log.2021-05-25T05-38-59 attached to SERVER-57379&quot;&gt;shard_cn_1.log.2021-05-25T05-38-59&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/li&gt;
		&lt;li&gt;Building index error :&#160;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/319825/319825_shard_cn_1.log.2021-05-26T05-23-19&quot; title=&quot;shard_cn_1.log.2021-05-26T05-23-19 attached to SERVER-57379&quot;&gt;shard_cn_1.log.2021-05-26T05-23-19&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;The logs of the repair operation.
	&lt;ul&gt;
		&lt;li&gt;I couldn&apos;t upload Log file because of it&apos;s size (260MB). Instead, I uploaded file which is included the end of original log file. :&#160;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/319823/319823_repair_20210611.log&quot; title=&quot;repair_20210611.log attached to SERVER-57379&quot;&gt;repair_20210611.log&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;The logs of any attempt to start mongod after the repair operation completed.
	&lt;ul&gt;
		&lt;li&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/319830/319830_shard_cn_1.log.2021-06-11T01-21-58&quot; title=&quot;shard_cn_1.log.2021-06-11T01-21-58 attached to SERVER-57379&quot;&gt;shard_cn_1.log.2021-06-11T01-21-58&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&lt;sup&gt;Regards,&lt;/sup&gt;&lt;/p&gt;</comment>
                            <comment id="3869211" author="JIRAUSER1257066" created="Wed, 9 Jun 2021 18:02:56 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jjongei0%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;jjongei0@gmail.com&quot;&gt;jjongei0@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for providing the output of the db.collection.validate command. It indeed confirms that there is collection level corruption. We look forward to hearing back to you with the results of running &lt;tt&gt;--repair&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;In the event that a &lt;tt&gt;--repair&lt;/tt&gt; operation is unsuccessful, then please also provide:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;The logs leading up to the first occurrence of any issue&lt;/li&gt;
	&lt;li&gt;The logs of the repair operation.&lt;/li&gt;
	&lt;li&gt;The logs of any attempt to start mongod after the repair operation completed.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Best,&lt;br/&gt;
Edwin&lt;/p&gt;</comment>
                            <comment id="3865445" author="jjongei0@gmail.com" created="Tue, 8 Jun 2021 05:14:06 +0000"  >&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;other collections validation result&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/319196/319196_shard_2021_06_08_1.log&quot; title=&quot;shard_2021_06_08_1.log attached to SERVER-57379&quot;&gt;shard_2021_06_08_1.log&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&lt;sup&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/319197/319197_shard_2021_06_08_2.log&quot; title=&quot;shard_2021_06_08_2.log attached to SERVER-57379&quot;&gt;shard_2021_06_08_2.log&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/sup&gt;&lt;/p&gt;

&lt;p&gt;&lt;sup&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/319196/319196_shard_2021_06_08_1.log&quot; title=&quot;shard_2021_06_08_1.log attached to SERVER-57379&quot;&gt;shard_2021_06_08_1.log&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/sup&gt;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;At the same database, I &lt;b&gt;made same collection and built index&lt;/b&gt; (previous collection was renamed) and I received below.&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;2021-06-08T09:01:04.852+0000&#160;I&#160;&#160;-&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&lt;span class=&quot;error&quot;&gt;&amp;#91;conn205&amp;#93;&lt;/span&gt;&#160;&#160;&#160;Index&#160;Build:&#160;scanning&#160;collection:&#160;65707300/1069318918&#160;6%&lt;br/&gt;
 2021-06-08T09:01:07.002+0000&#160;I&#160;&#160;-&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&lt;span class=&quot;error&quot;&gt;&amp;#91;conn205&amp;#93;&lt;/span&gt;&#160;&#160;&#160;Index&#160;Build:&#160;scanning&#160;collection:&#160;66091200/1069318918&#160;6%&lt;br/&gt;
 &#160;&lt;br/&gt;
 2021-06-08T09:01:08.099+0000&#160;E&#160;&#160;-&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;&lt;span class=&quot;error&quot;&gt;&amp;#91;conn205&amp;#93;&lt;/span&gt;&#160;&lt;b&gt;Assertion:&#160;Location10320:&#160;BSONElement:&#160;bad&#160;type&#160;67&#160;src/mongo/bson/bsonelement.cpp&#160;696&lt;/b&gt;&lt;br/&gt;
 &#160;&lt;br/&gt;
 2021-06-08T09:01:08.147+0000&#160;I&#160;&#160;STORAGE&#160;&#160;&lt;span class=&quot;error&quot;&gt;&amp;#91;conn205&amp;#93;&lt;/span&gt;&#160;Deferring&#160;table&#160;drop&#160;for&#160;index&#160;&apos;KWID_RANK_OF_RISINGKW_RANK_OF_NEWKW_RANK_OF_SPECIFICKW_RANK_OF_ASNKW_RANK_OF_ASNMNAKW_RANK_OF_TRIALKW_RANK_OF_SUITKW_COPAT_CNT_TOT_KW_CATEGORY_LOCALE&apos;&#160;on&#160;collection&#160;&apos;IPOFFICE_210201_CN.TBL_FEA_KW_KW.$KWID_RANK_OF_RISINGKW_RANK_OF_NEWKW_RANK_OF_SPECIFICKW_RANK_OF_ASNKW_RANK_OF_ASNMNAKW_RANK_OF_TRIALKW_RANK_OF_SUITKW_COPAT_CNT_TOT_KW_CATEGORY_LOCALE&#160;(5c2dbce3-4700-4507-8fe3-7b39d05ae7ee)&apos;.&#160;Ident:&#160;&apos;IPOFFICE_210201_CN/index-2--1086201584841391775&apos;,&#160;commit&#160;timestamp:&#160;&apos;Timestamp(1623142859,&#160;1)&apos;&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&#160;And then I ran &lt;b&gt;collection validation&lt;/b&gt; and found &lt;b&gt;corrupted records.&lt;/b&gt;&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;2021-06-08T10:22:41.741+0000 I STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn225&amp;#93;&lt;/span&gt; WiredTiger progress WT_SESSION.verify 8775600&lt;br/&gt;
 2021-06-08T10:22:41.746+0000 I STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn225&amp;#93;&lt;/span&gt; WiredTiger progress WT_SESSION.verify 8775700&lt;/p&gt;

&lt;p&gt;2021-06-08T10:24:14.892+0000 I STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn225&amp;#93;&lt;/span&gt; &lt;b&gt;document at location: RecordId(66267652) is corrupted&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;2021-06-08T10:28:34.848+0000 I ACCESS &lt;span class=&quot;error&quot;&gt;&amp;#91;conn238&amp;#93;&lt;/span&gt; Successfully authenticated as principal __system on local from client 192.168.0.165:47190&lt;br/&gt;
 2021-06-08T10:28:51.515+0000 I STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn225&amp;#93;&lt;/span&gt; &lt;b&gt;document at location: RecordId(241390443) is corrupted&lt;/b&gt;&lt;br/&gt;
 2021-06-08T10:28:57.786+0000 I STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn225&amp;#93;&lt;/span&gt; &lt;b&gt;document at location: RecordId(245063106) is corrupted&lt;/b&gt;&lt;br/&gt;
 2021-06-08T10:32:30.522+0000 I STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn225&amp;#93;&lt;/span&gt; &lt;b&gt;document at location: RecordId(361048613) is corrupted&lt;/b&gt;&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Now I am doing &quot;mongod --repair&quot;.&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="3864440" author="jjongei0@gmail.com" created="Tue, 8 Jun 2021 02:07:07 +0000"  >&lt;p&gt;Hi Edwin Zhou.&lt;/p&gt;

&lt;p&gt;Thank you for your reply.&lt;/p&gt;

&lt;p&gt;I ran db.collection.validate.&lt;/p&gt;

&lt;p&gt;Then I got message below. (I attached log file at the bottom.)&lt;/p&gt;

&lt;p&gt;And file size (&quot;collection-20-7619064565122544008.wt&quot;&#160;which faced error in log) is&#160;482GB.&lt;/p&gt;

&lt;p&gt;Is there anything to do for it?&lt;/p&gt;

&lt;p&gt;Regards,&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;2021-06-08T01:26:17.551+0000 I STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn18&amp;#93;&lt;/span&gt; WiredTiger progress WT_SESSION.verify 2562300&lt;br/&gt;
 2021-06-08T01:26:17.563+0000 I STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn18&amp;#93;&lt;/span&gt; WiredTiger progress WT_SESSION.verify 2562400&lt;br/&gt;
 2021-06-08T01:26:17.570+0000 E STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn18&amp;#93;&lt;/span&gt; WiredTiger error (0) &lt;span class=&quot;error&quot;&gt;&amp;#91;1623115577:570311&amp;#93;&lt;/span&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;15886:0x7f2d2b6a3700&amp;#93;&lt;/span&gt;, &lt;a href=&quot;file://ipoffice_210201_cn/collection-20-7619064565122544008.wt&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;file:IPOFFICE_210201_CN/collection-20-7619064565122544008.wt&lt;/a&gt;, WT_SESSION.verify: __wt_block_read_off, 274: IPOFFICE_210201_CN/collection-20-7619064565122544008.wt: read checksum error for 16384B block at offset 39527702528: calculated block checksum doesn&apos;t match expected checksum&lt;br/&gt;
 2021-06-08T01:26:17.570+0000 E STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn18&amp;#93;&lt;/span&gt; WiredTiger error (0) &lt;span class=&quot;error&quot;&gt;&amp;#91;1623115577:570399&amp;#93;&lt;/span&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;15886:0x7f2d2b6a3700&amp;#93;&lt;/span&gt;, &lt;a href=&quot;file://ipoffice_210201_cn/collection-20-7619064565122544008.wt&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;file:IPOFFICE_210201_CN/collection-20-7619064565122544008.wt&lt;/a&gt;, WT_SESSION.verify: __wt_bm_corrupt_dump, 135: {39527702528, 16384, 0x578c57a3}: (chunk 1 of 16): 0 ....&lt;/p&gt;

&lt;p&gt;...&lt;/p&gt;

&lt;p&gt;2021-06-08T01:26:18.422+0000 E STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn18&amp;#93;&lt;/span&gt; WiredTiger error (0) &lt;span class=&quot;error&quot;&gt;&amp;#91;1623115578:422514&amp;#93;&lt;/span&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;15886:0x7f2d2b6a3700&amp;#93;&lt;/span&gt;, &lt;a href=&quot;file://ipoffice_210201_cn/collection-20-7619064565122544008.wt&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;file:IPOFFICE_210201_CN/collection-20-7619064565122544008.wt&lt;/a&gt;, WT_SESSION.verify: __verify_ckptfrag_chk, 526: checkpoint ranges never verified: 375&lt;br/&gt;
 2021-06-08T01:26:18.762+0000 E STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn18&amp;#93;&lt;/span&gt; WiredTiger error (0) &lt;span class=&quot;error&quot;&gt;&amp;#91;1623115578:762395&amp;#93;&lt;/span&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;15886:0x7f2d2b6a3700&amp;#93;&lt;/span&gt;, &lt;a href=&quot;file://ipoffice_210201_cn/collection-20-7619064565122544008.wt&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;file:IPOFFICE_210201_CN/collection-20-7619064565122544008.wt&lt;/a&gt;, WT_SESSION.verify: __verify_filefrag_chk, 438: file ranges never verified: 375&lt;br/&gt;
 2021-06-08T01:26:18.762+0000 E STORAGE &lt;span class=&quot;error&quot;&gt;&amp;#91;conn18&amp;#93;&lt;/span&gt; verify() returned WT_ERROR: non-specific WiredTiger error. This indicates structural damage. Not examining individual documents.&lt;br/&gt;
 2021-06-08T01:26:18.769+0000 I COMMAND &lt;span class=&quot;error&quot;&gt;&amp;#91;ShardServerCatalogCacheLoader-2&amp;#93;&lt;/span&gt; command config.cache.collections command: find { find: &quot;cache.collections&quot;, filter:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { _id}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;, ntoreturn: 1, singleBatch: true, $db: &quot;config&quot; } planSummary: IDHACK keysExamined:1 docsExamined:1 cursorExhausted:1 numYields:0 nreturned:1 reslen:555 locks:{ ReplicationStateTransition: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { w}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, Global: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { r}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, Database: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { r}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, Collection: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { r}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, Mutex: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { r}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;} } storage:{ data:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { bytesRead}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;, timeWaitingMicros: { schemaLock: 164161664 } } protocol:op_msg 164164ms&lt;br/&gt;
 2021-06-08T01:26:18.771+0000 I WRITE &lt;span class=&quot;error&quot;&gt;&amp;#91;ShardServerCatalogCacheLoader-1&amp;#93;&lt;/span&gt; update config.cache.databases command: { q:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { _id}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;, u: { $set: { _id: &quot;config&quot;, primary: &quot;config&quot;, partitioned: true, version:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { uuid}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;} }, multi: false, upsert: true } planSummary: IDHACK keysExamined:1 docsExamined:1 nMatched:1 nModified:1 numYields:1 locks:{ ParallelBatchWriterMode: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { r}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, ReplicationStateTransition: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { w}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, Global: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { w}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, Database: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { w}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, Collection: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { w}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;}, Mutex: { acquireCount:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { r}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;} } flowControl:{ acquireCount: 2, timeAcquiringMicros: 1 } storage:{ data:&lt;/p&gt;

&lt;div class=&quot;error&quot;&gt;&lt;span class=&quot;error&quot;&gt;Unknown macro: { bytesRead}&lt;/span&gt; &lt;/div&gt;

&lt;p&gt;, timeWaitingMicros: { schemaLock: 164163963 } } 164166ms&lt;/p&gt;

&lt;p&gt;...&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;nobr&quot;&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/attachment/319188/319188_shard_2021_06_08.log&quot; title=&quot;shard_2021_06_08.log attached to SERVER-57379&quot;&gt;shard_2021_06_08.log&lt;sup&gt;&lt;img class=&quot;rendericon&quot; src=&quot;https://jira.mongodb.org/images/icons/link_attachment_7.gif&quot; height=&quot;7&quot; width=&quot;7&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/sup&gt;&lt;/a&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Regards,&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="3862911" author="JIRAUSER1257066" created="Mon, 7 Jun 2021 16:59:07 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jjongei0%40gmail.com&quot; class=&quot;user-hover&quot; rel=&quot;jjongei0@gmail.com&quot;&gt;jjongei0@gmail.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;It appears there may be some corruption on the collection level. I advise you to first make a complete copy of the database&apos;s $dbpath directory to safeguard so that you can work off of the current $dbpath.&lt;/p&gt;

&lt;p&gt;Then, can you run &lt;a href=&quot;https://docs.mongodb.com/v4.2/reference/method/db.collection.validate/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;db.collection.validate&lt;/a&gt; on the affected collection?&lt;/p&gt;

&lt;p&gt;Best,&lt;br/&gt;
Edwin&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="319820" name="after_repair_operation_first_start.log" size="2397" author="jjongei0@gmail.com" created="Fri, 11 Jun 2021 01:50:40 +0000"/>
                            <attachment id="318555" name="mongod-log.log" size="196922" author="jjongei0@gmail.com" created="Thu, 3 Jun 2021 07:17:59 +0000"/>
                            <attachment id="319821" name="mongod_build_index_error_2021_05_26.log" size="791575" author="jjongei0@gmail.com" created="Fri, 11 Jun 2021 01:50:41 +0000"/>
                            <attachment id="319822" name="mongod_config_server_not_work_2021_05_25.log" size="53828" author="jjongei0@gmail.com" created="Fri, 11 Jun 2021 01:50:41 +0000"/>
                            <attachment id="319823" name="repair_20210611.log" size="550697" author="jjongei0@gmail.com" created="Fri, 11 Jun 2021 01:50:41 +0000"/>
                            <attachment id="319188" name="shard_2021_06_08.log" size="2669877" author="jjongei0@gmail.com" created="Tue, 8 Jun 2021 02:05:26 +0000"/>
                            <attachment id="319196" name="shard_2021_06_08_1.log" size="1776595" author="jjongei0@gmail.com" created="Tue, 8 Jun 2021 05:13:08 +0000"/>
                            <attachment id="319198" name="shard_2021_06_08_2-1.log" size="5385983" author="jjongei0@gmail.com" created="Tue, 8 Jun 2021 05:13:15 +0000"/>
                            <attachment id="319197" name="shard_2021_06_08_2.log" size="5385983" author="jjongei0@gmail.com" created="Tue, 8 Jun 2021 05:13:09 +0000"/>
                            <attachment id="319824" name="shard_cn_1.log.2021-05-25T05-38-59" size="53828" author="jjongei0@gmail.com" created="Fri, 11 Jun 2021 01:50:41 +0000"/>
                            <attachment id="319825" name="shard_cn_1.log.2021-05-26T05-23-19" size="791575" author="jjongei0@gmail.com" created="Fri, 11 Jun 2021 01:50:42 +0000"/>
                            <attachment id="319830" name="shard_cn_1.log.2021-06-11T01-21-58" size="2262" author="jjongei0@gmail.com" created="Fri, 11 Jun 2021 01:56:00 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 7 Jun 2021 16:59:07 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 34 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>jjongei0@gmail.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 34 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>edwin.zhou@mongodb.com</customfieldvalue>
            <customfieldvalue>jjongei0@gmail.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzjgdz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hz42en:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[edwin.zhou@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzj2n3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>