<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:36:17 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-77685] Server can return CollectionUUIDMismatch with actual collection null if collection exists</title>
                <link>https://jira.mongodb.org/browse/SERVER-77685</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In mongosync, I am seeing that in some cases, on v7.0+ and in sharded clusters, that Collection UUID errors are being returned like the following:&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[j1:cl1:s1:prim] | 2023-05-31T20:56:32.038+00:00 D1 ASSERT   23074   [conn96] &quot;User assertion&quot;,&quot;attr&quot;:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;{&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    error: &quot;CollectionUUIDMismatch{ db: \&quot;test\&quot;, collectionUUID: UUID(\&quot;86ac3d10-3d4b-4951-927c-156c87640af7\&quot;), expectedCollection: \&quot;use_cases\&quot;, actualCollection: null }: Collection UUID does not match that specified&quot;,&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    file: &quot;src/mongo/db/catalog/collection_uuid_mismatch.cpp&quot;,&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    line: 70&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;In these errors the &lt;b&gt;actualCollection&lt;/b&gt; field is null, which should only happen if the collection hasn&apos;t been created yet, or has been dropped. However, the collection with this UUID should exist on the destination cluster and there is no drop that should be happening for the collection with that UUID.&lt;/p&gt;

&lt;p&gt;The following rough order of events happen in the BF I am seeing:&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;the previous collection use_cases is dropped&lt;/li&gt;
	&lt;li&gt;a temporary collection mongosync.tmp.&amp;lt;uuid&amp;gt; is created&lt;/li&gt;
	&lt;li&gt;the temporary collection is renamed to use_cases&lt;/li&gt;
	&lt;li&gt;a document tries to be inserted by mongosync into the new collection use_cases and fails with the CollectionUUIDMismatch error&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;Note that 2, 3 and 4 don&apos;t necessarily happen one after the other (they can happen at different relative times because of the parallelism in mongosync). Because of step 3, we expect some CollectionUUIDMismatch errors, but in this scenario, I either expect it not to throw one, or I expect it to have the actual collection field be mongosync.tmp.&amp;lt;uuid&amp;gt;, and not null. The end result of this is that mongosync ends up not applying some CRUD events (typically inserts) and has data inconsistency between the source and destination clusters.&lt;/p&gt;

&lt;p&gt;The logs are here: &lt;a href=&quot;https://parsley.mongodb.com/resmoke/98b363f8d81a6e1ffbe259c78c5b19f9/all?bookmarks=0,1089885,1090430,1090446,1090603,1090721,1090805,1091107,1193110&amp;amp;highlights=CollectionUUIDMismatch.%2Ause_cases,use_cases&amp;amp;shareLine=1090721&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://parsley.mongodb.com/resmoke/98b363f8d81a6e1ffbe259c78c5b19f9/all?bookmarks=0,1089885,1090430,1090446,1090603,1090721,1090805,1091107,1193110&amp;amp;highlights=CollectionUUIDMismatch.%2Ause_cases,use_cases&amp;amp;shareLine=1090721&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This is blocking mongosync v7.0 support, and is causing some BFs on our waterfall (because we run some integration tests on latest).&lt;/p&gt;</description>
                <environment></environment>
        <key id="2355952">SERVER-77685</key>
            <summary>Server can return CollectionUUIDMismatch with actual collection null if collection exists</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="jordi.olivares-provencio@mongodb.com">Jordi Olivares Provencio</assignee>
                                    <reporter username="rohan.sharan@mongodb.com">Rohan Sharan</reporter>
                        <labels>
                    </labels>
                <created>Thu, 1 Jun 2023 14:14:54 +0000</created>
                <updated>Sun, 29 Oct 2023 21:20:37 +0000</updated>
                            <resolved>Thu, 6 Jul 2023 10:07:17 +0000</resolved>
                                                    <fixVersion>7.1.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>13</watches>
                                                                                                                <comments>
                            <comment id="5562210" author="JIRAUSER1264979" created="Wed, 12 Jul 2023 22:02:58 +0000"  >&lt;p&gt;It seems that the problem was actually just a new test added in v7.0 that is exposing a pre-existing limitation of mongosync. Sorry for the confusion.&lt;/p&gt;</comment>
                            <comment id="5559068" author="JIRAUSER1264979" created="Tue, 11 Jul 2023 22:34:03 +0000"  >&lt;p&gt;I need to keep looking into this tomorrow. I have not yet figured out what is going on, but let me message you when I have more of an idea.&lt;/p&gt;</comment>
                            <comment id="5557102" author="JIRAUSER1257318" created="Tue, 11 Jul 2023 13:17:01 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=rohan.sharan%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;rohan.sharan@mongodb.com&quot;&gt;rohan.sharan@mongodb.com&lt;/a&gt;&lt;br/&gt;
I&apos;ve been staring at the &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77685?focusedCommentId=5555049&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-5555049&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;logs of the latest repro you posted&lt;/a&gt;. The CollectionUUIDMismatch with &quot;actualCollection: null&quot;:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;j5:cl1:s0:prim] | 2023-07-10T15:25:31.281+00:00 D1 ASSERT   23074   [conn80] &quot;User assertion&quot;,&quot;attr&quot;:{&quot;error&quot;:&quot;CollectionUUIDMismatch{ db: \&quot;test63_fsmdb0\&quot;, collectionUUID: UUID(\&quot;d262857c-09cb-4704-a502-1388ef411142\&quot;), expectedCollection: \&quot;unsharded_coll_0\&quot;, actualCollection: null }: Collection UUID does not match that specified&quot;,&quot;file&quot;:&quot;src/mongo/db/catalog/collection_uuid_mismatch.cpp&quot;,&quot;line&quot;:70}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;appears to be expected, given the sequence of DDL operations performed by mongosync.&lt;/p&gt;


&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;// Create collection &apos;test63_fsmdb0.mongosync.tmp.5911a15b-4a13-45da-9402-fc96746bf334&apos;. It gets uuid &apos;d262857c-09cb-4704-a502-1388ef411142&apos;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[j5:cl1:s0:prim] | 2023-07-10T15:25:05.341+00:00 I  STORAGE  20320   [conn467] &quot;createCollection&quot;,&quot;attr&quot;:{&quot;namespace&quot;:&quot;test63_fsmdb0.mongosync.tmp.5911a15b-4a13-45da-9402-fc96746bf334&quot;,&quot;uuidDisposition&quot;:&quot;generated&quot;,&quot;uuid&quot;:{&quot;uuid&quot;:{&quot;$uuid&quot;:&quot;d262857c-09cb-4704-a502-1388ef411142&quot;}},&quot;options&quot;:{}}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;// Rename &apos;test63_fsmdb0.mongosync.tmp.5911a15b-4a13-45da-9402-fc96746bf334&apos; to &apos;test63_fsmdb0.unsharded_coll_0&apos;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[j5:cl1:s0] | 2023-07-10T15:25:23.219+00:00 I  COMMAND  51803   [conn999] &quot;Slow query&quot;,&quot;attr&quot;:{&quot;type&quot;:&quot;command&quot;,&quot;ns&quot;:&quot;test63_fsmdb0.mongosync.tmp.5911a15b-4a13-45da-9402-fc96746bf334&quot;,&quot;appName&quot;:&quot;mongosync, v: 1.4.1-43-g6a8dcfd3, i: shard-rs0, cl: dst, cm: 6a8dcfd3, g: go1.19.10, cp: gc, p: 00000000-0000-0000-0000-00000000&quot;,&quot;command&quot;:{&quot;renameCollection&quot;:&quot;test63_fsmdb0.mongosync.tmp.5911a15b-4a13-45da-9402-fc96746bf334&quot;,&quot;to&quot;:&quot;test63_fsmdb0.unsharded_coll_0&quot;,&quot;dropTarget&quot;:false,&quot;collectionUUID&quot;:{&quot;$uuid&quot;:&quot;d262857c-09cb-4704-a502-1388ef411142&quot;},&quot;writeConcern&quot;:{&quot;w&quot;:&quot;majority&quot;,&quot;j&quot;:true,&quot;wtimeout&quot;:120000},&quot;lsid&quot;:{&quot;id&quot;:{&quot;$uuid&quot;:&quot;8738e0f2-1cfe-4397-82c2-38d288b5271a&quot;}},&quot;$clusterTime&quot;:{&quot;clusterTime&quot;:{&quot;$timestamp&quot;:{&quot;t&quot;:1689002722,&quot;i&quot;:167}},&quot;signature&quot;:{&quot;hash&quot;:{&quot;$binary&quot;:{&quot;base64&quot;:&quot;AAAAAAAAAAAAAAAAAAAAAAAAAAA=&quot;,&quot;subType&quot;:&quot;0&quot;}},&quot;keyId&quot;:0}},&quot;maxTimeMS&quot;:300000,&quot;$db&quot;:&quot;admin&quot;},&quot;numYields&quot;:0,&quot;reslen&quot;:163,&quot;readConcern&quot;:{&quot;level&quot;:&quot;local&quot;,&quot;provenance&quot;:&quot;implicitDefault&quot;},&quot;cpuNanos&quot;:299726,&quot;remote&quot;:&quot;127.0.0.1:48968&quot;,&quot;protocol&quot;:&quot;op_msg&quot;,&quot;durationMillis&quot;:228}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;// Rename &apos;test63_fsmdb0.unsharded_coll_0&apos; to &apos;test63_fsmdb0.mongosync.tmp.d262857c-09cb-4704-a502-1388ef411142&apos;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[j5:cl1:s1] | 2023-07-10T15:25:25.494+00:00 I  COMMAND  51803   [conn904] &quot;Slow query&quot;,&quot;attr&quot;:{&quot;type&quot;:&quot;command&quot;,&quot;ns&quot;:&quot;test63_fsmdb0.unsharded_coll_0&quot;,&quot;appName&quot;:&quot;mongosync, v: 1.4.1-43-g6a8dcfd3, i: shard-rs0, cl: dst, cm: 6a8dcfd3, g: go1.19.10, cp: gc, p: 00000000-0000-0000-0000-00000000&quot;,&quot;command&quot;:{&quot;renameCollection&quot;:&quot;test63_fsmdb0.unsharded_coll_0&quot;,&quot;to&quot;:&quot;test63_fsmdb0.mongosync.tmp.d262857c-09cb-4704-a502-1388ef411142&quot;,&quot;dropTarget&quot;:false,&quot;collectionUUID&quot;:{&quot;$uuid&quot;:&quot;d262857c-09cb-4704-a502-1388ef411142&quot;},&quot;writeConcern&quot;:{&quot;w&quot;:&quot;majority&quot;,&quot;j&quot;:true,&quot;wtimeout&quot;:120000},&quot;lsid&quot;:{&quot;id&quot;:{&quot;$uuid&quot;:&quot;8738e0f2-1cfe-4397-82c2-38d288b5271a&quot;}},&quot;$clusterTime&quot;:{&quot;clusterTime&quot;:{&quot;$timestamp&quot;:{&quot;t&quot;:1689002725,&quot;i&quot;:106}},&quot;signature&quot;:{&quot;hash&quot;:{&quot;$binary&quot;:{&quot;base64&quot;:&quot;AAAAAAAAAAAAAAAAAAAAAAAAAAA=&quot;,&quot;subType&quot;:&quot;0&quot;}},&quot;keyId&quot;:0}},&quot;maxTimeMS&quot;:300000,&quot;$db&quot;:&quot;admin&quot;},&quot;numYields&quot;:0,&quot;reslen&quot;:163,&quot;readConcern&quot;:{&quot;level&quot;:&quot;local&quot;,&quot;provenance&quot;:&quot;implicitDefault&quot;},&quot;cpuNanos&quot;:295560,&quot;remote&quot;:&quot;127.0.0.1:39128&quot;,&quot;protocol&quot;:&quot;op_msg&quot;,&quot;durationMillis&quot;:140}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;// Drop &apos;test63_fsmdb0.mongosync.tmp.d262857c-09cb-4704-a502-1388ef411142&apos; (which had uuid d262857c-09cb-4704-a502-1388ef411142)&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[j5:cl1:s0] | 2023-07-10T15:25:26.573+00:00 I  COMMAND  51803   [conn999] &quot;Slow query&quot;,&quot;attr&quot;:{&quot;type&quot;:&quot;command&quot;,&quot;ns&quot;:&quot;test63_fsmdb0.mongosync.tmp.d262857c-09cb-4704-a502-1388ef411142&quot;,&quot;appName&quot;:&quot;mongosync, v: 1.4.1-43-g6a8dcfd3, i: shard-rs0, cl: dst, cm: 6a8dcfd3, g: go1.19.10, cp: gc, p: 00000000-0000-0000-0000-00000000&quot;,&quot;command&quot;:{&quot;drop&quot;:&quot;mongosync.tmp.d262857c-09cb-4704-a502-1388ef411142&quot;,&quot;collectionUUID&quot;:{&quot;$uuid&quot;:&quot;d262857c-09cb-4704-a502-1388ef411142&quot;},&quot;writeConcern&quot;:{&quot;w&quot;:&quot;majority&quot;,&quot;j&quot;:true,&quot;wtimeout&quot;:120000},&quot;lsid&quot;:{&quot;id&quot;:{&quot;$uuid&quot;:&quot;8738e0f2-1cfe-4397-82c2-38d288b5271a&quot;}},&quot;$clusterTime&quot;:{&quot;clusterTime&quot;:{&quot;$timestamp&quot;:{&quot;t&quot;:1689002726,&quot;i&quot;:140}},&quot;signature&quot;:{&quot;hash&quot;:{&quot;$binary&quot;:{&quot;base64&quot;:&quot;AAAAAAAAAAAAAAAAAAAAAAAAAAA=&quot;,&quot;subType&quot;:&quot;0&quot;}},&quot;keyId&quot;:0}},&quot;maxTimeMS&quot;:300000,&quot;$db&quot;:&quot;test63_fsmdb0&quot;},&quot;numYields&quot;:0,&quot;reslen&quot;:163,&quot;readConcern&quot;:{&quot;level&quot;:&quot;local&quot;,&quot;provenance&quot;:&quot;implicitDefault&quot;},&quot;cpuNanos&quot;:295124,&quot;remote&quot;:&quot;127.0.0.1:48968&quot;,&quot;protocol&quot;:&quot;op_msg&quot;,&quot;remoteOpWaitMillis&quot;:62,&quot;durationMillis&quot;:62}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;// From this point, there&apos;s no collection with uuid &apos;d262857c-09cb-4704-a502-1388ef411142&apos; anymore. So it is expected that commands with expectedUUID=d262857c-09cb-4704-a502-1388ef411142 will fail with CollectionUUIDMismatch(...actualCollection=null).&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[j5:cl1:s0:prim] | 2023-07-10T15:25:31.281+00:00 D1 ASSERT   23074   [conn80] &quot;User assertion&quot;,&quot;attr&quot;:{&quot;error&quot;:&quot;CollectionUUIDMismatch{ db: \&quot;test63_fsmdb0\&quot;, collectionUUID: UUID(\&quot;d262857c-09cb-4704-a502-1388ef411142\&quot;), expectedCollection: \&quot;unsharded_coll_0\&quot;, actualCollection: null }: Collection UUID does not match that specified&quot;,&quot;file&quot;:&quot;src/mongo/db/catalog/collection_uuid_mismatch.cpp&quot;,&quot;line&quot;:70}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Given the above sequence of events, this does not look like a repro of the bug &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=jordi.olivares-provencio%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;jordi.olivares-provencio@mongodb.com&quot;&gt;jordi.olivares-provencio@mongodb.com&lt;/a&gt; fixed in this ticket. Something else must be going on. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=rohan.sharan%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;rohan.sharan@mongodb.com&quot;&gt;rohan.sharan@mongodb.com&lt;/a&gt;, could you check if the above actions are consistent with what mongosync should be doing?&lt;/p&gt;</comment>
                            <comment id="5555049" author="JIRAUSER1264979" created="Mon, 10 Jul 2023 17:05:13 +0000"  >&lt;p&gt;I think I am seeing an instance of this reoccuring:&#160;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://parsley.mongodb.com/resmoke/f5b7d0b751acd20c94aa6ee9fb3e4d9d/all?bookmarks=0,2267132,2279719,2334854&amp;amp;filters=000unsharded_coll_0.%2AactualCollection%253A%2520null,000create.%2Atest63_fsmdb0.%2Aunsharded_coll_0,000test63_fsmdb0.%2Aunsharded_coll_0.%2ADDL%2520applier,000DDL%2520applier%2520received%2520drop%2520event%2520for%2520a%2520collection,1009273a981-84cd-4666-a363-012762cbfa38&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://parsley.mongodb.com/resmoke/f5b7d0b751acd20c94aa6ee9fb3e4d9d/all?bookmarks=0,2267132,2279719,2334854&amp;amp;filters=000unsharded_coll_0.%2AactualCollection%253A%2520null,000create.%2Atest63_fsmdb0.%2Aunsharded_coll_0,000test63_fsmdb0.%2Aunsharded_coll_0.%2ADDL%2520applier,000DDL%2520applier%2520received%2520drop%2520event%2520for%2520a%2520collection,1009273a981-84cd-4666-a363-012762cbfa38&lt;/a&gt;&#160;&lt;/p&gt;

&lt;p&gt;It is happening on&#160;&lt;br/&gt;
&quot;version&quot;:&quot;7.0.0-rc6-48-ga732e55&quot;&lt;br/&gt;
Which is after the backport &lt;a href=&quot;https://evergreen.mongodb.com/version/mongodb_mongo_v7.0_5c2d756fe06b6c936e9e69c3e3a09ac813baa933&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;5c2d7&lt;/a&gt; was merged.&lt;/p&gt;</comment>
                            <comment id="5545340" author="xgen-internal-githook" created="Wed, 5 Jul 2023 18:41:10 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Jordi Olivares Provencio&apos;, &apos;email&apos;: &apos;jordi.olivares-provencio@mongodb.com&apos;, &apos;username&apos;: &apos;jordiolivares&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77685&quot; title=&quot;Server can return CollectionUUIDMismatch with actual collection null if collection exists&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-77685&quot;&gt;&lt;del&gt;SERVER-77685&lt;/del&gt;&lt;/a&gt; Move collection uuid mismatch check after shard check&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/5ba0cfd0f6dc81c7c3d9083d85f2b0bc6337d65d&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/5ba0cfd0f6dc81c7c3d9083d85f2b0bc6337d65d&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5531056" author="JIRAUSER1264163" created="Wed, 28 Jun 2023 14:36:35 +0000"  >&lt;p&gt;We&apos;re meeting tomorrow during triage where we&apos;ll decide on it. In theory if it all goes well and the ticket gets merged this sprint the backport will be performed quickly after. However, we&apos;ve identified that the logic is also flawed with the lock-free reads path which uses different checks. As a result it&apos;s taking a bit longer than expected since the logic has also diverged now between master and 7.0.&lt;/p&gt;</comment>
                            <comment id="5530965" author="JIRAUSER1264979" created="Wed, 28 Jun 2023 14:15:51 +0000"  >&lt;p&gt;Do we have a timeline for when this will be fixed and backported to v7.0?&lt;/p&gt;</comment>
                            <comment id="5520464" author="JIRAUSER1264163" created="Fri, 23 Jun 2023 09:28:34 +0000"  >&lt;p&gt;I&apos;m starting to agree with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=gregory.noma%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;gregory.noma@mongodb.com&quot;&gt;gregory.noma@mongodb.com&lt;/a&gt; on this. Previously (6.0) the AutoGetCollection didn&apos;t perform a UUID check, leaving it to happen &lt;a href=&quot;https://github.com/mongodb/mongo/blob/16892cef720e4ef0aa3ea719e7564e71ad27fd69/src/mongo/db/ops/write_ops_exec.cpp#L538&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;after acquiring it&lt;/a&gt; and all the sharding checks have occurred.&lt;/p&gt;

&lt;p&gt;As of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-68337&quot; title=&quot;Perform collectionUUID check inside of AutoGetCollection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-68337&quot;&gt;&lt;del&gt;SERVER-68337&lt;/del&gt;&lt;/a&gt; however, this check is performed in the AutoGetter before the other sharding checks. As this occurs in 7.0 I&apos;ll request a backport in order to fix this.&lt;/p&gt;</comment>
                            <comment id="5520420" author="JIRAUSER1264163" created="Fri, 23 Jun 2023 09:06:39 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=rohan.sharan%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;rohan.sharan@mongodb.com&quot;&gt;rohan.sharan@mongodb.com&lt;/a&gt; Sorry for the delay, the team got pulled into other tickets but we are now getting to this issue.&lt;/p&gt;</comment>
                            <comment id="5519529" author="JIRAUSER1264979" created="Thu, 22 Jun 2023 20:20:49 +0000"  >&lt;p&gt;Is there any update here? This is still blocking mongosync 7.0 support.&lt;/p&gt;</comment>
                            <comment id="5495638" author="gregory.noma" created="Tue, 13 Jun 2023 15:42:22 +0000"  >&lt;p&gt;We may just need to swap the order of the collection UUID mismatch check and the shard version check in AutoGetCollection&lt;/p&gt;</comment>
                            <comment id="5467343" author="JIRAUSER1264979" created="Thu, 1 Jun 2023 14:17:36 +0000"  >&lt;p&gt;In talking to Gregory, he mentioned that this might be related to how the sharding API never guaranteed that the primary shard of a sharded cluster would have info on all collections in the cluster (something may have changed that made this no longer the case in v7.0).&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                        <issuelink>
            <issuekey id="2383673">SERVER-78663</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="2375533">SERVER-78375</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="2375533">SERVER-78375</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25578"><![CDATA[v7.0]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 5 Jun 2023 17:02:12 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        30 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            30 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>gregory.noma@mongodb.com</customfieldvalue>
            <customfieldvalue>jordi.olivares-provencio@mongodb.com</customfieldvalue>
            <customfieldvalue>jordi.serra-torrens@mongodb.com</customfieldvalue>
            <customfieldvalue>rohan.sharan@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2bs73:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1u0pc:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="7534">Execution EMEA Team 2023-06-26</customfieldvalue>
    <customfieldvalue id="7535">Execution EMEA Team 2023-07-10</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i2becf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>