<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:47:06 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-37814] db.fsyncLock() do not prevent writes to database on a separate filesystem</title>
                <link>https://jira.mongodb.org/browse/SERVER-37814</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I&apos; trying to develop Mongodb filesystem snapshots.&lt;/p&gt;

&lt;p&gt;I have 2 filesystems (XFS)&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;/mongodb (main dir)&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;/database (one database on a separate filesystem - for performace, symlinked at /mongodb/database)&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I tried scenario decribed at &lt;a href=&quot;https://docs.mongodb.com/manual/tutorial/backup-with-filesystem-snapshots/:&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.mongodb.com/manual/tutorial/backup-with-filesystem-snapshots/:&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;1) db.fsyncLock()&lt;/p&gt;

&lt;p&gt;2) tar /mongodb&lt;/p&gt;

&lt;p&gt;3) tar /database&lt;/p&gt;

&lt;p&gt;4) db.fsyncUnlock()&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;All is done on a 4th node of reaplica set, that is :&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&quot;arbiterOnly&quot; : false,&lt;br/&gt;
 &quot;buildIndexes&quot; : true,&lt;br/&gt;
 &quot;hidden&quot; : true,&lt;br/&gt;
 &quot;priority&quot; : 0,&lt;br/&gt;
 &quot;tags&quot; : {},&lt;br/&gt;
 &quot;slaveDelay&quot; : NumberLong(0),&lt;br/&gt;
 &quot;votes&quot; : 1&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I discovered, that even after running db.fsyncLock() (and 100% ensured it is active by running db.CurrentOp() ), tar command complains about files in /database being changed at doing backup. Modification times of a complained files are a log after db.fsyncLock() was executed, so tar is right about those files changed. I also see some writes to this /database filesystem at the time tar was running (by watching dstat output)&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I&apos;m I missing something here?&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Best regards&lt;/p&gt;

&lt;p&gt;Piotr Rybicki&lt;/p&gt;</description>
                <environment></environment>
        <key id="623429">SERVER-37814</key>
            <summary>db.fsyncLock() do not prevent writes to database on a separate filesystem</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="daniel.hatcher@mongodb.com">Danny Hatcher</assignee>
                                    <reporter username="merituspl">Piotr Rybicki</reporter>
                        <labels>
                    </labels>
                <created>Wed, 24 Oct 2018 11:52:45 +0000</created>
                <updated>Wed, 6 Feb 2019 14:16:52 +0000</updated>
                            <resolved>Thu, 8 Nov 2018 16:58:50 +0000</resolved>
                                    <version>3.6.8</version>
                                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="2055861" author="daniel.hatcher" created="Thu, 8 Nov 2018 16:58:42 +0000"  >&lt;p&gt;After further attempts, I am unable to repro the problem. When using 3.6.8, the `db.fsyncLock()` command freezes all writes to the hidden node:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Hatcher:test danielhatcher$ ls -l data/replset/rs3/db/foo/*&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;-rw-------  1 danielhatcher  staff    345 Nov  8 11:44 data/replset/rs3/db/foo/collection-25--5574008208168300515.wt&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;-rw-------  1 danielhatcher  staff  16384 Nov  8 11:40 data/replset/rs3/db/foo/index-26--5574008208168300515.wt&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Hatcher:test danielhatcher$ ls -l data/replset/rs1/db/foo/*&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;-rw-------  1 danielhatcher  staff  36864 Nov  8 11:53 data/replset/rs1/db/foo/collection-19-4977384934404339527.wt&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;-rw-------  1 danielhatcher  staff  36864 Nov  8 11:53 data/replset/rs1/db/foo/index-20-4977384934404339527.wt&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;
&lt;p&gt;As we are unable to reproduce, I will close the ticket as &quot;Cannot Reproduce&quot; for now. If anyone experiences this issue again, please leave a comment here with details of your environment and we will investigate further.&lt;/p&gt;</comment>
                            <comment id="2048925" author="daniel.hatcher" created="Thu, 1 Nov 2018 14:51:28 +0000"  >&lt;p&gt;Hello Piotr,&lt;/p&gt;

&lt;p&gt;Thank you for your response. I&apos;ll keep trying to reproduce the problem. Please do not hesitate to let us know if you do have a chance to test.&lt;/p&gt;

&lt;p&gt;Have a great day,&lt;/p&gt;

&lt;p&gt;Danny&lt;/p&gt;</comment>
                            <comment id="2046921" author="merituspl" created="Tue, 30 Oct 2018 22:19:48 +0000"  >&lt;p&gt;Thank You Daniel.&lt;/p&gt;

&lt;p&gt;I solved this issue indeed, by stopping mongod process. In the end, I don&apos;t see any important advantage over freezing - either way mongod has to apply oplog, that is accumulated during offline/freeze. Adding 4th hidden node to the RS gets files backup job done without any additional impact on a running RS.&lt;/p&gt;

&lt;p&gt;I&apos;m currently on 3.6.8. Plan to migrate to 4.0 till end of this year. I can test it then, but i suppose just for curiosity really.&lt;/p&gt;

&lt;p&gt;And yes - i&apos;m using&#160;&#160;storage.directoryPerDB: true and also&#160;directoryForIndexes: true&lt;/p&gt;

&lt;p&gt;Best regards&lt;/p&gt;

&lt;p&gt;Piotr Rybicki&lt;/p&gt;</comment>
                            <comment id="2046095" author="daniel.hatcher" created="Tue, 30 Oct 2018 14:03:21 +0000"  >&lt;p&gt;Hello Piotr,&lt;/p&gt;

&lt;p&gt;I greatly apologize; there was a miscommunication on our end about the status of this ticket. You are correct that this appears to be a bug or an undocumented aspect of the system.&lt;/p&gt;

&lt;p&gt;Can you please elaborate on the following? I have been unable to reproduce the problem through some quick testing but I may not be sufficiently replicating your environment.&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;You mention that you solved the issue. Did you do so by simply stopping the process to &lt;tt&gt;tar&lt;/tt&gt; the files or did you follow some other method?&lt;/li&gt;
	&lt;li&gt;Which version did you experience this issue in? If it was not 4.0.3, are you able to reproduce on that version?&lt;/li&gt;
	&lt;li&gt;By &quot;one database on a separate filesystem&quot;, are you using our &quot;directory per database&quot; feature or some other implementation to reach the desired result?&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;I apologize again for the confusion.&lt;/p&gt;

&lt;p&gt;Danny&lt;/p&gt;

&lt;p&gt;Edit: Ah, I see that you reported 3.6.8 as the initial problem version. Would it be possible for you to test against an instance of 4.0.3?&lt;/p&gt;</comment>
                            <comment id="2044614" author="merituspl" created="Mon, 29 Oct 2018 10:57:50 +0000"  >&lt;p&gt;As I said before, I don&apos;t need support and I&apos;m reporting a bug.&lt;/p&gt;

&lt;p&gt;I&apos;ve already managed to solve this issue by myself.&lt;/p&gt;

&lt;p&gt;If You really do not consider this as a bug (at least misinformation in doc saying that db.rsyncLock() will lock all writes - which is not always true), than please close this report with apropriate status.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Kind Regards,&lt;/p&gt;

&lt;p&gt;Piotr Rybicki&lt;/p&gt;</comment>
                            <comment id="2044580" author="trafficbot" created="Mon, 29 Oct 2018 09:33:28 +0000"  >&lt;p&gt;Hello,&lt;/p&gt;

&lt;p&gt;This is an automated reminder that this ticket is awaiting your response. Please respond to this ticket so that we can continue working on this issue with you, or let us know if the ticket can be resolved.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
MongoDB Support&lt;/p&gt;</comment>
                            <comment id="2043314" author="karl.denby" created="Fri, 26 Oct 2018 14:42:17 +0000"  >&lt;p&gt;Hi Piotr,&lt;/p&gt;

&lt;p&gt;After consulting with the server team, they believe this is not a bug and it should be treated as a support issue.&lt;/p&gt;

&lt;p&gt;It appears that you do not currently have an active subscription with access to our core database support team, that provide support on issues like this one. Support is offered via the MongoDB Support Portal (support.mongodb.com), which includes access to create cases, our knowledge base, and more. If this is something that you are interested in, I&apos;d be happy to put you in touch with your MongoDB Account Executive to discuss our subscription option:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://www.mongodb.com/products/mongodb-enterprise-advanced&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;MongoDB Enterprise Advanced (includes product support)&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;This option would provide you with a guaranteed level of support response from MongoDB engineers, and a commitment to work through all support issues you have from beginning to end so that you are successful with MongoDB.&lt;/p&gt;

&lt;p&gt;Please let me know if you would like me to put you in touch with a MongoDB Account Executive to discuss our subscription offerings.&lt;/p&gt;

&lt;p&gt;Kind Regards,&lt;br/&gt;
Karl&lt;/p&gt;</comment>
                            <comment id="2043051" author="merituspl" created="Fri, 26 Oct 2018 09:32:36 +0000"  >&lt;p&gt;Thank You Karl,&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;but I think I don&apos;t need support. I just discovered situation where mongodb server does not behave as documented (db.fsyncLock() does not guarantee database locking - apart from diagnostic.data). And I believe it is a bug, so I&apos;m reporting it here&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Best regards&lt;/p&gt;

&lt;p&gt;Piotr Rybicki&lt;/p&gt;</comment>
                            <comment id="2042035" author="karl.denby" created="Thu, 25 Oct 2018 16:10:14 +0000"  >&lt;p&gt;Hi Piotr,&lt;/p&gt;

&lt;p&gt;We noticed that you created this issue in the SERVER JIRA project, and have moved it to the SUPPORT JIRA project so we can further determine if you have a support subscription.&lt;/p&gt;

&lt;p&gt;Support is offered via the MongoDB Support Portal (support.mongodb.com), which includes access to create cases, our knowledge base, and more.  I believe you have used it to create cases related to your Cloud Manager Project in the past.&lt;/p&gt;

&lt;p&gt;Alternatively, our &lt;a href=&quot;https://groups.google.com/forum/#!forum/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-user community forum&lt;/a&gt; is the best place to ask your query. The MongoDB team tries to help out in this forum, and in addition other community members might be able to share their experience.&lt;/p&gt;

&lt;p&gt;Please create a case via the support portal so that we may assist you.&lt;/p&gt;

&lt;p&gt;Kind Regards,&lt;br/&gt;
Karl&lt;/p&gt;</comment>
                            <comment id="2041680" author="merituspl" created="Thu, 25 Oct 2018 11:49:46 +0000"  >&lt;p&gt;There is also a backup agent from Mongo Cloud Manager used to do backups of RS, if that could be of some hint here&lt;/p&gt;</comment>
                            <comment id="2041573" author="merituspl" created="Thu, 25 Oct 2018 08:49:41 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-32132&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-32132&lt;/a&gt;&#160; is another case&lt;/p&gt;

&lt;p&gt;In my scenario, I have separate filesystem /database (with collection and index sub-dirs in it), and those files are changed. There is no diagnostic.data or journal or local directory there.&lt;/p&gt;

&lt;p&gt;I believe that journal was also changed during that db.fsyncLock() - I&apos;ll check that later&lt;/p&gt;</comment>
                            <comment id="2041186" author="daniel.hatcher" created="Wed, 24 Oct 2018 21:37:04 +0000"  >&lt;p&gt;Hello Piotr,&lt;/p&gt;

&lt;p&gt;Per &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-32132&quot; title=&quot;db.fsyncLock() does not stop writes to FTDC / diagnostic.data files&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-32132&quot;&gt;&lt;del&gt;SERVER-32132&lt;/del&gt;&lt;/a&gt;, the diagnostic.data folder will contain active writes to disk even if the &lt;tt&gt;db.fsyncLock()&lt;/tt&gt; command has been run. If you exclude that folder from your backup, are you still seeing complaints above file changes?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="684697">DOCS-12429</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>12.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 24 Oct 2018 17:54:39 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 13 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 13 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.hatcher@mongodb.com</customfieldvalue>
            <customfieldvalue>karl.denby@mongodb.com</customfieldvalue>
            <customfieldvalue>trafficbot</customfieldvalue>
            <customfieldvalue>merituspl</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hub40v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu1e2v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huaqa7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>