<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:22:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-50476] gdb.lookup_type on dynamically linked mongo programs can be prohibitively expensive</title>
                <link>https://jira.mongodb.org/browse/SERVER-50476</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Calls to &lt;tt&gt;gdb.lookup_type&lt;/tt&gt; when using gdb are ubiquitous. Perhaps most notably is when &lt;a href=&quot;https://github.com/gcc-mirror/gcc/blob/3eeede6de7f6021ad726f034401872f6d58b343d/libstdc%2B%2B-v3/python/libstdcxx/v6/printers.py#L1701&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;pretty printing strings&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Running gdb on a dynamically linked mongo program is struggling mightily compared to a statically linked program with the following invocations:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;gdb.lookup_type(&quot;std::__cxx11::string&quot;)&lt;/li&gt;
	&lt;li&gt;gdb.lookup_type(&quot;std::__cxx11::wstring&quot;)&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;On a mongo shell, the former call took maybe a minute? (pegs a single CPU) and consumed ~6-7GB of memory before succeeding. A follow-up call succeeded immediately (I assume the result was cached by gdb).&lt;/p&gt;

&lt;p&gt;However when examining &lt;tt&gt;mongod&lt;/tt&gt; on a spawn host, the lookup consumes all of the system memory and OOMs the gdb program before it gets a chance to succeed.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1449563">SERVER-50476</key>
            <summary>gdb.lookup_type on dynamically linked mongo programs can be prohibitively expensive</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="daniel.moody@mongodb.com">Daniel Moody</assignee>
                                    <reporter username="daniel.gottlieb@mongodb.com">Daniel Gottlieb</reporter>
                        <labels>
                            <label>post-v4-toolchain</label>
                    </labels>
                <created>Sat, 22 Aug 2020 03:35:52 +0000</created>
                <updated>Sun, 29 Oct 2023 22:04:07 +0000</updated>
                            <resolved>Tue, 17 Jan 2023 18:47:15 +0000</resolved>
                                                    <fixVersion>6.3.0-rc0</fixVersion>
                                    <component>Build</component>
                                        <votes>2</votes>
                                    <watches>12</watches>
                                                                                                                <comments>
                            <comment id="5118509" author="xgen-internal-githook" created="Tue, 17 Jan 2023 18:33:20 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Daniel Moody&apos;, &apos;email&apos;: &apos;daniel.moody@mongodb.com&apos;, &apos;username&apos;: &apos;dmoody256&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-50476&quot; title=&quot;gdb.lookup_type on dynamically linked mongo programs can be prohibitively expensive&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-50476&quot;&gt;&lt;del&gt;SERVER-50476&lt;/del&gt;&lt;/a&gt; created gdb_index scons tool to generate gdb_index&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/15e9bc63835c7c0c441607e10034841b3a157e3c&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/15e9bc63835c7c0c441607e10034841b3a157e3c&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5090719" author="JIRAUSER1253549" created="Wed, 4 Jan 2023 18:43:39 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb@mongodb.com&quot;&gt;daniel.gottlieb@mongodb.com&lt;/a&gt; Thanks for looking into it, so gdb-index was just silently not being used with lld because of the lack of pubnames option? Seems like gdb should have said some kind of warning about the bad index. Same goes for the gold case as well instead of just using it and running into errors.&lt;/p&gt;

&lt;p&gt;I will test it out and reproduce the tests you mentioned, and investigate pubnames a bit more and if it looks good, fix it under this ticket.&lt;/p&gt;</comment>
                            <comment id="5090631" author="daniel.gottlieb@10gen.com" created="Wed, 4 Jan 2023 18:07:54 +0000"  >&lt;p&gt;On a whim I tried the combination of gold + -ggnu-pubnames. gdb&apos;s startup time was reduced to 30 seconds as well (from the 1 minute baseline for gold + gdb-index - pubnames). GDB also succeeded in doing the lookup_type on the problematic names.&lt;/p&gt;

&lt;p&gt;It looks like David Blaikie&apos;s assessment that pubnames is required for proper gdb-indexing by the linker is correct:&lt;br/&gt;
 &lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;a id=&quot;424481_thumb&quot; href=&quot;https://jira.mongodb.org/secure/attachment/424481/424481_screenshot-1.png&quot; title=&quot;screenshot-1.png&quot; file-preview-type=&quot;image&quot; file-preview-id=&quot;424481&quot; file-preview-title=&quot;screenshot-1.png&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/thumbnail/424481/_thumb_424481.png&quot; style=&quot;border: 0px solid black&quot; role=&quot;presentation&quot;/&gt;&lt;/a&gt;&lt;/span&gt; &lt;/p&gt;</comment>
                            <comment id="5090413" author="daniel.gottlieb@10gen.com" created="Wed, 4 Jan 2023 17:05:29 +0000"  >&lt;p&gt;I can confirm the issue still exists when using &lt;tt&gt;--linker=gold&lt;/tt&gt; and goes away with lld. I&apos;m happy for this ticket to be closed.&lt;/p&gt;

&lt;p&gt;I did get curious and did some follow-up investigation:&lt;/p&gt;

&lt;p&gt;I noticed that the startup time for gdb when using lld was ~3 minutes (`execfile` is empty):&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;dgottlieb@chimichurri ~/xgen/mongo[master]$ time gdb --exec=execfile --batch ./bin/mongod ./core&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;real	2m56.794s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;user	3m50.890s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;sys	0m7.840s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;while it was ~1 minute for gold:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;real	1m12.225s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;user	2m6.107s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;sys	0m8.053s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;So to me that meant we were not building a gdb index with lld. However, I did see it being built:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;dgottlieb@chimichurri ~/xgen/mongo[master]$ readelf -S bin/mongod | grep -A 4 index&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  [32] .gdb_index        PROGBITS         0000000000000000  00007890&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;       00000000000020c8  0000000000000000           0     0     1&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  [33] .symtab           SYMTAB           0000000000000000  00009958&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;       0000000000000468  0000000000000018          35    26     8&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  [34] .shstrtab         STRTAB           0000000000000000  00009dc0&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;I stumbled across this &lt;a href=&quot;https://github.com/llvm/llvm-project/issues/34168&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;bug report&lt;/a&gt; that says lld won&apos;t hit the goals of a gdb index if the compilation units are not compiled with &lt;tt&gt;-ggnu-pubnames&lt;/tt&gt;. So I ran an experiment to add that. My scons -&amp;gt; ninja invocation:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;/opt/mongodbtoolchain/v4/bin/python3 ./buildscripts/scons.py CCFLAGS=&quot;-ggnu-pubnames&quot; --link-model=dynamic --dbg --opt=off --variables-files=etc/scons/mongodbtoolchain_stable_clang.vars VARIANT_DIR=newninja --variables-files=etc/scons/developer_versions.vars --modules=&quot;&quot; MONGO_GIT_HASH=&quot;unknown&quot; ICECC=icecc CCACHE=ccache DESTDIR=$(pwd) --ninja generate-ninja NINJA_PREFIX=dynamic.pubnames&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;gdb startup time went down to ~30 second:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;real	0m33.357s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;user	1m27.930s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;sys	0m6.297s&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;The existence of the index seemed to continue to work as expected:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(gdb) python print(gdb.lookup_type(&quot;std::stringstream&quot;))&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;std::stringstream&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.moody%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;daniel.moody@mongodb.com&quot;&gt;daniel.moody@mongodb.com&lt;/a&gt; I&apos;m happy to close this ticket out. But a question/ask that I would appreciate a response on:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Most of that bug report is jargon to me. From your understanding are we impacted by that bug/failure to deliver desired behavior (the index reduces startup times)?&lt;/li&gt;
	&lt;li&gt;Can you reproduce my experiment and results?&lt;/li&gt;
	&lt;li&gt;If so, should we be filing a new ticket to add -ggnu-pubnames to builds where the linker is being asked to create a gdb index?&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="5085782" author="JIRAUSER1253549" created="Tue, 3 Jan 2023 06:27:20 +0000"  >&lt;p&gt;This seems to have been resolved by switching to lld linker as the default linker (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-72324&quot; title=&quot;Toolchain v4 GCC uses system lld when building with -fuse-ld=lld&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-72324&quot;&gt;&lt;del&gt;SERVER-72324&lt;/del&gt;&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;I tested&#160;&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;python print(gdb.lookup_type(&quot;std::string&quot;))&lt;/li&gt;
	&lt;li&gt;python print(gdb.lookup_type(&quot;std::stringstream&quot;))&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;And both were near instantaneous, if any else interested could corroborate, that would be great, otherwise I will close this issue.&lt;/p&gt;</comment>
                            <comment id="4934077" author="luke.pearson" created="Thu, 27 Oct 2022 23:29:48 +0000"  >&lt;p&gt;I ran into this on the coredump from &lt;a href=&quot;https://spruce.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_80_64_bit_dynamic_required_jstestfuzz_concurrent_replication_write_conflicts_2_linux_enterprise_488e4cb26af738ea73fb02b5357be7db5f77ebeb_22_10_19_03_39_50/files?execution=0&amp;amp;sortBy=STATUS&amp;amp;sortDir=ASC&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this failure&lt;/a&gt; when trying to do a &lt;tt&gt;type_lookup&lt;/tt&gt; on &lt;tt&gt;uint8_t&lt;/tt&gt;, similarly for 16/32/64 bits. Disabling pretty printers in this instance didn&apos;t work around the issue. I did manage to work around it using a void pointer, which allowed me do WiredTiger insert list key conversion.&lt;/p&gt;

&lt;p&gt;This command should reproduce the issue on that coredump in the fourth frame.&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;(gdb) p  ((WT_CURSOR_BTREE*)c)-&amp;gt;ref-&amp;gt;page.modify.u2.row_leaf.insert&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt; 

&lt;p&gt;Thanks &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb@mongodb.com&quot;&gt;daniel.gottlieb@mongodb.com&lt;/a&gt; for assisting me with this issue.&lt;/p&gt;</comment>
                            <comment id="4499518" author="acm" created="Fri, 22 Apr 2022 11:58:50 +0000"  >&lt;p&gt;Some potentially good news on this front: &lt;a href=&quot;https://tromey.com/blog/?p=1084&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://tromey.com/blog/?p=1084&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4391733" author="acm" created="Fri, 4 Mar 2022 16:05:19 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt; - This comment on the gold bug suggests that at least some version of GDB has workarounds for the duplicated entries that ld.gold produces: &lt;a href=&quot;https://sourceware.org/bugzilla/show_bug.cgi?id=15646#c5&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://sourceware.org/bugzilla/show_bug.cgi?id=15646#c5&lt;/a&gt;. I will look into whether the version of GDB arriving with the v4 toolchain contains the workaround (along with the fix to the workaround indicated in the comment).&lt;/p&gt;

&lt;p&gt;I&apos;m going to enqueue this ticket into the epic where we are doing post-v4 tasks so we can investigate what to do here more thoroughly. The &lt;tt&gt;gdb-add-index&lt;/tt&gt; script just invokes &lt;tt&gt;gdb&lt;/tt&gt; and uses the &lt;tt&gt;save gdb-index&lt;/tt&gt; command, but does a lot of other things too. I&apos;d want to take a closer look at what all it was doing. I&apos;m a little hesitant to start needing to invoke &lt;tt&gt;gdb&lt;/tt&gt; as part of our build process. There is also &lt;tt&gt;ld.lld&lt;/tt&gt; in the mix, which I think we can start using in more situations once we get to v4. We had disabled it for many cases in &lt;tt&gt;v3&lt;/tt&gt; due to bugs.&lt;/p&gt;</comment>
                            <comment id="4389387" author="max.hirschhorn@10gen.com" created="Thu, 3 Mar 2022 15:44:33 +0000"  >&lt;blockquote&gt;
&lt;p&gt;Unfortunately, there isn&apos;t much we can do until GDB itself addresses this issue.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I&apos;m not sure GDB is really to blame here. After reading through&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;1. The version number, currently 8. Versions 1, 2 and 3 are obsolete. Version 4 uses a different hashing function from versions 5 and 6. Version 6 includes symbols for inlined functions, whereas versions 4 and 5 do not. Version 7 adds attributes to the CU indices in the symbol table. Version 8 specifies that symbols from DWARF type units (&apos;&lt;tt&gt;DW_TAG_type_unit&lt;/tt&gt;&apos;) refer to the type unit&#8217;s symbol table and not the compilation unit (&lt;tt&gt;DW_TAG_comp_unit&lt;/tt&gt;) using the type.&lt;/p&gt;

&lt;p&gt;GDB will only read version 4, 5, or 6 indices by specifying &lt;tt&gt;set use&amp;#45;deprecated&amp;#45;index&amp;#45;sections on&lt;/tt&gt;. GDB has a workaround for potentially broken version 7 indices so it is currently not flagged as deprecated.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://sourceware.org/gdb/onlinedocs/gdb/Index-Section-Format.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://sourceware.org/gdb/onlinedocs/gdb/Index-Section-Format.html&lt;/a&gt;&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;and tracking down &lt;a href=&quot;https://www.sourceware.org/legacy-ml/gdb-patches/2013-08/msg00055.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://www.sourceware.org/legacy-ml/gdb-patches/2013-08/msg00055.html&lt;/a&gt; and &lt;a href=&quot;https://sourceware.org/bugzilla/show_bug.cgi?id=25234&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://sourceware.org/bugzilla/show_bug.cgi?id=25234&lt;/a&gt;, I think the issue with the gold linker and how it generates the .gdb&amp;#95;index section format. Steps I wonder if we could take from here would be to either&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;(a) Disable using &amp;#45;&amp;#45;gdb&amp;#45;index with gold (effectively disabling it in practice because gold is our preferred linker).&lt;/li&gt;
	&lt;li&gt;(b) Use the &lt;tt&gt;gdb&amp;#45;add&amp;#45;index&lt;/tt&gt; CLI tool or the &lt;tt&gt;save gdb&amp;#45;index&lt;/tt&gt; GDB command to generate the .gdb&amp;#95;index section instead of gold (see &lt;a href=&quot;https://sourceware.org/gdb/onlinedocs/gdb/Index-Files.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://sourceware.org/gdb/onlinedocs/gdb/Index-Files.html&lt;/a&gt;). Based on &lt;a href=&quot;https://github.com/bminor/binutils-gdb/blob/gdb-8.3.1-release/gdb/dwarf2read.c#L3515-L3521&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/bminor/binutils-gdb/blob/gdb-8.3.1-release/gdb/dwarf2read.c#L3515-L3521&lt;/a&gt;, it seemed like the performance issue only affects .gdb&amp;#95;index sections generated by gold and not .gdb&amp;#95;index sections of the same version format when they were generated by GDB.&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="3353874" author="acm" created="Sun, 23 Aug 2020 17:37:40 +0000"  >&lt;p&gt;Some very interesting notes here: &lt;a href=&quot;https://sourceware.org/gdb/wiki/SymbolHandling&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://sourceware.org/gdb/wiki/SymbolHandling&lt;/a&gt;. In particular, note 8 seems relevant to our situation. Unfortunately no workaround is suggested.&lt;/p&gt;</comment>
                            <comment id="3353872" author="daniel.gottlieb@10gen.com" created="Sun, 23 Aug 2020 17:35:53 +0000"  >&lt;p&gt;I first ran into this on a coredump from a relatively recent version on our standard shared library rhel 6.2 (linked separately). I reproduced this locally using clang with the following invocation:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;alias build_hygienic_dynamic=&apos;/opt/mongodbtoolchain/v3/bin/python3 ./buildscripts/scons.py --link-model=dynamic --dbg --opt=on --variables-files=etc/scons/mongodbtoolchain_stable_clang.vars VARIANT_DIR=newninja MONGO_VERSION=$(git describe --abbrev=0 | sed -e &quot;s/^r//&quot;) --modules=&quot;&quot; MONGO_GIT_HASH=&quot;unknown&quot; ICECC=icecc CCACHE=ccache --install-mode=hygienic --ninja=next generate-ninja NINJA_PREFIX=dynamic.hygienic&apos;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt; was able to confirm the same problem on his existing mongod. I suspect the repro steps up top are sufficient to observe the same behavior for a wide variety of build settings.&lt;/p&gt;

&lt;p&gt;Are you familiar with what &lt;tt&gt;gdb.lookup_type&lt;/tt&gt; has to accomplish? When seeing the same problem on the shell (which required 6-7GB of memory before succeeding), the memory was freed and subsequent calls returned immediately. It&apos;s not clear to me that it&apos;s necessary for &lt;tt&gt;lookup_type&lt;/tt&gt; to aggregate so much data to return its output. Where I&apos;m going with this: is it plausible that what we&apos;re seeing is some unoptimized code path that is maybe improved upon in a later version of gdb?&lt;/p&gt;</comment>
                            <comment id="3353871" author="acm" created="Sun, 23 Aug 2020 17:12:51 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt; - Thanks for running those experiments. I was somewhat hopeful that we would only observe this behavior with the mongodbtoolchain because it would have suggested that &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-48291&quot; title=&quot;Avoid including a static copy of the GCC and C++ runtimes in every shared library&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-48291&quot;&gt;&lt;del&gt;SERVER-48291&lt;/del&gt;&lt;/a&gt; was responsible, and we are working on a fix for that. However, since the behavior seems to also occur with the system toolchain, where &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-48291&quot; title=&quot;Avoid including a static copy of the GCC and C++ runtimes in every shared library&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-48291&quot;&gt;&lt;del&gt;SERVER-48291&lt;/del&gt;&lt;/a&gt; is not a factor, that hypothesis seems eliminated. That leaves the less palatable hypothesis on the table that the issue is due to the sheer number of &lt;tt&gt;std::string&lt;/tt&gt; instantiations across the large number of shared libraries in a dynamic build. We don&apos;t have a short term fix for reducing the number of libraries, though we do have longer term goal to do so. In the meantime, we will investigate where there are GCC settings around debug info that we can adjust, or whether there are additional startup commands we could execute with GDB to lessen the impact. Note that we are already using &lt;tt&gt;&amp;#45;&amp;#45;Wl,&amp;#45;&amp;#45;gdb-index&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;Meanwhile could you provide details on exactly which build variant and branch, etc., you were using when you encountered this issue?&lt;/p&gt;</comment>
                            <comment id="3353808" author="daniel.gottlieb@10gen.com" created="Sun, 23 Aug 2020 03:48:20 +0000"  >&lt;p&gt;&lt;tt&gt;mongo::mutablebson::Document&lt;/tt&gt; worked fine on the toolchain dynamic build and the system gcc. The cxx11::string against the system gcc consumed a quarter of my desktop memory before I interrupted the command.&lt;/p&gt;</comment>
                            <comment id="3353416" author="acm" created="Sat, 22 Aug 2020 14:09:32 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt; - I don&apos;t have another workaround at the moment but I do have an idea what may be happening. To confirm, can you please, on the same build where you observe the poor performance of &lt;tt&gt;gdb.lookup_type&lt;/tt&gt; with &lt;tt&gt;std::string&lt;/tt&gt;, see if you observe the same poor performance for &lt;tt&gt;gdb.lookup_type&lt;/tt&gt; of some type in the &lt;tt&gt;mongo&lt;/tt&gt; namespace? Ideally a simple non-template class with a primarily out-of-line definition like &lt;tt&gt;mongo::mutablebson::Document&lt;/tt&gt;? Then, especially if the performance of &lt;tt&gt;gdb.lookup_type&lt;/tt&gt; shows that it works OK for the &lt;tt&gt;mongo::&lt;/tt&gt; type, could you please repeat the experiment with both &lt;tt&gt;std::string&lt;/tt&gt; and the &lt;tt&gt;mongo::&lt;/tt&gt; type, but this time on a build of the binaries produced with the system GCC rather than the &lt;tt&gt;mongodbtoolchain&lt;/tt&gt;?&lt;/p&gt;</comment>
                            <comment id="3353329" author="daniel.gottlieb@10gen.com" created="Sat, 22 Aug 2020 03:54:19 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=acm&quot; class=&quot;user-hover&quot; rel=&quot;acm&quot;&gt;acm&lt;/a&gt;, I can presumably &quot;workaround this&quot; by &lt;a href=&quot;https://github.com/gcc-mirror/gcc/blob/3eeede6de7f6021ad726f034401872f6d58b343d/libstdc%2B%2B-v3/python/libstdcxx/v6/printers.py#L1727-L1729&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;not registering the &quot;offending&quot; pretty printers&lt;/a&gt;. That said, I&apos;d highly appreciate an expedited triage process if it can result in a different workaround I can apply which doesn&apos;t sacrifice the ability to pretty print strings. I suspect you already have an idea of what&apos;s probably happening and might have some ideas of what we can try.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="2179472">SERVER-71211</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1452112">SERVER-50537</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2175431">SERVER-71098</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2103439">SERVER-70839</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="424481" name="screenshot-1.png" size="9515" author="daniel.gottlieb@mongodb.com" created="Wed, 4 Jan 2023 18:07:47 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>15.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25130"><![CDATA[Server Development Platform]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Sat, 22 Aug 2020 14:09:32 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 3 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2392</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 3 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andrew.morrow@mongodb.com</customfieldvalue>
            <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>daniel.moody@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>luke.pearson@mongodb.com</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hy1njb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr1man:d4</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;gdb ./path/to/mongod&lt;br/&gt;
$ b main&lt;br/&gt;
$ r&lt;br/&gt;
$ python print(gdb.lookup_type(&quot;std::__cxx11::string&quot;))&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hy19sn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>