<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 09:06:10 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[KAFKA-343] Improve schema inference for documents nested in arrays</title>
                <link>https://jira.mongodb.org/browse/KAFKA-343</link>
                <project id="16285" key="KAFKA">Kafka Connector</project>
                    <description>&lt;p&gt;Schema inference for documents nested in arrays falls back to &quot;string&quot; when any difference is detected in the schemas for the nested documents.  This is necessary because Kafka schemas can not handle arrays with elements of different type.  But we can improve the schema inference to detect some cases where the schemas for the nested documents are actually compatible:&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;Where the field is present in one document but missing in another&lt;/li&gt;
	&lt;li&gt;Where the field is present in one document but null in another&lt;/li&gt;
	&lt;li&gt;Where the field types conflict (in this case we can push the conflict down to the schema for the field)&lt;/li&gt;
	&lt;li&gt;Where the field is an array with elements of some type in one document but an empty array in another&lt;/li&gt;
&lt;/ol&gt;
</description>
                <environment></environment>
        <key id="2222741">KAFKA-343</key>
            <summary>Improve schema inference for documents nested in arrays</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="10300" iconUrl="https://jira.mongodb.org/images/icons/priorities/medium.svg">Unknown</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="jeff.yemin@mongodb.com">Jeffrey Yemin</assignee>
                                    <reporter username="jeff.yemin@mongodb.com">Jeffrey Yemin</reporter>
                        <labels>
                    </labels>
                <created>Tue, 3 Jan 2023 14:44:28 +0000</created>
                <updated>Sat, 28 Oct 2023 10:46:19 +0000</updated>
                            <resolved>Mon, 9 Jan 2023 15:21:50 +0000</resolved>
                                                    <fixVersion>1.9.0</fixVersion>
                                    <component>Source</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="5099869" author="xgen-internal-githook" created="Mon, 9 Jan 2023 15:21:30 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Jeff Yemin&apos;, &apos;email&apos;: &apos;jeff.yemin@mongodb.com&apos;, &apos;username&apos;: &apos;jyemin&apos;}
&lt;p&gt;Message: Combine compatible schemas for array elements (#128)&lt;/p&gt;

&lt;p&gt; 1. Where two struct types differ only where one has a field that the other does not.  In this case, the&lt;br/&gt;
    extra field is added to the combined schema&lt;br/&gt;
 2. Where two struct types differ only where one has a field that the other has whose value is null in&lt;br/&gt;
    the source document. In this case, the extra field is added to the combined schema with the type of the&lt;br/&gt;
    field that has the non-null value.&lt;br/&gt;
 3. Where two struct types differ where corresponding fields in the struct have a type conflict.  In this case, the type &lt;br/&gt;
    conflict is pushed down to the field, and the type of that field is what becomes string. &lt;br/&gt;
 4. Where two array types differ only in that one of the arrays is empty. In this case, the value schema for the&lt;br/&gt;
    empty array is changed to the one for the non-empty array&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/KAFKA-343&quot; title=&quot;Improve schema inference for documents nested in arrays&quot; class=&quot;issue-link&quot; data-issue-key=&quot;KAFKA-343&quot;&gt;&lt;del&gt;KAFKA-343&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Co-authored-by: Ross Lawley &amp;lt;ross.lawley@gmail.com&amp;gt;&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo-kafka/commit/5530385ee5432559be558279532c1c21157c91fa&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-kafka/commit/5530385ee5432559be558279532c1c21157c91fa&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5086682" author="ross@10gen.com" created="Tue, 3 Jan 2023 14:51:06 +0000"  >&lt;p&gt;Linking a similar issue from Spark&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                                                <inwardlinks description="is documented by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2241641">KAFKA-349</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1546000">KAFKA-175</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2169174">SPARK-375</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10257" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Documentation Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10250"><![CDATA[Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i17dbc:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>