Fix transient "Encountered errors while storing offsets" when commitAsync is called

XMLWordPrintableJSON

    • Atlas Streams
    • Fully Compatible
    • ALL
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      This reproes in various tests in kafka.js during the initial start of the processor. The error eventually goes away. We should fix it to avoid the confusing warning logs.

      this issue comes from us calling commitAsync before the kafka consumer has fully started up.

      We have two options for the fix:

      1. (quick hack) Just don't print a warning for the first X seconds.
      2. While initializing the kafka consumer group, wait for it to be ready.

       

      From Devika:

      Noticed these errors during a prod deploy, over 300 times for one SP.

       

      https://splunk.corp.mongodb.com/en-US/app/cloud/search?earliest=1742932440&latest=1742934480&q=search%20index%3Dmhouse%2066ed941512c18dd39b39f46d%20%20%22Encountered%20error%20while%20storing%20offsets%22&display.page.search.mode=smart&dispatch.sample_ratio=1&display.events.type=list&display.page.search.tab=events&display.general.type=events&display.visualizations.charting.chart=column&sid=1742940789.7454688

            Assignee:
            Jada Lilleboe
            Reporter:
            Devika Pisharoty
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: