Fix transient "Encountered errors while storing offsets" when commitAsync is called

XMLWordPrintableJSON

    • Atlas Streams
    • Fully Compatible
    • ALL
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      This reproes in various tests in kafka.js during the initial start of the processor. The error eventually goes away. We should fix it to avoid the confusing warning logs.

      this issue comes from us calling commitAsync before the kafka consumer has fully started up.

      We have two options for the fix:

      1. (quick hack) Just don't print a warning for the first X seconds.
      2. While initializing the kafka consumer group, wait for it to be ready.

       

      From Devika:

      Noticed these errors during a prod deploy, over 300 times for one SP.

       

      https://splunk.corp.mongodb.com/en-US/app/cloud/search?earliest=1742932440&latest=1742934480&q=search%20index%3Dmhouse%2066ed941512c18dd39b39f46d%20%20%22Encountered%20error%20while%20storing%20offsets%22&display.page.search.mode=smart&dispatch.sample_ratio=1&display.events.type=list&display.page.search.tab=events&display.general.type=events&display.visualizations.charting.chart=column&sid=1742940789.7454688

              Assignee:
              Jada Lilleboe
              Reporter:
              Devika Pisharoty
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: