• decrease the threshold for our OOM tracker on SP2 and SP5.
• check if there are any other librdkafka source and sink config we can set to reduce the memory librdkafka consumes. check if we need to set these values lower on SP2 / SP5.
• Call into KillAllMemoryUsageMonitor::onMemoryUsageIncreased more frequently to account for librdkafka’s memory allocations. It’s probably simplest to just add code in KafkaPartitionConsumer to call into this more frequently and force checking the OS reported resident byters.

======

https://10gen.pagerduty.com/incidents/Q1C7GG6OCHW833?utm_campaign=channel&utm_source=slack

There was a customer sp that was trying out the sp2 tier for their processor and continuously ran into uncaptured OOMs. We should lower how much wiggle room for memory or find a more accurate way to capture these OOMs.

tenantId 68c834f46597ed514b941d05,

projectId 6863bff9fe21057a20c03238,

name: Siesales_Contact_Kafka_Atlas_SP