Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-90387

Non-deterministic behavior in transaction_too_large_for_cache/temporarily_unavailable_on_secondary_transaction_application.js

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 8.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Storage Execution
    • Fully Compatible
    • Execution Team 2024-08-05, Execution Team 2024-09-02
    • 0

      Both transaction_too_large_for_cache_on_secondary_transaction_application.js and temporarily_unavailable_on_secondary_transaction_application.js both:

      • Reduce the cache size on a secondary.
      • In a parallel shell, execute a large insert expected to deterministically & repeatedly throw because it overfills the WT cache.
      • It expects the exceptions to be turned into WriteConflicts so they are retried.
      • Expect the secondary serverStatus to report the exception / insert is retried 5 times before re-increasing the cache size, so the insert can successfully commit in the parallel shell.

      However, WT behavior is not deterministic when performing a large insert into a main collection, then its indexes in a WT transaction. Sometimes the WT transaction succeeds with a cache filling insert, which causes the tests to fail.

      luke.pearson@mongodb.com wrote a non-deterministic reproducer of the WT behavior. When the two second sleep is removed, the test will mostly pass (not throw an error)

      import wiredtiger, wttest
      from wtscenario import make_scenarios
      from time import sleep
      
      class test_tlfc(wttest.WiredTigerTestCase):
          #1MB cache.
          conn_config = 'statistics=(all),cache_size=1MB'
          key_format_values = [
              ('integer-row', dict(key_format='i')),
          ]
          scenarios = make_scenarios(key_format_values)
      
          def test_tlfc(self):
              uri = "table:test_tlfc"
              uri2 = "table:index"
              create_params = 'value_format=S,key_format={}'.format(self.key_format)
              # 10MB string.
              valuebig = 'e' * 1024 * 1024 * 10
              self.session.create(uri, create_params)
              self.session.create(uri2, create_params)
              cursor = self.session.open_cursor(uri)
              cursor2 = self.session.open_cursor(uri2)
      
              self.session.begin_transaction()
              cursor[1] = valuebig
              # Comment out the below line and this test will start passing, mostly.
              sleep(2)
              cursor2[2] = "abc"
      

      The tests have also led to wired tiger cache eviction getting stuck. See WT-12931 for more details.

      Goal: Determine whether these tests should be removed, modified, or new tests should be added.

            Assignee:
            shinyee.tan@mongodb.com Shin Yee Tan
            Reporter:
            haley.connelly@mongodb.com Haley Connelly
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: