Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-8413

Open WT connection cause "too many open files" issue

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: None
    • Labels:
      None
    • Environment:
      Centos7, WT 10.0.0
    • Story Points:
      3
    • Sprint:
      Storage - Ra 2021-11-29, Storage - Ra 2021-12-13

      Description

      Hi there,

      We are doing some testing with the new wiredtiger 10.0.0 and we have encountered some issues with reopening the connection to the wiredtiger DB.

      [1637046703:105009][2351769:0x7f8277ad73c0], wt, file:r6CJAHkcscZQvemdL8RLMThIxoU3CyLEGs5DAQRRo1s_full.wt, txn rollback_to_stable: __posix_open_file, 808: /var/lib/wiredtiger/r6CJAHkcscZQvemdL8RLMThIxoU3CyLEGs5DAQRRo1s_full.wt: handle-open: open: Too many open files
      [1637046703:105204][2351769:0x7f8277ad73c0], wt, txn-recover: __wt_txn_recover, 1054: Recovery failed: Too many open files
      [1637046703:118307][2351769:0x7f8277ad73c0], wt, connection: __wt_cache_destroy, 364: cache server: exiting with 1103 pages in memory and 0 pages evicted
      [1637046703:118342][2351769:0x7f8277ad73c0], wt, connection: __wt_cache_destroy, 367: cache server: exiting with 1937629 image bytes in memory
      [1637046703:118347][2351769:0x7f8277ad73c0], wt, connection: __wt_cache_destroy, 370: cache server: exiting with 2122738 bytes in memory
      [1637046703:118351][2351769:0x7f8277ad73c0], wt, connection: __wt_cache_destroy, 376: cache server: exiting with 352 bytes dirty and 1 pages dirty
      wt: WT_PANIC: WiredTiger library panic

      Setting the service 'LimitNOFILE' and 'LimitNPROC' to high value solves the issue.
      But we have noticed that all the files that are in the database directory are being opened when we are restarting our service (and opening the WT connection).

      Is there any reason for the use of so many open fds when opening the connection?
      We didn't encounter this issue on version 3.2.1.
      Thank you for your help.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              haseeb.bokhari Haseeb Bokhari
              Reporter:
              tom@jetapps.com Tom M
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated: