Details

    • Type: Task
    • Status: Closed
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.6.0
    • Labels:

      Description

      From @markbenvenuto:

      I hit errors in test/format when I disabled fallocate in Windows. To try to narrow this down, I setup the same repro on Linux to see if it is platform specific and I hit the same error on my Linux machine.

      Fundamentally, either
      1. WiredTiger must not ask fallocate possible truncate the file.
      2. The Windows port layer must either not implement fallocate or track the end of the file itself. For Windows fallocate, either it calls SetEndOfFile (which could truncate), or WriteFile (it would have to write something). As we discussed, I cannot think of way to ensure WriteFile, and SendEndOfFile do not race without locking assuming that WT uses __wt_write to grow the file at times.

      [1428707407:255984][25414:0x7f865619a700], t, file:wt, session.verify: file ranges never verified: 7122
      t: session.verify: file:wt: post-bulk verify: WT_ERROR: non-specific WiredTiger error
       
      diff --git a/src/os_posix/os_fallocate.c b/src/os_posix/os_fallocate.c
      index 1423747..568210c 100644
      --- a/src/os_posix/os_fallocate.c
      +++ b/src/os_posix/os_fallocate.c
      @@ -29,14 +29,6 @@ __wt_fallocate_config(WT_SESSION_IMPL *session, WT_FH *fh)
               * start off requiring locking, we'll relax that requirement once we
               * know which system calls work with the handle's underlying filesystem.
               */
      -#if defined(HAVE_FALLOCATE) || defined(HAVE_POSIX_FALLOCATE)
      -       fh->fallocate_available = WT_FALLOCATE_AVAILABLE;
      -       fh->fallocate_requires_locking = 1;
      -#endif
      -#if defined(__linux__) && defined(SYS_fallocate)
      -       fh->fallocate_available = WT_FALLOCATE_AVAILABLE;
      -       fh->fallocate_requires_locking = 1;
      -#endif
       }
      

      Configuration:
      ./configure CFLAGS="-g -Og" -disable-shared --with-berkeleydb=/usr/include/libdb
      gcc 4.9.2
      Linux 3.19.1-201.fc21.x86_64

      ############################################
      #  RUN PARAMETERS
      ############################################
      abort=0
      auto_throttle=1
      firstfit=0
      bitcnt=6
      bloom=1
      bloom_bit_count=40
      bloom_hash_count=18
      bloom_oldest=0
      cache=23
      checkpoints=1
      checksum=uncompressed
      chunk_size=2
      compaction=0
      compression=none
      data_extend=1
      data_source=file
      delete_pct=0
      dictionary=0
      evict_max=5
      file_type=variable-length column-store
      backups=0
      huffman_key=0
      huffman_value=0
      insert_pct=19
      internal_key_truncation=1
      internal_page_max=9
      isolation=snapshot
      key_gap=2
      key_max=43
      key_min=13
      leak_memory=0
      leaf_page_max=9
      logging=1
      logging_archive=1
      logging_prealloc=0
      lsm_worker_threads=4
      merge_max=11
      mmap=1
      ops=626512
      prefix_compression=1
      prefix_compression_min=7
      repeat_data_pct=86
      reverse=0
      rows=270388
      runs=0
      split_pct=84
      statistics=0
      statistics_server=0
      threads=3
      timer=20
      value_max=3537
      value_min=17
      wiredtiger_config=
      write_pct=0
      ############################################
      

        Issue Links

          Activity

          Hide
          keith.bostic Keith Bostic added a comment -

          @markbenvenuto, the changes in the WT-1873 branch make ftruncate work for me on FreeBSD (I was able to duplicate the failure you saw, so I don't think Linux is required). Anyway, I'm hoping that branch allows you to turn off fallocate entirely on Windows and calls to ftruncate will be safe, we'll never overwrite existing blocks.

          Show
          keith.bostic Keith Bostic added a comment - @markbenvenuto, the changes in the WT-1873 branch make ftruncate work for me on FreeBSD (I was able to duplicate the failure you saw, so I don't think Linux is required). Anyway, I'm hoping that branch allows you to turn off fallocate entirely on Windows and calls to ftruncate will be safe, we'll never overwrite existing blocks.
          Hide
          keith.bostic Keith Bostic added a comment -

          Closed with the merge of WT-1873 and WT-1874.

          Show
          keith.bostic Keith Bostic added a comment - Closed with the merge of WT-1873 and WT-1874 .
          Hide
          ramon.fernandez Ramon Fernandez added a comment -

          Additional ticket information from GitHub

          This ticket was referenced in the following commits:
          Show
          ramon.fernandez Ramon Fernandez added a comment - Additional ticket information from GitHub This ticket was referenced in the following commits: f26f1c1c59d5cbbc8f5f543215d8fc636e7175d2 61a7d81ad26db1f2bfb65258d9b8ae4a4ca25b34

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                2 years, 5 weeks, 5 days ago
                Date of 1st Reply: