Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-2035

index read leads to read of main table

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: WT2.6.1
    • Fix Version/s: WT2.7.0
    • Labels:
      None

      Description

      Hi!

      We have a table with index and sometimes want to read only index.
      I.e. we're only interested in pairs (index key, primary key)

      So we tried to use index cursor projection as described here

      But it seems that WiredTiger nevertheless reads the main table although it doesn't need to (I think).

      The problem is that our main table is quite big (several GB's) and scanning through index lead to it being read non-sequentially which is quite slow.

      Small demo example attached (should be build with -rdynamic)
      Output I get:

      $ ./read_index
      Filling DB...
      total 481716
      -rw-r--r-- 1 sbn eng    757760 Aug  7 15:03 main_index.wti
      -rw-r--r-- 1 sbn eng 491974656 Aug  7 15:03 main.wt
      -rw-r--r-- 1 sbn eng        45 Aug  7 15:03 WiredTiger
      -rw-r--r-- 1 sbn eng       298 Aug  7 15:03 WiredTiger.basecfg
      -rw-r--r-- 1 sbn eng        21 Aug  7 15:03 WiredTiger.lock
      -rw-r--r-- 1 sbn eng       938 Aug  7 15:03 WiredTiger.turtle
      -rw-r--r-- 1 sbn eng     36864 Aug  7 15:03 WiredTiger.wt
       
      Reading index using index: cursor...
      total 0
      lrwx------ 1 sbn eng 64 Aug  7 15:03 0 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 1 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 2 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 3 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/WiredTiger.lock
      lrwx------ 1 sbn eng 64 Aug  7 15:03 4 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/WiredTiger.wt
      lrwx------ 1 sbn eng 64 Aug  7 15:03 5 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/main_index.wti
      lrwx------ 1 sbn eng 64 Aug  7 15:03 6 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/main.wt
      key_format = i, value_format = i
      pread() fd stats:
      5:      745472
      6:      491962368
       
      Reading index using file: cursor...
      total 0
      lrwx------ 1 sbn eng 64 Aug  7 15:03 0 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 1 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 2 -> /dev/pts/2
      lrwx------ 1 sbn eng 64 Aug  7 15:03 3 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/WiredTiger.lock
      lrwx------ 1 sbn eng 64 Aug  7 15:03 4 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/WiredTiger.wt
      lrwx------ 1 sbn eng 64 Aug  7 15:03 5 -> /home/sbn/src/wiredtiger/examples/c/read_index.db/main_index.wti
      key_format = ii, value_format = 
      pread() fd stats:
      5:      745472
      

      Thanks!

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                donald.anderson Donald Anderson
                Reporter:
                Dmitri Shubin Dmitri Shubin
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: