Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-21767

Index sizes differ significantly when different block compression is specified

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: Storage, WiredTiger
    • Labels:
      None
    • Fully Compatible
    • ALL
    • Hide

      git clone https://github.com/asya999/iibench-mongodb.git
      cd iibench-mongodb
      sed -i "s/export NUM_LOADER_THREADS=.*/export NUM_LOADER_THREADS=20/" run.simple.bash
      sed -i "s/export MAX_ROWS=.*/export MAX_ROWS=300000000/" run.simple.bash
      ./run.simple.bash

      Show
      git clone https://github.com/asya999/iibench-mongodb.git cd iibench-mongodb sed -i "s/export NUM_LOADER_THREADS=.*/export NUM_LOADER_THREADS=20/" run.simple.bash sed -i "s/export MAX_ROWS=.*/export MAX_ROWS=300000000/" run.simple.bash ./run.simple.bash

      Running 

      ../mongo-latest/bin/mongod --version
db version v3.2.0-rc4-21-g86e7b69
git version: 86e7b69a6c52c926d28a60d816faefa6db81eb96
OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
allocator: tcmalloc
modules: enterprise
build environment:
    distmod: ubuntu1404
    distarch: x86_64
    target_arch: x86_64

      which I believe is equivalent to what ended up being 3.2.0-rc5 I'm observing the following at the end of a run that inserts 200M documents:

      When server is started with default (snappy) block compression, the stats at the end of the run are approximately:

      	"db" : "iibench",
	"collections" : 1,
	"objects" : 200000000,
	"avgObjSize" : 1119,
	"dataSize" : 208.4299921989441,
	"storageSize" : 51.31330490112305,
	"numExtents" : 0,
	"indexes" : 4,
	"indexSize" : 22.347862243652344,
	"ok" : 1

      When the same run is done with block compression zlib, the stats are:

      	"db" : "iibench",
	"collections" : 1,
	"objects" : 200000000,
	"avgObjSize" : 1119,
	"dataSize" : 208.4299921989441,
	"storageSize" : 21.949535369873047,
	"numExtents" : 0,
	"indexes" : 4,
	"indexSize" : 2.2369346618652344,
	"ok" : 1

      On longer run the difference was only 2x but I would not have expected any size difference since we don't apply block compression to indexes.

            Assignee:
            keith.bostic@mongodb.com Keith Bostic (Inactive)
            Reporter:
            asya.kamsky@mongodb.com Asya Kamsky
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:
              Resolved: