[SERVER-16604] Improve integration with rocks Created: 19/Dec/14  Updated: 18/Sep/15  Resolved: 23/Jan/15

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.0.0-rc7

Type: Improvement Priority: Major - P3
Reporter: Igor Canadi Assignee: Matt Kangas
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-15271 rocks engine doesn't return capped in... Closed
depends on SERVER-15744 RocksDB storage engine does not pass ... Closed
depends on SERVER-15932 Fix build break of rocks storage engine Closed
depends on SERVER-16204 rocks storage engine should deal with... Closed
depends on SERVER-16259 rocks storage engine needs to support... Closed
depends on SERVER-16603 Fix linking of WriteConflictException... Closed
depends on SERVER-14352 prototype and implement rocksdb stora... Closed
depends on SERVER-16632 Change WiredTiger index key format to... Closed
Related
is related to SERVER-16850 Remove support for touch command for ... Closed
is related to SERVER-13256 Remove pch.h Closed
is related to SERVER-16979 [Rocks] Correctly handle errors retur... Closed
Backwards Compatibility: Fully Compatible
Participants:

 Description   

Goals for completion of this ticket:

  1. Different format for unique indexes. Probably the same thing that wiredtiger is doing – storing duplicates as a list (as oposed to storing RecordIDs as part of keys). This should also enable using bloom filters for duplicate key checks.
  2. Phase-out old keys in RocksTransaction. Currently all written keys ever are stored in RocksTransaction. We need to remove keys that are not being touched by any currently running transactions.
  3. Improve rocks_engine metadata. Introduce DB version to support changes in disk format in the future.
  4. Configure RocksDB + expose some config options to users.

(Copied from email from the submitter)



 Comments   
Comment by Githook User [ 26/Jan/15 ]

Author:

{u'username': u'igorcanadi', u'name': u'Igor Canadi', u'email': u'icanadi@fb.com'}

Message: SERVER-16604 [rocks] New format for unique indexes

For unique indexes, we now encode RecordId in value instead of as part
of the key. We keep duplicate RecordIds (when dupsAllowed == true) as a
list in the value. This is the same format as WiredTiger's unique
indexes.

This format enables us to use bloom filters to avoid doing reads for
secondary index maintenance.

This patch changes Mongo+Rocks' disk format.

Signed-off-by: Benety Goh <benety@mongodb.com>
(cherry picked from commit 7cb3be1c8c35b6f3b7903a3a2efa64b1468fd514)
Branch: v3.0
https://github.com/mongodb/mongo/commit/bf1998071a6c6650508e12d8b25cbf035d388dad

Comment by Benety Goh [ 23/Jan/15 ]

igor,

Please update the description to reflect the current functionality in the RocksDB storage engine. Further enhancements to the RocksDB storage engine should be reflected in new SERVER tickets.

Thank you and sdong for all your contributions!

Comment by Githook User [ 23/Jan/15 ]

Author:

{u'username': u'igorcanadi', u'name': u'Igor Canadi', u'email': u'icanadi@fb.com'}

Message: SERVER-16604 [rocks] New format for unique indexes

For unique indexes, we now encode RecordId in value instead of as part
of the key. We keep duplicate RecordIds (when dupsAllowed == true) as a
list in the value. This is the same format as WiredTiger's unique
indexes.

This format enables us to use bloom filters to avoid doing reads for
secondary index maintenance.

This patch changes Mongo+Rocks' disk format.

Signed-off-by: Benety Goh <benety@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/7cb3be1c8c35b6f3b7903a3a2efa64b1468fd514

Comment by Githook User [ 17/Jan/15 ]

Author:

{u'username': u'igorcanadi', u'name': u'Igor Canadi', u'email': u'icanadi@fb.com'}

Message: SERVER-16604 Use KeyString in Rocks index

This patch changes disk format for the Rocks storage engine.

Signed-off-by: Benety Goh <benety@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/2d7050b6c3b9d35590e7e874e6285f0a1fcb5f3d

Comment by Githook User [ 08/Jan/15 ]

Author:

{u'username': u'igorcanadi', u'name': u'Igor Canadi', u'email': u'icanadi@fb.com'}

Message: SERVER-16604 Phase out old keys in RocksTransaction

Signed-off-by: Matt Kangas <matt.kangas@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/174fe03770738005c08ed6ec61f1a95ce5542dbe

Comment by Ramon Fernandez Marina [ 21/Dec/14 ]

Thanks igor, please let us know when all patches for a given JIRA ticket have been merged in so we can resolve the ticket and set the right fixVersion.

Comment by Githook User [ 19/Dec/14 ]

Author:

{u'username': u'igorcanadi', u'name': u'Igor Canadi', u'email': u'icanadi@fb.com'}

Message: SERVER-16604 Stop special-casing oplog in capped collections

Signed-off-by: Matt Kangas <matt.kangas@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/53dbff5aa4407ada6b6f2416f6702be02ad26037

Comment by Igor Canadi [ 19/Dec/14 ]

Sure. I'm opening this just so I can tag my patches with JIRA ticket

Comment by Ramon Fernandez Marina [ 19/Dec/14 ]

igor, I understand this ticket and SERVER-16603 are placeholders, so I'm marking them as "debugging with submitter" pending further input from you.

Generated at Thu Feb 08 03:41:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.