[SERVER-20704] Make sure that storage engine without backup support can pass all the tests Created: 30/Sep/15 Updated: 06/Dec/22 Resolved: 15/Apr/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Igor Canadi | Assignee: | Backlog - Storage Execution Team |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Storage Execution
|
||||||||
| Participants: | |||||||||
| Description |
|
RocksDB has its own way of doing backups, so we didn't plan to support backups from MongoDB API at this time (http://blog.parse.com/learn/engineering/strata-open-source-library-for-efficient-mongodb-backups/). However, currently a storage engine without backup support fails a lot of tests. An example: https://logkeeper.mongodb.org/build/560bf35e90413021b19a2209/test/560bf3a2be07c4135f9a2607 Can we skip all the tests that depend on backup functionality if the storage engine doesn't implement it? |
| Comments |
| Comment by Sara Williamson [ 15/Apr/19 ] |
|
In the future we will be able to use tags for this, so closing as gone away. |
| Comment by Igor Canadi [ 03/Oct/15 ] |
|
RocksDB should now pass all the tests that depend on fsyncLock() functionality: https://github.com/mongodb-partners/mongo-rocks/commit/2c8f0b657f303cae24ba6613e26400ec3da7ccb7 |
| Comment by Igor Canadi [ 01/Oct/15 ] |
|
I just had a discussion with David and it turns out that the actual fsyncLock() guarantees are not very hard to implement on RocksDB side. So I'll fix it on our side to get all tests passing. However, it would be still good if a storage engine could pass all the tests without fsyncLock support. That way we keep the barrier to entry low for new storage engines. |
| Comment by David Hows [ 01/Oct/15 ] |
|
There are sort of a few facets to this as fsyncLock makes one guarantee and has 2 uses:
We have tests that use this one command for each of the above purposes. The changes made in The tests that are failing are those which want the lock, but don't care about the guarantee of on disk consistency when locked. In the jstests/gle/gle_explicit_optime.js test linked above, we lock the secondary to prevent replication (want the lock, not the fsync). Assuming that blocking writes is enough to guarantee that there will be no on disk changes in Rocks, then we can simply overload the begin/end backup functions in the mongo-rocks api to return Status::OK(), effectively saying that the lock alone meets the guarantee. The same is likely true in both the devNull and inMemory engines for similar reasons. A longer term solution here may be to look at a way to de-couple lock from fsync for the above tests. |
| Comment by Ramon Fernandez Marina [ 30/Sep/15 ] |
|
I guess the clean way to do this is to have an API call that returns whether the engine supports backups or not. Sending ticket for triaging... |