[SERVER-33992] Mobile SE: Test validate functionality is doing the right thing Created: 19/Mar/18  Updated: 27/Oct/23  Resolved: 11/Jul/18

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Sulabh Mahajan Assignee: Sulabh Mahajan
Resolution: Works as Designed Votes: 0
Labels: nonnyc, storage-engines
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-33740 Add Evergreen task for running powerc... Closed
Duplicate
is duplicated by SERVER-32993 Mobile SE: Data inconsistencies with ... Closed
is duplicated by SERVER-34953 MobileSE: validate on mobile should r... Closed
Backport Requested:
v4.0
Sprint: Storage Non-NYC 2018-07-16
Participants:

 Description   

Validate for Mobile SE is not tested yet. It might not be doing the right thing.
Implementation of validate() also depends on how the size Storer gets implemented.

This ticket tracks work needed to check if validate is actually finding corruption, and implementing as per size Storer's implementation.

Also make sure that validate() gets retried if SQLite returns a busy error.



 Comments   
Comment by Sulabh Mahajan [ 11/Jul/18 ]

I do not see any tests being impacted by validate() functionality anymore.
SERVER-36062 will stop running parallel and parallel-compatibility and SERVER-35473 will take care of concurrency and concurrency_simultaneous once mapreduce issue is resolved.

Marking this ticket closed as all tasks tracked by this ticket are complete.

Comment by Sulabh Mahajan [ 11/Jul/18 ]

I verified and validate() is doing the following:

  • Iterate over the collection/index and verify if the number of records and size of data cached tallies with what is in the database
  • If full validate is performed, SQLite's PRAGMA integrity_check is run over the whole database.
    I discussed with the team and we think we would like to keep the sizeStorer as it is today. With that validate doesn't need more improvements.

Another point to consider is if we want to retry validate if a write conflict is encountered. I checked what WiredTiger is doing and it doesn't retry today. Validate in SQLite only does reads, so ideally shouldn't even get a database locked issue that translates to write conflict. I intend to keep the behaviour consistent with WiredTiger here.

Effectively that means as far as implementation is concerned validate() seems to be doing the right thing and doesn't need any changes.

Comment by Sulabh Mahajan [ 20/Jun/18 ]

SERVER-32993 was filed for disabling concurrency and concurrency_simultaneous suites. Validate() functionality prevents successful running of a few tests in those suites. Once validate is implemented, re-enable those tests and the ones tagged "SERVER-32993".

Generated at Thu Feb 08 04:35:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.