[SERVER-74519] The SamplingBasedSplitPolicyTest test writes to /data/db/tmp unconditionally Created: 01/Mar/23  Updated: 29/Oct/23  Resolved: 10/May/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 7.0.0-rc1

Type: Bug Priority: Minor - P4
Reporter: Kaloian Manassiev Assignee: Cheahuychou Mao
Resolution: Fixed Votes: 0
Labels: new-eng, sharding-nyc-subteam3
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Assigned Teams:
Sharding NYC
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.0
Steps To Reproduce:

Ensure that /data doesn't exist.

mkdir --parents $HOME/Temp/Data/resmoke
 
python3.10 ./build/install/bin/resmoke.py run \
  -j$(nproc --all) \
  --log file \
  --mrlog \
  --dbpathPrefix $HOME/Temp/Data/resmoke \
  "--suites=unittests"

Sprint: Sharding NYC 2023-05-15
Participants:

 Description   

My workstation's / drive has very little space so I run resmoke with --dbpathPrefix $HOME/Temp/Data/resmoke.
However, the SamplingSucceedsWithLimitedMemoryForSortOperation unit-test reliably fails because it is trying to use /data/db/tmp.

This test should obey the root of the testing hierarchy.



 Comments   
Comment by Githook User [ 10/May/23 ]

Author:

{'name': 'Cheahuychou Mao', 'email': 'mao.cheahuychou@gmail.com', 'username': 'cheahuychou'}

Message: SERVER-74519 Make ShardingTestFixtureCommon use unittest::TempDir for dbpath

(cherry picked from commit 380259f880032aa15fe999d7ec812b0ce9d72b16)
Branch: v7.0
https://github.com/mongodb/mongo/commit/e9ff37f8765dd5bda9484ab7e4ef77fbd80694d5

Comment by Githook User [ 10/May/23 ]

Author:

{'name': 'Cheahuychou Mao', 'email': 'mao.cheahuychou@gmail.com', 'username': 'cheahuychou'}

Message: SERVER-74519 Make ShardingTestFixtureCommon use unittest::TempDir for dbpath
Branch: master
https://github.com/mongodb/mongo/commit/380259f880032aa15fe999d7ec812b0ce9d72b16

Comment by Max Hirschhorn [ 01/Mar/23 ]

SamplingBasedSplitPolicyTest is a test case which was written as part of resharding. Arguably the initial split of resharding is owned by Sharding EMEA (see SERVER-68050) but I feel like this one would be fine for NYC to do.

I feel like there is some confusion because C++ tests do not accept a --dbpath argument like mongod or dbtest do. The mongo::unittest::TempDir class used for C++ tests is configured through TMPDIR, TMP, TEMP, or TEMPDIR, and falling back to /tmp. However there is a problem where the SamplingBasedSplitPolicyTest fixture derives from ShardedAggTestFixture which derives CatalogCacheTestFixture, etc. which not derive from ServiceContextMongoDTest and therefore doesn't assign storageGlobalParams.dbpath to a temporary directory. The SamplingSucceedsWithLimitedMemoryForSortOperation test case when it runs under the SamplingBasedSplitPolicyTest fixture spills to /data/data/tmp due to storageGlobalParams.dbpath being left as its production default value.

Tangentially related, a git grep 'storageGlobalParams.dbpath +' suggests SamplingBasedSplitPolicy::_makePipelineDocumentSource() should maybe be spilling to an _tmp/ directory (underscore prefix) within the dbpath for consistency.

Generated at Thu Feb 08 06:27:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.