[SERVER-72227] Investigate how ESE WiredTiger configuration string arguments may appear in replicated catalog data Created: 16/Dec/22  Updated: 31/Jul/23  Resolved: 31/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Spencer Jackson Assignee: Jordi Olivares Provencio
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-72229 Implement FCV upgrade step to remove ... Closed
Duplicate
is duplicated by SERVER-72228 Investigate how ESE WiredTiger config... Closed
Related
is related to SERVER-68122 Investigate replicating the collectio... Closed
is related to SERVER-79496 Ban encryption options being provided... Closed
Assigned Teams:
Storage Execution
Sprint: Execution EMEA Team 2023-07-24, Execution EMEA Team 2023-08-07
Participants:
Linked BF Score: 13

 Description   

In SERVER-68122 and linked tickets we received reports that, under some circumstances, config string arguments specific to the encrypted storage engine could become durably persisted in the replicated catalog metadata, without being specified in a create command.

SERVER-68122 intends to resolve this issue by ignoring ESE options stored in the catalog. Still, we should investigate how this situation happened, and figure out how ephemeral config string arguments became durable.



 Comments   
Comment by Jordi Olivares Provencio [ 31/Jul/23 ]

Opened SERVER-79496 in order to investigate banning the encryption options. This should prevent the creation config string from being propagated invalidly

Comment by Jordi Olivares Provencio [ 31/Jul/23 ]

The creation string is shown to the user if performing a collStats command on the collection as can be seen here:

> db.runCommand({collStats: db.test.getName()})
{
  ns: 'test.test',
  ...
  wiredTiger: {
    metadata: { formatVersion: 1 },
    creationString: 'access_pattern_hint=none,allocation_size=4KB,app_metadata=(formatVersion=1),assert=(commit_timestamp=none,durable_timestamp=none,read_timestamp=none,write_timestamp=on),block_allocation=best,block_compressor=snappy,cache_resident=false,checksum=on,colgroups=,collator=,columns=,dictionary=0,encryption=(keyid=,name=),exclusive=false,extractor=,format=btree,huffman_key=,huffman_value=,ignore_in_memory_cache_size=false,immutable=false,import=(compare_timestamp=oldest_timestamp,enabled=false,file_metadata=,metadata_file=,repair=false),internal_item_max=0,internal_key_max=0,internal_key_truncate=true,internal_page_max=4KB,key_format=q,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=32KB,leaf_value_max=64MB,log=(enabled=true),lsm=(auto_throttle=true,bloom=true,bloom_bit_count=16,bloom_config=,bloom_hash_count=8,bloom_oldest=false,chunk_count_limit=0,chunk_max=5GB,chunk_size=10MB,merge_custom=(prefix=,start_generation=0,suffix=),merge_max=15,merge_min=0),memory_page_image_max=0,memory_page_max=10m,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=false,prefix_compression_min=4,source=,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=90,tiered_storage=(auth_token=,bucket=,bucket_prefix=,cache_directory=,local_retention=300,name=,object_target_size=0),type=file,value_format=u,verbose=[write_timestamp],write_timestamp_usage=none',
    type: 'file',
    uri: 'statistics:table:collection-7-9017635690571954474',
    ...

Considering the previous comment and in light of this information I suspect some external tooling must've been involved in order to first create a backup of the collection using collStats or some other way of obtaining the creationString. Then a subsequent restore would recreate the collection using the same creationString as the original.

As such, closing this ticket as Cannot Reproduce since we currently cannot reproduce this issue and deem it to be caused due to external actors.

Comment by Jordi Olivares Provencio [ 21/Jul/23 ]

I can't seem to reproduce this in 3.2+. My main issue with this is that the only way I can see for the metadata to contain the encryption config is if the collection is explicitly created with this option in the createCommand arguments. As far as I can tell the encryption config is only hooked in while building the creation config string. This string is never leaked into what goes into the collection catalog since it only stores what the user provided CollectionOptions contains.

This has been the case since 3.2 as far as I can tell.

Comment by Jordi Olivares Provencio [ 21/Jul/23 ]

I've been investigating how this could end up inside of the catalog information. There seems to have been a previous case of this (TOOLS-2038) in May 2018 so this has been an issue since then at the very least. I'm going to center my efforts into the 3.X series of MongoDB to see how this occurred in the first place.

Generated at Thu Feb 08 06:21:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.