[SERVER-44841] Revisit log levels for new migration protocol Created: 26/Nov/19  Updated: 29/Oct/23  Resolved: 02/Mar/20

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 4.3.4

Type: Improvement Priority: Major - P3
Reporter: Matthew Saltz (Inactive) Assignee: Alexander Taskov (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-47227 rename LogComponent MIGRATION to MIGRATE Closed
Backwards Compatibility: Fully Compatible
Sprint: Sharding 2020-03-09
Participants:

 Description   

It's helpful to have log level 0 for the new components we're adding for a little while to aid in debugging test failures in evergreen but we should more carefully decide which to keep at level 0 and which to decrease.

Also, we should decide whether to use the thread name as an identifier for log lines from range deletion code, or whether to introduce a new log component.



 Comments   
Comment by Githook User [ 28/Apr/20 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-44841 change log component string MIGRATION => MIGRATE

(cherry picked from commit 2638bf93b6af37b1c71dcbd60947f4a1973222a7)
Branch: v4.4
https://github.com/mongodb/mongo/commit/451c11f3e483fb89fdc2f7d4368d8c8d1d2f4cc4

Comment by Billy Donahue [ 01/Apr/20 ]

Created SERVER-47227 to track just the rename of MIGRATION to MIGRATE specifically.

I was being lazy and hoping to avoid a separate ticket, but I'll need to track backporting and don't want to keep spamming this larger and closed ticket.

Comment by Billy Donahue [ 01/Apr/20 ]

There's still an 8-character limitation, it's just not tested anymore.

This is not a technical limitation, but we have opted to pad the first few json fields and having one that's 9 chars harshes the feng shui of the logs.

We still have code that uses the v1 API and some converter helpers. Until that's no longer the case we have to keep the v1 and v2 enums in sync.

Comment by Githook User [ 01/Apr/20 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-44841 change log component string MIGRATION => MIGRATE
Branch: master
https://github.com/mongodb/mongo/commit/2638bf93b6af37b1c71dcbd60947f4a1973222a7

Comment by Alexander Taskov (Inactive) [ 01/Apr/20 ]

It's unfortunate that the new component name has to be bound by the limitations of the legacy one. Out of curiosity, why do they have to match? I thought the plan was to remove the legacy component when everything has been migrated to the v2 component. The preferred name is MIGRATION however for the legacy component we had to compromise to MIGRATE because of the 8 character limitation.

Comment by Billy Donahue [ 01/Apr/20 ]

LogComponent "MIGRATION" => "MIGRATE" change
CR https://mongodbcr.appspot.com/563350001/

Comment by Billy Donahue [ 01/Apr/20 ]

A new LogComponent was created by the commit for this ticket, in both the logger (log v1) and logv2 libraries.
The logger::LogComponent is logged as "MIGRATE".
The logv2::LogComponent is logged as "MIGRATION".

The logger and the logv2 have to agree on the name of the component.
Also, we need this field to be 8 characters or less.

So I'm going to change the logv2 component to appear as "MIGRATE" to match the logger component.
I assume this won't mess anything up on your end. Please let me know if I'm wrong about that.
Thanks.

Comment by Githook User [ 29/Feb/20 ]

Author:

{'name': 'Alex Taskov', 'username': 'alextaskov', 'email': 'alex.taskov@mongodb.com'}

Message: SERVER-44841 Revisit log levels for new migration protocol
Branch: master
https://github.com/mongodb/mongo/commit/e4673d051856772b44642abf4ebef38a3086a188

Comment by Esha Maharishi (Inactive) [ 24/Feb/20 ]

There are a few things I think we could do to improve logging for the range deleter:

  1. Add a new range-deleter specific log component. It looks like this has the list of the new logging system's (logv2) components, and this is the list of the old logging system's components
  2. Change the default log component in migration_coordinator.cpp, migration_util.cpp, range_deletion_util.cpp to the new component
  3. It looks like the new logging system has a way to set the log component just for a specific log line in a file, though I haven't tried it yet. We could also audit the log lines in the MigrationSourceManager, CollectionShardingRuntime, and MetadataManager and override the log component for any specific lines that are related to range deletion. For example, this and this in the MetadataManager
  4. Lower the verbosity of range deletion log lines from level 0 to level 1 or 2 (depending on the line) so that they do not flood users' logs
  5. However, so that we can still see all the log lines in our tests, make our tests set the verbosity for the new range deletion component to level 2 (see what we similarly did for the transaction log component)
Generated at Thu Feb 08 05:07:08 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.