[SERVER-78241] truncated log line shows number of remaining strings instead of truncated strings Created: 20/Jun/23  Updated: 14/Dec/23  Resolved: 08/Dec/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.3.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Asya Kamsky Assignee: Erwin Pe
Resolution: Fixed Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by TOOLS-3443 Investigate changes in SERVER-78241: ... Closed
Documented
is documented by DOCS-16534 Investigate changes in SERVER-78241: ... Closed
Related
is related to SERVER-55276 Log gets incorrectly truncated in cas... Backlog
Assigned Teams:
Server Security
Backwards Compatibility: Fully Compatible
Sprint: Security 2023-07-10, Security 2023-07-24, Security 2023-08-07, Security 2023-08-21, Security 2023-10-16, Security 2023-11-13, Security 2023-11-27, Security 2023-12-11
Participants:

 Description   

When running a simple find command with over 5000 elements in $in clause, the output in the log line is: 

 ... _id:{$in:[ <truncated list of 377 out of 5000 values> ] ...
{
    "truncated" : {
        "command" : {
            "filter" : {
                "_id" : {
                    "$in" : {
                        "377" : {
                            "type" : "string",
                            "size" : 29
                        }
                    }
                }
            }
        }
    },

I'd love to know how many actual values were truncated which is what I thought it would show but it seems instead to show how many are written.



 Comments   
Comment by Githook User [ 07/Dec/23 ]

Author:

{'name': 'Erwin Pe', 'email': 'erwin.pe@mongodb.com', 'username': 'erwee'}

Message: SERVER-78241 Show how many elements were omitted in truncation report

GitOrigin-RevId: e6ef4ac429231d7b95eeb66eb9cb50bf81e4eef7
Branch: master
https://github.com/mongodb/mongo/commit/c55abab3acd7057c5d70e62c340c5900391f0451

Comment by Erwin Pe [ 20/Jul/23 ]

Right now, I don't think we put any information at all in the truncation report about how much was truncated.

Based on the technical design doc, the "truncated" field is only supposed to tell you where the truncation occurred, and not what fields got omitted.  The sibling "size" field tells you the size of the entire BSON object before it was converted to JSON, but that does not help either.

Comment by Asya Kamsky [ 13/Jul/23 ]

I don't understand how index 377 helps since I can see 377 elements already in the log line. Where do I find the information about how much was truncated? What does
> the "size" field in the truncation report indicates the size of the value that would have been written if truncation had not occurred at that BSON element
how is this useful? It's the size of the exact element where we truncated with no information how much information was actually omitted from the logs?

Comment by Erwin Pe [ 30/Jun/23 ]

Changed from "bug" to "improvement", as I think we just need to add more information in the truncation report that would inform the user of which fields got omitted.

Comment by Erwin Pe [ 26/Jun/23 ]

The numbers in the above example neither shows how much was truncated from the entire BSON object, nor how much was written.  Instead, the "size" field in the truncation report indicates the size of the value that would have been written if truncation had not occurred at that BSON element.  In the above example, the truncation happened at index "377" of the $in array, and the truncated value at that index had a string of size 29.

I don't think this is a bug that was introduced in SERVER-66841, since that bugfix was only applicable to raw string truncations, whereas this is a truncation that occurs in a stringified BSONObj.

Comment by Asya Kamsky [ 21/Jun/23 ]

I'm told security owns logs so switching assigned team - LMK if any other information is needed for triage.

 

Generated at Thu Feb 08 06:37:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.