[GODRIVER-2469] NewInsufficientBytesError should not create a stack trace Created: 23/Jun/22  Updated: 28/Oct/23  Resolved: 06/Jul/22

Status: Closed
Project: Go Driver
Component/s: BSON
Affects Version/s: 1.9.0
Fix Version/s: 1.10.0

Type: Improvement Priority: Major - P3
Reporter: David Golden Assignee: Matt Dale
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File 2022-06-23_10-21.png    
Epic Link: Reduce Allocations
Quarter: FY23Q2, FY23Q3

 Description   

Summary

NewInsufficientBytesError creates a stack trace, which allocates a lot of memory. For some reason, this is called a lot during normal operations, particularly by the Elements function.

When ADL does format conversions, such as to Parquet, we recursively traverse documents and have observed trace memory allocations significantly greater than the underlying data traversed. This may be related to OOM problems we've seen in production. See attached pprof image.

Please provide the version of the driver. If applicable, please provide the MongoDB server version and topology (standalone, replica set, or sharded cluster).

1.9.0

How to Reproduce

Traverse a large amount of BSON documents with Elements and look at memory allocations with the profiler.

Additional Background

The stack trace appears to have been introduced in the original codebase. The function to retrieve it, ErrorStack, does not appear to be used in the codebase. Because bsoncore is under bsonx, it should be safe to remove the stack trace functionality as it doesn't affect external, stable APIs.



 Comments   
Comment by Githook User [ 30/Jun/22 ]

Author:

{'name': 'Matt Dale', 'email': '9760375+matthewdale@users.noreply.github.com', 'username': 'matthewdale'}

Message: GODRIVER-2469 Remove the stack trace from the InsufficientBytesError type. (#1002)
Branch: master
https://github.com/mongodb/mongo-go-driver/commit/2d577b30d5cae1471af134c6b1a5f4f979c10c37

Comment by Matt Dale [ 30/Jun/22 ]

david.golden@mongodb.com thanks for the additional context. This still seems like a straightforward improvement. A side benefit is we get to remove the github.com/go-stack/stack dependency. PR https://github.com/mongodb/mongo-go-driver/pull/1002 is approved, waiting for tests to complete before merge. I'm marking this for release with Go Driver v1.10.0.

Comment by David Golden [ 28/Jun/22 ]

It was a bug, where Elements was being called on a nil pointer and the error was being ignored. So fixing that bug avoids the stack traces, but nevertheless, it seemed better to not have them anyway.

Generated at Thu Feb 08 08:38:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.