-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: 1.13.1
-
Component/s: None
-
None
-
Go Drivers
-
Not Needed
-
Detailed steps to reproduce the problem?
create a simple app that use zstd as Compressors
ClientOptions := options.Client().ApplyURI(str).SetCompressors([]string{"zstd"}).SetBSONOptions(bsonOpts).SetServerSelectionTimeout(30 * time.Second) var err error config.Client, err = mongo.Connect(context.Background(), ClientOptions) collection := config.Client.Database(config.MainConf.Main_db).Collection("flexyoperations") changeStream, errChangeStream := collection.Watch(context.Background(), mongo.Pipeline{ bson.D{{Key: "$match", Value: bson.M{"operationType": "insert"}}}, })
a simple function that save a struct
func SaveServer() { mutexSave.Lock() defer mutexSave.Unlock() server.LastSeen = time.Now() logger.DBL.Info("DB Server Save => Called ", zap.Any("server", server)) if config.Client == nil { ServerStatus.Set("Pending") dialog.ShowError(errors.New("not Connected"), config.Window) logger.DBL.Error("DB Server Load => Client is nil ") return } _, err := config.Client.Database(config.MainConf.Main_db).Collection("flexyservers").UpdateOne(context.TODO(), bson.M{"server_name": config.MainConf.Server_name}, bson.M{"$set": server}) if err != nil { logger.DBL.Error("DB Server SaveServer => Error: ", zap.Any("err", err)) return } }
just after calling saveserver 5- 10 time i can see huge memory usage , it's can go up to 2.5gb after my app run for few hours when i start debuging using pprof i can see the following just calling save few times
flat flat% sum% cum cum% 256MB 77.50% 77.50% 256MB 77.50% github.com/klauspost/compress/zstd.(*fastBase).ensureHist (inline) 48.86MB 14.79% 92.29% 48.86MB 14.79% github.com/klauspost/compress/zstd.encoderOptions.encoder 10.99MB 3.33% 95.62% 10.99MB 3.33% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer 2.26MB 0.68% 96.31% 2.26MB 0.68% github.com/klauspost/compress/zstd.(*blockEnc).init 2MB 0.61% 96.91% 4MB 1.21% github.com/go-text/typesetting/opentype/tables.(*Glyph).parseData 0 0% 96.91% 32MB 9.69% flexyserver/Repositories.LoadOffers 0 0% 96.91% 32.57MB 9.86% flexyserver/Repositories.LoadServer 0 0% 96.91% 161.70MB 48.95% flexyserver/Repositories.SaveServer 0 0% 96.91% 81.36MB 24.63% flexyserver/services/db.Connect 0 0% 96.91% 11.91MB 3.61% flexyserver/ui.ShowMainUI
simply changing from zstd to snappy or zlib resolve my issue and my app doesn't exceed 150mb memory usage
flat flat% sum% cum cum% 13932.58kB 38.75% 38.75% 13932.58kB 38.75% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer 3584.50kB 9.97% 48.72% 3584.50kB 9.97% github.com/go-text/typesetting/opentype/tables.(*SimpleGlyph).parsePoints 2707.76kB 7.53% 56.26% 2707.76kB 7.53% compress/flate.NewWriter 2560.18kB 7.12% 63.38% 6656.70kB 18.52% github.com/go-text/typesetting/opentype/tables.(*Glyph).parseData 2384.64kB 6.63% 70.01% 2384.64kB 6.63% github.com/go-text/typesetting/opentype/api/font/cff.(*cffParser).parseIndex 1131.52kB 3.15% 73.16% 7788.22kB 21.66% github.com/go-text/typesetting/opentype/tables.ParseGlyf 632.14kB 1.76% 74.92% 1144.27kB 3.18% fyne.io/fyne/v2/internal/cache.SetCanvasForObject 600.58kB 1.67% 76.59% 600.58kB 1.67% github.com/go-playground/validator/v10.map.init.4 574.34kB 1.60% 78.18% 574.34kB 1.60% flexyserver/services/serviceat.(*ATService).readUntilEmpty 570.04kB 1.59% 79.77% 17542.25kB 48.79% fyne.io/fyne/v2/internal/cache.Renderer
Definition of done: what must be done to consider the task complete?
i created an issue to
https://github.com/klauspost/compress/issues/928
he said the issue is with the mongodb client library and not with his library
The exact Go version used, with patch level:
$ go version
go version go1.21.1 linux/amd64
The exact version of the Go driver used:
$ go list -m go.mongodb.org/mongo-driver
go.mongodb.org/mongo-driver v1.13.1
Describe how MongoDB is set up. Local vs Hosted, version, topology, load balanced, etc.
mongodb server local ( replicaset )
The operating system and version (e.g. Windows 7, OSX 10.8, ...)
ubuntu 22.04