-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: 1.13.1
-
Component/s: None
-
None
-
Go Drivers
-
Not Needed
-
Detailed steps to reproduce the problem?
create a simple app that use zstd as Compressors
ClientOptions := options.Client().ApplyURI(str).SetCompressors([]string{"zstd"}).SetBSONOptions(bsonOpts).SetServerSelectionTimeout(30 * time.Second)
var err error
config.Client, err = mongo.Connect(context.Background(), ClientOptions)
collection := config.Client.Database(config.MainConf.Main_db).Collection("flexyoperations")
changeStream, errChangeStream := collection.Watch(context.Background(), mongo.Pipeline{
bson.D{{Key: "$match", Value: bson.M{"operationType": "insert"}}},
})
a simple function that save a struct
func SaveServer() {
mutexSave.Lock()
defer mutexSave.Unlock()
server.LastSeen = time.Now()
logger.DBL.Info("DB Server Save => Called ", zap.Any("server", server))
if config.Client == nil {
ServerStatus.Set("Pending")
dialog.ShowError(errors.New("not Connected"), config.Window)
logger.DBL.Error("DB Server Load => Client is nil ")
return
}
_, err := config.Client.Database(config.MainConf.Main_db).Collection("flexyservers").UpdateOne(context.TODO(), bson.M{"server_name": config.MainConf.Server_name},
bson.M{"$set": server})
if err != nil {
logger.DBL.Error("DB Server SaveServer => Error: ", zap.Any("err", err))
return
}
}
just after calling saveserver 5- 10 time i can see huge memory usage , it's can go up to 2.5gb after my app run for few hours when i start debuging using pprof i can see the following just calling save few times
flat flat% sum% cum cum%
256MB 77.50% 77.50% 256MB 77.50% github.com/klauspost/compress/zstd.(*fastBase).ensureHist (inline)
48.86MB 14.79% 92.29% 48.86MB 14.79% github.com/klauspost/compress/zstd.encoderOptions.encoder
10.99MB 3.33% 95.62% 10.99MB 3.33% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer
2.26MB 0.68% 96.31% 2.26MB 0.68% github.com/klauspost/compress/zstd.(*blockEnc).init
2MB 0.61% 96.91% 4MB 1.21% github.com/go-text/typesetting/opentype/tables.(*Glyph).parseData
0 0% 96.91% 32MB 9.69% flexyserver/Repositories.LoadOffers
0 0% 96.91% 32.57MB 9.86% flexyserver/Repositories.LoadServer
0 0% 96.91% 161.70MB 48.95% flexyserver/Repositories.SaveServer
0 0% 96.91% 81.36MB 24.63% flexyserver/services/db.Connect
0 0% 96.91% 11.91MB 3.61% flexyserver/ui.ShowMainUI
simply changing from zstd to snappy or zlib resolve my issue and my app doesn't exceed 150mb memory usage
flat flat% sum% cum cum%
13932.58kB 38.75% 38.75% 13932.58kB 38.75% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer
3584.50kB 9.97% 48.72% 3584.50kB 9.97% github.com/go-text/typesetting/opentype/tables.(*SimpleGlyph).parsePoints
2707.76kB 7.53% 56.26% 2707.76kB 7.53% compress/flate.NewWriter
2560.18kB 7.12% 63.38% 6656.70kB 18.52% github.com/go-text/typesetting/opentype/tables.(*Glyph).parseData
2384.64kB 6.63% 70.01% 2384.64kB 6.63% github.com/go-text/typesetting/opentype/api/font/cff.(*cffParser).parseIndex
1131.52kB 3.15% 73.16% 7788.22kB 21.66% github.com/go-text/typesetting/opentype/tables.ParseGlyf
632.14kB 1.76% 74.92% 1144.27kB 3.18% fyne.io/fyne/v2/internal/cache.SetCanvasForObject
600.58kB 1.67% 76.59% 600.58kB 1.67% github.com/go-playground/validator/v10.map.init.4
574.34kB 1.60% 78.18% 574.34kB 1.60% flexyserver/services/serviceat.(*ATService).readUntilEmpty
570.04kB 1.59% 79.77% 17542.25kB 48.79% fyne.io/fyne/v2/internal/cache.Renderer
Definition of done: what must be done to consider the task complete?
i created an issue to
https://github.com/klauspost/compress/issues/928
he said the issue is with the mongodb client library and not with his library
The exact Go version used, with patch level:
$ go version
go version go1.21.1 linux/amd64
The exact version of the Go driver used:
$ go list -m go.mongodb.org/mongo-driver
go.mongodb.org/mongo-driver v1.13.1
Describe how MongoDB is set up. Local vs Hosted, version, topology, load balanced, etc.
mongodb server local ( replicaset )
The operating system and version (e.g. Windows 7, OSX 10.8, ...)
ubuntu 22.04