[GODRIVER-1176] Data race somewhere in bsoncodec Created: 01/Jul/19  Updated: 03/Jul/19  Resolved: 02/Jul/19

Status: Closed
Project: Go Driver
Component/s: BSON, JSON & ExtJSON
Affects Version/s: 1.0.3
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Anton Kucherov Assignee: Divjot Arora (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

go version go1.12.6 darwin/amd64



 Description   

Hi. I'm not sure, maybe I have some problems with my code. But it seems something wrong in bson encoder/decoder.
At least for this moment, I couldn't represent this race with a simple test.  But I'll try to do it. Here is race detector log.

==================
WARNING: DATA RACE
Write at 0x00c000534080 by goroutine 86:
  reflect.typedmemmove()
      /usr/local/Cellar/go/1.12.6/libexec/src/runtime/mbarrier.go:177 +0x0
  reflect.Value.Set()
      /usr/local/Cellar/go/1.12.6/libexec/src/reflect/value.go:1475 +0xe5
  reflect.Append()
      /usr/local/Cellar/go/1.12.6/libexec/src/reflect/value.go:1964 +0x153
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.DefaultValueDecoders.SliceDecodeValue()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/default_value_decoders.go:782 +0x629
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.DefaultValueDecoders.SliceDecodeValue-fm()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/default_value_decoders.go:745 +0xa3
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.ValueDecoderFunc.DecodeValue()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/bsoncodec.go:156 +0xab
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.(*StructCodec).DecodeValue()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/struct_codec.go:204 +0x6e6
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.(*Decoder).Decode()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/decoder.go:92 +0x2e3
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.unmarshalFromReader()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/unmarshal.go:100 +0x137
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.UnmarshalWithRegistry()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/unmarshal.go:45 +0x1b5
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler/collector.connectAndCollect()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/mongo/cursor.go:123 +0x76f
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler/collector.start()
      pmm-agent/agents/mongodb/internal/profiler/collector/collector.go:186 +0x10ePrevious read at 0x00c000534080 by goroutine 88:
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.DefaultValueEncoders.SliceEncodeValue()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/default_value_encoders.go:390 +0x3ba
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.DefaultValueEncoders.SliceEncodeValue-fm()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/default_value_encoders.go:372 +0x83
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.ValueEncoderFunc.EncodeValue()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/bsoncodec.go:142 +0x8b
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.(*StructCodec).EncodeValue()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/struct_codec.go:97 +0x6ff
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.(*Encoder).Encode()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/encoder.go:79 +0x210
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.MarshalExtJSONAppendWithContext()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/marshal.go:150 +0x247
  github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.MarshalExtJSON()
      pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/marshal.go:113 +0xe5
  github.com/percona/pmm-agent/vendor/github.com/percona/percona-toolkit/src/go/mongolib/stats.(*Stats).Add()
      pmm-agent/vendor/github.com/percona/percona-toolkit/src/go/mongolib/stats/stats.go:78 +0xf4f
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler/aggregator.(*Aggregator).Add()
      pmm-agent/agents/mongodb/internal/profiler/aggregator/aggregator.go:111 +0x374
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler/parser.start()
      pmm-agent/agents/mongodb/internal/profiler/parser/parser.go:141 +0x245Goroutine 86 (running) created at:
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler/collector.(*Collector).Start()
      pmm-agent/agents/mongodb/internal/profiler/collector/collector.go:94 +0x593
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler.(*monitor).Start()
      pmm-agent/agents/mongodb/internal/profiler/monitor.go:77 +0x25b
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler.(*monitors).MonitorAll()
      pmm-agent/agents/mongodb/internal/profiler/monitors.go:89 +0x4d8
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler.start()
      pmm-agent/agents/mongodb/internal/profiler/profiler.go:207 +0xaaGoroutine 88 (running) created at:
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler/parser.(*Parser).Start()
      pmm-agent/agents/mongodb/internal/profiler/parser/parser.go:71 +0x352
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler.(*monitor).Start()
      pmm-agent/agents/mongodb/internal/profiler/monitor.go:85 +0x43b
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler.(*monitors).MonitorAll()
      pmm-agent/agents/mongodb/internal/profiler/monitors.go:89 +0x4d8
  github.com/percona/pmm-agent/agents/mongodb/internal/profiler.start()
      pmm-agent/agents/mongodb/internal/profiler/profiler.go:207 +0xaa
==================



 Comments   
Comment by Anton Kucherov [ 03/Jul/19 ]

Yeah, you are right. I've found concurrent call. Thanks.

Comment by Divjot Arora (Inactive) [ 02/Jul/19 ]

dexterhd Based on the stack traces, it seems like there are concurrent calls to bson.MarshalExtJSON and bson.UnmarshalWithRegistry, which is triggering a data race on a slice field in that struct. This is not supported behavior, so I'm closing this ticket.

Comment by Anton Kucherov [ 01/Jul/19 ]

Can't edit a ticket. Here is full DATARACE log with normal formatting.

==================
WARNING: DATA RACE
Write at 0x00c000534080 by goroutine 86:
 reflect.typedmemmove()
 /usr/local/Cellar/go/1.12.6/libexec/src/runtime/mbarrier.go:177 +0x0
 reflect.Value.Set()
 /usr/local/Cellar/go/1.12.6/libexec/src/reflect/value.go:1475 +0xe5
 reflect.Append()
 /usr/local/Cellar/go/1.12.6/libexec/src/reflect/value.go:1964 +0x153
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.DefaultValueDecoders.SliceDecodeValue()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/default_value_decoders.go:782 +0x629
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.DefaultValueDecoders.SliceDecodeValue-fm()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/default_value_decoders.go:745 +0xa3
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.ValueDecoderFunc.DecodeValue()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/bsoncodec.go:156 +0xab
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.(*StructCodec).DecodeValue()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/struct_codec.go:204 +0x6e6
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.(*Decoder).Decode()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/decoder.go:92 +0x2e3
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.unmarshalFromReader()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/unmarshal.go:100 +0x137
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.UnmarshalWithRegistry()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/unmarshal.go:45 +0x1b5
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler/collector.connectAndCollect()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/mongo/cursor.go:123 +0x76f
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler/collector.start()
 pmm-agent/agents/mongodb/internal/profiler/collector/collector.go:186 +0x10e
 
Previous read at 0x00c000534080 by goroutine 88:
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.DefaultValueEncoders.SliceEncodeValue()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/default_value_encoders.go:390 +0x3ba
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.DefaultValueEncoders.SliceEncodeValue-fm()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/default_value_encoders.go:372 +0x83
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.ValueEncoderFunc.EncodeValue()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/bsoncodec.go:142 +0x8b
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec.(*StructCodec).EncodeValue()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/bsoncodec/struct_codec.go:97 +0x6ff
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.(*Encoder).Encode()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/encoder.go:79 +0x210
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.MarshalExtJSONAppendWithContext()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/marshal.go:150 +0x247
 github.com/percona/pmm-agent/vendor/go.mongodb.org/mongo-driver/bson.MarshalExtJSON()
 pmm-agent/vendor/go.mongodb.org/mongo-driver/bson/marshal.go:113 +0xe5
 github.com/percona/pmm-agent/vendor/github.com/percona/percona-toolkit/src/go/mongolib/stats.(*Stats).Add()
 pmm-agent/vendor/github.com/percona/percona-toolkit/src/go/mongolib/stats/stats.go:78 +0xf4f
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler/aggregator.(*Aggregator).Add()
 pmm-agent/agents/mongodb/internal/profiler/aggregator/aggregator.go:111 +0x374
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler/parser.start()
 pmm-agent/agents/mongodb/internal/profiler/parser/parser.go:141 +0x245
 
Goroutine 86 (running) created at:
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler/collector.(*Collector).Start()
 pmm-agent/agents/mongodb/internal/profiler/collector/collector.go:94 +0x593
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler.(*monitor).Start()
 pmm-agent/agents/mongodb/internal/profiler/monitor.go:77 +0x25b
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler.(*monitors).MonitorAll()
 pmm-agent/agents/mongodb/internal/profiler/monitors.go:89 +0x4d8
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler.start()
 pmm-agent/agents/mongodb/internal/profiler/profiler.go:207 +0xaa
 
Goroutine 88 (running) created at:
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler/parser.(*Parser).Start()
 pmm-agent/agents/mongodb/internal/profiler/parser/parser.go:71 +0x352
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler.(*monitor).Start()
 pmm-agent/agents/mongodb/internal/profiler/monitor.go:85 +0x43b
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler.(*monitors).MonitorAll()
 pmm-agent/agents/mongodb/internal/profiler/monitors.go:89 +0x4d8
 github.com/percona/pmm-agent/agents/mongodb/internal/profiler.start()
 pmm-agent/agents/mongodb/internal/profiler/profiler.go:207 +0xaa
==================

Generated at Thu Feb 08 08:35:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.