[SERVER-47714] Secondary asserts on system.profile collection with WiredTigerRecordStore::insertRecord 95: Operation not supported Created: 23/Apr/20  Updated: 29/Oct/23  Resolved: 15/Jul/20

Status: Closed
Project: Core Server
Component/s: Catalog
Affects Version/s: 4.2.8
Fix Version/s: 4.2.9

Type: Bug Priority: Major - P3
Reporter: Dmitry Ryabtsev Assignee: Eric Milkie
Resolution: Fixed Votes: 5
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File metrics.2020-05-21T00-57-03Z-00000     File mongod.log.bak    
Issue Links:
Depends
is depended on by SERVER-49531 forward port profile.js to master branch Closed
Duplicate
is duplicated by SERVER-48210 create profile collection failed Closed
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

1. Create two member replica set
2. Set priority to 0 on the member that we're going to reproduce the problem on

var cfg = rs.conf(); cfg.members[0].priority = 0; rs.reconfig(cfg)
db.getSIblingDB("test").t.insert({a});

3. On the member with _id: 0 add this to the configuration file:

operationProfiling:
  mode: "slowOp"
  slowOpThresholdMs: 0

4. Stop and re-start the _id: 0 member
5. On the restarted member run a query to trigger the system.profile collection creation

rs.slaveOk()
db.getSiblingDB("test").t.find()

6. Check if the system.profile collection was created:

db.getSiblingDB("test").system.profile.stats()

7. Check if you see messages similar to the following in the mongod log

2020-04-23T15:09:36.353+1000 I  STORAGE  [conn28] createCollection: test.system.profile with generated UUID: 0ab5602f-7076-4986-938b-8d0dc1bd7c36 and options: { capped: true, size: 1048576 }
2020-04-23T15:09:36.354+1000 E  STORAGE  [conn28] WiredTiger error (95) [1587618576:353988][3843:0x7ff7c8688700], file:_mdb_catalog.wt, WT_CURSOR.insert: __wt_txn_modify, 418: Transactions with ignore_prepare=true cannot perform updates: Operation not supported Raw: [1587618576:353988][3843:0x7ff7c8688700], file:_mdb_catalog.wt, WT_CURSOR.insert: __wt_txn_modify, 418: Transactions with ignore_prepare=true cannot perform updates: Operation not supported
2020-04-23T15:09:36.354+1000 E  -        [conn28] Assertion: UnknownError: WiredTigerRecordStore::insertRecord 95: Operation not supported src/mongo/db/catalog/database_impl.cpp 662
2020-04-23T15:09:36.382+1000 W  -        [conn28] Caught Assertion while trying to profile msg against test.t: UnknownError: WiredTigerRecordStore::insertRecord 95: Operation not supported

Sprint: Execution Team 2020-06-15, Execution Team 2020-06-29, Execution Team 2020-07-13, Execution Team 2020-07-27
Participants:
Case:
Linked BF Score: 5

 Description   

I've managed to reproduce this behaviour on v4.2.1 and 4.2.6 (latest).

If a replica set member is started with the database profiler enabled in the configuration it will keep asserting failing to create system.profile collection.

Turning profiling on and off makes the errors cease:

db.setProfilingLevel(0,100);
db.setProfilingLevel(1,0);



 Comments   
Comment by Dejun Xia [ 16/Jul/20 ]

@Eric Milkie Thanks for for the work, we will upgrade to the new version soon.

Comment by Githook User [ 15/Jul/20 ]

Author:

{'name': 'Eric Milkie', 'email': 'milkie@10gen.com', 'username': 'milkie'}

Message: SERVER-47714 fix test
Branch: v4.2
https://github.com/mongodb/mongo/commit/02e0fc0714f54e4a53895699411c3d19f776dffe

Comment by Githook User [ 15/Jul/20 ]

Author:

{'name': 'Eric Milkie', 'email': 'milkie@10gen.com', 'username': 'milkie'}

Message: SERVER-47714 explicitly set the prepare conflict behavior prior to creating a system.profile collection
Branch: v4.2
https://github.com/mongodb/mongo/commit/6377d98fadd8ab5c10eb1e0bb240dc4151fda316

Comment by Eric Milkie [ 14/Jul/20 ]

I can reproduce the issue and am now working on a fix.

Comment by Dejun Xia [ 10/Jul/20 ]

Hello, everyone,

I wonder when this issue will be fixed in 4.2 bugfix version, we have encountered many times of this problem in Alibaba MongoDB Service.Looking forward to your reply, thanks.

Comment by Cen Zheng [ 21/May/20 ]

Hi, 

We found not only secondary nodes will encounter this issue, primary node will also. I have uploaded both mongod.log and related FTDC file. Please help investigate this issue.

Thanks!

Generated at Thu Feb 08 05:15:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.