[SERVER-60594] mongos 4.4.8 slow on mongos,but fast on mongod Created: 11/Oct/21 Updated: 14/May/22 Resolved: 28/Jan/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | jing xu | Assignee: | Edwin Zhou |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Operating System: | ALL |
| Participants: |
| Description |
|
i upgrade from 4.4.4 to 4.4.8 cluster. i check io,mem,cpu, there is no stress on it. i guess exists problem on mongos. mongos slow log: ,"s":"I", "c":"COMMAND", "id":51803, "ctx":"conn3042892","msg":"Slow query","attr":{"type":"command","ns":"wzjMonitor.expWzjMonitor","appName":"wzj_op","command":{"find":"expWzjMonitor","filter": {"no":"YT3224052005840"},"projection": {"no":1,"del":1,"curT":1,"oType":1,"matOrg":1,"_id":0},"$db":"wzjMonitor","$clusterTime":{"clusterTime":{"$timestamp":{"t":1633958124,"i":152}},"signature":{"hash":{"$binary":{"base64":"XbmmyAeBvyjB0IIpxpqKars52yg=","subType":"0"}},"keyId":6958985885933109266}},"lsid":{"id": {"$uuid":"5b593b65-b82d-40e8-8c94-682d9e4abc3d"}}},"nShards":1,"cursorExhausted":true,"numYields":0,"nreturned":1,"reslen":323,"protocol":"op_msg","durationMillis":1080}} but i execute it just 2ms. ).projection( {"no":1,"del":1,"curT":1,"oType":1,"matOrg":1,"_id":0}).explain("executionStats") , }, , }, , } , }, , , , } |
| Comments |
| Comment by jing xu [ 14/May/22 ] |
|
hi edwin zhou: ,"s":"I", "c":"COMMAND", "id":51803, ,{"createTime":{"$gte": {"$date":"2022-04-14T05:24:20.968Z"}}}, }}]},"sort": {"createTime":-1}, more shard1.log ,"s":"I", "c":"COMMAND", "id":51803, "ctx":"conn410512","msg":"Slow query", , }},{"createTime":{"$lte": {"$date":"2022-05-15T05:24:20.968Z"}}}]}, , |
| Comment by Edwin Zhou [ 28/Jan/22 ] |
|
Hi 601290552@qq.com, We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket. Best, |
| Comment by Edwin Zhou [ 07/Jan/22 ] |
|
Hi 601290552@qq.com, We still need additional information to diagnose the problem. If this is still an issue for you, would you please let us know if performance has improved after disabling the serviceExecutro setting? Best, |
| Comment by Edwin Zhou [ 22/Dec/21 ] |
|
Hi 601290552@qq.com, I tremendously appreciate your patience and I apologize for the wait as I investigate this issue. I believe that the reason for increased latency on the mongos is because serviceExecutor is set to adaptive. I recommend removing the serviceExecutor setting on all of your config files. We're interesting in hearing if performance improves after this setting is disabled. Best, |
| Comment by jing xu [ 18/Dec/21 ] |
|
Hi Edwin: |
| Comment by jing xu [ 20/Nov/21 ] |
|
Hi Edwin: for example: [root@~]# curl -X POST https://upload.box.com/api/2.0/files/content \
|
| Comment by Edwin Zhou [ 19/Nov/21 ] |
|
Hi 601290552@qq.com, I deeply apologize for the delay in my response. Unfortunately, I wasn't able to find any diagnostic data in the upload portal and it appears that the link has now expired. Here is another support uploader link. Best, |
| Comment by jing xu [ 12/Nov/21 ] |
|
Hi Edwin: |
| Comment by jing xu [ 06/Nov/21 ] |
|
collection expInfos read write latencyStats for all shard less than 1ms. , , , } , , , } , , , } , , , } , , , } , , , } |
| Comment by jing xu [ 06/Nov/21 ] |
|
Hi Edwin: for example the collection is expInfos from 2021.11.05 00:00:00 to 2021.11.06 00:00:00 there are six shards,every shard slow log is near the same. so mongos has bottleneck. |
| Comment by jing xu [ 05/Nov/21 ] |
|
Hi Edwin: |
| Comment by Edwin Zhou [ 04/Nov/21 ] |
|
Hi 601290552@qq.com, We still need additional information to diagnose the problem. If this is still an issue for you, would you please upload mongod.log files from the requested nodes covering the incident and the $dbpath/diagnostic.data? Best, |
| Comment by Edwin Zhou [ 18/Oct/21 ] |
|
Hi 601290552@qq.com, To investigate this as a possible bug, we'd like information from the following nodes in the cluster:
For each of these nodes, please archive (tar or zip) the mongod.log files covering the incident and the $dbpath/diagnostic.data directory (the contents are described here) I've created a secure upload portal for you. Files uploaded to this portal are hosted on Box, are visible only to MongoDB employees. Best, |
| Comment by jing xu [ 11/Oct/21 ] |
|
i upgrade from 4.4.4 to 4.4.8 cluster. i check io,mem,cpu, there is no stress on it. i guess exists problem on mongos. mongos slow log: ,"s":"I", "c":"COMMAND", "id":51803, "ctx":"conn3042892","msg":"Slow query","attr":{"type":"command","ns":"wzjMonitor.expWzjMonitor","appName":"wzj_op","command":{"find":"expWzjMonitor","filter": {"no":"103224052005840"},"projection": {"no":1,"del":1,"curT":1,"oType":1,"matOrg":1,"_id":0},"$db":"wzjMonitor","$clusterTime":{"clusterTime":{"$timestamp":{"t":1633958124,"i":152}},"signature":{"hash":{"$binary":{"base64":"XbmmyAeBvyjB0IIpxpqKars52yg=","subType":"0"}},"keyId":6958985885933109266}},"lsid":{"id": {"$uuid":"5b593b65-b82d-40e8-8c94-682d9e4abc3d"}}},"nShards":1,"cursorExhausted":true,"numYields":0,"nreturned":1,"reslen":323,"protocol":"op_msg","durationMillis":1080}} but i execute it just 2ms. ).projection( {"no":1,"del":1,"curT":1,"oType":1,"matOrg":1,"_id":0}).explain("executionStats") , }, , }, , } , }, , , , } |