[SERVER-68771] Error while update with pymongo driver Created: 12/Aug/22  Updated: 12/Aug/22  Resolved: 12/Aug/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 3.2.8
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: jipeng N/A Assignee: Chris Kelly
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File backtrace.log    
Issue Links:
Duplicate
is duplicated by SERVER-68770 Error while updating with pymongo driver Closed
Operating System: ALL
Participants:

 Description   

pymongo log

 

  File "/usr/lib/python2.7/site-packages/zbs/lib/mongo/proxy.py", line 89, in __call__
    return self.method(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 2237, in update
    check_keys, multi, manipulate, write_concern)
  File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 710, in _update
    _check_write_command_response([(0, result)])
  File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 301, in _check_write_command_response
    raise WriteError(error.get("errmsg"), error.get("code"), error) 

 

 

mongod.log

 

2022-08-09T16:42:22.920+0800 I -        [conn355] Assertion failure type == cbindata src/mongo/db/storage/mmap_v1/btree/key.cpp 612
2022-08-09T16:42:22.929+0800 I CONTROL  [conn355] 
 0xaaaab7715e9c 0xaaaab76b7b50 0xaaaab76a4230 0xaaaab74514e0 0xaaaab74420e0 0xaaaab744e668 0xaaaab744f140 0xaaaab7439b10 0xaaaab710fe80 0xaaaab6f64f18 0xaaaab7091d9c 0xaaaab7092710 0xaaaab7293df4 0xaaaab729428c 0xaaaab7294394 0xaaaab70129a4 0xaaaab701446c 0xaaaab70147e8 0xaaaab7017438 0xaaaab702fa78 0xaaaab70307c8 0xaaaab6f997c0 0xaaaab712d628 0xaaaab712f824 0xaaaab6e3b070 0xaaaab76c6c54 0xffff7fa47d38 0xffff7f98f5f0
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"AAAAB69F0000","o":"D25E9C","s":"_ZN5mongo15printStackTraceERSo"},{"b":"AAAAB69F0000","o":"CC7B50","s":"_ZN5mongo10logContextEPKc"},{"b":"AAAAB69F0000","o":"CB4230","s":"_ZN5mongo12verifyFailedEPKcS1_j"},{"b":"AAAAB69F0000","o":"A614E0","s":"_ZNK5mongo5KeyV18dataSizeEv"},{"b":"AAAAB69F0000","o":"A520E0","s":"_ZN5mongo10BtreeLogicINS_13BtreeLayoutV1EE15_packedDataSizeEPNS_13BtreeBucketV1Ei"},{"b":"AAAAB69F0000","o":"A5E668","s":"_ZN5mongo10BtreeLogicINS_13BtreeLayoutV1EE23mayBalanceWithNeighborsEPNS_16OperationContextEPNS_13BtreeBucketV1ENS_7DiskLocE"},{"b":"AAAAB69F0000","o":"A5F140","s":"_ZN5mongo10BtreeLogicINS_13BtreeLayoutV1EE7unindexEPNS_16OperationContextERKNS_7BSONObjERKNS_7DiskLocE"},{"b":"AAAAB69F0000","o":"A49B10"},{"b":"AAAAB69F0000","o":"71FE80","s":"_ZN5mongo17IndexAccessMethod6updateEPNS_16OperationContextERKNS_12UpdateTicketEPl"},{"b":"AAAAB69F0000","o":"574F18","s":"_ZN5mongo10Collection14updateDocumentEPNS_16OperationContextERKNS_8RecordIdERKNS_11SnapshottedINS_7BSONObjEEERKS7_bbPNS_7OpDebugERNS_20oplogUpdateEntryArgsE"},{"b":"AAAAB69F0000","o":"6A1D9C","s":"_ZN5mongo11UpdateStage18transformAndUpdateERKNS_11SnapshottedINS_7BSONObjEEERNS_8RecordIdE"},{"b":"AAAAB69F0000","o":"6A2710","s":"_ZN5mongo11UpdateStage4workEPm"},{"b":"AAAAB69F0000","o":"8A3DF4","s":"_ZN5mongo12PlanExecutor11getNextImplEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE"},{"b":"AAAAB69F0000","o":"8A428C","s":"_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE"},{"b":"AAAAB69F0000","o":"8A4394","s":"_ZN5mongo12PlanExecutor11executePlanEv"},{"b":"AAAAB69F0000","o":"6229A4","s":"_ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE"},{"b":"AAAAB69F0000","o":"62446C","s":"_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE"},{"b":"AAAAB69F0000","o":"6247E8","s":"_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE"},{"b":"AAAAB69F0000","o":"627438","s":"_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderE"},{"b":"AAAAB69F0000","o":"63FA78","s":"_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE"},{"b":"AAAAB69F0000","o":"6407C8","s":"_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE"},{"b":"AAAAB69F0000","o":"5A97C0","s":"_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE"},{"b":"AAAAB69F0000","o":"73D628"},{"b":"AAAAB69F0000","o":"73F824","s":"_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE"},{"b":"AAAAB69F0000","o":"44B070","s":"_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE"},{"b":"AAAAB69F0000","o":"CD6C54","s":"_ZN5mongo17PortMessageServer17handleIncomingMsgEPv"},{"b":"FFFF7FA40000","o":"7D38"},{"b":"FFFF7F8B0000","o":"DF5F0"}],"processInfo":{ "mongodbVersion" : "3.2.8", "gitVersion" : "ed70e33130c977bda0024c125b56d159573dbaf0", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.18.0-193.28.1.el7.smartx.2.aarch64", "version" : "#1 SMP Fri Feb 26 06:17:05 UTC 2021", "machine" : "aarch64" }, "somap" : [ { "b" : "AAAAB69F0000", "elfType" : 3, "buildId" : "F73E521CCE2BB96C75C343ECD64F905C403E99DD" }, { "b" : "FFFF80D30000", "elfType" : 3, "buildId" : "4E2598E59BD41F139BC3DB092D23E32EEDCB424D" }, { "b" : "FFFF80CD0000", "path" : "/lib64/libboost_filesystem.so.1.59.0", "elfType" : 3, "buildId" : "9B3E7D2DF68DE2ACEB9D5A338680E1C5A3837DF2" }, { "b" : "FFFF80CA0000", "path" : "/lib64/libboost_system.so.1.59.0", "elfType" : 3, "buildId" : "F67FFFE6D3CF8A1872BCCB22678615EA33AC0551" }, { "b" : "FFFF80C70000", "path" : "/lib64/libboost_chrono.so.1.59.0", "elfType" : 3, "buildId" : "956FA00583F66289CAAA79525D19E714CD6E1A9E" }, { "b" : "FFFF80BD0000", "path" : "/lib64/libboost_program_options.so.1.59.0", "elfType" : 3, "buildId" : "1E8766D28CC18C2C5133FCD97838F0D6965767FA" }, { "b" : "FFFF80AB0000", "path" : "/lib64/libboost_regex.so.1.59.0", "elfType" : 3, "buildId" : "FFF84CCDB4F673936E4CBFB82BC9AA30ED19A83F" }, { "b" : "FFFF80A70000", "path" : "/lib64/libboost_thread.so.1.59.0", "elfType" : 3, "buildId" : "EE1CACED02B2AD6993B46A69926FA00D1BF6483D" }, { "b" : "FFFF80A10000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "8051CD689AD871A6D003310DDFCDF275B76002D0" }, { "b" : "FFFF809E0000", "path" : "/lib64/libpcrecpp.so.0", "elfType" : 3, "buildId" : "83D91599D2979737C68104F11A278D5E0A1196D5" }, { "b" : "FFFF80940000", "path" : "/lib64/libboost_program_options-mt.so.1.53.0", "elfType" : 3, "buildId" : "AC1D20F6802B2E9A85AD8C3D7B07889740F8CA3F" }, { "b" : "FFFF80900000", "path" : "/lib64/libboost_filesystem-mt.so.1.53.0", "elfType" : 3, "buildId" : "E03C7FABFC031EBD33C84604152C0B8C1B15E9B4" }, { "b" : "FFFF808C0000", "path" : "/lib64/libboost_thread-mt.so.1.53.0", "elfType" : 3, "buildId" : "CF8E4F141239561BCE99324B6A321F6B4D771B4B" }, { "b" : "FFFF80890000", "path" : "/lib64/libboost_system-mt.so.1.53.0", "elfType" : 3, "buildId" : "14F60C7977ED4590A5C28934162277D9873192B6" }, { "b" : "FFFF80860000", "path" : "/lib64/libboost_chrono-mt.so.1.53.0", "elfType" : 3, "buildId" : "CE4D9411D5807398B69168B70ADEA2DC76A42291" }, { "b" : "FFFF80740000", "path" : "/lib64/libboost_regex-mt.so.1.53.0", "elfType" : 3, "buildId" : "D2170B8FEBF5B8EF072126761F3C9430DA892635" }, { "b" : "FFFF806A0000", "path" : "/lib64/libyaml-cpp.so.0.5", "elfType" : 3, "buildId" : "2AD8533C091300A162DD3DD8CD9D7E3D4539E656" }, { "b" : "FFFF80660000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "EB69FAD02E1DBBC57B8732FD6FF9ACE3C119D565" }, { "b" : "FFFF805F0000", "path" : "/lib64/libstemmer.so.0", "elfType" : 3, "buildId" : "6FC348A358E1AF0F190725CEF0A6A90162A28F55" }, { "b" : "FFFF7FFE0000", "path" : "/lib64/libmozjs-38.so", "elfType" : 3, "buildId" : "19418685C71AE68D0BDF19573FC1F328F5F37A5F" }, { "b" : "FFFF7FFB0000", "path" : "/lib64/libsnappy.so.1", "elfType" : 3, "buildId" : "C60FD94C122062DF86A05A5DAB2399F646B13DFE" }, { "b" : "FFFF7FF20000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "6A10B1CE159E1A75A3C4EB4AD966570577E61C7E" }, { "b" : "FFFF7FD10000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "9486A480187119C2A55240E6CF0FB4CC276447EB" }, { "b" : "FFFF7FCE0000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "10D04AB351A5B9109B261B2E011A334565E4A14C" }, { "b" : "FFFF7FCB0000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "29BF038F8FAFD184ACE6C6FB45ABE9963B8956F9" }, { "b" : "FFFF7FB80000", "path" : "/lib64/libstdc++.so.6", "elfType" : 3, "buildId" : "188CF7F6CF7FFD2078ECFDA395FA5849AC153946" }, { "b" : "FFFF7FAC0000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "44CD1FBCDA5368A3D90AF239FEEBAB59530F6C56" }, { "b" : "FFFF7FA80000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "4C75FE128BFD08DB49A4712B92400DFA83079A33" }, { "b" : "FFFF7FA40000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "3589E1F6842B5C6C4AA29B356FBE7947E4997B27" }, { "b" : "FFFF7F8B0000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "6D939E12FFD3F6F61FF3DE0B0E5D82FA10BF3E19" }, { "b" : "FFFF80D40000", "path" : "/lib/ld-linux-aarch64.so.1", "elfType" : 3, "buildId" : "2AC0990B34BC9094B563813C0F10CCF8CA32740B" }, { "b" : "FFFF7E4B0000", "path" : "/lib64/libicudata.so.50", "elfType" : 3, "buildId" : "26984DD9089E365B7A27ABA4C3AE90780F6DFDBF" }, { "b" : "FFFF7E290000", "path" : "/lib64/libicui18n.so.50", "elfType" : 3, "buildId" : "9CF07E3FC924687F88E0F56EB3BC98003CA221DE" }, { "b" : "FFFF7E0F0000", "path" : "/lib64/libicuuc.so.50", "elfType" : 3, "buildId" : "2C7497527887C581F126A5387483B4FE73F3233C" }, { "b" : "FFFF7E0C0000", "path" : "/lib64/libplds4.so", "elfType" : 3, "buildId" : "D5B40662A6888D1F6315FBB426F334745D09FB2B" }, { "b" : "FFFF7E090000", "path" : "/lib64/libplc4.so", "elfType" : 3, "buildId" : "C40AB22E65CEBF60866363A657B9268135171BD5" }, { "b" : "FFFF7E030000", "path" : "/lib64/libnspr4.so", "elfType" : 3, "buildId" : "7A534F7B3F86E011894C038A96CE620ECA9CE9AB" }, { "b" : "FFFF7DFC0000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "5FFEA9D93B7940093C2F69213F7FE93EB0293747" }, { "b" : "FFFF7DEB0000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "73412DAF21A3B9805915383746C7013DA4E8F8E8" }, { "b" : "FFFF7DE80000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "2D8692824EA0862EAB8A0F020D3BF53F96A88C5C" }, { "b" : "FFFF7DE20000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "99F5AB86F3CEC1F86421AA1A91FBBF86F4138C2A" }, { "b" : "FFFF7DDF0000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "B95DA6E2733FDD102C6B15CD8FC5A79F9718366A" }, { "b" : "FFFF7DDD0000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "D460CF6B956BE51643802523054EAF4CBBBF9376" }, { "b" : "FFFF7DD90000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "B7C5E25B2D981DBF38E8780B745F5FB755399B7B" }, { "b" : "FFFF7DD40000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "B790A085B8A07572F377F7548F23F58BFBDD4EE6" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x48) [0xaaaab7715e9c]
 mongod(_ZN5mongo10logContextEPKc+0x140) [0xaaaab76b7b50]
 mongod(_ZN5mongo12verifyFailedEPKcS1_j+0xC0) [0xaaaab76a4230]
 mongod(_ZNK5mongo5KeyV18dataSizeEv+0x13C) [0xaaaab74514e0]
 mongod(_ZN5mongo10BtreeLogicINS_13BtreeLayoutV1EE15_packedDataSizeEPNS_13BtreeBucketV1Ei+0x94) [0xaaaab74420e0]
 mongod(_ZN5mongo10BtreeLogicINS_13BtreeLayoutV1EE23mayBalanceWithNeighborsEPNS_16OperationContextEPNS_13BtreeBucketV1ENS_7DiskLocE+0x4C) [0xaaaab744e668]
 mongod(_ZN5mongo10BtreeLogicINS_13BtreeLayoutV1EE7unindexEPNS_16OperationContextERKNS_7BSONObjERKNS_7DiskLocE+0x130) [0xaaaab744f140]
 mongod(+0xA49B10) [0xaaaab7439b10]
 mongod(_ZN5mongo17IndexAccessMethod6updateEPNS_16OperationContextERKNS_12UpdateTicketEPl+0xC0) [0xaaaab710fe80]
 mongod(_ZN5mongo10Collection14updateDocumentEPNS_16OperationContextERKNS_8RecordIdERKNS_11SnapshottedINS_7BSONObjEEERKS7_bbPNS_7OpDebugERNS_20oplogUpdateEntryArgsE+0x8F0) [0xaaaab6f64f18]
 mongod(_ZN5mongo11UpdateStage18transformAndUpdateERKNS_11SnapshottedINS_7BSONObjEEERNS_8RecordIdE+0x890) [0xaaaab7091d9c]
 mongod(_ZN5mongo11UpdateStage4workEPm+0x5B4) [0xaaaab7092710]
 mongod(_ZN5mongo12PlanExecutor11getNextImplEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE+0x420) [0xaaaab7293df4]
 mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_8RecordIdE+0x48) [0xaaaab729428c]
 mongod(_ZN5mongo12PlanExecutor11executePlanEv+0x58) [0xaaaab7294394]
 mongod(_ZN5mongo18WriteBatchExecutor10execUpdateERKNS_12BatchItemRefEPNS_7BSONObjEPPNS_16WriteErrorDetailE+0x56C) [0xaaaab70129a4]
 mongod(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0x24C) [0xaaaab701446c]
 mongod(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x198) [0xaaaab70147e8]
 mongod(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderE+0x234) [0xaaaab7017438]
 mongod(_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE+0x448) [0xaaaab702fa78]
 mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE+0x438) [0xaaaab70307c8]
 mongod(_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE+0x1E8) [0xaaaab6f997c0]
 mongod(+0x73D628) [0xaaaab712d628]
 mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x5CC) [0xaaaab712f824]
 mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortE+0xD4) [0xaaaab6e3b070]
 mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x304) [0xaaaab76c6c54]
 libpthread.so.0(+0x7D38) [0xffff7fa47d38]
 libc.so.6(+0xDF5F0) [0xffff7f98f5f0]
-----  END BACKTRACE  -----

 

 

MongoDB version:

db version v3.2.8
git version: ed70e33130c977bda0024c125b56d159573dbafo
OpenSSL version: OpenSSL 1.0.le-fips 11 Feb 2013
allocator: system
modules: none
build environment:
distarch: arm64
target_arch: arm64

 

Linux version:{}

Linux  nodel4.18.0 aarch64



 Comments   
Comment by Chris Kelly [ 12/Aug/22 ]

Hi Jipeng,

MongoDB 3.2 reached end of life in September 2018 and is no longer supported. I would recommend upgrading to 4.2+ or newer for support. The MMAPv1 storage engine was removed in 4.2.

In the short term, there is a chance this could be related to corruption, and you could consider running validate on your collection to see if it turns up any issues. Here's the legacy doc reference on that for 3.2. We will be unable to provide further input until you upgrade to a supported version, however you can also ask our community for help by posting on the MongoDB Developer Community Forums.

Regards,
Christopher

Comment by jipeng N/A [ 12/Aug/22 ]

We used the Pymongo driver to manage the mongodb, but recently we were suddenly unable to update the data,detailed error as above.

Generated at Thu Feb 08 06:11:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.