[SERVER-9457] has not default index of _id after execute "compact" ,and ensureIndex with unique:true,dropDups:true fails Created: 25/Apr/13  Updated: 20/May/13  Resolved: 17/May/13

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 2.0.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: handyyu Assignee: Thomas Rueckstiess
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

linux x86_64


Attachments: JPEG File 1.jpg     JPEG File 2.jpg    
Operating System: Linux
Steps To Reproduce:

the result of executing validate collection is OK,no error
why the default index of '_id' disappear
and the parameter 'dropDups' has no action?

Participants:

 Description   

problem is that:



 Comments   
Comment by handyyu [ 17/May/13 ]

Ok, I get the answer that i want .
Thank you very much .

by the way, you can close it.

Regards.

Comment by Thomas Rueckstiess [ 16/May/13 ]

This says that the BSON document didn't pass the validation, because the size was larger than 16MB. This doesn't mean that the size of the BSON document actually was larger, it can also happen if the BSON document got corrupted and the field that stores the size at the beginning of a BSON document has changed to a larger number.

You can also see some other errors in your logfile, for example

Wed May 8 10:37:30 [conn979] Assertion: 13655:BSONElement: bad type 115

Type 115 doesn't exist, so this really looks like your data got corrupted and the BSON representation didn't make sense anymore. The corruption can occur through disk or network related problems for example.

Does that answer your question?

Regards,
Thomas

Comment by handyyu [ 15/May/13 ]

Hi,
Thank you for your reply.

Wed May 8 10:37:30 [conn979] Assertion: 10321:Invalid dbref/code/string/symbol size: 1947205632
0x584722 0x507903 0x5113b8 0x51154f 0x51154f 0xaa209d 0xaa4466 0xaa508f 0xaa5cc7 0x9793e4 0x97a9df 0x962de5 0x966171 0x883877 0x88a09c 0xaa00c6 0x635bb7 0x32570062f7 0x32564d1b6d
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x112) [0x584722]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo11BSONElement8validateEv+0x223) [0x507903]
...

I have a question about the contents of the above,what it tells us?
what's kind of the error?

Regards.

Comment by Thomas Rueckstiess [ 15/May/13 ]

I'm glad the index build worked for you in the end. I am not sure why it didn't work with "dropDups", this may be an exception for the _id index. Like I said, it's not normal procedure to drop the _id index, so you should never have to create one manually.

My guess is that the problem was a corruption in your data (see the assertions in your logfile) and somehow there were duplicate _id entries, which should never happen. That's what made the index build fail and hence you didn't have an _id index anymore.

As such, since we can't reproduce this problem, I would suggest to close this ticket now. You can re-open it at any time if this problem occurs again. Or is there anything else you'd like me to do on the ticket?

Regards,
Thomas

Comment by handyyu [ 13/May/13 ]

It failed by using 'dropDups',why?
It's puzzle.
That's normal when executing commands with 'dropDups' on key except for '_id'.

Regards

Comment by handyyu [ 13/May/13 ]

Hi,
I could remove duplicates manually,and then it's successful that building the '_id' index.

Comment by Thomas Rueckstiess [ 10/May/13 ]

Hi,

I'm not aware of any way to drop the _id index, it is not supported in MongoDB. Since you have several documents with the same _id in your collection, can you remove duplicates manually and then try building the index again?

Thomas

Comment by handyyu [ 10/May/13 ]

hi,
could you give me some suggests or answers to solve the problem,
thank you!

Comment by handyyu [ 08/May/13 ]

Hi,

it didn't work when i rebuilt/built the _id index with dropdups ,why?

can i drop the _id index for simulating the Situation by some way?

Comment by handyyu [ 08/May/13 ]

Hi,

The logfile errors refer to a collection VTR in the database RDB. In your screenshots, the database was "RELATIONDB" and the collection was "REALTOVIRTUAL". Are these two different databases/collections?
RELATIONDB is the same as RDB,i just rename RELATIONDB to RDB in the log
this error has appeared many times in different instances
it doesn't matter the different from the names

Can you run a find with the offending _id and see if it returns more than one document?
i had excuted db.VIRTUALTOREAL.find({_id:"1903935564_14401"}) before, it returned two document

Can you also tell me the current state? Were you able to rebuilt the _id index in the mean time or is it still missing?
you can see it in 1.jpg of Attachments, 2.jpg can show you it didn't work

by the way, i run "repair" ,then solve it

Comment by Thomas Rueckstiess [ 08/May/13 ]

Hi,

when a compact starts, it first drops all the indexes (including _id). This is normal behavior. They are normally rebuilt at the end of the compact.

In your case, it seems the compact did not finish but stopped with an error:

Wed May 8 10:14:00 [conn979] compact dropping indexes
...
Wed May 8 10:40:45 [conn979] compact RDB.VTR end (with error)
Wed May 8 10:40:45 [conn979] command RDB.$cmd command: { compact: "VTR" } ntoreturn:1 exception: E11000 duplicate key error index: RDB.VTR.$id dup key: { : "1903935564_14401" } code:11000 reslen:228 1605378ms

The logfile errors refer to a collection VTR in the database RDB. In your screenshots, the database was "RELATIONDB" and the collection was "REALTOVIRTUAL". Are these two different databases/collections?

The missing _id index can be explained by the failure of the compact command. It doesn't explain though how you could have a duplicate key violation on the _id index. This may be related to the termination (kill -9) of your mongod and deletion of the journal file afterwards.

Can you run a find with the offending _id and see if it returns more than one document?

use RDB
db.VTR.find({_id: "1903935564_14401"})

Can you also tell me the current state? Were you able to rebuilt the _id index in the mean time or is it still missing?

Regards,
Thomas

Comment by handyyu [ 08/May/13 ]

Has the compact been interrupted in any way?
I had stopped the update,maybe has some query

Also, did you up- or downgrade mongodb at any time recently?
no,i had used the version 2.2.0

Comment by handyyu [ 08/May/13 ]

Wed May 8 10:14:00 [conn979] compact RDB.VTR begin
Wed May 8 10:14:00 [conn979] compact 34 extents
Wed May 8 10:14:00 [conn979] compact orphan deleted lists
Wed May 8 10:14:00 [FileAllocator] allocating new datafile /data/mongodb/27035/data/RDB.20, filling with zeroes...
Wed May 8 10:14:00 [conn979] compact dropping indexes
Wed May 8 10:14:14 [conn979] compact extent #0
Wed May 8 10:14:14 [conn979] compact paging in len=0.012288MB
Wed May 8 10:14:14 [conn979] compact copying records
Wed May 8 10:14:14 [conn979] compact 61 documents 0.009175MB
Wed May 8 10:14:14 [conn979] compact extent #1
Wed May 8 10:14:14 [conn979] compact paging in len=0.049152MB
Wed May 8 10:14:14 [conn979] compact copying records
Wed May 8 10:14:14 [conn979] compact 246 documents 0.037MB
Wed May 8 10:14:14 [conn979] compact extent #2
Wed May 8 10:14:14 [conn979] compact paging in len=0.196608MB
Wed May 8 10:14:14 [conn979] compact copying records
Wed May 8 10:14:14 [conn979] compact 973 documents 0.146603MB
Wed May 8 10:14:14 [conn979] compact extent #3
Wed May 8 10:14:14 [conn979] compact paging in len=0.786432MB
Wed May 8 10:14:14 [conn979] compact copying records
Wed May 8 10:15:26 [FileAllocator] done allocating datafile /data/mongodb/27035/data/RDB.20, size: 2047MB, took 85.529 secs
Wed May 8 10:15:26 [conn979] compact 3958 documents 0.59901MB
Wed May 8 10:15:26 [conn979] compact extent #4
Wed May 8 10:15:26 [conn979] compact paging in len=3.14573MB
Wed May 8 10:15:26 [conn979] compact copying records
Wed May 8 10:15:30 [conn979] compact 15867 documents 2.39871MB
Wed May 8 10:15:30 [conn979] compact extent #5
Wed May 8 10:15:30 [conn979] compact paging in len=12.5829MB
Wed May 8 10:15:30 [conn979] compact copying records
Wed May 8 10:15:49 [conn979] compact 61844 documents 9.41811MB
Wed May 8 10:15:49 [conn979] compact extent #6
Wed May 8 10:15:49 [conn979] compact paging in len=15.102MB
Wed May 8 10:15:49 [conn979] compact copying records
Wed May 8 10:16:26 [conn979] compact 71758 documents 10.9891MB
Wed May 8 10:16:26 [conn979] compact extent #7
Wed May 8 10:16:26 [conn979] compact paging in len=18.1248MB
Wed May 8 10:16:26 [conn979] compact copying records
Wed May 8 10:17:07 [conn979] compact 85978 documents 13.1956MB
Wed May 8 10:17:07 [conn979] compact extent #8
Wed May 8 10:17:07 [conn979] compact paging in len=21.7498MB
Wed May 8 10:17:07 [conn979] compact copying records
Wed May 8 10:18:14 [conn979] compact 102434 documents 15.7507MB
Wed May 8 10:18:14 [conn979] compact extent #9
Wed May 8 10:18:14 [conn979] compact paging in len=26.0997MB
Wed May 8 10:18:14 [conn979] compact copying records
Wed May 8 10:19:19 [conn979] compact 119768 documents 18.4455MB
Wed May 8 10:19:19 [conn979] compact extent #10
Wed May 8 10:19:19 [conn979] compact paging in len=31.3221MB
Wed May 8 10:19:19 [conn979] compact copying records
Wed May 8 10:20:56 [conn979] compact 137508 documents 21.2983MB
Wed May 8 10:20:56 [conn979] compact extent #11
Wed May 8 10:20:56 [conn979] compact paging in len=37.589MB
Wed May 8 10:20:56 [conn979] compact copying records
Wed May 8 10:24:14 [conn979] compact 165505 documents 25.5954MB
Wed May 8 10:24:14 [conn979] compact extent #12
Wed May 8 10:24:14 [conn979] compact paging in len=45.1092MB
Wed May 8 10:24:14 [conn979] compact copying records
Wed May 8 10:26:16 [conn979] compact 195271 documents 30.3789MB
Wed May 8 10:26:16 [conn979] compact extent #13
Wed May 8 10:26:16 [conn979] compact paging in len=54.1327MB
Wed May 8 10:26:16 [conn979] compact copying records
Wed May 8 10:26:30 [conn979] compact 229908 documents 35.9221MB
Wed May 8 10:26:30 [conn979] compact extent #14
Wed May 8 10:26:30 [conn979] compact paging in len=64.9626MB
Wed May 8 10:26:30 [conn979] compact copying records
Wed May 8 10:26:42 [conn979] compact 271709 documents 42.6945MB
Wed May 8 10:26:42 [conn979] compact extent #15
Wed May 8 10:26:42 [conn979] compact paging in len=77.9551MB
Wed May 8 10:26:42 [conn979] compact copying records
Wed May 8 10:26:50 [conn979] compact 331680 documents 51.8669MB
Wed May 8 10:26:50 [conn979] compact extent #16
Wed May 8 10:26:50 [conn979] compact paging in len=93.5485MB
Wed May 8 10:26:50 [conn979] compact copying records
Wed May 8 10:27:05 [conn979] compact 389844 documents 61.3996MB
Wed May 8 10:27:05 [conn979] compact extent #17
Wed May 8 10:27:05 [conn979] compact paging in len=112.259MB
Wed May 8 10:27:05 [conn979] compact copying records
Wed May 8 10:27:17 [conn979] compact 469820 documents 74.1947MB
Wed May 8 10:27:17 [conn979] compact extent #18
Wed May 8 10:27:17 [conn979] compact paging in len=134.713MB
Wed May 8 10:27:17 [conn979] compact copying records
Wed May 8 10:27:35 [conn979] compact 550736 documents 87.821MB
Wed May 8 10:27:35 [conn979] compact extent #19
Wed May 8 10:27:35 [conn979] compact paging in len=161.657MB
Wed May 8 10:27:35 [conn979] compact copying records
Wed May 8 10:27:57 [conn979] compact 638725 documents 103.556MB
Wed May 8 10:27:57 [conn979] compact extent #20
Wed May 8 10:27:57 [conn979] compact paging in len=193.991MB
Wed May 8 10:27:57 [conn979] compact copying records
Wed May 8 10:28:20 [conn979] compact 768699 documents 124.682MB
Wed May 8 10:28:20 [conn979] compact extent #21
Wed May 8 10:28:20 [conn979] compact paging in len=232.792MB
Wed May 8 10:28:20 [conn979] compact copying records
Wed May 8 10:28:54 [conn979] compact 904364 documents 149.523MB
Wed May 8 10:28:54 [conn979] compact extent #22
Wed May 8 10:28:54 [conn979] compact paging in len=279.351MB
Wed May 8 10:28:54 [conn979] compact copying records
Wed May 8 10:29:25 [conn979] compact 1067559 documents 180.128MB
Wed May 8 10:29:25 [conn979] compact extent #23
Wed May 8 10:29:25 [conn979] compact paging in len=335.225MB
Wed May 8 10:29:25 [conn979] compact copying records
Wed May 8 10:30:09 [conn979] compact 1268898 documents 216.736MB
Wed May 8 10:30:09 [conn979] compact extent #24
Wed May 8 10:30:09 [conn979] compact paging in len=402.272MB
Wed May 8 10:30:09 [conn979] compact copying records
Wed May 8 10:30:35 [conn979] compact 1489346 documents 261.587MB
Wed May 8 10:30:35 [conn979] compact extent #25
Wed May 8 10:30:35 [conn979] compact paging in len=482.73MB
Wed May 8 10:30:35 [conn979] compact copying records
Wed May 8 10:31:12 [conn979] compact 1660145 documents 319.751MB
Wed May 8 10:31:12 [conn979] compact extent #26
Wed May 8 10:31:12 [conn979] compact paging in len=579.277MB
Wed May 8 10:31:12 [conn979] compact copying records
Wed May 8 10:31:41 [FileAllocator] allocating new datafile /data/mongodb/27035/data/RDB.21, filling with zeroes...
Wed May 8 10:31:54 [FileAllocator] done allocating datafile /data/mongodb/27035/data/RDB.21, size: 2047MB, took 12.044 secs
Wed May 8 10:32:05 [conn979] compact 1802310 documents 385.447MB
Wed May 8 10:32:05 [conn979] compact extent #27
Wed May 8 10:32:05 [conn979] compact paging in len=695.132MB
Wed May 8 10:32:05 [conn979] compact copying records
Wed May 8 10:32:58 [conn979] compact 1975166 documents 424.759MB
Wed May 8 10:32:58 [conn979] compact extent #28
Wed May 8 10:32:58 [conn979] compact paging in len=834.159MB
Wed May 8 10:32:58 [conn979] compact copying records
Wed May 8 10:34:19 [conn979] compact 2601982 documents 528.211MB
Wed May 8 10:34:19 [conn979] compact extent #29
Wed May 8 10:34:19 [conn979] compact paging in len=1000.99MB
Wed May 8 10:34:19 [conn979] compact copying records
Wed May 8 10:35:39 [conn979] compact 2791594 documents 674.125MB
Wed May 8 10:35:39 [conn979] compact extent #30
Wed May 8 10:35:39 [conn979] compact paging in len=1201.19MB
Wed May 8 10:35:39 [conn979] compact copying records
Wed May 8 10:36:20 [FileAllocator] allocating new datafile /data/mongodb/27035/data/RDB.22, filling with zeroes...
Wed May 8 10:36:41 [FileAllocator] done allocating datafile /data/mongodb/27035/data/RDB.22, size: 2047MB, took 20.759 secs
Wed May 8 10:37:30 [conn979] Assertion: 10321:Invalid dbref/code/string/symbol size: 1947205632
0x584722 0x507903 0x5113b8 0x51154f 0x51154f 0xaa209d 0xaa4466 0xaa508f 0xaa5cc7 0x9793e4 0x97a9df 0x962de5 0x966171 0x883877 0x88a09c 0xaa00c6 0x635bb7 0x32570062f7 0x32564d1b6d
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x112) [0x584722]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo11BSONElement8validateEv+0x223) [0x507903]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo7BSONObj5validEv+0x88) [0x5113b8]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo7BSONObj5validEv+0x21f) [0x51154f]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo7BSONObj5validEv+0x21f) [0x51154f]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo13compactExtentEPKcPNS_16NamespaceDetailsENS_7DiskLocEiRKN5boost12scoped_arrayINS_9IndexSpecEEERNS6_INS_12SortPhaseOneEEEib+0x3dd) [0xaa209d]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo8_compactEPKcPNS_16NamespaceDetailsERSsbRNS_14BSONObjBuilderE+0x15e6) [0xaa4466]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo7compactERKSsRSsbRNS_14BSONObjBuilderE+0x1cf) [0xaa508f]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo10CompactCmd3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x267) [0xaa5cc7]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x6a4) [0x9793e4]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6ff) [0x97a9df]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x35) [0x962de5]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0x11e1) [0x966171]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod [0x883877]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x55c) [0x88a09c]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x76) [0xaa00c6]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x287) [0x635bb7]
/lib64/libpthread.so.0 [0x32570062f7]
/lib64/libc.so.6(clone+0x6d) [0x32564d1b6d]
Wed May 8 10:37:30 [conn979] compact skipping invalid object
Wed May 8 10:37:30 [conn979] Assertion: 13655:BSONElement: bad type 115
0x584722 0x5072d0 0x511390 0x51154f 0x51154f 0xaa209d 0xaa4466 0xaa508f 0xaa5cc7 0x9793e4 0x97a9df 0x962de5 0x966171 0x883877 0x88a09c 0xaa00c6 0x635bb7 0x32570062f7 0x32564d1b6d
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x112) [0x584722]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo11BSONElement4sizeEi+0x210) [0x5072d0]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo7BSONObj5validEv+0x60) [0x511390]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo7BSONObj5validEv+0x21f) [0x51154f]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZNK5mongo7BSONObj5validEv+0x21f) [0x51154f]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo13compactExtentEPKcPNS_16NamespaceDetailsENS_7DiskLocEiRKN5boost12scoped_arrayINS_9IndexSpecEEERNS6_INS_12SortPhaseOneEEEib+0x3dd) [0xaa209d]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo8_compactEPKcPNS_16NamespaceDetailsERSsbRNS_14BSONObjBuilderE+0x15e6) [0xaa4466]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo7compactERKSsRSsbRNS_14BSONObjBuilderE+0x1cf) [0xaa508f]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo10CompactCmd3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x267) [0xaa5cc7]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x6a4) [0x9793e4]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6ff) [0x97a9df]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x35) [0x962de5]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0x11e1) [0x966171]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod [0x883877]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x55c) [0x88a09c]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x76) [0xaa00c6]
/data/mongodb/mongodb-linux-x86_64-2.0.2/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x287) [0x635bb7]
/lib64/libpthread.so.0 [0x32570062f7]
/lib64/libc.so.6(clone+0x6d) [0x32564d1b6d]
Wed May 8 10:37:30 [conn979] compact skipping invalid object
Wed May 8 10:37:30 [conn979] compact 3699328 documents 809.102MB
Wed May 8 10:37:30 [conn979] compact extent #31
Wed May 8 10:37:30 [conn979] compact paging in len=1441.43MB
Wed May 8 10:37:30 [conn979] compact copying records
Wed May 8 10:37:43 [conn979] compact 477853 documents 121.954MB
Wed May 8 10:37:43 [conn979] compact extent #32
Wed May 8 10:37:43 [conn979] compact paging in len=1729.72MB
Wed May 8 10:37:43 [conn979] compact copying records
Wed May 8 10:38:24 [conn979] old journal file will be removed: /data/mongodb/27035/data/journal/j._0
Wed May 8 10:40:08 [conn979] compact 4301388 documents 920.222MB
Wed May 8 10:40:08 [conn979] compact extent #33
Wed May 8 10:40:08 [conn979] compact paging in len=2075.66MB
Wed May 8 10:40:08 [conn979] compact copying records
Wed May 8 10:40:08 [conn979] compact 2045 documents 0.604563MB
Wed May 8 10:40:08 [conn979] compact create index

{ _id: 1 }

Wed May 8 10:40:08 [conn979] build index RDB.VTR

{ _id: 1 }

378500/28654268 1%
1731500/28654268 6%
3601200/28654268 12%
Wed May 8 10:40:45 [conn979] compact RDB.VTR end (with error)
Wed May 8 10:40:45 [conn979] command RDB.$cmd command:

{ compact: "VTR" }

ntoreturn:1 exception: E11000 duplicate key error index: RDB.VTR.$id dup key: { : "1903935564_14401" } code:11000 reslen:228 1605378ms

Comment by handyyu [ 08/May/13 ]

i will answer your questions step by step:

the collection is not a capped collection, i use version 2.2.0

first my mongo is killed(-9) by someone, i use journal default,
but i found some errors during journal in the log, then i remove the journal.
it seems like normal at the begging, it is unnormal after running some days, the query or the update is slowly(about cost 20s or 30s).
so i try to excute "compact".
"assertion" : "E11000 duplicate key error index" appears in the end

Comment by Thomas Rueckstiess [ 06/May/13 ]

Hi,

Is the collection a capped collection? Prior to version 2.2, capped collections did not have an index on _id.

If this is not a capped collection and if it had an index on _id before, can you explain step by step what you did?

Has the compact been interrupted in any way?

Also, did you up- or downgrade mongodb at any time recently?

Can you also please attach the logfile from a point before starting the compact command to a point after the compact has finished? If the logfile contains sensitive information, please let me know and I can provide a way to upload the logfile securely to our server.

Regards,
Thomas

Generated at Thu Feb 08 03:20:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.