[SERVER-2186] mongorestore doesn't restore indexes (osx) Created: 07/Dec/10  Updated: 12/Jul/16  Resolved: 13/Dec/10

Status: Closed
Project: Core Server
Component/s: Tools
Affects Version/s: 1.6.4
Fix Version/s: 1.7.4

Type: Bug Priority: Major - P3
Reporter: Visnu Pitiyanuvath Assignee: Richard Kreuter (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

osx 10.6.5


Attachments: File system.indexes.bson    
Operating System: ALL
Participants:

 Description   

indexes don't get created using mongorestore.

I've only experienced this consistently on my development machine, which is osx 10.6.5. in linux, I've seen indexes restored correctly. this is a mongodump on linux, copied to my osx machine, then mongorestore'd.

below is the output from running mongorestore:

[visnup@qi ? 11:33 AM ~/Downloads/syn_production-20101206124133/syn_production]
$ mongorestore --drop --indexesLast -d syn_development .
connected to: 127.0.0.1
Tue Dec 7 11:33:57 ./activities.bson
Tue Dec 7 11:33:57 going into namespace [syn_development.activities]
Tue Dec 7 11:33:57 dropping
Tue Dec 7 11:33:58 4581 objects found
Tue Dec 7 11:33:58 ./changes.bson
Tue Dec 7 11:33:58 going into namespace [syn_development.changes]
Tue Dec 7 11:33:58 dropping
26812356/966500518 2%
43780626/966500518 4%
59037582/966500518 6%
70340640/966500518 7%
78848874/966500518 8%
90603729/966500518 9%
111478889/966500518 11%
135888448/966500518 14%
155931866/966500518 16%
173213887/966500518 17%
189886416/966500518 19%
207615774/966500518 21%
223109175/966500518 23%
234331933/966500518 24%
245661693/966500518 25%
257975390/966500518 26%
269330150/966500518 27%
280035356/966500518 28%
289414343/966500518 29%
300844527/966500518 31%
313036656/966500518 32%
324098827/966500518 33%
332027701/966500518 34%
342203880/966500518 35%
352078194/966500518 36%
363737030/966500518 37%
375278667/966500518 38%
387282648/966500518 40%
402093245/966500518 41%
419602331/966500518 43%
438455816/966500518 45%
452377195/966500518 46%
469667675/966500518 48%
476543503/966500518 49%
490550065/966500518 50%
504931431/966500518 52%
523103700/966500518 54%
540838099/966500518 55%
557674638/966500518 57%
576200908/966500518 59%
587706308/966500518 60%
600408295/966500518 62%
615653817/966500518 63%
630180005/966500518 65%
641175655/966500518 66%
651158936/966500518 67%
662458554/966500518 68%
670275912/966500518 69%
682284915/966500518 70%
693770746/966500518 71%
703934034/966500518 72%
714622250/966500518 73%
729391831/966500518 75%
737134510/966500518 76%
749751327/966500518 77%
764419345/966500518 79%
778512066/966500518 80%
793389181/966500518 82%
813628353/966500518 84%
827386509/966500518 85%
837774509/966500518 86%
849880895/966500518 87%
858468949/966500518 88%
874579049/966500518 90%
889011199/966500518 91%
902740902/966500518 93%
914523326/966500518 94%
926271424/966500518 95%
937596872/966500518 97%
950370050/966500518 98%
962235955/966500518 99%
Tue Dec 7 11:37:35 2509232 objects found
Tue Dec 7 11:37:35 ./contacts.bson
Tue Dec 7 11:37:35 going into namespace [syn_development.contacts]
Tue Dec 7 11:37:35 dropping
29509090/238492536 12%
43347912/238492536 18%
65850892/238492536 27%
92157341/238492536 38%
111299436/238492536 46%
137499691/238492536 57%
166426587/238492536 69%
196846077/238492536 82%
214657884/238492536 90%
Tue Dec 7 11:38:04 197481 objects found
Tue Dec 7 11:38:04 ./data_migrations.bson
Tue Dec 7 11:38:04 going into namespace [syn_development.data_migrations]
Tue Dec 7 11:38:04 dropping
Tue Dec 7 11:38:04 27 objects found
Tue Dec 7 11:38:04 ./emails.bson
Tue Dec 7 11:38:04 going into namespace [syn_development.emails]
Tue Dec 7 11:38:04 dropping
7389689/114124096 6%
18220669/114124096 15%
29809886/114124096 26%
40428925/114124096 35%
50986113/114124096 44%
58475985/114124096 51%
68237263/114124096 59%
76902885/114124096 67%
84227246/114124096 73%
92610612/114124096 81%
101530384/114124096 88%
106062706/114124096 92%
113672572/114124096 99%
Tue Dec 7 11:38:43 558255 objects found
Tue Dec 7 11:38:43 ./feed_items.bson
Tue Dec 7 11:38:43 going into namespace [syn_development.feed_items]
Tue Dec 7 11:38:43 dropping
Tue Dec 7 11:38:43 1045 objects found
Tue Dec 7 11:38:43 ./feeds.bson
Tue Dec 7 11:38:43 going into namespace [syn_development.feeds]
Tue Dec 7 11:38:43 dropping
Tue Dec 7 11:38:43 20 objects found
Tue Dec 7 11:38:43 ./notes.bson
Tue Dec 7 11:38:43 going into namespace [syn_development.notes]
Tue Dec 7 11:38:43 dropping
Tue Dec 7 11:38:43 4 objects found
Tue Dec 7 11:38:43 ./system.profile.bson
Tue Dec 7 11:38:43 skipping
Tue Dec 7 11:38:43 ./system.indexes.bson
Tue Dec 7 11:38:43 going into namespace [syn_development.system.indexes]
Tue Dec 7 11:38:43 dropping
Tue Dec 7 11:38:43 30 objects found

[visnup@qi ? 11:38 AM ~/Downloads/syn_production-20101206124133/syn_production]
$ mongo syn_development
MongoDB shell version: 1.6.4
connecting to: syn_development
> db.system.indexes.find()
{ "name" : "id", "ns" : "syn_development.system.users", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_development.activities", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_development.changes", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_development.contacts", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_development.data_migrations", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_development.emails", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_development.feed_items", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_development.feeds", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_development.notes", "key" :

{ "_id" : 1 }

}
> db.system.indexes.count()
9

the indexes on the machine the dump came from are:

visnu@app03:~$ mongo mong02.sequoiacap.com/syn_production
MongoDB shell version: 1.6.4
connecting to: mong02.sequoiacap.com/syn_production
> db.system.indexes.find()
{ "name" : "id", "ns" : "syn_production.data_migrations", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_production.changes", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_production.contacts", "key" :

{ "_id" : 1 }

}
{ "name" : "id", "ns" : "syn_production.emails", "key" :

{ "_id" : 1 }

}
{ "background" : true, "key" :

{ "person_id" : 1 }

, "ns" : "syn_production.changes", "name" : "person_id_1" }
{ "unique" : false, "key" :

{ "admin_id" : 1 }

, "ns" : "syn_production.contacts", "name" : "admin_id_1" }
{ "unique" : false, "key" :

{ "domain" : 1 }

, "ns" : "syn_production.contacts", "name" : "domain_1" }
{ "unique" : false, "key" :

{ "linkedin_id" : 1 }

, "ns" : "syn_production.contacts", "name" : "linkedin_id_1" }
{ "unique" : false, "key" :

{ "email_addresses.address" : 1 }

, "ns" : "syn_production.contacts", "name" : "email_addresses.address_1" }
{ "unique" : false, "key" :

{ "name" : 1 }

, "ns" : "syn_production.contacts", "name" : "name_1" }
{ "name" : "id", "ns" : "syn_production.activities", "key" :

{ "_id" : 1 }

}
{ "background" : true, "ns" : "syn_production.activities", "name" : "contact_id_1", "key" :

{ "contact_id" : 1 }

}
{ "ns" : "syn_production.activities", "unique" : false, "name" : "outbound_company_ids_1", "key" :

{ "outbound_company_ids" : 1 }

}
{ "ns" : "syn_production.activities", "unique" : false, "name" : "inbound_company_ids_1", "key" :

{ "inbound_company_ids" : 1 }

}
{ "ns" : "syn_production.contacts", "unique" : false, "name" : "physical_address.latlng_2d", "key" :

{ "physical_address.latlng" : "2d" }

}
{ "background" : true, "ns" : "syn_production.contacts", "name" : "partner_ids_1", "key" :

{ "partner_ids" : 1 }

}
{ "ns" : "syn_production.contacts", "unique" : false, "name" : "needs_admin_sync_1", "key" :

{ "needs_admin_sync" : 1 }

}
{ "ns" : "syn_production.contacts", "unique" : false, "name" : "user_1", "key" :

{ "user" : 1 }

}
{ "name" : "id", "ns" : "syn_production.feed_items", "key" :

{ "_id" : 1 }

}
{ "ns" : "syn_production.feed_items", "unique" : false, "name" : "tags_1_created_at_-1", "key" :

{ "tags" : 1, "created_at" : -1 }

}
has more
> it
{ "ns" : "syn_production.feed_items", "unique" : false, "name" : "search_1_created_at_-1", "key" :

{ "search" : 1, "created_at" : -1 }

}
{ "name" : "id", "ns" : "syn_production.feeds", "key" :

{ "_id" : 1 }

}
{ "key" :

{ "positions.company_id" : 1, "recent_connection_date" : -1, "linkedin_id" : -1 }

, "ns" : "syn_production.contacts", "unique" : false, "name" : "positions.company_id_1_recent_connection_date_-1_linkedin_id_-1" }
{ "key" :

{ "deal" : 1, "stage" : 1, "region" : 1, "follow_up_date" : 1, "name" : 1 }

, "ns" : "syn_production.contacts", "unique" : false, "name" : "deal_1_stage_1_region_1_follow_up_date_1_name_1" }
{ "key" :

{ "role" : 1, "name" : 1 }

, "ns" : "syn_production.contacts", "unique" : false, "name" : "role_1_name_1" }
{ "key" :

{ "search" : 1, "_type" : 1, "recent_connection_date" : -1 }

, "ns" : "syn_production.contacts", "unique" : false, "name" : "search_1_type_1_recent_connection_date-1" }
{ "key" :

{ "name_downcase" : 1 }

, "ns" : "syn_production.contacts", "unique" : false, "name" : "name_downcase_1" }
{ "key" :

{ "intermediary" : 1, "stage" : 1, "region" : 1, "follow_up_date" : 1, "name" : 1 }

, "ns" : "syn_production.contacts", "unique" : false, "name" : "intermediary_1_stage_1_region_1_follow_up_date_1_name_1" }
{ "name" : "id", "ns" : "syn_production.notes", "key" :

{ "_id" : 1 }

}
{ "unique" : false, "key" :

{ "contact_id" : 1, "source" : 1, "created_at" : -1 }

, "ns" : "syn_production.changes", "name" : "contact_id_1_source_1_created_at_-1" }
> db.system.indexes.count()
30



 Comments   
Comment by Visnu Pitiyanuvath [ 13/Dec/10 ]

whoo hoo!

Comment by Richard Kreuter (Inactive) [ 13/Dec/10 ]

Fixed in git commits mentioned in comments.

Comment by auto [ 10/Dec/10 ]

Author:

{u'login': u'kreuter', u'name': u'Richard Kreuter', u'email': u'richard@10gen.com'}

Message: Add a test for SERVER-2186
https://github.com/mongodb/mongo/commit/0b23cc1bc38f5fa29c65b6b9f18876d53d240aab

Comment by auto [ 09/Dec/10 ]

Author:

{u'login': u'kreuter', u'name': u'Richard Kreuter', u'email': u'richard@10gen.com'}

Message: Fixup ns when inserting index & check for errors. SERVER-2186.
https://github.com/mongodb/mongo/commit/2d44cc04200fb1cd80f3d2255148aa3120ebbd3c

Comment by Visnu Pitiyanuvath [ 07/Dec/10 ]

you are correct! when I don't specify a differently named db to restore into, indexes are created correctly.

Comment by Visnu Pitiyanuvath [ 07/Dec/10 ]

unfortunately, no error message. below is the output from the log while I ran restore. let me try importing into a same-named db.

Tue Dec 7 15:13:40 [initandlisten] connection accepted from 127.0.0.1:58860 #24
Tue Dec 7 15:13:40 [conn24] CMD: drop syn_development.activities
Tue Dec 7 15:13:40 [conn24] building new index on

{ _id: 1 }

for syn_development.activities
Tue Dec 7 15:13:40 [conn24] done for 0 records 0.005secs
Tue Dec 7 15:13:41 [conn24] CMD: drop syn_development.changes
Tue Dec 7 15:13:41 [conn24] building new index on

{ _id: 1 }

for syn_development.changes
Tue Dec 7 15:13:41 [conn24] done for 0 records 0.002secs
Tue Dec 7 15:13:54 [conn24] insert syn_development.changes 192ms
Tue Dec 7 15:14:01 [conn24] insert syn_development.changes 192ms
Tue Dec 7 15:14:32 [conn24] insert syn_development.changes 174ms
Tue Dec 7 15:15:02 [conn24] insert syn_development.changes 182ms
Tue Dec 7 15:15:32 [conn24] CMD: drop syn_development.contacts
Tue Dec 7 15:15:32 [conn24] building new index on

{ _id: 1 }

for syn_development.contacts
Tue Dec 7 15:15:32 [conn24] done for 0 records 0.001secs
Tue Dec 7 15:15:32 [conn24] insert syn_development.contacts 262ms
Tue Dec 7 15:15:48 [conn24] CMD: drop syn_development.data_migrations
Tue Dec 7 15:15:48 [conn24] building new index on

{ _id: 1 }

for syn_development.data_migrations
Tue Dec 7 15:15:48 [conn24] done for 0 records 0.005secs
Tue Dec 7 15:15:48 [conn24] CMD: drop syn_development.emails
Tue Dec 7 15:15:48 [conn24] building new index on

{ _id: 1 }

for syn_development.emails
Tue Dec 7 15:15:48 [conn24] done for 0 records 0.023secs
Tue Dec 7 15:15:58 [conn24] insert syn_development.emails 134ms
Tue Dec 7 15:16:03 [conn24] insert syn_development.emails 294ms
Tue Dec 7 15:16:14 [conn24] CMD: drop syn_development.feed_items
Tue Dec 7 15:16:14 [conn24] building new index on

{ _id: 1 }

for syn_development.feed_items
Tue Dec 7 15:16:14 [conn24] done for 0 records 0secs
Tue Dec 7 15:16:14 [conn24] CMD: drop syn_development.feeds
Tue Dec 7 15:16:14 [conn24] building new index on

{ _id: 1 }

for syn_development.feeds
Tue Dec 7 15:16:14 [conn24] done for 0 records 0secs
Tue Dec 7 15:16:14 [conn24] CMD: drop syn_development.notes
Tue Dec 7 15:16:14 [conn24] building new index on

{ _id: 1 }

for syn_development.notes
Tue Dec 7 15:16:14 [conn24] done for 0 records 0secs
Tue Dec 7 15:16:14 [conn24] CMD: drop syn_development.system.indexes
Tue Dec 7 15:16:14 [conn24] end connection 127.0.0.1:58860

Comment by Richard Kreuter (Inactive) [ 07/Dec/10 ]

I think the actual bug has to do with using a different database name than that against which mongodump was run.

Can you tell me if you see error messages in the log file "exception 10097 bad table to index name on add index attempt"?

Comment by Visnu Pitiyanuvath [ 07/Dec/10 ]

system.indexes.bson attached.

Comment by Richard Kreuter (Inactive) [ 07/Dec/10 ]

Could you attach the system.indexes.bson file from your dump directory to this case?

Generated at Thu Feb 08 02:59:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.