[COMPASS-3643] Compass fails to import a JSON file that can be imported with no issues with mongoimport Created: 14/May/19  Updated: 21/May/20  Resolved: 16/Aug/19

Status: Closed
Project: Compass
Component/s: Import/Export
Affects Version/s: 1.17.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Dmitry Ryabtsev Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File import.json.gz     PNG File scr.png    
Issue Links:
Tested
tested by COMPASS-3824 Test behavior of new feature in relat... Closed

 Description   

When I try to attach the file in question, Compass inserts some of the documents but then quickly fails with "Unexpected end of JSON input".

At the same time, this file can be imported with no issues with mongoimport:

Show all

dmitry@mubuntu:/data$ mongoimport --host nubuntu -u admin -p 123 --authenticationDatabase admin -d test -c importTest import.json 
2019-05-14T14:28:50.007+1000	connected to: nubuntu
2019-05-14T14:28:52.979+1000	[........................] test.importTest	16.0MB/444MB (3.6%)
2019-05-14T14:28:55.979+1000	[#.......................] test.importTest	33.6MB/444MB (7.6%)
2019-05-14T14:28:58.979+1000	[##......................] test.importTest	50.1MB/444MB (11.3%)
2019-05-14T14:29:01.979+1000	[###.....................] test.importTest	62.1MB/444MB (14.0%)
2019-05-14T14:29:04.979+1000	[####....................] test.importTest	74.5MB/444MB (16.8%)
2019-05-14T14:29:07.979+1000	[####....................] test.importTest	83.4MB/444MB (18.8%)
2019-05-14T14:29:10.979+1000	[#####...................] test.importTest	97.6MB/444MB (22.0%)
2019-05-14T14:29:13.979+1000	[######..................] test.importTest	111MB/444MB (25.0%)
2019-05-14T14:29:16.979+1000	[######..................] test.importTest	126MB/444MB (28.4%)
2019-05-14T14:29:19.979+1000	[#######.................] test.importTest	133MB/444MB (30.0%)
2019-05-14T14:29:22.979+1000	[########................] test.importTest	149MB/444MB (33.6%)
2019-05-14T14:29:25.979+1000	[########................] test.importTest	160MB/444MB (36.1%)
2019-05-14T14:29:28.979+1000	[#########...............] test.importTest	169MB/444MB (38.2%)
2019-05-14T14:29:31.979+1000	[##########..............] test.importTest	187MB/444MB (42.2%)
2019-05-14T14:29:34.980+1000	[##########..............] test.importTest	200MB/444MB (45.1%)
2019-05-14T14:29:37.979+1000	[###########.............] test.importTest	217MB/444MB (48.8%)
2019-05-14T14:29:40.979+1000	[############............] test.importTest	231MB/444MB (52.1%)
2019-05-14T14:29:43.979+1000	[############............] test.importTest	237MB/444MB (53.5%)
2019-05-14T14:29:46.979+1000	[#############...........] test.importTest	253MB/444MB (57.1%)
2019-05-14T14:29:49.981+1000	[##############..........] test.importTest	263MB/444MB (59.3%)
2019-05-14T14:29:52.979+1000	[##############..........] test.importTest	274MB/444MB (61.7%)
2019-05-14T14:29:55.979+1000	[###############.........] test.importTest	289MB/444MB (65.2%)
2019-05-14T14:29:58.979+1000	[################........] test.importTest	303MB/444MB (68.2%)
2019-05-14T14:30:01.979+1000	[################........] test.importTest	311MB/444MB (70.1%)
2019-05-14T14:30:04.979+1000	[#################.......] test.importTest	323MB/444MB (72.8%)
2019-05-14T14:30:07.979+1000	[##################......] test.importTest	340MB/444MB (76.6%)
2019-05-14T14:30:10.979+1000	[###################.....] test.importTest	353MB/444MB (79.5%)
2019-05-14T14:30:13.979+1000	[###################.....] test.importTest	369MB/444MB (83.1%)
2019-05-14T14:30:16.979+1000	[####################....] test.importTest	386MB/444MB (86.9%)
2019-05-14T14:30:19.979+1000	[#####################...] test.importTest	392MB/444MB (88.4%)
2019-05-14T14:30:22.979+1000	[#####################...] test.importTest	401MB/444MB (90.4%)
2019-05-14T14:30:25.979+1000	[######################..] test.importTest	410MB/444MB (92.5%)
2019-05-14T14:30:28.979+1000	[#######################.] test.importTest	427MB/444MB (96.3%)
2019-05-14T14:30:31.979+1000	[#######################.] test.importTest	438MB/444MB (98.7%)
2019-05-14T14:30:33.205+1000	[########################] test.importTest	444MB/444MB (100.0%)
2019-05-14T14:30:33.205+1000	imported 1000000 documents

To add more to it, if, after loading data with mongoimport, I export it with Compass, then the Compass will fail to load the data that was just exported back into the MongoDB instance. The error is the same - "Unexpected end of JSON import".

Tested against MongoDB v4.0.9.



 Comments   
Comment by Massimiliano Marcon [ 16/Aug/19 ]

We are reimplementing the import/export feature so we won't fix existing bugs. However, we will use existing bugs as a guideline for testing the new implementation.

Comment by Brian Blevins [ 16/Jul/19 ]

Hi Conor,

Thank you for the feedback on this Jira ticket.

As a workaround, you also have the option of using the mongoimport command line tool to import documents from a JSON file into your cluster or replica set.

Comment by Conor McGrath [ 16/Jul/19 ]

I've tried breaking the file up and it works if you do a smaller number of rows (so its not my formatting), but I have a file with 10000 rows so I don't want to have to upload smaller batches.  Is this not a critical bug?  Because how am I suppose to get my data into the db without this functionality?  

Generated at Wed Feb 07 22:33:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.