[COMPASS-4610] Invaid characters on JSON import Created: 03/Feb/21  Updated: 27/Oct/23  Resolved: 28/Feb/22

Status: Closed
Project: Compass
Component/s: Import/Export
Affects Version/s: None
Fix Version/s: No version

Type: Bug Priority: Major - P3
Reporter: John Smith Assignee: Unassigned
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screen Shot 2022-02-23 at 10.27.45 AM.png     PNG File bug.png     PNG File screenshot-1.png     PNG File screenshot-2.png    
Documentation Changes: Not Needed

 Description   

Problem Description

While importing a JSON file in compass it is possible to have invalid characters in the document which causes issues later on for example accessing that information is not possible

Steps to Reproduce

Start Compass

Create DB, Collection

Select Import JSON file

Pick a malformed file

Expected Results

Import should fail providing an error about invalid data inside the file being imported

Actual Results

Malformed JSON file gets imported as is

Additional Notes

Example file data:

{

"asx:abap": {
"@xmlns:asx": "http://www.sap.com/abapxml",
"@version": "1.0",
"asx:values": {}

}}



 Comments   
Comment by Rhys Howell [ 28/Feb/22 ]

unfold1911@gmail.com Sounds good - thanks for checking out Compass and opening up the issue! If you've got more feedback we've got a feedback site setup that allows for upvoting, there are a number of posts around import/export in there if you want explore and leave any input - https://feedback.mongodb.com/forums/924283-compass/category/370861-import-export

Comment by John Smith [ 28/Feb/22 ]

Hi Rhys Howell

Thank you for explanation and based on that i consider this issue resolved.

Comment by Rhys Howell [ 28/Feb/22 ]

Hi unfold1911@gmail.com

In the example provided in your comment it looks like an expected syntax rejection, there is no value for the field specified.

db.test.find({
  "asx:abap.@version": "1.0"
})

Is an example of a valid find on the @version field of the asx:abap embedded document.

For the second part of your question, it's a bit before my time on the team so I'm not totally sure why it isn't on by default, I think we were considering the cases where users would import a large amount of documents. In those cases it's easier to have a list of the errors that occurred when importing, like which documents are malformed or fail a validation, than it is to have the export stop on the first occurrence and have to re-import all of the document and deduplicate the already imported ones. We are starting to look into revisiting and improving the user experience of the import/export flow in Compass soon, so down the line we might have some more powerful features here and probably explain what happens in each case a bit better.

Comment by John Smith [ 23/Feb/22 ]

Hi Rhys Howell

I think i misunderstood the issue, you see, my problem was i encountered a json file with a format like the example i've mentioned, since it had special characters i thought it will be considered a malformed json file, but i've just checked with a validator and apparently it's not.

First of all, the main problem for me is when a field has colon in it's name i can't specify it, for example:

db.test.find({'asx:values': {})

Will give me a syntax error

SyntaxError: Unexpected token (1:28)
 
> 1 | db.test.find({'asx:values': {})
    |                             ^

I believed it's because i've used mongo compass to import and the json is bad but now i think, it's either a compass\mogosh problem or i just don't know how to handle the situation.

 

The second thing i wanted to ask is, if there is a checkbox for stop on error.

Why isn't it on by default? and what gets imported without it? a cleaned file?

Comment by Rhys Howell [ 23/Feb/22 ]

Hi unfold1911@gmail.com - I'm not reproducing any unexpected results using the supplied document. Maybe this has been fixed? Closing, please feel free to reopen.

In the export flow we have the option to stop on errors which should handle those cases where the json is malformed (screenshot attached).

Generated at Wed Feb 07 22:36:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.