[COMPASS-4332] Compass doesn't export all existing fields in CSV Created: 23/Jun/20  Updated: 10/Jan/24  Resolved: 07/Apr/22

Status: Closed
Project: Compass
Component/s: Import/Export
Affects Version/s: 1.21.2, 1.22.0, 1.28.1, 1.31.0
Fix Version/s: 1.26.0

Type: Bug Priority: Major - P3
Reporter: Alejo Goñez Assignee: Anna Henningsen
Resolution: Fixed Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screen Shot 2020-06-23 at 09.33.49.png    
Issue Links:
Duplicate
is duplicated by COMPASS-4593 CSV export field selection is broken Closed
Related
related to COMPASS-5700 CSV export doesn't contain fields man... Closed
is related to COMPASS-4971 CSV export omits explicitly included ... Closed
Documentation Changes: Not Needed
Sprint: Iteration Uglifruit

 Description   

The user reports that Compass is not exporting all fields even when he add the fields manually.
I've tested it doing a comparison with mongoexport and confirm that Compass is not showing all fields as you can see in the attached screenshot.
Exported format: CSV
Tested bot exporting all documents and with the query option.

Tested on:

  • Version 1.21.2
  • Version 1.22.0-beta.2

Intercom chat

User's cluster

User's explanation:
For example, let's say some of my documents have about 50 variables. Others have the same 50 variables, plus 30 more. Some website users only fill out 50 of the survey questions. Others fill out all 80. I want to export all 80 fields, but when I use the export function for the entire collection of documents the tool will only find 50 fields! This is causing me to miss a lot of data.

If I only export the documents for the website users that filled out all 80, then the tool will find all 80. However, if the export includes any of the users who only filled out 50, then the tool will only find 50. When I add the other 30, the tool still only exports those 50 even for the users who filled out all 80. It's maddening.

Is there a way I can override this and force it to download all my data for the entire collection?



 Comments   
Comment by Jason R. Coombs [ 08/Apr/22 ]

Note that in COMPASS-4971, this issue was substantially fixed in 1.31, at least for explicitly included fields, but possibly not for inferred fields.

Comment by Tomáš Laubr [ 07/Apr/22 ]

Here you are COMPASS-5700

Btw sorry, I didn't expect the Open issue button will work for me.

Comment by Anna Henningsen [ 07/Apr/22 ]

jaraco@jaraco.com tomas.laubr@smart-software.cz felicia.hsieh

I’ll close this ticket, since this particular issue has indeed been fixed. It might have regressed, or you may be experiencing a similar, different issue.

I would ask that you open new tickets, and that you include the following information, which is important for investigating these problems:

  • Compass version and operating system
  • Database version
  • Compass log (Menu bar → Help → Open Log File → Extract and open as .txt) after the failed export happened
  • If applicable, the filter you applied before exporting documents
  • If possible, screenshots of the fields/documents that are missing in the collection itself
  • If possible, screenshots of the Compass field selection screen during export
  • If possible, the exported file
Comment by Tomáš Laubr [ 07/Apr/22 ]

Same issue here in 1.30.1. So definitely not fixed in 1.26.0.

In the export dialog I have manually added fields. But they didn't appear in exported CSV.

Comment by Jason R. Coombs [ 24/Jul/21 ]

Please re-open. I have this issue too. I have a collection where some fields weren't present in the early documents but they're present in the later documents. Running an export on the full collection (a) doesn't show the fields present in later documents and (b) even if you manually add that field to the dialog, the CSV omits that field entirely. I'd instead expect that field to be blank if it's not present in the document. This is with 1.28.1.

Comment by Maurizio Casimirri [ 22/Jan/21 ]

Going to reopen this since is not fixed yet, the field selection dialog in 1.25 displays more fields, but the export keeps using only those from the first document.

Comment by Maurizio Casimirri [ 22/Jan/21 ]

This is supposedly solved in 1.25, however there are still issues with finding all the fields to export. Are all the docs having all the 50 fields?

Comment by ritika srivastava [ 22/Jan/21 ]

Is this resolved?

I faced same issue , compass is only exporting 7 fields by default and not even showing remaining fields in the selection panel(I have more than 50 fields and cannot enter them manually). I could not find any way to export complete collection with all the fields. Please let me know is there is a work around.

Generated at Wed Feb 07 22:36:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.