Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-6528

Consistent progress support for analyzeCSVFields, importCSV and importJSON, etc

    XMLWordPrintableJSON

Details

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • No version
    • None
    • None
    • None
    • 2
    • Not Needed
    • Iteration Velvet Crab

    Description

      For importing CSV I think the progress event worked fine because you could see how many docs have been processed and an earlier step already returned the number of docs when we analysed the file. This is enough to accurately show the progress as designed.

      For importing JSON we don't know how many docs there are in advance.

      So I'll have to augment the progress callback with how many bytes have been processed. Then the progress can be the number of bytes out of the total.Add it to both importCSV() and importJSON() so they can remain compatible.

      ie. you would have the number of docs and number of bytes processed. Total bytes in the file can be read outside the function and for CSV we have the additional advantage that we already have the total number of docs.

      Also check other import/export functions that have a progress callback.

      Attachments

        Activity

          People

            leroux.bodenstein@mongodb.com Le Roux Bodenstein
            leroux.bodenstein@mongodb.com Le Roux Bodenstein
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: