Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-343

GridStore.stream() returns an invalid Writable Stream

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.0.10
    • Component/s: None
    • Labels:

      (it happens with 2.0.12, but this version doesn't exist in Jira...)

      var mongodb = require('mongodb');
      var gridstream = require('gridfs-stream');
      var fs = require('fs');
      
      var filename = 'zero.100MB';
      
      mongodb.MongoClient.connect('mongodb://localhost:27017/test', function(err, db) {
      
        var start = process.hrtime();
      
        var fileId = new mongodb.ObjectID();
      
        var gridStore = new mongodb.GridStore(db, fileId, filename, "w");
        var stream = gridStore.stream();
        var fileStream = fs.createReadStream(filename);
      
        var size = 0;
        fileStream.on('data', function logInputData(data) {
          size += data.length;
          console.log('fs input data sent', size);
        });
      
        fileStream.pipe(stream)
          .once('end', function gfsDone() {
      
            var diff = process.hrtime(start),
                time = diff[0]*1e9+diff[1];
            console.log('mongodb driver 2.0 gridfs-stream write complete for "%s", total time was %dns.',filename,time);
      
            db.close();
          });
      });
      

      This blocks after the second fileStream 'data' event. When removing the fileStream.on('data'), or moving it after the pipe(stream), then it works.

      It should probably work in all cases.

      When patching node-mongodb-native to implement the real Writable stream API (ie renaming GridStoreStream.prototype.write to _write, and calling callback(err) in gs.write() callbacks), then the above code works.

      (The proper patch needs more fixes, like not returning false, and in case of write after end we probably should call the callback with an error, and maybe not directly emit the error, but this requires some testing which I haven't done).

            Assignee:
            christkv Christian Amor Kvalheim
            Reporter:
            trsystran Thomas Riccardi
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: