[JAVA-185] add OutputStream capability to GridFSInputFile Created: 12/Oct/10  Updated: 22/Dec/10  Resolved: 09/Dec/10

Status: Closed
Project: Java Driver
Component/s: API
Affects Version/s: None
Fix Version/s: 2.4

Type: New Feature Priority: Minor - P4
Reporter: Guy K. Kloss Assignee: Antoine Girbal
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

feature usable for all environments



 Description   

For the implementation of certain types of servers it is
desirable to obtain a handle to a newly created file in the form
of a java.io.OutputStream, which can be used to successively
write to until the file is closed. A similar request already came
in June this year, see Peter's email here:

http://www.listware.net/201006/mongodb-user/17457-mongodb-user-java-driver-obtaining-an-outputstream-for-a-gridfs-file.html

or simpler: http://ur1.ca/1vfuf

I have finished an implementing this addition, and it would be
good to see it getting into the MongoDB Java API. The
implementation is compatible to the current Git HEAD and pass all
unit tests (including two newly made ones for the new feature).

A Github repository containing the patch along with a pull
request into the mongo-java-driver main branch can be found here:

Some information on the implementation:

  • The implementation has been aimed at staying backwards compatible with
    previous versions.
  • The class GridFS now contains two more createFile() methods, taking no
    parameters and just taking a file name (String). This way I do not have the
    necessity anymore of having to provide a file/InputStream/byte array object.
  • The GridFSInputFile class now offers a public method getOutputStream(). As
    soon as this output stream is obtained, a save() or saveChunks() call will
    only persist the already available input stream data, but the object will
    remain writable on the retrieved output stream until the stream's close()
    method is called.
  • Two further unit tests in GridFSTest are provided to test the output stream
    functionality.
  • For all new API methods and all of GridFSInputFile proper Javadoc is
    provided (it was really tedious to find out things in the beginning with the
    currently very sparse API documentation).
  • All wild card imports in the three files have been removed in favour of a
    clean list of individual imports. (Thanks to Eclipse's "organise imports" that
    was a breeze.)


 Comments   
Comment by Guy K. Kloss [ 09/Dec/10 ]

Sounds good, looking forward to trying it out on my project.

Comment by Antoine Girbal [ 09/Dec/10 ]

Thanks Guy, in the end I took your code since it mirrors what we have for GridFSDBFile.
Some tweaks:

  • ability to choose chunkSize before obtaining and OutputStream
  • fix to previous commit: do not save file object in saveChunks()
  • prevent mixed use of inputStream / outputStream in GridFSInputFile
  • better comments on functions
  • added test for writing a gridfs file of a size aligned with chunkSize
Comment by Guy K. Kloss [ 08/Dec/10 ]

The reason for the implementation like this was two fold: (A) to keep the modifications localised to just the tree files, and (B) to retain maximum backwards compatibility. Feel free to improve, much appreciated. It just seemed to me more likely to be accepted with meeting the conditions (A) and (B).

Comment by Antoine Girbal [ 08/Dec/10 ]

I feel like the patch implementation is functional but ends up being messy and confusing for most users, basically it is possible to mix inputstream, outputstream, save(), ...
I will keep GridFSInputFile and add separate GridFSOutputStream, since method are mostly distinct.
Also some common code will be moved to GridFSFile.

Comment by Han [ 08/Nov/10 ]

It will be a wonderful feature.

Comment by Guy K. Kloss [ 12/Oct/10 ]

On imports: I agree, that should be discussed elsewhere.

Kept it compact, only three files:

src/main/com/mongodb/gridfs/GridFS.java
src/main/com/mongodb/gridfs/GridFSInputFile.java
src/test/com/mongodb/gridfs/GridFSTest.java

The commits were to keep up with the change in your HEAD plus the edits to conform to common style.

Hope that helps! :o)

Comment by Eliot Horowitz (Inactive) [ 12/Oct/10 ]

Not going to comment on wild card on this ticket - have to have some semblance of cleanness As I said, I'm open to discussion.

Will look at commit again

Which files did you edit?

Having trouble reading through since its a number of commits.

Up for squashing?

Comment by Guy K. Kloss [ 12/Oct/10 ]

I've tried to manually fix the coding style (re. horizontal white space and imports) into compliance with the rest of the code.

Regarding the import scheme: This is not really part of this JIRA issue, but as we're discussing it partly already ...

The common Java guides as well as common practice state that wildcard imports are not well suited. Particularly for the reason as they get expanded at compile time. So when, say, an import "import com.mongodb.*;" is compiled, it is expanded to import all classes (used or not) into the name space of the code. If now the API of something imported like that changes, the code making use of the wildcard imports will potentially break. So one gains convenience in development for a decreased robustness against used external references. This can be resolved by a simple recompilation (unless other things have changed incompatibly as well), but nonetheless, the binary compatibility has been broken, even though the change in the API wouldn't necessarily require that to happen. It seems like other developer's on the mongodb-dev list are feeling equally to encourage a change here. After all, most IDEs nowadays provide support for cleaning up the imports automatically without manual intervention.

Comment by Eliot Horowitz (Inactive) [ 12/Oct/10 ]

Re: wild cards, the absolute requirement is that the entire code base be consistent. so changing in some files or with another commit will never get in.

i'm open to discussion about making a global change - though i'm personally opposed.

Comment by Guy K. Kloss [ 12/Oct/10 ]

Yes, I did see as well as use the Eclipse formatting provided by Scott in the repo.

I don't know where the alternative formatting comes from, but I can fix at least the indentation/spaces issue. That's an easy one.

With the wildcard imports ... Well, that takes manual tinkering, as that's not done by Eclipse, but rather the reverse.

On that topic as well: Wildcard imports are largely regarded as harmful, so what's the religious adherence towards using them, if they make the code more likely to break? See also here:

http://javadude.com/articles/importondemandisevil.html
http://www.javaperformancetuning.com/news/qotm031.shtml

If there'd be some documentation on what to wildcard import, and what not, I could much more easily resolve this.

Comment by Eliot Horowitz (Inactive) [ 12/Oct/10 ]

did you use the eclipse formatting files in the repo?
this doesn't obey most of them
4 spaces
no tabs
wildcard imports

Generated at Thu Feb 08 08:51:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.