[JAVA-345] FileNotFoundException - too many open files Created: 03/May/11  Updated: 25/Jun/13  Resolved: 03/May/11

Status: Closed
Project: Java Driver
Component/s: API
Affects Version/s: 2.5.3
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Chris Kaempfe Assignee: Scott Hernandez (Inactive)
Resolution: Done Votes: 0
Labels: GridFSInputFile, insert
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux 64 Bit, Eclipse Helios Service Release 2



 Description   

I'm getting the following error:
<<<
java.io.FileNotFoundException: /home/user/image.jpg (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at com.mongodb.gridfs.GridFS.createFile(GridFS.java:239)
<<<

I'm migrating a Oracle DB to MongoDB. For the work I'm using several threads which reads Orcale and put the information to MongoDB.
Therefor im crawling a filesystem for different files, which need to be inserted in MongoDB.
My Code - within the thread - where I'm receiving the exception:

File file = new File(filePath);
GridFSInputFile inputFile = null;
try {
if(!file.exists())

{ throw new FileNotFoundException("File: " + filePath + " does not exist."); }

inputFile = grid.createFile(file);
inputFile.put("media", "file");
inputFile.save();
} catch (IOException e)

{ log.error("MedienDBWorker addFile() - ID: " + workID + " ******* ", e); }

Is seams that GridFSInputFile is creating an InputStream and after saving the file, the InputStream isn't closed nor is there a method to close the stream.
How should this be handled?

My workaround is:
File file = new File(filePath);
GridFSInputFile inputFile = null;
try {
if(!file.exists())

{ throw new FileNotFoundException("File: " + filePath + " does not exist."); }

final InputStream is = new FileInputStream(file);
try

{ inputFile = grid.createFile(is,file.getName()); inputFile.put("media", "file"); inputFile.save(); }

finally

{ is.close(); }


} catch (IOException e)

{ log.error("MedienDBWorker addFile() - ID: " + workID + " ******* ", e); }

 Comments   
Comment by Antoine Girbal [ 03/May/11 ]

thanks for the report we will add the close().
Another workaround may be to force gc() so that the file handles get finalized and closed.
Also make sure that you are using only 1 mongo instance for all threads: if you use 1 mongo per thread you may end up using too many connections.

Comment by Scott Hernandez (Inactive) [ 03/May/11 ]

This is a system issue with the number of open files and has nothing to do with the driver.

You should increase the number of open file handles that your process is allowed to use; use ulimit -n to do this, or talk to your sysadmin.

Generated at Thu Feb 08 08:52:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.