[CDRIVER-72] more conveniant interface for providing extra meta data for gridfiles Created: 29/Jun/11  Updated: 05/Dec/13  Resolved: 14/Nov/13

Status: Closed
Project: C Driver
Component/s: None
Affects Version/s: 0.3
Fix Version/s: 0.90.0

Type: Improvement Priority: Minor - P4
Reporter: Gustaf Neumann Assignee: Mira Carey
Resolution: Done Votes: 2
Labels: gridfs
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

all



 Description   

My interpretation of gridfile_get_metadata() is to obtain user-defined metadata, where the user-defined metadata is embedded in the "file-system" specific metadata (such as md5, upload date, content type, ...). At least, gridfile_get_metadata() tries to access the object named "metadata" and returns its content, which is - without other operations - empty.

Currently, i use the plain mongo db interface to add the additional metadata with separate operations. It would be nice, to pass this additional (user-defined) meta-data as bson field to the following file-creating operations.

gridfs_store_buffer()
gridfs_insert_file()

if backward compatibility is already an issue, providing new names for this functions would help.



 Comments   
Comment by Christian Hergert [ 14/Nov/13 ]

Code complete, includes mongoc_gridfs_file_set_metadata() for actual implementation.

Comment by Christian Hergert [ 14/Nov/13 ]

Hi everyone,

We are approaching the release of our 0.90 implementation (a complete rewrite, which should include all features of the current driver and much, much, more). It has a new streaming gridfs implementation that also allows robust control over metadata for a file.

See mongoc_gridfs_file_set_metadata().

I'm closing this ticket since it is code complete, but note that the driver wont be released for a few weeks.

Cheers!

Comment by Mira Carey [ 30/Oct/13 ]

Apologies about the spurious issue close.

Nothing changed for the gridfs metadata handling between 0.8.0 and 0.8.1. This issue is still outstanding.

Comment by Frank J. Lhota [ 09/Oct/13 ]

The C driver supports 3 methods for creating a GRID file:

  1. By writing chunks using gridfile_writer_init() / gridfile_write_buffer() / gridfile_writer_done();
  2. By storing a regular file using gridfs_store_file(), and
  3. By storing a memory buffer using gridfs_store_buffer().

AFAICT, the version 0.8.1 C driver still does not provide any way to set the metadata for GRID files created by any of these 3 methods. The C driver has supported reading the existing metadata (using gridfile_get_metadata()), but the 0.8.1 driver still does not have a function for updating that GRID file's metadata.

Please clarify what was fixed for this issue.

Comment by Frank J. Lhota [ 04/Jan/13 ]

My company has modified the 0.7 C driver to support custom metadata. There are 3 external functions that create a GridFile: gridfile_writer_done(), gridfs_store_buffer(), and gridfs_store_file(). To each of these functions, we added the additional parameter

const bson* metadata

If this parameter is non-null, then this metadata will be included as the custom metadata for the GridFile. To implement this change, we added this parameter to the static function gridfs_insert_file(), and implemented the desired functionality there.

Other drivers permit users to set the GridFile metadata. The lack of a straightforward way to do this with the C driver is a serious limitation. We would be happy to share our modifications for inclusion in the next version.

Comment by Gustaf Neumann [ 10/May/12 ]

Dan,

below, i write the example in the Tcl syntax, which maps straightworward to
the C code. let me know, if i should write the c driver call in more details:

Create a file

  set gridFS [::mongo::gridfs::open $mongoConn myfs fs]
  set fn README
  set r [::mongo::gridfs::store_file $gridFS $fn $fn text/plain]

The result is a boolean value (MONGO_OK or not).
It would be nice to return the bson data, containing _id, content_length, etc. Otherwise it has to be obtained with a separate call.

Get data:

   set f [mongo::gridfile::open $gridFS README]
   puts stderr "\nOpened grid file '$f'"
   puts stderr "Metadata: [mongo::gridfile::get_metadata $f]"
   puts stderr "ContentLength: [mongo::gridfile::get_contentlength $f]"
   puts stderr "ContentType: [mongo::gridfile::get_contenttype $f]"

We have no metadata from "get_metadata" returned, all values are as expected.
Now we get the file named README from gridfs via plain query interface:

   set atts [lindex [::mongo::query $mongoConn myfs.fs.files \
		      [list \$query object {filename string README}] \
		      -limit 1] 0]
   puts "Attributes of file README:\n$atts\n"

Extract the oid from the bson attributes

   foreach {name type value} $atts {
     if {$name eq "_id"} {
       set oid $value
       break
     }
   }

Add a dc:creator to the bson attributes as an object named "metadata" ...

   lappend atts metadata object { dc:creator string "Gustaf Neumann"}

... and update the entry in the gridfs

   ::mongo::update $mongoConn myfs.fs.files [list _id oid $oid] $atts

Now we can use the gridfs interface to obtain the additional
metadata as well:

   set f [mongo::gridfile::open $gridFS README]
   puts stderr "Metadata: [mongo::gridfile::get_metadata $f]"
   mongo::gridfile::close $f

Does this help?

Comment by Dan Myers [ 10/May/12 ]

Gustaf,
I'm not sure that I understand the context but in general I support this request.

Can you give examples of how you add your own metadata now?

Is there a mongodb reserved attribute(s) at the doc and also doc-attribute level for a description?

Generated at Wed Feb 07 21:08:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.