[SERVER-1083] mongoimport: option to support an array of json objects Created: 04/May/10 Updated: 12/Jul/16 Resolved: 14/Jun/10 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 1.5.1 |
| Fix Version/s: | 1.5.3 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Alex Sharp | Assignee: | Mathias Stearn |
| Resolution: | Done | Votes: | 2 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
os x |
||
| Participants: |
| Description |
|
Many public API's (twitter, for example) return a collection of json objects for a request for a collection of resources. The mongoimport tool only understands newline-delimited json objects in a file. In other words, it only understands one json object at a time. It seems like mongoimport should know how to understand these collections natively. For example, trying the following will result in an error: curl http://api.twitter.com//1/statuses/user_timeline.json?screen_name=noradio > noradio.twitter.json |
| Comments |
| Comment by Eliot Horowitz (Inactive) [ 01/Sep/10 ] |
|
@maxime if you want to increase the limit, can you open a new case. this one shouldn't be modified anymore |
| Comment by Maxime Bizon [ 26/Aug/10 ] |
|
Hi Mathias, I just hit the array size limit. The generated data comes from a perl script, where it is handy to put everything in a array or hash, and at the end JSON::encode the whole thing. I can certainly put the loop in the perl script to generate one line per entry, but it doesn't seem natural. Any chance this limit could be raised ? Thanks, |
| Comment by Mathias Stearn [ 14/Jun/10 ] |
|
Right now the array must be <4MB. Let me know if that is an issue. |
| Comment by auto [ 14/Jun/10 ] |
|
Author: {'login': 'RedBeard0531', 'name': 'Mathias Stearn', 'email': 'redbeard0531@gmail.com'}Message: test for --jsonArray |
| Comment by auto [ 14/Jun/10 ] |
|
Author: {'login': 'RedBeard0531', 'name': 'Mathias Stearn', 'email': 'redbeard0531@gmail.com'}Message: --jsonArray for mongoexport |
| Comment by auto [ 14/Jun/10 ] |
|
Author: {'login': 'RedBeard0531', 'name': 'Mathias Stearn', 'email': 'redbeard0531@gmail.com'}Message: --jsonArray for mongoimport |