-
Type: Bug
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: 1.6.5
-
Component/s: JavaScript, Stability
-
None
-
Environment:Linux qa-mongo1 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:34:28 EST 2008 x86_64 GNU/Linux
Amazon EC2 VM
-
Linux
Version data:
db version v1.6.6-pre-, pdfile version 4.5
git hash: nogitversion
sys info: Linux bobek-a0 2.6.32-5-amd64 #1 SMP Thu Nov 25 18:02:11 UTC 2010 x86_64 BOOST_LIB_VERSION=1_42
uptime: 1011 seconds
Trying to run the following map/reduce job from the mongo console:
m = function() {
var self = this;
this.regionIds.forEach(function(rid) {
emit(rid,
);
});
};
r = function(k,vals) {
var tweets = [];
for (var i in vals)
tweets.sort(createdAtComparer);
if (tweets > maxTweets)
return
{"tweets": tweets};
};
options = {
out: "tweetsByRegion",
scope: { maxTweets: 50,
createdAtComparer: function(a,b) {
return b.fetch().createdAt - a.fetch().createdAt;
}}
};
res = db.tweets.mapReduce(m,r,options);
As you can guess we are processing geo-located tweets where each tweet has a list of "region ids" associated with them. The Map/Reduce job basically inverts the relationship to group tweets (up to 50) by the region that they appear in. Also, note that I'm using a DBRef and not including the tweet directly. The only fields accesses are _id, and createdAt via a fetch().
The errors I have gotten are (and the errors are not always the same):
Wed Dec 22 15:00:30 uncaught exception: map reduce failed: {
"assertion" : "assertion scripting/engine_spidermonkey.cpp:286",
"errmsg" : "db assertion failure",
"ok" : 0
}
And
Wed Dec 22 14:36:22 uncaught exception: map reduce failed: {
"assertion" : "assertion scripting/engine_spidermonkey.cpp:512",
"errmsg" : "db assertion failure",
"ok" : 0
}
One seems to indicate missing "properties" and the other some kind of character encoding issue.
Here's an example record:
{
"_id" : NumberLong("17320712503037953"),
"user" :
,
"regionIds" : [
100888,
2635458,
113373,
2635448,
113299,
17641,
113322
],
"text" : "Hey Guy skaha is dead calm http://myloc.me/fxtc1",
"original" : "{\"in_reply_to_status_id_str\":null,\"place\":{\"country_code\":\"\",\"url\":\"http:\\/\\/api.twitter.com\\/1\\/geo\\/id\\/89436cc68723693a.json\",\"bounding_box\":
,\"attributes\":{},\"full_name\":\"Okanagan-Similkameen, British Columbia\",\"country\":\"Canada\",\"name\":\"Okanagan-Similkameen\",\"id\":\"89436cc68723693a\",\"place_type\":\"city\"},\"in_reply_to_user_id\":null,\"text\":\"Hey Guy skaha is dead calm http:\\/\\/myloc.me\\/fxtc1\",\"contributors\":null,\"coordinates\":
{\"type\":\"Point\",\"coordinates\":[-119.61187,49.44703]},\"retweet_count\":0,\"in_reply_to_user_id_str\":null,\"id_str\":\"17320712503037953\",\"retweeted\":false,\"in_reply_to_status_id\":null,\"source\":\"
u003Ca href=\\\"http:\\/\\/www.ubertwitter.com\\/bb\\/download.php\\\" rel=\\\"nofollow\\\"\\u003E\\u00dcberTwitter\\u003C\\/a\\u003E\",\"created_at\":\"Tue Dec 21 20:49:14 +0000 2010\",\"truncated\":false,\"geo\":
,\"favorited\":false,\"user\":
{\"profile_link_color\":\"0084B4\",\"location\":\"\\u00dcT: 49.40707,-119.60601\",\"verified\":false,\"favourites_count\":0,\"profile_sidebar_border_color\":\"C0DEED\",\"id_str\":\"214251894\",\"friends_count\":5,\"is_translator\":false,\"show_all_inline_media\":false,\"geo_enabled\":true,\"profile_use_background_image\":true,\"description\":\"kayak and bicycle enthusiast\",\"contributors_enabled\":false,\"profile_background_color\":\"C0DEED\",\"url\":\"http:\\/\\/rockinrobin.posterous.com\",\"profile_image_url\":\"http:\\/\\/a2.twimg.com\\/profile_images\\/1164994911\\/madbee_normal.jpg\",\"profile_background_image_url\":\"http:\\/\\/a3.twimg.com\\/a\\/1292883740\\/images\\/themes\\/theme1\\/bg.png\",\"created_at\":\"Wed Nov 10 23:24:16 +0000 2010\",\"followers_count\":6,\"follow_request_sent\":null,\"screen_name\":\"BicycleRobin\",\"profile_text_color\":\"333333\",\"protected\":false,\"lang\":\"en\",\"statuses_count\":37,\"notifications\":null,\"profile_sidebar_fill_color\":\"DDEEF6\",\"name\":\"Robin Dunham\",\"following\":null,\"profile_background_tile\":false,\"time_zone\":\"Pacific Time (US & Canada)\",\"id\":214251894,\"listed_count\":5,\"utc_offset\":-28800},\"id\":17320712503037953,\"entities\":{\"urls\":[
{\"indices\":[27,48],\"expanded_url\":null,\"url\":\"http:\\/\\/myloc.me\\/fxtc1\"}],\"hashtags\":[],\"user_mentions\":[]},\"in_reply_to_screen_name\":null}",
"createdAt" : "Tue Dec 21 2010 15:49:14 GMT-0500 (EST)",
"source" : "<a href=\"http://www.ubertwitter.com/bb/download.php\" rel=\"nofollow\">ĆberTwitter</a>",
"entities" : {
"urls" : [
]
}
}
The number of records being operated on is about 65K (not a lot really).
The collection, 'tweets', is being actively written to while the M/R job runs.
Help.