[CSHARP-2049] MapReduce does not work on DB version 3.4.5 Created: 29/Sep/17  Updated: 27/Oct/23  Resolved: 04/Oct/17

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: 2.4.4
Fix Version/s: None

Type: Bug Priority: Blocker - P1
Reporter: Paolo Laurenti Assignee: Robert Stam
Resolution: Works as Designed Votes: 0
Labels: map_reduce, mapreduce
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
is caused by SERVER-31374 If map function string ends in a semi... Closed
Related

 Description   

MapReduce when run on db with 3.4.5. version does not produce any result. (Both with inline and output collection).

I tried with C# driver 2.4.4:

  • on 3.4.5 version the MapReduce does not work
  • on 3.0.2 version everything works fine.

Here's a simple snippet to use to reproduce the bug:

// Example of input documents
// {
//    "_id" : ObjectId("59c52b3cb602cb6397c2ec9d"),
//    "Timestamp" : NumberLong(1505860144116),
//    "Value" : 14,
//    "Date" : ISODate("2017-09-19T22:29:04.116Z")
// }
 
public class Program
{
    private const string MapJs = @"function mapF() {
    const key = this.Date.getFullYear();
    const valuePerYear = { total: 1};
 
    emit(key, valuePerYear);
}; ";
 
    private const string ReduceJS = @"function reduceF(year, values) {
    let sum = 0;
    values.forEach(v => {
        sum += v.total;
    });
    return {total: NumberInt(sum)};
}";
 
    public static void Main()
    {
        string mongoConnectionString = "my-connection-string";
        MongoUrl mongoUrl = MongoUrl.Create(mongoConnectionString);
        MongoClient client = new MongoClient(mongoConnectionString);
        IMongoDatabase db = client.GetDatabase("ny_database_name");
        IMongoCollection<BsonDocument> collection = db.GetCollection<BsonDocument>("DocInput");
        BsonJavaScript map = new BsonJavaScript(MapJs);
        BsonJavaScript reduce = new BsonJavaScript(ReduceJS);
        FilterDefinitionBuilder<BsonDocument> filterBuilder = new FilterDefinitionBuilder<BsonDocument>();
        FilterDefinition<BsonDocument> filter = filterBuilder.Empty;
        MapReduceOptions<BsonDocument, BsonDocument> options = new MapReduceOptions<BsonDocument, BsonDocument>
        {
            Filter = filter,
            MaxTime = TimeSpan.FromMinutes(1),
            OutputOptions = MapReduceOutputOptions.Reduce("Result", nonAtomic: true),
            Verbose = true
        };
        try
        {
            collection.MapReduce(map, reduce, options).ToList();
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Exception occurred {ex.Message}");
        }
    }
}



 Comments   
Comment by Robert Stam [ 04/Oct/17 ]

Thanks for confirming that removing the semicolon worked for you.

Comment by Paolo Laurenti [ 04/Oct/17 ]

You are right.
I removed the trailing semicolon in the map and reduce functions and everything worked fine.

I agree with you that the silent fail is a problem.

Thank you very much.

Comment by Robert Stam [ 03/Oct/17 ]

I've created a SERVER ticket for this.

I'm not necessarily surprised that a trailing semicolon would not work, but failing silently is a problem.

Comment by Robert Stam [ 03/Oct/17 ]

After a lot of trail and error, it appears that the issue is caused by the trailing semicolon in the string constants for the map and reduce functions:

        private const string MapJs = @"function mapF() {
    const key = this.Date.getFullYear();
    const valuePerYear = { total: 1};
 
    emit(key, valuePerYear);
};";    
 
        private const string ReduceJS = @"function reduceF(year, values) {
    let sum = 0;
    values.forEach(v => {
        sum += v.total;
    });
    return {total: NumberInt(sum)};
};";

If you remove the trailing semi colon it works.

Can you confirm?

Comment by Paolo Laurenti [ 03/Oct/17 ]

Awesome!
Thank you very much.

Comment by Robert Stam [ 03/Oct/17 ]

OK. I'm focusing on testing with 3.4.9 (should be the same as 3.4.5, if not I'll download 3.4.5 and test against that also).

Since you find that it works with 3.0.2 (after rewriting the reduce function to not use arrow functions) I won't test against 3.0.2.

I have reproduced that when run from C# with 3.4.9 the outcome is an empty Result collection.

While single stepping I see that that server has returned the following result to the mapreduce command:

"{ \"result\" : { \"db\" : \"test\", \"collection\" : \"Result\" }, \"timeMillis\" : 58, \"counts\" : { \"input\" : 1, \"emit\" : 0, \"reduce\" : 0, \"output\" : 0 }, \"ok\" : 1.0 }"

Note that the emit count is 0. When run from the shell the emit count is 1.

So that's a clue to the problem there.

I'll continue investigating.

Comment by Paolo Laurenti [ 03/Oct/17 ]

Yes, you're right. With 3.0.2 you need to change the anonymous functions from arrow functions to standard function.
No problem with that.

The focus of the issue I am reporting is about running MapReduce from C# .NET code on dbs with versions >= 3.4.5.

Comment by Paolo Laurenti [ 03/Oct/17 ]

Sorry Robert, probably my description was not very clear.
I try to better explain the issue.

The subject of this ticket is the C# .NET Driver so, I am reporting a problem running MapReduce from C# code.
In my tests the MapReduce run from the shell worked every time.

The problem I am facing occurs when I try to run MapReduce from C# code.

As I stated in my description:

I tried with C# driver 2.4.4:
on 3.4.5 version the MapReduce does not work
on 3.0.2 version everything works fine.

I've never tried with 3.4.9 version.

Let me know if now the problem is more clear and if I can do something else to help you.

Comment by Robert Stam [ 03/Oct/17 ]

Thanks for the quick response. In your original description you stated that it worked with 3.0.2 but not with 3.4.5.

Based on your latest information I think it never worked with 3.0.2 (most likely because the Javascript interpreter present in that version of the server does not support arrow functions).

So the remaining question is whether it works correctly with 3.4.5 (or 3.4.9 which is the current version).

When you run it from the shell does it work correctly? (i.e. does it result in a non-empty Result collection?). That would help determine whether the issue is with the map/reduce functions or with the C# driver.

Edit: when I run the most recent example in the shell using the single sample document in the InputCollection I get the following Result collection:

> db.Result.find()
{ "_id" : 2017, "value" : { "total" : 1 } }
>

I'll attempt again to reproduce using C# now. But only against 3.4.9. I don't think there's any chance this will work on 3.0.2 because of the arrow function.

Comment by Paolo Laurenti [ 03/Oct/17 ]

I tried the following code in my mongoDB shell (3.4.5) and everything works fine.
Here's the code you can paste in your shell (changing the input and output collection).

function mapF() {
    const key = this.Date.getFullYear();
    const valuePerYear = { total: 1};
 
    emit(key, valuePerYear);
};    
 
function reduceF(year, values) {
    let sum = 0;
    values.forEach(v => {
        sum += v.total;
    });
    return {total: NumberInt(sum)};
};
 
db.getCollection('InputCollection').mapReduce(mapF, reduceF, { out: { reduce: "Result", nonAtomic: true }});

I copied and pasted the "mapF" and "reduceF" function in my C# code (driver 2.4.4) and the output is an empty "Result" collection (no exception raised).

Here's the C# code:

using System;
using MongoDB.Bson;
using MongoDB.Driver;
 
namespace TestMongoDB
{
 
    public class Program
    {
        private const string MapJs = @"function mapF() {
    const key = this.Date.getFullYear();
    const valuePerYear = { total: 1};
 
    emit(key, valuePerYear);
};";    
 
        private const string ReduceJS = @"function reduceF(year, values) {
    let sum = 0;
    values.forEach(v => {
        sum += v.total;
    });
    return {total: NumberInt(sum)};
};";
 
        public static void Main()
        {
            string mongoConnectionString = "my-connection-string";
            MongoClient client = new MongoClient(mongoConnectionString);
            IMongoDatabase db = client.GetDatabase("stat");
            IMongoCollection<BsonDocument> collection = db.GetCollection<BsonDocument>("InputCollection");
            BsonJavaScript map = new BsonJavaScript(MapJs);
            BsonJavaScript reduce = new BsonJavaScript(ReduceJS);
            FilterDefinitionBuilder<BsonDocument> filterBuilder = new FilterDefinitionBuilder<BsonDocument>();
            FilterDefinition<BsonDocument> filter = filterBuilder.Empty;
            MapReduceOptions<BsonDocument, BsonDocument> options = new MapReduceOptions<BsonDocument, BsonDocument>
            {
                Filter = filter,
                OutputOptions = MapReduceOutputOptions.Reduce("Result", nonAtomic: true),
            };
            try
            {
                collection.MapReduce(map, reduce, options).ToList();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Exception occurred {ex.Message}");
            }
        }
    }
}

Let me know if I can make some other test in order to help you.

Thanks
Paolo

Comment by Robert Stam [ 03/Oct/17 ]

I tried to reproduce this and for me it doesn't work with 3.0.2 either.

The problem seems to be with the ReduceJS string constant. If I paste it into the mongo shell it doesn't parse correctly:

PS C:\mongodb\v3.0.2\bin> .\mongo
MongoDB shell version: 3.0.2
connecting to: test
> function reduceF(year, values) {
...     let sum = 0;
...     values.forEach(v => {
...         sum += v.total;
...     });
...     return {total: NumberInt(sum)};
... }
2017-10-03T11:33:57.704-0400 E QUERY    SyntaxError: Unexpected identifier
>

Can you try debugging your map reduce in the shell first and then pasting the functions back into your C# test program?

Generated at Wed Feb 07 21:41:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.