Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17889

Using eval command to run mapReduce with non-inline "out" option triggers fatal assertion failure

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.0.1
    • Fix Version/s: 3.0.3, 3.1.2
    • Component/s: MapReduce
    • Labels:
      None
    • Backwards Compatibility:
      Minor Change
    • Operating System:
      ALL
    • Backport Completed:
    • Steps To Reproduce:
      Hide

      PHP 5.6.7 (cli) (built: Mar 23 2015 01:51:22)
      Copyright (c) 1997-2015 The PHP Group
      Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
      with Xdebug v2.3.2, Copyright (c) 2002-2015, by Derick Rethans

      Mongo Driver Version 1.6.4

      Here is my code https://gist.github.com/OzzyCzech/f8201b22eadcabf5a8c6

      Show
      PHP 5.6.7 (cli) (built: Mar 23 2015 01:51:22) Copyright (c) 1997-2015 The PHP Group Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies with Xdebug v2.3.2, Copyright (c) 2002-2015, by Derick Rethans Mongo Driver Version 1.6.4 Here is my code https://gist.github.com/OzzyCzech/f8201b22eadcabf5a8c6
    • Sprint:
      Quint Iteration 3.1.2

      Description

      Issue Status as of Apr 29, 2015

      ISSUE SUMMARY
      Changes introduced in the 3.0 cycle make the mapReduce command incompatible with execution inside the the eval command, in the case where the out option of mapReduce specifies a collection. In affected versions of the server, this leads to a fatal assertion. Given the deprecation of eval in 3.0, initiating mapReduce operations from within eval is now explicitly forbidden and produces an error message.

      USER IMPACT
      Running mapReduce in eval where the out option specifies a collection would lead to a fatal assertion. With this change, these operations are not allowed and lead to an operation error.

      WORKAROUNDS
      Run mapReduce operations directly in the client and do not use eval.

      AFFECTED VERSIONS
      3.0.0, 3.0.1, and 3.0.2

      FIX VERSION
      The fix is included in the 3.0.3 production release.

      Original description

      2015-04-05T06:49:13.681+0200 I CONTROL  [initandlisten] db version v3.0.1
      2015-04-05T06:49:13.681+0200 I CONTROL  [initandlisten] git version: nogitversion
      2015-04-05T06:49:13.681+0200 I CONTROL  [initandlisten] build info: Darwin miniyosemite.local 14.1.0 Darwin Kernel Version 14.1.0: Thu Feb 26 19:26:47 PST 2015; root:xnu-2782.10.73~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
      2015-04-05T06:49:13.681+0200 I CONTROL  [initandlisten] allocator: system
      2015-04-05T06:49:13.681+0200 I CONTROL  [initandlisten] options: { config: "/usr/local/etc/mongod.conf", net: { bindIp: "127.0.0.1" }, storage: { dbPath: "/usr/local/var/mongodb" }, systemLog: { destination: "file", logAppend: true, path: "/usr/local/var/log/mongodb/mongo.log" } }
      2015-04-05T06:49:14.383+0200 I NETWORK  [initandlisten] waiting for connections on port 27017
      2015-04-05T06:49:17.361+0200 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:49729 #1 (1 connection now open)
      2015-04-05T06:49:17.424+0200 E QUERY    [conn1] ReferenceError: db is not defined
          at _funcs1:11:1
      2015-04-05T06:49:17.474+0200 I COMMAND  [conn1] CMD: drop testomato.tmp.mr.projects_0
      2015-04-05T06:49:17.477+0200 I -        [conn1] Invariant failure !txn->lockState()->isLocked() src/mongo/db/commands/mr.cpp 550
      2015-04-05T06:49:17.488+0200 I CONTROL  [conn1] 
       0x10d1268da 0x10d0d343e 0x10d0bf734 0x10cb09670 0x10cb13acd 0x10cb7198a 0x10cb72b97 0x10cb73c45 0x10cd5a4d8 0x10cc55588 0x10cb83da8 0x10cb83f92 0x10c9e8ea3 0x10c9d0af3 0x10cb84153 0x10d0b06c3 0x10d0a3642 0x10d22fb76 0x1f020bf06362 0x1f020bf42ac2 0x1f020bf42727 0x1f020bf41418
      ----- BEGIN BACKTRACE -----
      {"backtrace":[{"b":"10C97E000","o":"7A88DA"},{"b":"10C97E000","o":"75543E"},{"b":"10C97E000","o":"741734"},{"b":"10C97E000","o":"18B670"},{"b":"10C97E000","o":"195ACD"},{"b":"10C97E000","o":"1F398A"},{"b":"10C97E000","o":"1F4B97"},{"b":"10C97E000","o":"1F5C45"},{"b":"10C97E000","o":"3DC4D8"},{"b":"10C97E000","o":"2D7588"},{"b":"10C97E000","o":"205DA8"},{"b":"10C97E000","o":"205F92"},{"b":"10C97E000","o":"6AEA3"},{"b":"10C97E000","o":"52AF3"},{"b":"10C97E000","o":"206153"},{"b":"10C97E000","o":"7326C3"},{"b":"10C97E000","o":"725642"},{"b":"10C97E000","o":"8B1B76"},{"b":"0","o":"1F020BF06362"},{"b":"0","o":"1F020BF42AC2"},{"b":"0","o":"1F020BF42727"},{"b":"0","o":"1F020BF41418"}],"processInfo":{ "mongodbVersion" : "3.0.1", "gitVersion" : "nogitversion", "uname" : { "sysname" : "Darwin", "release" : "14.1.0", "version" : "Darwin Kernel Version 14.1.0: Thu Feb 26 19:26:47 PST 2015; root:xnu-2782.10.73~1/RELEASE_X86_64", "machine" : "x86_64" }, "somap" : [ { "path" : "/usr/local/opt/mongodb/bin/mongod", "machType" : 2, "b" : "10C97E000", "buildId" : "5CF6C5C246FF31509ACE3D6A17797C07" }, { "path" : "/usr/lib/libSystem.B.dylib", "machType" : 6, "b" : "7FFF86A72000", "buildId" : "90B107BCFF7432CCB1CF4E02F544D957" }, { "path" : "/usr/lib/libc++.1.dylib", "machType" : 6, "b" : "7FFF90328000", "buildId" : "1B9530FD989B3174BB1CBDC159501710" }, { "path" : "/usr/lib/system/libcache.dylib", "machType" : 6, "b" : "7FFF8749C000", "buildId" : "45E9A2E799C436B2BEE30C4E11614AD1" }, { "path" : "/usr/lib/system/libcommonCrypto.dylib", "machType" : 6, "b" : "7FFF911C8000", "buildId" : "D381EBC669D831D380845A80A32CB748" }, { "path" : "/usr/lib/system/libcompiler_rt.dylib", "machType" : 6, "b" : "7FFF8887F000", "buildId" : "BF8FC133EE103DA69B9092039E28678F" }, { "path" : "/usr/lib/system/libcopyfile.dylib", "machType" : 6, "b" : "7FFF911E0000", "buildId" : "0C68D3A6ACDD3EF3991ACC82C32AB836" }, { "path" : "/usr/lib/system/libcorecrypto.dylib", "machType" : 6, "b" : "7FFF92431000", "buildId" : "E178980139853949B7366B3378873301" }, { "path" : "/usr/lib/system/libdispatch.dylib", "machType" : 6, "b" : "7FFF8D12B000", "buildId" : "502CF32B669B3709886208188225E4F0" }, { "path" : "/usr/lib/system/libdyld.dylib", "machType" : 6, "b" : "7FFF94CF6000", "buildId" : "4E33E416F1D83598B8CC6863E2ECD0E6" }, { "path" : "/usr/lib/system/libkeymgr.dylib", "machType" : 6, "b" : "7FFF864D0000", "buildId" : "77845842DE703CC5BD01C3D14227CED5" }, { "path" : "/usr/lib/system/liblaunch.dylib", "machType" : 6, "b" : "7FFF8AA4B000", "buildId" : "DFCDEBDF82473DC79879E7E497DDA4B4" }, { "path" : "/usr/lib/system/libmacho.dylib", "machType" : 6, "b" : "7FFF8EAAD000", "buildId" : "126CA2EDDE91308F8881B9DAEC3C63B6" }, { "path" : "/usr/lib/system/libquarantine.dylib", "machType" : 6, "b" : "7FFF8E6F8000", "buildId" : "DC0416272D92361CBABFA869A5C72293" }, { "path" : "/usr/lib/system/libremovefile.dylib", "machType" : 6, "b" : "7FFF8E3A4000", "buildId" : "3485B5F46CE83C628DFD8736ED6E8531" }, { "path" : "/usr/lib/system/libsystem_asl.dylib", "machType" : 6, "b" : "7FFF86C90000", "buildId" : "F153AC5B0542356E88C820A62CA704E2" }, { "path" : "/usr/lib/system/libsystem_blocks.dylib", "machType" : 6, "b" : "7FFF86657000", "buildId" : "9615D10AFCA73BE4AA1A1B195DACE1A1" }, { "path" : "/usr/lib/system/libsystem_c.dylib", "machType" : 6, "b" : "7FFF91D2B000", "buildId" : "199ED5EB77A13D43AA5181779CE0A742" }, { "path" : "/usr/lib/system/libsystem_configuration.dylib", "machType" : 6, "b" : "7FFF895FC000", "buildId" : "5E14864E089A3D8485A4980B776427A8" }, { "path" : "/usr/lib/system/libsystem_coreservices.dylib", "machType" : 6, "b" : "7FFF8D2D5000", "buildId" : "41B7C5785A5331C8A96FC73E030B0938" }, { "path" : "/usr/lib/system/libsystem_coretls.dylib", "machType" : 6, "b" : "7FFF9241F000", "buildId" : "3EAED90A7AA0323CA52BE16477981D59" }, { "path" : "/usr/lib/system/libsystem_dnssd.dylib", "machType" : 6, "b" : "7FFF8AE6A000", "buildId" : "62B70ECAE40D3C63896E7F00EC386DDB" }, { "path" : "/usr/lib/system/libsystem_info.dylib", "machType" : 6, "b" : "7FFF86C67000", "buildId" : "B85A85D585303A93B0C34DEC41F79478" }, { "path" : "/usr/lib/system/libsystem_kernel.dylib", "machType" : 6, "b" : "7FFF88D28000", "buildId" : "97CD7ACDEA0C3434BEFCFCD013D6BB73" }, { "path" : "/usr/lib/system/libsystem_m.dylib", "machType" : 6, "b" : "7FFF8D1B0000", "buildId" : "1E12AB456D9636D0A226F24D9FB0D9D6" }, { "path" : "/usr/lib/system/libsystem_malloc.dylib", "machType" : 6, "b" : "7FFF8608B000", "buildId" : "19BCC25757173502A71F95D65AFA861B" }, { "path" : "/usr/lib/system/libsystem_network.dylib", "machType" : 6, "b" : "7FFF872A3000", "buildId" : "2EC3A005473F3C36A665F88B5BACC7F0" }, { "path" : "/usr/lib/system/libsystem_networkextension.dylib", "machType" : 6, "b" : "7FFF88630000", "buildId" : "29AB225BD7FB30ED960065D44B9A9442" }, { "path" : "/usr/lib/system/libsystem_notify.dylib", "machType" : 6, "b" : "7FFF864D1000", "buildId" : "61147800F3203DAA850CBADF33855F29" }, { "path" : "/usr/lib/system/libsystem_platform.dylib", "machType" : 6, "b" : "7FFF8CB79000", "buildId" : "64E34079D7123D669CE2418624A5C040" }, { "path" : "/usr/lib/system/libsystem_pthread.dylib", "machType" : 6, "b" : "7FFF8B24C000", "buildId" : "3103AA7F3BAE3673964947FFD7E15C97" }, { "path" : "/usr/lib/system/libsystem_sandbox.dylib", "machType" : 6, "b" : "7FFF8E56C000", "buildId" : "95312E09DA28324AA084F3E574D0210E" }, { "path" : "/usr/lib/system/libsystem_secinit.dylib", "machType" : 6, "b" : "7FFF95096000", "buildId" : "581DAD0F6B633A48B63B917AF799ABAA" }, { "path" : "/usr/lib/system/libsystem_stats.dylib", "machType" : 6, "b" : "7FFF8EE8E000", "buildId" : "9B8CCF24DDDB399A92374BEC225D2E8C" }, { "path" : "/usr/lib/system/libsystem_trace.dylib", "machType" : 6, "b" : "7FFF910D2000", "buildId" : "A9E6B7D8C3273742AC5486C94218B1DF" }, { "path" : "/usr/lib/system/libunc.dylib", "machType" : 6, "b" : "7FFF94CF5000", "buildId" : "5676F7EAC1DF329FB006D2C3022B7D70" }, { "path" : "/usr/lib/system/libunwind.dylib", "machType" : 6, "b" : "7FFF91D25000", "buildId" : "BE7E51A0B6EA3A549CCA9D88F683A6D6" }, { "path" : "/usr/lib/system/libxpc.dylib", "machType" : 6, "b" : "7FFF90EB4000", "buildId" : "876216DCD5D3381E8AF949AE464E5107" }, { "path" : "/usr/lib/libobjc.A.dylib", "machType" : 6, "b" : "7FFF8F185000", "buildId" : "759E155DBC423D4E869B6F57D477177C" }, { "path" : "/usr/lib/libauto.dylib", "machType" : 6, "b" : "7FFF89558000", "buildId" : "A260789BD4D8316A9490254767B8A5F1" }, { "path" : "/usr/lib/libc++abi.dylib", "machType" : 6, "b" : "7FFF8F103000", "buildId" : "88A22A0F87C63002BFBAAC0F2808B8B9" }, { "path" : "/usr/lib/libDiagnosticMessagesClient.dylib", "machType" : 6, "b" : "7FFF91DFF000", "buildId" : "2EE8E4365CDC34C599595BA218D507FB" } ] }}
       mongod(_ZN5mongo15printStackTraceERNSt3__113basic_ostreamIcNS0_11char_traitsIcEEEE+0x3A) [0x10d1268da]
       mongod(_ZN5mongo10logContextEPKc+0x10E) [0x10d0d343e]
       mongod(_ZN5mongo15invariantFailedEPKcS1_j+0x1B4) [0x10d0bf734]
       mongod(_ZN5mongo2mr5State21postProcessCollectionEPNS_16OperationContextEPNS_5CurOpERNS_19ProgressMeterHolderE+0x190) [0x10cb09670]
       mongod(_ZN5mongo2mr16MapReduceCommand3runEPNS_16OperationContextERKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEERNS_7BSONObjEiRSA_RNS_14BSONObjBuilderEb+0x125D) [0x10cb13acd]
       mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEERNS_7BSONObjEiRSA_RNS_14BSONObjBuilderEb+0x3A) [0x10cb7198a]
       mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xC77) [0x10cb72b97]
       mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x735) [0x10cb73c45]
       mongod(_ZN5mongo8runQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERKNS_15NamespaceStringERNS_5CurOpES3_+0xF18) [0x10cd5a4d8]
       mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xA58) [0x10cc55588]
       mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE+0xA8) [0x10cb83da8]
       mongod(_ZThn152_N5mongo14DBDirectClient4callERNS_7MessageES2_bPNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEE+0x12) [0x10cb83f92]
       mongod(_ZN5mongo14DBClientCursor4initEv+0x93) [0x10c9e8ea3]
       mongod(_ZN5mongo12DBClientBase5queryERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEENS_5QueryEiiPKNS_7BSONObjEii+0xA3) [0x10c9d0af3]
       mongod(_ZN5mongo14DBDirectClient5queryERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEENS_5QueryEiiPKNS_7BSONObjEii+0x53) [0x10cb84153]
       mongod(_ZN5mongo9mongoFindEPNS_7V8ScopeERKN2v89ArgumentsE+0x2C3) [0x10d0b06c3]
       mongod(_ZN5mongo7V8Scope10v8CallbackERKN2v89ArgumentsE+0x52) [0x10d0a3642]
       mongod(_ZN2v88internalL21Builtin_HandleApiCallENS0_12_GLOBAL__N_116BuiltinArgumentsILNS0_21BuiltinExtraArgumentsE1EEEPNS0_7IsolateE+0x1F6) [0x10d22fb76]
       ??? [0x1f020bf06362]
       ??? [0x1f020bf42ac2]
       ??? [0x1f020bf42727]
       ??? [0x1f020bf41418]
      -----  END BACKTRACE  -----
      2015-04-05T06:49:17.488+0200 I -        [conn1] 
       
      ***aborting after invariant() failure
      

        Issue Links

          Activity

          Hide
          rassi J Rassi (Inactive) added a comment - - edited

          Hi,

          Thanks for reporting this issue. I can confirm that a fatal assertion failure is triggered when the "eval" command is used invoke the "mapReduce" command with a non-inline "out" option. This is a regression introduced in MongoDB 3.0.0.

          As a workaround, you can invoke the map-reduce functionality directly with the "mapReduce" command, instead of using the "eval" command. From your PHP application, use the MongoDB::command() helper instead of the MongoDB::execute() helper.

          Please continue to watch this ticket for updates on when a fix will be available.

          ~ Jason Rassi

          Show
          rassi J Rassi (Inactive) added a comment - - edited Hi, Thanks for reporting this issue. I can confirm that a fatal assertion failure is triggered when the "eval" command is used invoke the "mapReduce" command with a non-inline "out" option. This is a regression introduced in MongoDB 3.0.0. As a workaround, you can invoke the map-reduce functionality directly with the "mapReduce" command, instead of using the "eval" command. From your PHP application, use the MongoDB::command() helper instead of the MongoDB::execute() helper. Please continue to watch this ticket for updates on when a fix will be available. ~ Jason Rassi
          Hide
          OzzyCzech Roman added a comment -

          Hi,

          thanks for quick response.

          Will be there any way how to execute simple Javascripts from PHP since eval is deprecated? Or we will need rewrite everything to command or mapReduce to cluttering PHP arrays?

          Roman

          Show
          OzzyCzech Roman added a comment - Hi, thanks for quick response. Will be there any way how to execute simple Javascripts from PHP since eval is deprecated? Or we will need rewrite everything to command or mapReduce to cluttering PHP arrays? Roman
          Hide
          jmikola Jeremy Mikola added a comment -

          Will be there any way how to execute simple Javascripts from PHP since eval is deprecated?

          Roman: Since the service command itself is being deprecated and ultimately removed in 3.2, no driver will be able to do that when running against a 3.2+ server. Since drivers do need to support older server versions and preserve their existing API, the eval command methods will still be around. PHP's API is unfortunate in this respect, as our helper method is historically named execute(), which means some users aren't aware it is a wrapper for eval. We have added a warning to MongoDB::execute()'s documentation to note that the server command is deprecated.

          Or we will need rewrite everything to command or mapReduce to cluttering PHP arrays?

          Unless you're using a driver abstraction (e.g. Doctrine MongoDB) that implements a convenience method for mapReduce, you will need to use MongoDB::command(). The usage examples include a demonstration of executing mapReduce. If you're concerned about code clutter (assuming you invoke mapReduce in numerous places), you could always abstract that into a method in your application that mimics the shell's db.collection.mapReduce() method.

          Show
          jmikola Jeremy Mikola added a comment - Will be there any way how to execute simple Javascripts from PHP since eval is deprecated? Roman : Since the service command itself is being deprecated and ultimately removed in 3.2, no driver will be able to do that when running against a 3.2+ server. Since drivers do need to support older server versions and preserve their existing API, the eval command methods will still be around. PHP's API is unfortunate in this respect, as our helper method is historically named execute() , which means some users aren't aware it is a wrapper for eval . We have added a warning to MongoDB::execute()'s documentation to note that the server command is deprecated. Or we will need rewrite everything to command or mapReduce to cluttering PHP arrays? Unless you're using a driver abstraction (e.g. Doctrine MongoDB ) that implements a convenience method for mapReduce, you will need to use MongoDB::command() . The usage examples include a demonstration of executing mapReduce. If you're concerned about code clutter (assuming you invoke mapReduce in numerous places), you could always abstract that into a method in your application that mimics the shell's db.collection.mapReduce() method.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

          Message: SERVER-17889 MapReduceCommand::run() error out from DBDirectClient
          Branch: master
          https://github.com/mongodb/mongo/commit/93c8c001e4462a1f0c0eeed12901fadbdc1861d8

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'} Message: SERVER-17889 MapReduceCommand::run() error out from DBDirectClient Branch: master https://github.com/mongodb/mongo/commit/93c8c001e4462a1f0c0eeed12901fadbdc1861d8
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

          Message: SERVER-17889 MapReduceCommand::run() error out from DBDirectClient

          (cherry picked from commit 93c8c001e4462a1f0c0eeed12901fadbdc1861d8)
          Branch: v3.0
          https://github.com/mongodb/mongo/commit/4afe8edcb27537f6d2f13dd7ccae700cc3b4efb8

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'} Message: SERVER-17889 MapReduceCommand::run() error out from DBDirectClient (cherry picked from commit 93c8c001e4462a1f0c0eeed12901fadbdc1861d8) Branch: v3.0 https://github.com/mongodb/mongo/commit/4afe8edcb27537f6d2f13dd7ccae700cc3b4efb8

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                  Agile