[SERVER-12600] Failure to parse $regex $option 'x' in fromjson() Created: 04/Feb/14  Updated: 23/Feb/21  Resolved: 23/Feb/21

Status: Closed
Project: Core Server
Component/s: Shell
Affects Version/s: 2.6.3
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Ben Becker Assignee: DO NOT USE - Backlog - Platform Team
Resolution: Done Votes: 0
Labels: 26qa, json, move-sa, regex
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-27649 Don't error on serialization of $elem... Closed
is related to SERVER-21974 mongo shell can't explain insensible ... Closed
Operating System: ALL
Steps To Reproduce:

Run the following dbtest:

 
    // Base client helper
    struct ClientBase {
        ClientBase() { mongo::lastError.reset(new LastError()); }
        virtual ~ClientBase() { }
 
    protected:
        static bool error() {
            return !_client.getPrevError().getField( "err" ).isNull();
        }
        static DBDirectClient _client;
    };
    DBDirectClient ClientBase::_client;
 
    // Runner for generated queries
    template <typename Q>
    struct GeneratedQueryRunner : ClientBase {
        virtual void run(){
            auto_ptr<DBClientCursor> cursor = _client.query(kNameSpace, Q::getQuery(), 0);
            ASSERT(!error());
 
            int docCount = 0;
            BSONObj result;
            while (cursor->more()) {
                ++docCount;
                result = cursor->next().getOwned();
            }
            LOG(0) << "Document Count: " << docCount;
        }
    };
 
struct GeneratedQuery19 {
      static BSONObj getQuery() { return query; }
      static BSONObj* getProjection() { return NULL; }
      static BSONObj query;
  };
BSONObj GeneratedQuery19::query = fromjson("{\"d\":{\"$regex\":\"(s|t|y|n)\\..\",\"$options\":'mxisi'}}");
 
...
 
    class All : public Suite {
    public:
        All() : Suite( "GeneratedQueries" ) { }
        void setupTests() {
            add< GeneratedQueryRunner<GeneratedQuery19> >();
        }
    } myall;

Eventually this should included as part of the fuzz/nltk generated tests.

Participants:

 Description   

The following query throws a FailedToParse exception when passed to fromjson():

{"d":{"$regex":"(s|t|y|n)\..","$options":'mxsi'}}

libc++abi.dylib: terminating with uncaught exception of type mongo::MsgAssertionException: code FailedToParse: FailedToParse: Bad regex option: x: offset:47 of:{"d":{"$regex":"(s|t|y|n)\..","$options":'mxsi'}}



 Comments   
Comment by Asya Kamsky [ 23/Feb/21 ]

I can no longer reproduce this issue.

Comment by James Wahlin [ 19/Jan/17 ]

The issue reported in the comments of this ticket, where running .explain() with certain regexes (like the following), has been fixed under SERVER-27649.

> db.reg.find({category:{$regex:"^00",$options:"x"}}).explain()
2014-08-19T16:18:12.965-0400 Error: 16863 Error converting /^00/x in field 0 to a JS RegExp object: SyntaxError: Invalid flags supplied to RegExp constructor 'x' at src/mongo/shell/types.js:616

The originally reported issue, where fromjson() fails to parse a regex with "x" as an option is a different code path and is still an outstanding issue:

std::exception: code FailedToParse: FailedToParse: Bad regex option: x: offset:48 of:{"d":{"$regex":"(s|t|y|n)\..","$options":'mxisi'}}

Comment by Thomas Rueckstiess [ 19/Aug/14 ]

And this is the stack trace from 2.7.4:

MongoDB shell version: 2.7.4
connecting to: test
Server has startup warnings:
2014-08-19T16:24:06.174-0400 [initandlisten]
2014-08-19T16:24:06.174-0400 [initandlisten] ** NOTE: This is a development version (2.7.4) of MongoDB.
2014-08-19T16:24:06.174-0400 [initandlisten] **       Not recommended for production.
2014-08-19T16:24:06.174-0400 [initandlisten]
 
// === no explain, works
> db.reg.find({category:{$regex:"^00",$options:"s"}})
{ "_id" : ObjectId("53f3b019806d1287d7277918"), "category" : "0012345" }
 
// === explain, blows up
> db.reg.find({category:{$regex:"^00",$options:"s"}}).explain()
2014-08-19T16:24:22.479-0400 Assertion: 16686:error converting js type to Utf8Value
2014-08-19T16:24:22.481-0400
 0x10018ec8f 0x10014b180 0x10013a861 0x10010d756 0x1000fcc08 0x1000fc9b9 0x1000fc9b9 0x1000faad3 0x1003d75d7 0x1003d6e86 0x100446c0d 0x10035d92a 0x1003567f5 0x45df1e06362
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"100000000","o":"18EC8F"},{"b":"100000000","o":"14B180"},{"b":"100000000","o":"13A861"},{"b":"100000000","o":"10D756"},{"b":"100000000","o":"FCC08"},{"b":"100000000","o":"FC9B9"},{"b":"100000000","o":"FC9B9"},{"b":"100000000","o":"FAAD3"},{"b":"100000000","o":"3D75D7"},{"b":"100000000","o":"3D6E86"},{"b":"100000000","o":"446C0D"},{"b":"100000000","o":"35D92A"},{"b":"100000000","o":"3567F5"},{"b":"0","o":"45DF1E06362"}],"processInfo":{ "mongodbVersion" : "2.7.4", "gitVersion" : "74daefb6f2330ee42ff035cc593472f41f3f8ed8", "uname" : { "sysname" : "Darwin", "release" : "13.3.0", "version" : "Darwin Kernel Version 13.3.0: Tue Jun  3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64", "machine" : "x86_64" }, "somap" : [ { "path" : "/usr/local/bin/mongo", "machType" : 2, "b" : "100000000", "buildId" : "9B7CEFFF810C3389BE24AC9738018455" }, { "path" : "/usr/lib/libSystem.B.dylib", "machType" : 6, "b" : "7FFF88D0B000", "buildId" : "E303F2F8A8CF3DF384B3F2D0EE41CCF6" }, { "path" : "/usr/lib/libstdc++.6.dylib", "machType" : 6, "b" : "7FFF89721000", "buildId" : "0241E6A4136833BE950BD0A175C41F54" }, { "path" : "/usr/lib/system/libcache.dylib", "machType" : 6, "b" : "7FFF8DA8F000", "buildId" : "549DA205700F353DA201458D99964F9C" }, { "path" : "/usr/lib/system/libcommonCrypto.dylib", "machType" : 6, "b" : "7FFF8E292000", "buildId" : "D1ECE8B9647E3DE08AC9B972E68E822C" }, { "path" : "/usr/lib/system/libcompiler_rt.dylib", "machType" : 6, "b" : "7FFF8E60B000", "buildId" : "2A77D859E2F73769BCACF6971A7CAF7A" }, { "path" : "/usr/lib/system/libcopyfile.dylib", "machType" : 6, "b" : "7FFF944FE000", "buildId" : "CF29DFF605893590834C82E2316612E8" }, { "path" : "/usr/lib/system/libcorecrypto.dylib", "machType" : 6, "b" : "7FFF8BD87000", "buildId" : "F3973C2814B63006BB2B00DD7F09ABC7" }, { "path" : "/usr/lib/system/libdispatch.dylib", "machType" : 6, "b" : "7FFF8FC07000", "buildId" : "C4E4A18D3C3B3C9C8709A4270D998DE7" }, { "path" : "/usr/lib/system/libdyld.dylib", "machType" : 6, "b" : "7FFF8E2E1000", "buildId" : "7C9EC3B7DDE333FF953F4067C743951D" }, { "path" : "/usr/lib/system/libkeymgr.dylib", "machType" : 6, "b" : "7FFF8BC6D000", "buildId" : "CF815A653F2D39D3A58A163574C43B12" }, { "path" : "/usr/lib/system/liblaunch.dylib", "machType" : 6, "b" : "7FFF924A8000", "buildId" : "A40A0C7B321639B48AE0B5D3BAF1DA8A" }, { "path" : "/usr/lib/system/libmacho.dylib", "machType" : 6, "b" : "7FFF920A6000", "buildId" : "22EA37A35E093ABBAA7748EFF64090A5" }, { "path" : "/usr/lib/system/libquarantine.dylib", "machType" : 6, "b" : "7FFF9345B000", "buildId" : "D8D3C33F032838948898D9766F888FE4" }, { "path" : "/usr/lib/system/libremovefile.dylib", "machType" : 6, "b" : "7FFF8BE0E000", "buildId" : "EE47FD189E3334C9869F0B91A5277C3F" }, { "path" : "/usr/lib/system/libsystem_asl.dylib", "machType" : 6, "b" : "7FFF8C25F000", "buildId" : "BA3FA877F584342D8DEE4B548B8B7233" }, { "path" : "/usr/lib/system/libsystem_blocks.dylib", "machType" : 6, "b" : "7FFF8B023000", "buildId" : "FB856CD12AEA39078E9B1E54B6827F82" }, { "path" : "/usr/lib/system/libsystem_c.dylib", "machType" : 6, "b" : "7FFF9679F000", "buildId" : "6FD3A4004BB23B95B90CBE6E9D0D78FA" }, { "path" : "/usr/lib/system/libsystem_configuration.dylib", "machType" : 6, "b" : "7FFF88D08000", "buildId" : "4998CB6A9D54390A9F575D1AC53C135C" }, { "path" : "/usr/lib/system/libsystem_dnssd.dylib", "machType" : 6, "b" : "7FFF9084A000", "buildId" : "17B03FFD92C532829981EBB28B456207" }, { "path" : "/usr/lib/system/libsystem_info.dylib", "machType" : 6, "b" : "7FFF8E2EE000", "buildId" : "6ABA528078A63BCEB28C8080CE5CDFF9" }, { "path" : "/usr/lib/system/libsystem_kernel.dylib", "machType" : 6, "b" : "7FFF8E648000", "buildId" : "873931CED1AF3596AADBD2E63C9AB29F" }, { "path" : "/usr/lib/system/libsystem_m.dylib", "machType" : 6, "b" : "7FFF93475000", "buildId" : "9937893C3E603B35B3957CAF6F2B4C77" }, { "path" : "/usr/lib/system/libsystem_malloc.dylib", "machType" : 6, "b" : "7FFF93538000", "buildId" : "A695B4E438E9332EA77229D31E3F1385" }, { "path" : "/usr/lib/system/libsystem_network.dylib", "machType" : 6, "b" : "7FFF916AA000", "buildId" : "ABD26C2EC3C0329BAB7AD4942783872E" }, { "path" : "/usr/lib/system/libsystem_notify.dylib", "machType" : 6, "b" : "7FFF92FFE000", "buildId" : "E9131A5BEBFA341FBE7B2084426F5435" }, { "path" : "/usr/lib/system/libsystem_platform.dylib", "machType" : 6, "b" : "7FFF9014B000", "buildId" : "3C3D3DA832B9324398ECD89B9A1670B3" }, { "path" : "/usr/lib/system/libsystem_pthread.dylib", "machType" : 6, "b" : "7FFF92FD1000", "buildId" : "9D0EBF77A6ED3D5192141DB9E8731900" }, { "path" : "/usr/lib/system/libsystem_sandbox.dylib", "machType" : 6, "b" : "7FFF928BB000", "buildId" : "0D0B13EA6B7A3AC8BE60B548543BEB77" }, { "path" : "/usr/lib/system/libsystem_stats.dylib", "machType" : 6, "b" : "7FFF8C271000", "buildId" : "4E51D5B092A03D0DB90E495A1ED3E391" }, { "path" : "/usr/lib/system/libunc.dylib", "machType" : 6, "b" : "7FFF8E23E000", "buildId" : "A7E7B983D29D355ABD9543189161B663" }, { "path" : "/usr/lib/system/libunwind.dylib", "machType" : 6, "b" : "7FFF8DA29000", "buildId" : "78DCC3582FC1302EB3950155B47CB547" }, { "path" : "/usr/lib/system/libxpc.dylib", "machType" : 6, "b" : "7FFF88C33000", "buildId" : "AB40CD57F4543FD4B41563B3C0D5C624" }, { "path" : "/usr/lib/libobjc.A.dylib", "machType" : 6, "b" : "7FFF91343000", "buildId" : "AD7FD984271E30F4A3616B20319EC73B" }, { "path" : "/usr/lib/libauto.dylib", "machType" : 6, "b" : "7FFF91537000", "buildId" : "F45C36E8B6063886B5B1B6745E757CA8" }, { "path" : "/usr/lib/libc++abi.dylib", "machType" : 6, "b" : "7FFF90D79000", "buildId" : "21A807D367323455B77F743E9F916DF0" }, { "path" : "/usr/lib/libc++.1.dylib", "machType" : 6, "b" : "7FFF912F0000", "buildId" : "2CB96D101E3E39B1B90DC56C3818321E" }, { "path" : "/usr/lib/libDiagnosticMessagesClient.dylib", "machType" : 6, "b" : "7FFF8C796000", "buildId" : "4CDB0F7BC0AF3424BC39495696F0DB1E" } ] }}
 mongo(_ZN5mongo15printStackTraceERSo+0x2F) [0x10018ec8f]
 mongo(_ZN5mongo10logContextEPKc+0xA0) [0x10014b180]
 mongo(_ZN5mongo11msgassertedEiPKc+0xE1) [0x10013a861]
 mongo(_ZN5mongo11toSTLStringERKN2v86HandleINS0_5ValueEEE+0x36) [0x10010d756]
 mongo(_ZN5mongo7V8Scope16mongoToV8ElementERKNS_11BSONElementEb+0x646) [0x1000fcc08]
 mongo(_ZN5mongo7V8Scope16mongoToV8ElementERKNS_11BSONElementEb+0x3F7) [0x1000fc9b9]
 mongo(_ZN5mongo7V8Scope16mongoToV8ElementERKNS_11BSONElementEb+0x3F7) [0x1000fc9b9]
 mongo(_ZN5mongoL8namedGetEN2v85LocalINS0_6StringEEERKNS0_12AccessorInfoE+0x253) [0x1000faad3]
 mongo(_ZN2v88internal8JSObject26GetPropertyWithInterceptorEPNS0_10JSReceiverEPNS0_6StringEP18PropertyAttributes+0x20B) [0x1003d75d7]
 mongo(_ZN2v88internal6Object23GetPropertyWithReceiverEPS1_PNS0_6StringEP18PropertyAttributes+0x80) [0x1003d6e86]
 mongo(_ZN2v88internal7Runtime17GetObjectPropertyEPNS0_7IsolateENS0_6HandleINS0_6ObjectEEES6_+0x157) [0x100446c0d]
 mongo(_ZN2v88internal11KeyedLoadIC4LoadENS0_16InlineCacheStateENS0_6HandleINS0_6ObjectEEES5_b+0x63C) [0x10035d92a]
 mongo(_ZN2v88internal16KeyedLoadIC_MissENS0_9ArgumentsEPNS0_7IsolateE+0x1C5) [0x1003567f5]
 ??? [0x45df1e06362]
-----  END BACKTRACE  -----
2014-08-19T16:24:22.483-0400 Error: 16686 error converting js type to Utf8Value
    at tojsonObject (src/mongo/shell/types.js:658:20)
    at tojson (src/mongo/shell/types.js:602:17)
    at tojsonObject (src/mongo/shell/types.js:666:57)
    at tojson (src/mongo/shell/types.js:602:17)
    at shellPrintHelper (src/mongo/shell/utils.js:398:15)
    at (shell2):1:19 at src/mongo/shell/types.js:658

Comment by Thomas Rueckstiess [ 19/Aug/14 ]

This is still an issue in 2.6.3 for the options x and s (at least running them through explain). Couldn't confirm if the same issue occurs with fromjson() as mentioned in ticket description.

> db.reg.find({category:{$regex:"^00",$options:"s"}}).explain()
2014-08-19T16:18:10.453-0400 Error: 16863 Error converting /^00/s in field 0 to a JS RegExp object: SyntaxError: Invalid flags supplied to RegExp constructor 's' at src/mongo/shell/types.js:616

> db.reg.find({category:{$regex:"^00",$options:"x"}}).explain()
2014-08-19T16:18:12.965-0400 Error: 16863 Error converting /^00/x in field 0 to a JS RegExp object: SyntaxError: Invalid flags supplied to RegExp constructor 'x' at src/mongo/shell/types.js:616

Comment by Asya Kamsky [ 04/Feb/14 ]

I also ran into something similar - without explain it works, with explain it fails:

test@local(2.5.5) > db.reg.find({category:{$regex:"^00",$options:"i"}}).explain()
{
	"cursor" : "BtreeCursor category_1",
	"isMultiKey" : false,
	"n" : 0,
	"nscannedObjects" : 0,
	"nscanned" : 2,
	"nscannedObjectsAllPlans" : 0,
	"nscannedAllPlans" : 2,
	"scanAndOrder" : false,
	"indexOnly" : false,
	"nYields" : 0,
	"nChunkSkips" : 0,
	"millis" : 0,
	"indexBounds" : {
		"category" : [
			[
				"",
				{
 
				}
			],
			[
				/^00/i,
				/^00/i
			]
		]
	},
	"server" : "asyasmacbook.local:27017"
}
test@local(2.5.5) > db.reg.find({category:{$regex:"^00",$options:"s"}}).explain()
2014-02-04T13:22:04.139-0800 Error: 16863 Error converting /^00/s in field 0 to a JS RegExp object: SyntaxError: Invalid flags supplied to RegExp constructor 's' at src/mongo/shell/types.js:616
test@local(2.5.5) > db.reg.find({category:{$regex:"^Fo",$options:"s"}})
{ "_id" : ObjectId("5182e1f9658b341284135a20"), "category" : "Foo/bar.baz" }
Fetched 1 record(s) in 2ms

Went to check what 2.4.x did with it and got a stack trace on the explain() but it's 2.4.3

Generated at Thu Feb 08 03:29:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.