When making memory.js ensure that the exceptions that it caused were actually out of memory errors in SERVER-36748 we exposed the fact that memory.js was not causing out of memory errors on ASAN builders, but instead timing out, albeit with the wrong error ( see SERVER-41911 ).
I was able to manually verify that memory.js can successfully run under an ASAN build of mongod, but it takes exceedingly long to run and required modifying hard coded timeouts ( https://github.com/mongodb/mongo/blob/master/src/mongo/db/matcher/expression_where.cpp#L90 )
So, I think the best thing to do is to just blacklist memory.js on ASAN builders.