Expression evaluation runs on the hot path and each step typically touches only a small, transient amount of memory that is released as soon as the evaluation finishes. With the current memory tracking, every one of those small add() / assertWithinMemory() calls propagates up the tracker's base chain to the operation-wide tracker, performs a memory-limit check, and can write the updated usage to CurOp. That per-operand coordination is far more expensive than the tiny allocation it accounts for, and its cost scales with the number of operands, so for cheap, high-frequency expressions the tracking overhead dominates.
The fix is to track locally and flush to the tracker chain periodically.