$graphLookup should spill to disk when the size of the frontier and the visited set exceeds the maximum memory usage. In particular, DocumentSourceGraphLookUp::checkMemoryUsage() should be modified to spill if the size of the two data structures is too large. Second, any place where the frontier or visited set is iterated will need to be updated to use a file iterator if it has spilled to disk.

The spilling should be implemented in a method

void spill(long long maximumMemoryUsage)

. The method should track the following metrics

• bool usedDisk : Set to true when the stage has spilled.
• long long spills : The number of times the stage spilled.
• long long spilledBytes : The size, in bytes, of the memory released with spilling.
• long long spilledDataStorageSize : The size, in bytes, of disk space used for spilling.

To track the metrics, we should add a new struct

struct DocumentSourceGraphLookupStats : public SpecificStats

 in plan_stats.h. The metrics should be reported in serverStatus and in explain execution stats.

Before spilling, the stage should make sure that there is enough disk space for spilling. This can be done using ensureSufficientDiskSpaceForSpilling and uassertStatusOK.

A second method, to retrieve the spilled data, should be added to allow the DocumentSourceGraphLookUp to execute reading data from disk. The method should make sure to keep the memory usage below the threshold at any moment.
The stage should release all memory and disk when it is closed.