I have, what seems to be, a relatively simple task, to take data from two collections and "merge" them into a single collection. I have successfully been working with the aggregation framework and the $lookup operator for various other tasks.
I guess maybe the closest analogy i can come up with is a full outer join, but $lookup is really analogous to a left outer join.
As an example, let's say i have the following two collections:
collection1:
{key: 1, val: '1'}
|
{key: 2, val: '2'}
|
collection2:
{key: 3, val: '3'}
|
{key: 4, val: '4'}
|
and the desired end effect is:
collection3:
{key: 1, val: '1'}
|
{key: 2, val: '2'}
|
{key: 3, val: '3'}
|
{key: 4, val: '4'}
|
I know i can accomplish this "client-side", but there could be millions of records in each collection, and i was trying to get this done with as little data movement as possible for efficiency reasons.
My real requirement may also involve some grouping around matching records, such as can be accomplished via the $group operator, but i think that is an orthogonal concern, so i'm leaving it aside for now.
does anyone with more experience than i, have any suggestions around this type of thing?