Keep memory allocations for containers in the pipeline dependency graph

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Optimization
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      This is a prospective performance optimization for the pipline dependency graph.

      We currently keep graph node allocations around when rebuilding the graph, but we destroy and discard data structures within those nodes. 

      For example, Field nodes have a hash_map with the field-level dependencies, Stages do too, and we have a hash_map of each Field in a Scope.

      We can tweak the code in NodeContainer, which uses a "soft-delete" path when resizing down.

      Instead of calling the ~Node destructor, we can soft-delete, which will .clear() the container (allocation remains). On the next graph rebuild, the Field/Scope/Stage node already has the backing memory for the containers is maintains.

            Assignee:
            Unassigned
            Reporter:
            Vesko Karaganev
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: