Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-65805

libdeps analyzers fails for some critical edges.

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 6.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • ALL
    • Dev Platform 2022-05-16

      Note unit tests pass with existing codebase but fail with changes below. Below we tried to get the critical edge:

      /opt/mongodbtoolchain/v4/bin/python3 buildscripts/libdeps/gacli.py --graph-file build/opt/libdeps/libdeps.graphml --critical-edges build/opt/mongo/util/util_test build/opt/mongo/scripting/libscripting.so
      Loading graph data...Loaded!
      
      
      Traceback (most recent call last):
        File "/home/ubuntu/mongo/buildscripts/libdeps/gacli.py", line 265, in <module>
          main()
        File "/home/ubuntu/mongo/buildscripts/libdeps/gacli.py", line 252, in main
          ga = libdeps_analyzer.LibdepsGraphAnalysis(analysis)
        File "/home/ubuntu/mongo/buildscripts/libdeps/libdeps/analyzer.py", line 702, in __init__
          analyzer.report(self._results)
        File "/home/ubuntu/mongo/buildscripts/libdeps/libdeps/analyzer.py", line 563, in report
          self._to_node])] = self.run()
        File "/home/ubuntu/mongo/buildscripts/libdeps/libdeps/analyzer.py", line 95, in check
          return func(*args, **kwargs)
        File "/home/ubuntu/mongo/buildscripts/libdeps/libdeps/analyzer.py", line 552, in run
          minimum_st_edge_cut(
        File "/opt/mongodbtoolchain/revisions/347afa180670768de1b57d19832e898216d433ed/stow/python3-v4.Ffz/lib/python3.9/site-packages/networkx/algorithms/connectivity/cuts.py", line 145, in minimum_st_edge_cut
          cut_value, partition = nx.minimum_cut(H, s, t, **kwargs)
        File "/opt/mongodbtoolchain/revisions/347afa180670768de1b57d19832e898216d433ed/stow/python3-v4.Ffz/lib/python3.9/site-packages/networkx/algorithms/flow/maxflow.py", line 458, in minimum_cut
          R = flow_func(flowG, _s, _t, capacity=capacity, value_only=True, **kwargs)
        File "/opt/mongodbtoolchain/revisions/347afa180670768de1b57d19832e898216d433ed/stow/python3-v4.Ffz/lib/python3.9/site-packages/networkx/algorithms/flow/edmondskarp.py", line 237, in edmonds_karp
          R = edmonds_karp_impl(G, s, t, capacity, residual, cutoff)
        File "/opt/mongodbtoolchain/revisions/347afa180670768de1b57d19832e898216d433ed/stow/python3-v4.Ffz/lib/python3.9/site-packages/networkx/algorithms/flow/edmondskarp.py", line 99, in edmonds_karp_impl
          raise nx.NetworkXError(f"node {str(t)} not in graph")
      networkx.exception.NetworkXError: node mongo/util/util_test not in graph 

      Then when we give networkx the full graph, it finds some critical edges:

      diff --git a/buildscripts/libdeps/libdeps/analyzer.py b/buildscripts/libdeps/libdeps/analyzer.py
      index c440fcf7d8f..a37c0d7ca50 100644
      --- a/buildscripts/libdeps/libdeps/analyzer.py
      +++ b/buildscripts/libdeps/libdeps/analyzer.py
      @@ -550,7 +550,7 @@ class CriticalEdges(Analyzer):
           # would encounter on a given path from the from_node to the to_node.
           min_cut_edges = list(
             minimum_st_edge_cut(
      -        G=self._dependents_graph.get_direct_nonprivate_graph().get_node_tree(self._to_node),
      +        G=self._dependents_graph,
               s=self._to_node, t=self._from_node))
           return [(edge[1], edge[0]) for edge in min_cut_edges]
      
      /opt/mongodbtoolchain/v4/bin/python3 buildscripts/libdeps/gacli.py --graph-file build/opt/libdeps/libdeps.graphml --critical-edges build/opt/mongo/util/util_test build/opt/mongo/scripting/libscripting.so
      Loading graph data...Loaded!
      
      Graph built from git hash:
      572cb2ea4d872432ce95c0de1c5cf4d37641b50a
      
      Graph Schema version:
      3
      
      Build invocation:
      "/opt/mongodbtoolchain/v4/bin/python3" "../scons/scripts/scons.py" "--link-model=dynamic" "--variables-files=etc/scons/mongodbtoolchain_v3_clang.vars" "ICECC=icecc" "CCACHE=ccache" "generate-libdeps-graph" "-j400"
      
      Nodes in Graph: 1035
      Edges in Graph: 127538
      Direct Edges in Graph: 7630
      Transitive Edges in Graph: 119908
      Direct Public Edges in Graph: 4024
      Public Edges in Graph: 123932
      Private Edges in Graph: 3580
      Interface Edges in Graph: 26
      Program Nodes in Graph: 178
      Library Nodes in Graph: 857
      
      Critical Edges:
      =>critical edges between mongo/util/util_test and mongo/scripting/libscripting.so:
        1: ('mongo/util/util_test', 'mongo/util/liblatch_analyzer.so')
        2: ('mongo/util/util_test', 'mongo/executor/libthread_pool_task_executor_test_fixture.so')
        3: ('mongo/util/util_test', 'mongo/executor/libtask_executor_test_fixture.so')
        4: ('mongo/util/util_test', 'mongo/executor/libnetwork_interface_mock.so')
      

            Assignee:
            daniel.moody@mongodb.com Daniel Moody
            Reporter:
            daniel.moody@mongodb.com Daniel Moody
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: