[SERVER-65805] libdeps analyzers fails for some critical edges. Created: 19/Apr/22  Updated: 29/Oct/23  Resolved: 05/May/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Daniel Moody Assignee: Daniel Moody
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Dev Platform 2022-05-16
Participants:

 Description   

Note unit tests pass with existing codebase but fail with changes below. Below we tried to get the critical edge:

/opt/mongodbtoolchain/v4/bin/python3 buildscripts/libdeps/gacli.py --graph-file build/opt/libdeps/libdeps.graphml --critical-edges build/opt/mongo/util/util_test build/opt/mongo/scripting/libscripting.so
Loading graph data...Loaded!
 
 
Traceback (most recent call last):
  File "/home/ubuntu/mongo/buildscripts/libdeps/gacli.py", line 265, in <module>
    main()
  File "/home/ubuntu/mongo/buildscripts/libdeps/gacli.py", line 252, in main
    ga = libdeps_analyzer.LibdepsGraphAnalysis(analysis)
  File "/home/ubuntu/mongo/buildscripts/libdeps/libdeps/analyzer.py", line 702, in __init__
    analyzer.report(self._results)
  File "/home/ubuntu/mongo/buildscripts/libdeps/libdeps/analyzer.py", line 563, in report
    self._to_node])] = self.run()
  File "/home/ubuntu/mongo/buildscripts/libdeps/libdeps/analyzer.py", line 95, in check
    return func(*args, **kwargs)
  File "/home/ubuntu/mongo/buildscripts/libdeps/libdeps/analyzer.py", line 552, in run
    minimum_st_edge_cut(
  File "/opt/mongodbtoolchain/revisions/347afa180670768de1b57d19832e898216d433ed/stow/python3-v4.Ffz/lib/python3.9/site-packages/networkx/algorithms/connectivity/cuts.py", line 145, in minimum_st_edge_cut
    cut_value, partition = nx.minimum_cut(H, s, t, **kwargs)
  File "/opt/mongodbtoolchain/revisions/347afa180670768de1b57d19832e898216d433ed/stow/python3-v4.Ffz/lib/python3.9/site-packages/networkx/algorithms/flow/maxflow.py", line 458, in minimum_cut
    R = flow_func(flowG, _s, _t, capacity=capacity, value_only=True, **kwargs)
  File "/opt/mongodbtoolchain/revisions/347afa180670768de1b57d19832e898216d433ed/stow/python3-v4.Ffz/lib/python3.9/site-packages/networkx/algorithms/flow/edmondskarp.py", line 237, in edmonds_karp
    R = edmonds_karp_impl(G, s, t, capacity, residual, cutoff)
  File "/opt/mongodbtoolchain/revisions/347afa180670768de1b57d19832e898216d433ed/stow/python3-v4.Ffz/lib/python3.9/site-packages/networkx/algorithms/flow/edmondskarp.py", line 99, in edmonds_karp_impl
    raise nx.NetworkXError(f"node {str(t)} not in graph")
networkx.exception.NetworkXError: node mongo/util/util_test not in graph 

Then when we give networkx the full graph, it finds some critical edges:

diff --git a/buildscripts/libdeps/libdeps/analyzer.py b/buildscripts/libdeps/libdeps/analyzer.py
index c440fcf7d8f..a37c0d7ca50 100644
--- a/buildscripts/libdeps/libdeps/analyzer.py
+++ b/buildscripts/libdeps/libdeps/analyzer.py
@@ -550,7 +550,7 @@ class CriticalEdges(Analyzer):
     # would encounter on a given path from the from_node to the to_node.
     min_cut_edges = list(
       minimum_st_edge_cut(
-        G=self._dependents_graph.get_direct_nonprivate_graph().get_node_tree(self._to_node),
+        G=self._dependents_graph,
         s=self._to_node, t=self._from_node))
     return [(edge[1], edge[0]) for edge in min_cut_edges]

/opt/mongodbtoolchain/v4/bin/python3 buildscripts/libdeps/gacli.py --graph-file build/opt/libdeps/libdeps.graphml --critical-edges build/opt/mongo/util/util_test build/opt/mongo/scripting/libscripting.so
Loading graph data...Loaded!
 
Graph built from git hash:
572cb2ea4d872432ce95c0de1c5cf4d37641b50a
 
Graph Schema version:
3
 
Build invocation:
"/opt/mongodbtoolchain/v4/bin/python3" "../scons/scripts/scons.py" "--link-model=dynamic" "--variables-files=etc/scons/mongodbtoolchain_v3_clang.vars" "ICECC=icecc" "CCACHE=ccache" "generate-libdeps-graph" "-j400"
 
Nodes in Graph: 1035
Edges in Graph: 127538
Direct Edges in Graph: 7630
Transitive Edges in Graph: 119908
Direct Public Edges in Graph: 4024
Public Edges in Graph: 123932
Private Edges in Graph: 3580
Interface Edges in Graph: 26
Program Nodes in Graph: 178
Library Nodes in Graph: 857
 
Critical Edges:
=>critical edges between mongo/util/util_test and mongo/scripting/libscripting.so:
  1: ('mongo/util/util_test', 'mongo/util/liblatch_analyzer.so')
  2: ('mongo/util/util_test', 'mongo/executor/libthread_pool_task_executor_test_fixture.so')
  3: ('mongo/util/util_test', 'mongo/executor/libtask_executor_test_fixture.so')
  4: ('mongo/util/util_test', 'mongo/executor/libnetwork_interface_mock.so')



 Comments   
Comment by Githook User [ 05/May/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-65805 added libdeps handling for no critical edges found
Branch: master
https://github.com/mongodb/mongo/commit/6cdc62195875a9d33a4ace5feb3826f1ff19a9e8

Generated at Thu Feb 08 06:03:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.