IBCONGEST(1)

NAME
ibcongest - STATIC CONGESTION ANALYSIS

SYNOPSYS
TOPO: ibcongest [-hdABC][-v lvl] -t <topo-file> [-n <sm-node>][-p <sm-port>]
                [-r ftree|pgft|osm>] [-z roots-file] [-R rep-hsd-file] [-Q rep-bw-file]
                [-O order-file] [-o order-file]
                {-a [-b path-bit][-j <min-shift> -k <max-shift>]|-s <schedule>}

LST: ibcongest [-hdABC][-v lvl] -l <lst> -f <fdbs> [-R rep-hsd-file] [-Q rep-bw-file]
                [-O order-file] [-o order-file]
                {-a [-b path-bit][-j <min-shift> -k <max-shift>]|-s <schedule>}

DESCRIPTION
Analyzes congestion for a traffic schedule provided in a "schedule-file" or use an automatically generated schedule of all-to-all-shift. The schedule file may define multiple stages of communication - each one of them is considered as infinite and the traffic switches together to the new stage.

Calculates routing for a given topology (topo-mode) or use exstracted lst/fdb files (lst-mode).

The utility provides both Host-Spot-Degree analysis, which is the count of number of flows per link, and bandwidth per-flow calculation.

For HSD this tool provides a histogram of number of links vs. HSD over all stages and a histogram of number of stages vs. the worst link HSD per stage.

Similarly for flow bandwidth a histogram of flows vs. bandwidth and stages vs. worst bandwidth per stage. Bandwidth can be calculated assuming perfect congestion control (as defined by the -C flag) or without any congestion control.

Detailed log files can be obtained by providing specific flags.

ARGUMENTS
In topo-mode the following arguments are required:

-t|--topology <file>
    Topology file describing the network. See man ibdm-topology-file.

In lst-mode the following optiontions are used:

-l|--lst-file <lst-file>
    LST file obtained from OpenSM or ibdiagnet.
-f|--fdbs-file <file>
    FDBs file obtained from OpenSM or ibdiagnet.

TOPO MODE OPTIONS
In topo-mode the following options are supported:

-r|--routing  ftree/pgft/osm
    Defines the routing engine to be used (osm).
-z|--roots <root nodes file>
    Provides a file holding roots of the topology - used by ftree routing engine
-n|--sm-node <name>
    Define the node name for the SM to run on (H-1/U1).
-p|--sm-port sm-port-num
    Define the port number of the sm-node the SM run from (1).

COMMON OPTIONS

-R|--rep-hsd  <file-name>
    Report each stage over subscription into the given file name
-Q|--rep-bw <file-name>
    Report each stage each flow BW (in csv format) into the given file name
-O|--out-order <file-name>
    Output the node order used by the algorithm (useful when defaul order is used).
-o|--order-file <file-name>
    A file holding host name in each line, defining their ordering. Description of the order file is in section ORDER_FILE. Default order is by host name sort H-<N> by the N
-a|--all-to-all
    Automatically generate an all-to-all schedule which containes all shift permutations. Mutex with the -s option.
-b|--path-bits <a number 0 to 127>
    Use the given path-bits for the -a option. This defines the specific LID within the range assigned to the destination that will be used by the all-to-all traffic.
-j|--from <min-shift> -k|--to <max-shift>
    Restrict the -a option to a range of shifts starting at <min-shift> ending <max-shift>
-s|--sched-file <file-name>
    A file defining the communication pattern. Mutex with the -a option. The format described in the below section SCHEDULE FILE.
-A|--all-paths
    Check all paths not just CA to CA
-B|--bw-mode
    Calculate and report actual bandwidth of each flow. Turned on also if -Q is provided.
-C|--cong-ctrl
    Assume Congestion Control. I.e. flow fairness and no back-pressure.
-v|--verbose <level>
    Verbsoe mode to use :
        1 -
        2 - order optimization
        3 - congestion analysis
        4 - fat tree routing
        5 - fat tree recognition
-h|--help
    Provide this help message

SCHEDULE FILE
The schedule file describes a communication sequence. There are 2 types of lines:

1. Stage line separates different stages of the schedule:
    ^STAGE(\s+[0-9]+)?

2. SRC DST line describes traffic in pairs. Optionally the destination may be given a specific 'path-bits' that specify exact DLID from a range of DLIDs a destination may have. Another option is the flow may be given a relative BW of the SRC node.
    From the first rank to the second rank:

    ^\s*[0-9]+\s+[0-9]+(:([0-9]+))?(\s+(0\.[0-9]+))?\s*

    Or from a port name to a port name:

    ^\s*(\S+)/P([0-9]+)\s+*(\S+)/P([0-9]+)(:([0-9]+))?(\s+(0\.[0-9]+))?\s*

    For example: A flow from node H-1 port 1 to H-100 port 1 that 50% of the BW of H-1

    H-1/U1/P1 H-100/U1/P1:1 0.5

ORDER FILE
The order file provides mapping between the rank (which is the line number) and a host in the topology. There are two type of lines supported:

1. OpenSM order file style holding LID and host name:
    ^\s*(0x[0-9a-fA-F]+)\s(\s+)\s+HCA-([0-9]+)

2. Port name only: simply declare the order of host names to be used
    ^\s*(\s+)\s+P?([0-9]+)\s*$

3. Dummy place holder. DUMMY stands for an empty rank. Just a placeholder - not a real HCA. Traffic from DUMMY or to DUMMY is not generated even if specified in the schedule
    ^DUMMY$

AUTHOR
Eitan Zahavi, Mellanox Technologies LTD, eitan@mellanox.com