B )`+D@sdZddlmZddlmZddlmZddlZddlZddlZddlZe dZ e dZ e dZ e d Z Gd d d eZd d ZddZddZddZddZddZddZddZddZddZd d!Zd"d#Zd$d%Zd&d'ZdS)(z7Command parsing module for TensorFlow Debugger (tfdbg).)absolute_import)division)print_functionNz \[[^\]]*\]z(\"[^\"]*\"|\'[^\']*\')z\s+z'[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?c@s(eZdZdZddZddZddZdS) Intervalz5Represents an interval between a start and end value.cCs||_||_||_||_dS)N)startstart_includedend end_included)selfrrrr r Y/home/dcms/DCMS/lib/python3.7/site-packages/tensorflow/python/debug/cli/command_parser.py__init__$szInterval.__init__cCs@||jks||jkr|jsdS||jks8||jkr<|jsLsz!parse_command..cSsg|] }|qSr )r)rrr r r rMscSsg|] }|qSr )r)rrr r r rOsrNc3s.|]&}|dko |dknVqdS)rNr )rinterval)rr r Zsz parse_command.."'r) strip_BRACKETS_PATTERNfinditer_QUOTES_PATTERN_WHITESPACE_PATTERNlenany startswithendswithappend)commandZbrackets_intervalsZquotes_intervalsZwhitespaces_intervals argumentsZidx0rargumentr )rr parse_command8s(    r-cCs|r|ddrtdn|r|ddryLt|dt|dkr^|ddr^d}n|ddd}|dd}Wn0tk r|ddd}|dd}YnXnt|dkr|ddkr|d}|dd}n|rV|dddkrV|dd}|dkr,|d|dd kr,d}n(|d|dd}|dd||d<nLt|dkr|ddr|d}|dd}|ddd|d<nd}||fS) a@Extract output file path from command arguments. Args: args: (list of str) command arguments. Returns: (list of str) Command arguments with the output file path part stripped. (str or None) Output file path (if any). Raises: SyntaxError: If there is no file path after the last ">" character. r>zRedirect file path is emptyr-Nr=)r( SyntaxErrorr'_parse_intervalr% ValueErrorcountindex)argsZoutput_file_pathZgt_indexr r r extract_output_file_pathhs6    r8cCsN|ddkr>|dr>|d|d}||dd}n|}d}||fS)aParse tensor name, potentially suffixed by slicing string. Args: in_str: (str) Input name of the tensor, potentially followed by a slicing string. E.g.: Without slicing string: "hidden/weights/Variable:0", with slicing string: "hidden/weights/Variable:0[1, :]" Returns: (str) name of the tensor (str) slicing string, if any. If no slicing string is present, return "". [r]N)r5r(r6)in_str tensor_nametensor_slicingr r r parse_tensor_name_with_slicings r?cCsttd|S)a7Validate a slicing string. Check if the input string contains only brackets, digits, commas and colons that are valid characters in numpy-style array slicing. Args: slicing_string: (str) Input slicing string to be validated. Returns: (bool) True if and only if the slicing string is valid. z^\[(\d|,|\s|:)+\]$)boolresearch)slicing_stringr r r validate_slicing_strings rDcCsg}x|dddD]p}|d}t|dkrJ|t|dqdt|krbdkrnn|tdd |Dqtd qWt|S) a-Construct a tuple of slices from the slicing string. The string must be a valid slicing string. Args: slicing_string: (str) Input slicing string to be parsed. Returns: tuple(slice1, slice2, ...) Raises: ValueError: If tensor_slicing is not a valid numpy ndarray slicing str. rr,:rcSs$g|]}|rt|ndqS)N)r int)rr6r r r rsz!_parse_slices..zInvalid tensor-slicing string.)splitr%r)rIr slicer4tuple)rCparsedZ slice_stringindicesr r r _parse_slicess   rOcCsBtdd|}|dr.|dr.|dd}dd|d DS) a7Parse a string representing indices. For example, if the input is "[1, 2, 3]", the return value will be a list of indices: [1, 2, 3] Args: indices_string: (str) a string representing indices. Can optionally be surrounded by a pair of brackets. Returns: (list of int): Parsed indices. z\s+r;r9r:rrcSsg|] }t|qSr )rI)relementr r r rsz!parse_indices..rE)rAsubr'r(rJ)Zindices_stringr r r parse_indicess rRcCs|}|sgSd|kr.tdttjj|}t|}t |t rVt |dt sV|g}xp|D]h}t |dkrvt dq\t |dt tfst dt|dq\t |dt tfs\t dt|dq\W|S)aParse a string representing numerical range(s). Args: range_string: (str) A string representing a numerical range or a list of them. For example: "[-1.0,1.0]", "[-inf, 0]", "[[-inf, -1.0], [1.0, inf]]" Returns: (list of list of float) A list of numerical ranges parsed from the input string. Raises: ValueError: If the input doesn't represent a range or a list of ranges. infrrGz%Incorrect number of elements in rangez.Incorrect type in the 1st element of range: %srz.Incorrect type in the 2nd element of range: %s)r rArQreprsys float_infomaxast literal_eval isinstancelistr%r4rIfloattype)Z range_stringrangesitemr r r parse_rangess$    r`cCsZt|}d}td}|jr$t|j}|jr4t|j}||krHtd|t||j||jS)aConvert a human-readable memory interval to a tuple of start and end value. Args: interval_str: (`str`) A human-readable str representing an interval (e.g., "[10kB, 20kB]", "<100M", ">100G"). Only the units "kB", "MB", "GB" are supported. The "B character at the end of the input `str` may be omitted. Returns: `Interval` object where start and end are in bytes. Raises: ValueError: if the input is not valid. rrSzUInvalid interval %s. Start of interval must be less than or equal to end of interval.) r3r\rparse_readable_size_strrr4rrr ) interval_str str_intervalinterval_start interval_endr r r parse_memory_intervals  rfcCsZt|}d}td}|jr$t|j}|jr4t|j}||krHtd|t||j||jS)a|Convert a human-readable time interval to a tuple of start and end value. Args: interval_str: (`str`) A human-readable str representing an interval (e.g., "[10us, 20us]", "<100s", ">100ms"). Supported time suffixes are us, ms, s. Returns: `Interval` object where start and end are in microseconds. Raises: ValueError: if the input is not valid. rrSz:Invalid interval %s. Start must be before end of interval.) r3r\rparse_readable_time_strrr4rrr )rbrcrdrer r r parse_time_interval:s  rhcCs|}|drPt|ddrDtdd|ddddStd||drt|d drtdd|d dddStd ||d rt|ddrt|ddddddStd ||d r,t|d dr t|d dddddStd||drD|dsPtd||d dd}t|dkr|td||d}t|std||d }t|std|t||ddk||ddkdS)a4Convert a human-readable interval to a tuple of start and end value. Args: interval_str: (`str`) A human-readable str representing an interval (e.g., "[1M, 2M]", "<100k", ">100ms"). The items following the ">", "<", ">=" and "<=" signs have to start with a number (e.g., 3.0, -2, .98). The same requirement applies to the items in the parentheses or brackets. Returns: Interval object where start or end can be None if the range is specified as "N" respectively. Raises: ValueError: if the input is not valid. z<=rGNFT)rrrr z%Invalid value string after <= in '%s'=z%Invalid value string after >= in '%s'r.z$Invalid value string after > in '%s')r9()r:)zRInvalid interval format: %s. Valid formats are: [min, max], (min, max), minrrEz\Incorrect interval format: %s. Interval should specify two values: [min, max] or (min, max).rz$Invalid first item in interval: '%s'z%Invalid second item in interval: '%s'r9r:) r r'_NUMBER_PATTERNmatchrr4r(rJr%)rbrZ start_itemZend_itemr r r r3WsR                   r3cCs|}|dr|dd}|r.t|S|drPtt|dddS|drrtt|dddS|drtt|ddd Std |dS) aConvert a human-readable str representation to number of bytes. Only the units "kB", "MB", "GB" are supported. The "B character at the end of the input `str` may be omitted. Args: size_str: (`str`) A human-readable str representing a number of bytes (e.g., "0", "1023", "1.1kB", "24 MB", "23GB", "100 G". Returns: (`int`) The parsed number of bytes. Raises: ValueError: on failure to parse the input `size_str`. BNrkiMiGi@z3Failed to parsed human-readable byte size str: "%s")r r(isdigitrIr\r4)Zsize_strr r r ras     racCs~dd}|}|dr.t||ddS|drPt||dddS|drrt||dd d St||S) aBParses a time string in the format N, Nus, Nms, Ns. Args: time_str: (`str`) string consisting of an integer time value optionally followed by 'us', 'ms', or 's' suffix. If suffix is not specified, value is assumed to be in microseconds. (e.g. 100us, 8ms, 5s, 100). Returns: Microseconds value. cSs t|}|dkrtd||S)Nrz-Invalid time %s. Time value must be positive.)r\r4)Z value_strrr r r parse_positive_floats  z5parse_readable_time_str..parse_positive_floatusNr/msg@@srg.A)r r(rI)Ztime_strrsr r r rgs    rgcCs |}t|std|t|S)a~Call eval on the slicing of a tensor, with validation. Args: tensor: (numpy ndarray) The tensor value. tensor_slicing: (str or None) Slicing of the tensor, e.g., "[:, 1]". If None, no slicing will be performed on the tensor. Returns: (numpy ndarray) The sliced tensor. Raises: ValueError: If tensor_slicing is not a valid numpy ndarray slicing str. zInvalid tensor-slicing string.)rDr4rO)Ztensorr>_r r r evaluate_tensor_slicesrxcCstj|tjd}|jdtdd|jdddtdd d |jd d d tddd |jdddddd|jddddd|jddtddd|S)zGet an ArgumentParser for a command that prints tensor values. Examples of such commands include print_tensor and print_feed. Args: description: Description of the ArgumentParser. Returns: An instance of argparse.ArgumentParser. ) descriptionusager=zwName of the tensor, followed by any slicing indices, e.g., hidden1/Wx_plus_b/MatMul:0, hidden1/Wx_plus_b/MatMul:0[1, :])r]helpz-nz--numbernumberrzV0-based dump number for the specified tensor. Required for tensor with multiple dumps.)destr]defaultr{z-rz--rangesr^r;zuNumerical ranges to highlight tensor elements in. Examples: -r 0,1e-8, -r [-0.1,0.1], -r "[[-inf, -0.1], [0.1, inf]]"z-az--allZ print_all store_truezs4       0/)B"