Pi'9 pddlZddlZddlmZddlmZddlmZmZddl Z ddl m Z ddl m Z ddlmZmZddlmZdd lmZd d lmZd Zd ZdZejeZedGddZde ddfdZdedeefdZ de!de!fdZ"de!de!de#fdZ$dee j%e j%ge j%fde!de!de!fdZ&Gdde j'j(Z)d eded!edefd"Z*d edefd#Z+edGd$d%Z,edGd&d'Z-d(e!de.fd)Z/d ede0ee1ee.e!e2e!fffd*Z3d+e0ee1ee.e!e2e j%ffd,e0ee1ee.e!e2e j%ffde0ee-ffd-Z4dS).N)Sequence) dataclass)CallableOptional) compute_sqnr)ExportedProgram) GraphModuleNode) NodeSource) functional)bfs_trace_with_node_processnumeric_debug_handlecustom from_nodeT)frozenc(eZdZUdZeed<eed<dS)NodeSourceDebugInfoz Contains node source information for locating the node in the original graph. This replaces the numeric debug handle approach with direct node source info. namegraph_idN)__name__ __module__ __qualname____doc__str__annotations__int/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchao/quantization/pt2e/_numeric_debugger.pyrrs0 IIIMMMMMrrepreturnc,t|ts$tdttddtjjddffd }dtjjddffd }t||dz t||dS) aj Attach numeric_debug_handle_id for all nodes in the graph module of the given ExportedProgram, like conv2d, squeeze, conv1d, etc, except for placeholder. Notice that nodes like getattr are out of scope since they are not in the graph. The graph nodes of input exported program are modified inplace. Here's an example of using debug handle quantize flow:: ep = torch.export.export(eager_model, example_inputs) generate_numeric_debug_handle(ep) m = ep.module() quantizer = XNNPACKQuantizer() m = prepare_pt2e(m, quantizer) m = convert_pt2e(m) z'Expected ep to be ExportedProgram, got rnoder"Nct|jtitddS)Nr)maxmetaget CUSTOM_KEYNUMERIC_DEBUG_HANDLE_KEYr$ unique_ids r _find_max_idz3generate_numeric_debug_handle.._find_max_idGs= ty}}Z44889QSTUU  rct|jvri|jt<t|jtvr!|jtt<dz dSdS)Nr )r)r'r*r+s r _assign_debug_handlez;generate_numeric_debug_handle.._assign_debug_handleMsV TY & &$&DIj ! #49Z+@ @ @>GDIj !": ; NIII A @rr ) isinstancer ValueErrortypetorchfxr r)r!r-r/r,s @r generate_numeric_debug_handler5,s( b/ * *  Md?6K6K M M   I 58= T      58=T L111 NI $899999rr$cdtddfd}dtdtfd}|jdks |jdkrdS||sdS||}t|j|j S) a Extract node source debug info from a node, or return None if the node does not need to be traced. Returns NodeSourceDebugInfo containing the name and graph_id from the node's greatest ancestor node source, or None if the node is not in the original graph. r$r"r c|jtd}t|jdkr%|jd}t|jdk%|S)Nr)r'r( FROM_NODE_KEYlenr)r$ node_sources r "_get_greatest_ancestor_node_sourcezK_extract_node_source_debug_info.._get_greatest_ancestor_node_sourcels]immM2226 +'((1,,%/3K+'((1,,rct|jvs|jtdS|jtdjdkrdSdS)NFr8z!ExportedProgram.module().unlift()T)r9r' pass_name)r$s r _is_node_in_original_graphzC_extract_node_source_debug_info.._is_node_in_original_graphtsO  ) )TY}-E-M5 Im $R ( 22 3 35tr placeholderoutputN)rr)r booloprrr)r$r<r?greatest_ancestor_node_sources r _extract_node_source_debug_inforEbs,$  w-47h#6#6t % %d + +t$F$Ft$L$L!  * /.7   rxcRd}t|tjr|}nut|tt fr#t |d|D}n6t|trd|D}n|}|S)Nc,g|]}t|Sr_detach).0es r z_detach..s2221GAJJ222rc4i|]\}}|t|SrrI)rKkrLs r z_detach..s$888daAwqzz888r) r0r3Tensordetachlisttupler2dictitems)rFdetacheds r rJrJsH!U\""88:: Ae} % %4772222233 At  88aggii888 Orycvt|tjr*t|tjr|j|jkSt|tt frCt|tt fr't dt||DSt|tr@t|tr+d}|D]$}|o||vot||||}%|St d||t|t|ko||kS)Nc3<K|]\}}t||VdSN)_tensor_shape_equals)rKe1e2s r z'_tensor_shape_equals..s1HHFB'B//HHHHHHrTz4Comparing non Tensors: %s and %s, they must be equal) r0r3rQshaperSrTallziprUr\logdebugr2)rFrX all_equalrOs r r\r\s&!U\"" -z!U\'B'B -w!'!! Ae} % % -*Qu *F*F -HHc!QiiHHHHHH At  -At!4!4-  T TA!Sa1fS2FqtQqT2R2RII H!QOOOAww$q''!,a1f,rlosscht|tjr`ttjrF|tjtjSt|t t frNtt t fr2t|fdt|DSt|tr6ttr!fd| DSdS)aThe returned loss will have the same structure as `x` and `y`, e.g. if both are Tensor, we'll return a Tensor if both are list, we'll return a list of Tensors if both are dict, we'll return a dict with the same key, and value being the loss between the two Tensors c8g|]\}}t||Sr_loss_fn)rKr]r^rfs r rMz_loss_fn..s)GGG62rr2..GGGrc Fi|]\}}|t||Srri)rKrOrLrfrXs r rPz_loss_fn..s/AAAtq!8D!QqT**AAArN) r0r3rQtofloat32rSrTr2rbrUrV)rfrFrXs` `r rjrjs!U\""z!U\'B'BtADD''em)<)<=== Ae} % %*Qu *F*FtAwwGGGGSAYYGGGHHH At  At!4!4AAAAAqwwyyAAAAtrc reZdZdZdZ d dedeedeeddffd Z d edefd Z defd Z xZ S) OutputLoggerz Base class for capturing output values for nodes in a GraphModule, it only captures Tensor output currently, but we can extend it to work for other types of inputs later if needed TN debug_info node_namenn_module_stackr"ct||_||_||_g|_dSr[)super__init__rqrrrpstats)selfrprqrr __class__s r ruzOutputLogger.__init__s< ".$#% rrFcT|jt||Sr[)rvappendrJ)rwrFs r forwardzOutputLogger.forwards# '!**%%%rc(d|jd|jdS)Nz debug_info=z , node_name=zF, nn_module_stack={self.nn_module_stack}, num_stats={len(self.stats)}))rprqrws r __extra_repr__zOutputLogger.__extra_repr__s3 S$/ S St~ S S S r)NN) rrrr _is_impurerrrobjectrur{r~ __classcell__)rxs@r roros J $(,0 & &' &C= &"&) &  & & & & & &F         rromodelrpc ddlm}|j|5||jd}||}t ||t ||j|jd|j ||fi}dddn #1swxYwYt|j }|D]}||ur| |||S)zFor a given node, adds an OutputLogger that observes the output of that node, and all its users use the OutputLogger output instead. The OutputLogger will contain the debug_info which can be used to compare graphs after transformsr)get_new_attr_name_with_prefix_loggerrrN)torchao.quantization.pt2e.utilsrgraphinserting_afterrsetattrror'r( call_modulerSuserskeysreplace_input_with) rr$rprget_new_attr_name logger_name logger_node orig_users user_nodes r _insert_loggerrsmNMMMMM  $ $T * *HH99TY:O:O:OPP''..    TY >O0P0P Q Q   k--kD7BGG HHHHHHHHHHHHHHHdjoo''((J88  # # $$T;7777 sA9B&&B*-B*ctj|}|jjD]$}t |x}t |||%||S)zAdd output loggers to unlifted node Args: model (GraphModule): original model Returns: a model with output loggers for all unlifted nodes )copydeepcopyrnodesrEr recompile)rnrps r "prepare_for_propagation_comparisonrsb M% E [ 119!<< %u|U\&BEL&PQ> >>>>  #    rrcXeZdZUeed<eed<eed<eed<eed<eeed<dS)NodeAccuracySummaryrpactual_node_nameactual_module_stack ref_node_nameref_module_stackresultsN)rrrrrrrrrrr rr=s`#### 2 333333rr module_stackc t|tst|St|}t |dkr|dd}t|St|S)zlSimplifies the stack from ("mod", "mod.foo", "mod.foo.0", "mod.foo.0.linear") to "mod.foo.0.linear" rr8)r0rUrrSvaluesr:)rmodule_values_list owning_modules r _module_stack_to_strrGs} lD ) )!<   l113344 ""*2.q1 =!!!<   rci}|D]N\}}t|tr4t|jdkr|j|j|jf||j<O|S)aFor a given model, extract the tensors stats and related information for each debug info. The reason we have a list of object, instead of Tensor is because the output of node may not be a Tensor, it could be (nested) list, tuple or dict as well. Returns: A dict is keyed by the NodeSourceDebugInfo and the values are a list of object recorded in loggers r)named_childrenr0ror:rvrqrrrp)rhandles_modules r extract_results_from_loggersrUsxVXG))++ 6 fl + + FL0A0AA0E0E & *GF% & Nr ref_resultsactual_resultsc i}|D]\}\}}}||vrtd|)||\}}} dt| |D} n+#t$r} t d|d|d|| d} ~ wwxYwt ||pdt||pdt|| ||<|S) aGiven two dict mapping from `NodeSourceDebugInfo` to list of tensors return a map from `NodeSourceDebugInfo` to `NodeAccuracySummary` that contains comparison information like SQNR, MSE etc. Args: ref_results (Dict[NodeSourceDebugInfo, Tuple[str, object, List[torch.Tensor]]]): reference results for each debug info actual_results (Dict[NodeSourceDebugInfo, Tuple[str, object, List[torch.Tensor]]]): actual results for each debug info Returns: Dict[NodeSourceDebugInfo, NodeAccuracySummary] zQCannot compare for debug info %s because it wasn't found in the transformed modelc6g|]\}}t||S))rr)r)rKabs r rMz#compare_results..s9Aq-A1===rzFor debug_info=z from ref node z and actual node N)rprrrrr)rVrcrdrb Exceptionr1rr) rr comparisonsrpref_name ref_stack ref_stats actual_name actual_stack actual_statsrrLs r compare_resultsrnsA&K8C8I8I8K8K  4 4Xy) ^ + + IIc    2@2L/ \<  i88GG   e*eeXeeXcee  #6!(.B 4\ B B".b1)<< # # # J s A(( B2B  B)5rloggingcollections.abcr dataclassesrtypingrrr3torch.ao.ns.fx.utilsr torch.exportrtorch.fxr r torch.fx.tracebackr torch.nnr r graph_utilsrr*r)r9 getLoggerrrcrr5rErrJrBr\rQrjnnModulerorrrrrrrUrTrSrrrrr rs $$$$$$!!!!!!%%%%%%%% ------((((((&&&&&&&&))))))$$$$$$4444441   g!! $        3:o3:$3:3:3:3:l.$.8CINT &     58?   @ "0C >kk$ $&&&&&&&&R $44444444 !v !# ! ! ! !  uXc]FDL%HI IJ21U8C=&$u|:L#LMM1U8C=&$u|:L#LMM 1 2 23111111r