`iB lddlZddlZddlZddlZddlmZmZddlmZddl m Z ddl m Z ddl mZmZer,ddlZddlmZd ZejrddlZd Znd Ze jeZd Zejd Zd edefdZdZ d#de ede de efdZ!d#de ede de efdZ"dej#fdZ$dZ%ejdZ&dZ'dZ(de efdZ) d$dede de fdZ*ed !e d%de ede de fd"Z+dS)&N)contextmanagerredirect_stdout)StringIO)Optional)logging)is_torch_availablerequires) save_fileFTctrtjsdStjdkS)z7Return True if rank=0 or we aren't running distributed.Tr)_torch_distributed_availabletorch distributedis_initializedget_rankv/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/model_debugging_utils.py _is_rank_zeror-s? (U->-M-M-O-Ot   % % ' '1 ,,rzobject at 0x[0-9A-Fa-f]+x_strreturnc8td|S)z Replace memory addresses in an object's repr with a stable placeholder so that beautiful JSON diffs won't be ruined by ephemeral addresses. zobject at 0xXXXXXXXX)MEMORY_ADDRESS_REGEXsub)rs r_sanitize_repr_for_diffr7s # #$:E B BBrcPtrdt|jSdS)z@Return a stable string representation for a DTensor-like object.zDTensor (rank0) -> zDTensor(non-rank0))rrepr _local_tensor)xs r _dtensor_reprr ?s,=H[27<< M:::m65++--4466::<<=xHHH((( RHRRMRRRSSSek""ek""  C  {u}emU^DDD /UZZ\\0B0BCC.tEIIKK/@/@AA.tEIIKK/@/@AA.tEIIKK/@/@AA       Jrct|ttfrfdt|DSt|tr"fd|DSt |drt|jSt|tj rt|Stt|S)a Recursively build a JSON-serializable Python structure from `value`. Tensors and DTensors become either sanitized repr strings, or are saved to disk as SafeTensors files and their relative paths are recorded in the returned Python structure. Lists/tuples/dicts are recursed into. All memory addresses are replaced with a stable placeholder. Args: value: Any Python object, often including torch Tensors, lists, dicts, etc. debug_path (`str`, *optional*, defaults to `None`): Directory to dump debug JSON and SafeTensors files. use_repr (bool, *optional*, defaults to `True`): Whether to save a `repr()`-ized version of the tensors as the `value` property in the asscoiated FULL_TENSORS.json file, or to store full tensors in separate SafeTensors files and store the relative path to that file in the `value` property. path_to_value (`str`, *optional*, defaults to `None`): The file name for the SafeTensors file holding the full tensor value if `use_repr=False`. Returns: A nested Python structure (list, dict, or sanitized string) that is safe to json.dump. c Fg|]\}}t|d|S_r!r"r# _serialize_io).0ivr!r#r"s r z!_serialize_io..sN   1 ! XXeUkUkhiUkUk l l l   rc Hi|]\}}|t|d|SrBrE)rGkrIr!r#r"s r z!_serialize_io..sP   1 }Q:[hXnXnklXnXnooo   rrrD) isinstancelisttuple enumeratedictitemshasattrr?rrTensorrr)r)r!r"r#s ```rrFrFws'(%$''       !%((    %            uo&& (  JYf    %&&w(:PXhuvvvv "4;; / //rr)cPtjddt5}t|5t ||}dddn #1swxYwYdddn #1swxYwYt |S)z Converts a tensor into a sanitized multi-line string representation. Args: value (`torch.Tensor`): The tensor to represent. Returns: `list[str]`: List of string lines representing the tensor. Tx)r% linewidthN)rr.rrprintgetvaluer splitlines)r)bufraws rr/r/s  DC8888 sOC00 e llnn #3 ' ' 2 2 4 44s4A<$A% A<%A) )A<,A) -A<<BBc|dr0|dd|dD]}t|dSdS)Nchildrenoutputs)getpopprune_outputs_if_children)nodechilds rrcrcse xx - D!!!*% - -E %e , , , ,-- - -rz (.*)\.(\d+)$ct|dd}|r|dsdS|dt fd|dDS)z Checks whether a node represents a layer block with submodules. Args: node (`dict`): A node from the call tree. Returns: `bool`: Whether the node is a layer block. module_pathr_Fc3PK|] }dd|ddvV!dS).rgrhNra)rGrenumbers r z!is_layer_block..s>[[}6}}} - < <<[[[[[[r)LAYER_SUFFIX_REmatchragroupany)rdrprms @ris_layer_blockrss}  ! !$((=""="= > >E ,,u [[^^F [[[[$zJZ[[[ [ [[rc>|dsdSdt|dD}t|dkr8d|ddDfdt|dD|d<|dD]}t|dS) z Recursively removes intermediate layers from the tree to improve readability. Keeps at least the first and last layers if many consecutive layers are present. Args: node (`dict`): The root or subnode to prune recursively. r_Nc:g|]\}}t|||fSr)rs)rGrHres rrJz-prune_intermediate_layers..s.ddd81en]bNcNcdQJdddrricg|]\}}|Srr)rGrHrCs rrJz-prune_intermediate_layers..s66641aQ666rrc"g|] \}}|v | Srr)rGrHre to_removes rrJz-prune_intermediate_layers..s)dddhaQRZcQcQcEQcQcQcr)rarQlenprune_intermediate_layers)rd layer_blocksrerys @rr{r{s 88J  ddyj9I/J/JdddL <166<"#5666 dddd)D.strip_values..cleans#t$$ &&&AE!HHHHC&&   DE$KKKK    rinputsr`r_rl)rdrer strip_valuess @rrz+log_model_debug_trace..strip_valuess      dhhx$$%%% dhhy"%%&&&XXj"--  E L      r)r1makedirsr2r3_debugger_module_dump_name Exceptionr7loggerinforc _call_treeopenjsondumploadsdumps) r!modelbasee full_path summary_pathf tree_copyrs @rlog_model_debug_tracers$@ X K T 2 2 2 27<< E,L},\]]DD X X XO*OOOPPVW W X/-? KK5$555666++I/)Le./// i  1 %"Aa0000111111111111111      4:e&67788IL lC *A )Qq))))******************s;>A A'A""A'C))C-0C-?E$$E(+E(rkdo_prune_layersc jj ddgd_g_ _r> t jdn&#t$r}tdd|d}~wwxYwfd} D]\}}|dkr || d|j tj   fd }|_ dS) a Attaches a debugging wrapper to every module in the model. This records structured inputs and outputs during the forward pass into a call tree. Args: model (`PreTrainedModel`, `nn.Module`): Model to wrap. debug_path (`str`): Optional directory to dump debug JSON files. do_prune_layers (`bool`, *optional*, defaults to `True`): Whether to prune intermediate layers. use_repr (bool, *optional*, defaults to `True`): Whether to save a `repr()`-ized version of the tensors as the `value` property in the associated FULL_TENSORS.json file, or to store full tensors in separate SafeTensors files and store the relative path to that file in the `value` property. Nrgrr`r_Tr~rrkcrjtjfd}|_dS)NctrH||dfdDt ddgd}j|t j5 |i|}dddn #1swxYwYtrt d Ddkrd|d<nt| d |d<j}|d s|d jr&jd d ||S) NargskwargscTi|]$}t|dk||%S)r)rz)rGrL dict_inputss rrMzY_attach_debugger_logic..wrap_forward..wrapped_forward..6s7aaaQ[YZ^I\I\_`I`I`q+a.I`I`I`r_inputsrDrc3K|]}dVdS)rNr)rGrCs rrnzX_attach_debugger_logic..wrap_forward..wrapped_forward..Gs"::Qq::::::rrr`_outputsr_rw) rrF_debugger_model_call_stackappendrno_gradsumnamed_childrenrb) inpskwsrdr>finishedrr!rrmodule orig_forwardr"s @rwrapped_forwardzE_attach_debugger_logic..wrap_forward..wrapped_forward2s >'+s;; aaaa+aaa #,+##-!))2&;&;&;  $ "  077=== 1 1"lD0C00 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 V::&"7"7"9"9:::::Q>>&*DOO&3#-!))2&<&<&< '''DO!;??AA +-LL,,,3V4R8DKKHUUUJs, BBB)forward functoolswraps)rrrrr!rr"s`` @r wrap_forwardz,_attach_debugger_logic..wrap_forward/sb~  & &% % % % % % % % %  ' &% N)rrhcJtr;dt||d ddgd}j| |i|}tr‰jrt| d|d<j}|djd<|djd<|d jd <fd t jDrtjt |S) Nz (top-level)rrrDrrr`rr_c`g|]*}j|j|d+S)N)rrb)rGrLrs rrJzG_attach_debugger_logic..top_wrapped_forward..s; m m mqY^YijkYl mU  ! !!T * * m m mr)r!r) rrFrrrbrrOkeysr{r) rrtop_noder>r class_namer!rrreal_top_forwardr"s rtop_wrapped_forwardz3_attach_debugger_logic..top_wrapped_forwardes ?? >",:::'!S11)%%/"8"8"8     H  , 3 3H = = =,,, ?? Fu? F"/%!!+555 ###HY  7;;==H)1();E X &*29*=E Y '+3J+?E Z ( m m m mD9I9N9N9P9P4Q4Q m m m m <)%*:;;; !Zu E E E E r) __class____name__rrrr1rrr7 named_modulesrrr) rr!rr"rrname submodulerrrs ```` @@r_attach_debugger_logicrsv&)J(2Td`bccE')E$'1E$X X K T 2 2 2 2 2 X X XO*OOOPPVW W X+)+)+)+)+)+)+)\!..0088i 2::  Y: 6 6 6 67777}_%&&#########'&#J(EMMMsA A*A%%A*)r)backendsc#Kd|D}|j||<t|||| |V|D] \}}||_ dS#|D] \}}||_ wxYw)a # Model addition debugger - context manager for model adders This context manager is a power user tool intended for model adders. It tracks all forward calls within a model forward and logs a slice of each input and output on a nested JSON file. If `use_repr=True` (the default), the JSON file will record a `repr()`-ized version of the tensors as a list of strings. If `use_repr=False`, the full tensors will be stored in separate SafeTensors files and the JSON file will provide a relative path to that file. To note, this context manager enforces `torch.no_grad()`. ## Usage add the context manager to a model to debug ```python import torch from PIL import Image from transformers import LlavaProcessor, LlavaForConditionalGeneration, model_addition_debugger_context torch.random.manual_seed(673) # load pretrained model and processor model_id = "llava-hf/llava-1.5-7b-hf" processor = LlavaProcessor.from_pretrained(model_id) model = LlavaForConditionalGeneration.from_pretrained(model_id) # create random image input random_image = Image.fromarray(torch.randint(0, 256, (224, 224, 3), dtype=torch.uint8).numpy()) # prompt prompt = "Describe this image." # process inputs inputs = processor(text=prompt, images=random_image, return_tensors="pt") # call forward method (not .generate!) with model_addition_debugger_context(model, debug_path="Your_debug_path", do_prune_layers=False): output = model.forward(**inputs) ``` c$i|] \}}||jSr)r)rGrCms rrMz3model_addition_debugger_context..s DDDdaQ DDDrN)rrrrS)rr!rr" orig_forwardsmodule_instanceforward_methods rmodel_addition_debugger_contextrsfEDe.A.A.C.CDDDM =M%5*oxHHH5 /rs  66666666<<<<<<<< )LLL++++++#(  %%'','''''+$#(   H % %---"rz"=>>C3C3CCCC   dh..}.7;.S[\_S`....b(0(0Xc](0T(0aijman(0(0(0(0V55555"---"*_--\\\")))((*hsm(*(*(*(*Z  |(|(|(|( |(|(|(|(~ :!%  8585 8585 858585858585r