`i` ddlZddlmZddlmZddlZddlmZejeZ dZ dZ Gdde Z d Zd eeejeejd ffd eeejeejd fffd Z ddedededeeeffdZ ddedededeeeeffdZeGddZdS)N) dataclass)Union)fxcZgfd}tj||}|fS)z\ Flatten the args into a list form and detach the tensors from computational graph. ct|tjrC||j}||S||SN) isinstancetorchTensordetachrequires_grad_ requires_gradappend)avalflat_detached_argss w/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torch/distributed/pipelining/_utils.pyextract_tensor_argsz0flatten_args_detach..extract_tensor_argssf a & & ((**++AO<.extract_tensor_args-srr)rrrs @r flatten_argsr'sMI G  rceZdZdZdS)PipeliningShapeErrorz5Shape mismatch between configured and runtime values.N)__name__ __module__ __qualname____doc__rrr!r!:s????rr!c|j|jks!t|d|jd|j|j|jks!t|d|jd|j||ks;t|d|d|dS)Nz has a shape mismatch: expected z actual z has a dtype mismatch: expected z! has a stride mismatch: expected )shaper!dtypestride)descexpectedgivens rvalidate_tensor_metadatar.>s >U[ ( (" Z ZX^ Z ZU[ Z Z    >U[ ( (" Z ZX^ Z ZU[ Z Z    ??    . ." a ahoo6G6G a aQVQ]Q]Q_Q_ a a    / .rexpected_tensors.actual_tensorsc ,t|t|kr2t|dt|dt|dtt|D]$}t|d|||||%dS)Nz: Number of values (z") does not match expected number ()z: value )lenr!ranger.)r+r/r0is rvalidate_tensors_metadatar6Ms  N 3 333" x x^)<)< x x`cdt`u`u x x x   3'(( ) )    Q "21"5~a7H      rlooppp_size num_stagesstylereturnc8i}|dkrt|D] }||z||< nv|dkr]||zdkrtd|d|dd}t|D]+}|||<|dz|zdkr||zdzdkr|dz }&|dz},ntd |d |S) z Compute the stage id to rank mapping for either a looped or V-style schedule. Most commonly num_stages == pp_size * 2, but this function can be used to compute the mapping for any number of stages per rank. r7vrz num_stages z% must be evenly divisible by pp_size z for V scheduleszStyle z is not supported.)r4 ValueError)r8r9r:mapping stage_index rank_indexs rgenerate_stage_to_rank_mappingrD\sG  ,, 9 9K#.#8GK  9 #  1 $ $hjhhwhhh   ,,  K#-GK a7*a//w&!+q00a a  ;%;;;<<< Nrct|||}i}|D])\}}||vrg||<|||*|D]}||S)a Compute the rank to stage id mapping for either a looped or V-style schedule. This function inverts the stage_to_rank_mapping to get which stages are assigned to each rank. Returns a dictionary mapping rank -> list of stage indices assigned to that rank. )rDitemsrvaluessort)r8r9r: stage_to_rankrank_to_stagesstage_idrankstagess rgenerate_rank_to_stage_mappingrN~s37JNNM,.N'--//..$ ~ % %#%N4 t##H----!'')) rc<eZdZUdZejed<eed<eed<dS)PipeInfoz> Captures information for a pipeline (`Pipe` object). graphr9has_loss_and_backwardN) r"r#r$r%rGraph__annotations__intboolr&rrrPrPs> 8OOOOOOrrP)r7)logging dataclassesrtypingrr r getLoggerr"loggerrr RuntimeErrorr!r.listr tupler6rUstrdictrDrNrPr&rrras!!!!!!   8 $ $(((0&@@@@@<@@@      D.elC6G0HHI  $u|,eEL#4E.FFG     17 !*- #s(^F17 !*- #tCy.4            r