`i' ddlZddlZddlmZddlmZmZmZddlZddl m Z ddl m Z ddlmZmZddlmZddlmZmZddlmZmZmZmZmZdd lmZmZmZdd l m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'ddl(m)Z)gdZ*Gdde'Z+GddeZ,dee-ej.fdee-ej.ffdZ/dee-ej.fdee-ej.ffdZ0e1dkr~GddeZ2ej3Z4e45de-dde2De2j6e45de-d !e45d"e-d#!e47Z8e9d$e8j:d%e8j;d&e8j<d'd(e8j:d)Z=e8j<e2j6j>krDej?@e8j:re0e8j:e8j;dSe9e=dSe8j<e2jAj>krDej?Be8j:re/e8j:e8j;dSe9e=dSeCd*e8j<dS)+N)Enum)castOptionalUnion)narrow_tensor_by_index)FileSystemReaderFileSystemWriter)flatten_state_dict)_EmptyStateDictLoadPlannerDefaultLoadPlanner)MetadataSTATE_DICT_TYPE STORAGE_TYPESTensorPropertiesTensorStorageMetadata) LoadItemTypeLoadPlan LoadPlanner)_create_chunk_list)_load_state_dict)_save_state_dict) StorageReader)Future)dcp_to_torch_savetorch_save_to_dcpBroadcastingTorchSaveReaderDynamicMetaLoadPlannerc2eZdZdZ ddeeeejfde ddfdZ de fdZ d e d ededfd Zd e d eddfdZd e de fdZdee dee fdZddeeejdfddfdZedeeejfdefdZdS)raI StorageReader for reading a Torch Save file. This reader will read the entire checkpoint on the coordinator rank, and then broadcast and shard each tensor to all ranks. . N.B. Intended to be used with DynamicMetaLoadPlanner .. warning:: Current implementation only supports loading Tensors. >>> # xdoctest: +SKIP("undefined vars") >>> sd = {"mode": model} >>> dcp.load( >>> sd, >>> storage_reader=BroadcastingTorchSaveReader(), >>> planner=DynamicMetaLoadPlanner(), >>> checkpoint_id="path_to_model.pt" >>> ) Nr checkpoint_idcoordinator_rankreturnc"||_||_dSN)rr )selfrr s }/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torch/distributed/checkpoint/format_utils.py__init__z$BroadcastingTorchSaveReader.__init__;s + 0c"tiS)zGExtends the default StorageReader to support building the metadata filestate_dict_metadata)r )r$s r% read_metadataz)BroadcastingTorchSaveReader.read_metadataCsB////r'planplannerc rtt|}|jr?|jJt j|jdd}|jrt |\}}nd}|jD]}|jtj kr2td|j j dt|jd|jrDtj}||j j |}n)t j|j|j j }tj||jdt/||j|j}||}||ks>@@M %%''6;;==888=s(== %%''==-3[[]]==988    ' ' '  ! !#} 5 5 5 5hh t r'metadatar7cp||_|jrtj|jksJ|jJdS*Implementation of the StorageReader methodN)r7rAget_rankr r)r$rXr7s r%set_up_storage_readerz1BroadcastingTorchSaveReader.set_up_storage_reader|sD,   <=??d&;;;;;!-----r'c|Sr[)r$r,s r%prepare_local_planz.BroadcastingTorchSaveReader.prepare_local_plans r' global_planc|Sr_r`)r$rbs r%prepare_global_planz/BroadcastingTorchSaveReader.prepare_global_plansr'c||_dSrZ)r)r$rs r%resetz!BroadcastingTorchSaveReader.resets*r'c@tj|Sr_)ospathisfile)clsrs r%validate_checkpoint_idz2BroadcastingTorchSaveReader.validate_checkpoint_idsw~~m,,,r')Nrr#)r@ __module__ __qualname____doc__rrstrrhPathLikeintr&r r+rrrrWboolr]ralistrdrf classmethodrlr`r'r%rr's*<@ !11c2;&6 7811  11110x0000 1h111111f.h..QU....xHtH~$x.++5bk4)?#@+D++++-5bk9I3J-t---[---r'rc FeZdZdZ d dedeededdffd ZxZ S) ra Extension of DefaultLoadPlanner, which creates a new Metadata object based on the passed in state dict, avoiding the need to read metadata from disk. This is useful when reading formats which don't have a metadata file, like Torch Save files. . N.B. Intended to be used with BroadcastingTorchSaveReader .. warning:: Current implementation only supports loading Tensors. >>> # xdoctest: +SKIP("undefined vars") >>> sd = {"mode": model} >>> dcp.load( >>> sd, >>> storage_reader=BroadcastingTorchSaveReader(), >>> planner=DynamicMetaLoadPlanner(), >>> checkpoint_id="path_to_model.pt" >>> ) NFrFrXr7r!c t|||i}|jD]\}}t j|s(t d|dt|jdtt|j | t|||<t||_dS)zdSetups of the planner, extnding default behavior by creating the Metadata object from the state dictr2r3r4)dtyper)N)superset_up_plannerrFr:r8 is_tensorr=r;r@rrrxrLrr rX)r$rFrXr7r*keyrT __class__s r%rzz%DynamicMetaLoadPlanner.set_up_planners z8^DDD8:?0022  KC?6** "YcYY$(JJ$7YYY (= v|444 "6**((  $ $ !5HIII r')NF) r@rmrnrorrr rsrz __classcell__)r}s@r%rrs.(,$ JJ#J8$J J  JJJJJJJJJJr'rdcp_checkpoint_dirtorch_save_pathci}t|t|tdtj||dS)aq Given a directory containing a DCP checkpoint, this function will convert it into a Torch save file. Args: dcp_checkpoint_dir: Directory containing the DCP checkpoint. torch_save_path: Filename to store the converted Torch save file. .. warning:: To avoid OOM, it's recommended to only run this function on a single rank. T)storage_readerr-no_distN)rrr r8save)rrsds r%rrsUB '(:;;*,,   Jr?#####r'cptj|d}t|t|ddS)aB Given the location of a torch save file, converts it into a DCP checkpoint. Args: torch_save_path: Filename of the Torch save file. dcp_checkpoint_dir: Directory to store the DCP checkpoint. .. warning:: To avoid OOM, it's recommended to only run this function on a single rank. F)r1T)storage_writerrN)r8r9rr )rrrFs r%rrsMO%@@@J#34F#G#GQUr'__main__ceZdZdZdZdS) FormatMode torch_to_dcp dcp_to_torchN)r@rmrn TORCH_TO_DCP DCP_TO_TORCHr`r'r%rrs% % r'rmodezConversion modecg|] }|j Sr`)value).0ms r% rs---Q---r')r;helpchoicesdefaultr5zPath to the source model)r;rdstzPath to the destination modelzConverting checkpoint from z to z using method: ''zNo checkpoint found at z. Skipping conversion.zUnknown conversion mode: )Dargparserhenumrtypingrrrr8torch.distributed distributedrAtorch.distributed._shard._utilsrtorch.distributed.checkpointrr )torch.distributed.checkpoint._nested_dictr ,torch.distributed.checkpoint.default_plannerr r %torch.distributed.checkpoint.metadatar rrrr$torch.distributed.checkpoint.plannerrrr,torch.distributed.checkpoint.planner_helpersr.torch.distributed.checkpoint.state_dict_loaderr-torch.distributed.checkpoint.state_dict_saverr$torch.distributed.checkpoint.storager torch.futuresr__all__rrrprqrrr@rArgumentParserparser add_argumentr parse_argsargsprintr5rrcheckpoint_missing_warningrrirjrisdir ValueErrorr`r'r%rs| (((((((((( BBBBBBKKKKKKKKHHHHHHUTTTTTTTTTKKKKKKKKKKKKJJJJJJ>>>>>>    l-l-l-l-l--l-l-l-^+J+J+J+J+J/+J+J+J\$c2;./$3 +,$$$$23 +,c2;./. z&&&&&T&&& %X $ & &F   --*---'  C.HIII C.MNNN     D EZdhZZDHZZdiZZZ C$(BBB yJ+111 7>>$( # # .  dh 1 1 1 1 1 E, - - - - - j-3 3 3 7== " " .  dh 1 1 1 1 1 E, - - - - -j@TY@@AAAIr'