Pi(/ddlZddlmZmZmZmZmZmZmZm Z m Z ddl Z ddl m Z edZ GddeZde jdeee jffd Zde jd e eeefe fddfd Zd ee d ededeefdZ ddeeefdeedeeeffdZdeeefde efdZe jdeeefdededeeeffdZejde jdedfdZ d d ee d ededeeedeeedeeedeeeddfdZdS)!N) AnyDict GeneratorListLiteralOptionalProtocolSetUnion)nn)q_projk_projv_proj output_projc*eZdZdZdeefdZdS) AdapterModulez Interface for an ``nn.Module`` containing adapter weights. Note that an adapter module does not have to explicitly implement this protocol, but it must define the ``adapter_params(self)`` method. returncdS)a Return a list of strings corresponding to the names of the ``nn.Parameter`` s in the model coming from the adapter. E.g. if an nn.Module has adapter ``self.proj = nn.Linear(in_dim, out_dim)``, then adapter_params should return ``['proj.weight', 'proj.bias']``. See LoRALinear's :func:`~torchtune.modules.peft.LoRALinear.adapter_params` for an example. N)selfs q/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchtune/modules/peft/_utils.pyadapter_paramszAdapterModule.adapter_paramss  N)__name__ __module__ __qualname____doc__rstrrrrrrrs=  S       rrmodelrci}|D]\}}t|drt|jr|}|dD]@\}}||vr7|r|d|n|}|||i||A|gksJd|d|S)a Return the subset of parameters from a model that correspond to an adapter. Assumes that any adapter class has defined the :func:`~torchtune.modules.peft.AdapterModule.adapter_params` method. Args: model (nn.Module): Instance of model class containing some adapter params. Returns: Dict[str, nn.Parameter]: the subset of model's state dict containing only adapter parameters. rT)recurse.zAdapter params z not converted) named_moduleshasattrcallablernamed_parametersupdateremove)rrkvcurrent_adapter_paramsnpfull_keys rget_adapter_paramsr/$sN##%% H H1 1& ' ' HHQ5E,F,F H%&%5%5%7%7 "**4*88 5 51...-.5!zzazzzAH"))8Q-888*11!444&",,,G!7GGG-,, rrch|D]\}}|||vdS)a Set trainable parameters for an nn.Module based on a state dict of adapter parameters. Args: model (nn.Module): Instance of model class containing some adapter params. adapter_params (Union[Dict[str, Any], Set]): State dict mapping adapter key names to their respective nn.Parameters (i.e. outputs of :func:`~torchtune.modules.peft.get_adapter_params`.) Returns: None N)r&requires_grad_)rrr)r*s rset_trainable_paramsr2AsI&&((..1 n,----..rlora_attn_modulesapply_lora_to_mlpapply_lora_to_outputcJ|}|r|gdz}|r|d|S)a Return a list of the names of modules in the model that have LoRA applied. Note that the names here are local to their modules and not the fully qualified names from the model state dict. Args: lora_attn_modules (List[LORA_ATTN_MODULES]): list of which linear layers LoRA should be applied to in each self-attention block. Options are ``{"q_proj", "k_proj", "v_proj", "output_proj"}``. apply_lora_to_mlp (bool): whether LoRA is applied to each MLP linear. apply_lora_to_output (bool): whether LoRA is applied to the final output projection. Returns: List[str]: list of module names in the model that have LoRA applied. )w1w2w3output)append)r3r4r5lora_module_keyss rget_lora_module_namesr=SsG*)A+.@.@.@@*))) rcpu state_dictdevicecNdfd|DS)a Return the subset of the full state_dict from a model that correspond to an adapter. Assumes that "lora" and "magnitude" are unique names for adapter parameters, and that the state_dict is not sharded. All returned parameters are moved to CPU. Args: state_dict (Dict[str, Any]): Full model state dict. device (Optional[str]): device to move adapter parameters to. Default: 'cpu' Returns: Dict[str, Any]: the subset of model's state dict containing only adapter parameters. cd|vpd|vS)Nlora magnituder)xs rz(get_adapter_state_dict..s6Q;#B+2BrcXi|]&\}}|||'Sr)to).0r)r*adapter_key_filterr@s r z*get_adapter_state_dict..s; T T T1>P>PQR>S>S TAqttF|| T T Tr)items)r?r@rJs `@rget_adapter_state_dictrMps9"CB T T T T T (8(8(:(: T T TTrcpd|D}td|DS)a Get the keys from a state dict that correspond to LoRALinear modules. For example, if state_dict is the state dict of model and model.x.y.z is a LoRALinear, this method will return "model.x.y.z", not "model.x.y.z.lora_a.weight" or "model.x.y.z.lora_b.weight". Args: state_dict (Dict[str, Any]): State dict from a model. Returns: Set[str]: Set of keys in the state dict that correspond to LoRA modules. c"g|] }d|vsd|v | S)rCrDrrIr)s r z%_get_lora_modules..s*QQQq1 q@P@P@P@P@Prcg|]@}|ddddddAS).lora_a.weight.lora_b.weight .magnitude)replacerPs rrQz%_get_lora_modules..sX    II& + + W%r * * W\2 & &   r)keysset)r? lora_keyss r_get_lora_modulesr[sRRQJOO--QQQI         rrankalphact|}|D]}||d}||d}||dd}|||d|j}||z |z|z} || z} tj|| zd} || z dd} | | z} | ||d<||d=n||dxx||z |z|zz cc<||d=||d=|S) a] Merge LoRA weights into the base model format for efficient inference. NOTE: This function modifies state_dict inplace. If you do not want to do that, make a copy prior to calling this function. For every LoRA module in the state dict, this function will convert its base weight then delete the LoRA-specific parameters. Args: state_dict (Dict[str, Any]): State dict from a model. rank (int): The rank of LoRA matrices. alpha (float): The alpha value used for scaling LoRA decompositions. Returns: Dict[str, Any]: The merged state dict. rSrUrVNz.weight)dim)r[getrHdtypetorchlinalgnormview) r?r\r] lora_modulesmodule lora_a_weight lora_b_weightlora_magnitude base_weight lora_weight merged_weight weight_normmag_norm_scales rget_merged_lora_ckptrrsz,%Z00L22"f#<#<#<= "f#<#<#<= #6(=(=(=tDD  %$%7%7%78;;M  * * * &000 1 &000 1 1 r)NNNc#K|D]@\}}t|dr+t|jrt|drd|_A dV|D]@\}}t|dr+t|jrt|drd|_AdS#|D]@\}}t|dr+t|jrt|drd|_AwxYw)a Temporarily disable the adapters in a model. For example, this can be used in DPO for treating the LoRA adapters as the policy model and disabling it to treat the base model as the reference model. This context manager goes through all modules in the provided neural network model, and if a module has an ``adapter_params`` attribute that is callable and a ``disabled`` attribute, it sets ``disabled`` to True. Then, the control is given back to caller. When exiting the context manager, it sets ``disabled`` back to False for all modules that were temporarily disabled. Args: model (nn.Module): The model whose adapters are to be temporarily disabled. Yields: None: This function yields control back to the caller, with the adapters disabled. Example: >>> with disable_adapter(model): ... # Perform operations with adapters disabled ... pass rdisabledTNF)r#r$r%rrt)r_ris rdisable_adapterrvsQ,((**## 6 F, - - #.// # ++ # #FO ( ,,.. ( (IAv 011 (V233 (FJ// ( #(  ( (,,.. ( (IAv 011 (V233 (FJ// ( #(  (s B44AD  base_missingbase_unexpected lora_missinglora_unexpectedc  t||| fd}|r#|D] }||std|d!|rtd|r#|D] }||rtd|d!|rtddS) a A more memory-efficient way to validate that LoRA state dict loading was done properly. This function uses a model's LoRA config to check that LoRA and/or base model weights are loaded into the full model correctly. This function relies only on the values of missing and unexpected as returned by the load_state_dict API with strict=False. This allows us to do the validation without any additional calls to .state_dict(), which use additional memory. Args: lora_attn_modules (List[LORA_ATTN_MODULES]): list of which linear layers LoRA should be applied to in each self-attention block. Options are ``{"q_proj", "k_proj", "v_proj", "output_proj"}``. apply_lora_to_mlp (bool): whether LoRA is applied to each MLP linear. apply_lora_to_output (bool): whether LoRA is applied to the final output projection. base_missing (Optional[List[str]]): List of missing keys when loading base model weights. Default: None base_unexpected (Optional[List[str]]): List of unexpected keys when loading base model weights. Default: None lora_missing (Optional[List[str]]): List of missing keys when loading LoRA weights. Default: None lora_unexpected (Optional[List[str]]): List of unexpected keys when loading LoRA weights. Default: None Returns: None Raises: AssertionError: If base_missing contains any base model keys, **or** if base_unexpected is nonempty, **or** if lora_missing contains any LoRA keys, **or** if lora_unexpected is nonempty. c<tfdDS)Nctg|]4}d|dgvpd|dgv5S)r"rCrD)join)rIr)rEs rrQzNvalidate_missing_and_unexpected_for_lora....)sY    HHa[ ! !Q & I#((A{3C*D*D*I   r)any)rErhs`rrFz:validate_missing_and_unexpected_for_lora..(s8c    !   rzMissing non-LoRA key z from base model dictz!Unexpected key loading base modelzMissing LoRA key z from adapter state dictzUnexpected key loading adapterN)r=AssertionError) r3r4r5rwrxryrz is_lora_paramr)rhs @r(validate_missing_and_unexpected_for_lorars T),.BLMW W WA =## W$%UQ%U%U%UVVV WB@AAAV V VA}Q V$%T%T%T%TUUU V?=>>>??r)r>)NNNN) contextlibtypingrrrrrrr r r rdr LORA_ATTN_MODULESrModuler Parameterr/r2boolr=rMr[no_gradintfloatrrcontextmanagerrvrrrrrsVVVVVVVVVVVVVVVVVVVVVV GH     H   &biDbl1B,C:. 9.&+DcNC,?&@. ....$-. #Y <9>UUS#XU(0 U #s(^UUUU*$sCx.SX20S#X0 0 0 #s(^ 0000f %(29%(3C)D%(%(%(%(X)-+/(,+/????-.??????49% ?? d3i( ?? 49% ??d3i(?? ????????????r