PiKddlmZddlZddlmZdddddd d d d d ddd Z ddeeejfdededededeeejff dZ ddeeejfdedededeeejff dZ dS))DictN)get_mapped_keyztok_embeddings.weightzlayers.{}.attn.q_proj.weightzlayers.{}.attn.k_proj.weightzlayers.{}.attn.v_proj.weightz!layers.{}.attn.output_proj.weightzlayers.{}.mlp.w1.weightzlayers.{}.mlp.w3.weightzlayers.{}.mlp.w2.weightzlayers.{}.sa_norm.scalezlayers.{}.mlp_norm.scalez norm.scalez output.weight) zmodel.embed_tokens.weightz'model.layers.{}.self_attn.q_proj.weightz'model.layers.{}.self_attn.k_proj.weightz'model.layers.{}.self_attn.v_proj.weightz'model.layers.{}.self_attn.o_proj.weightz$model.layers.{}.mlp.gate_proj.weightz"model.layers.{}.mlp.up_proj.weightz$model.layers.{}.mlp.down_proj.weightz&model.layers.{}.input_layernorm.weightz/model.layers.{}.post_attention_layernorm.weightzmodel.norm.weightz score.weight  state_dict num_heads num_kv_headsdimhead_dimreturnci}|zfd}|D]K\}}|dkr d|vrt|t} d|vr |||}nd|vr |||}||| <L|S)a Convert a state dict from HF's format to torchtune's format, which contains the weights of a reward model (i.e. a classifier with a single class). State dicts from multiple checkpoint files should be consolidated into a single state dict before calling this function. The logic is identical to :func:`~torchtune.models.convert_weights.hf_to_tune`, but with a different mapping. Eg of HF-format state dict can be found in the ``Ray2333/reward-model-Mistral-7B-instruct-Unified-Feedback`` repo in HF. Args: state_dict (Dict[str, torch.Tensor]): State dict in HF's format. num_heads (int): Number of heads in the model. num_kv_heads (int): Number of heads in the key/value projection layers. dim (int): Dimension of the model. head_dim (int): Dimension of the head. If not provided, it will be calculated as dim // num_heads. Returns: Dict[str, torch.Tensor]: State dict in torchtune's format. Nc||ddzdd|zSNview transposereshapetn_headsr r s y/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchtune/rlhf/utils/_convert_weights.py_permutez#reward_hf_to_tune.._permute=sB FF7Ax1}c 2 2 Yq!__ Wh(3 / / z score.biaszrotary_emb.inv_freqq_projk_proj)itemsr_REWARD) rrr r r converted_state_dictrkeyvaluenew_keys `` rreward_hf_to_tuner$s8)#      !&&(( . . U ,     + +$S'22G s??HUI..EE __HUL11E(-W%% rc  i}dtD}|z  fd}|D];\}}t||} d|vr |||}nd|vr |||}||| <<|S)ap Convert a state dict from torchtune's format to Hugging Face's format for a reward model. This function takes a state dictionary in torchtune's format, which contains the weights of a reward model (i.e. a classifier with a single class), and converts it into a format that can be loaded into a Hugging Face model. The logic is identical to :func:`~torchtune.models.convert_weights.tune_to_hf`, but with a different mapping. Args: state_dict (Dict[str, torch.Tensor]): State dict in torchtune's format. num_heads (int, optional): Number of heads in the model. Defaults to 32. num_kv_heads (int, optional): Number of heads in the key/value projection layers. Defaults to 32. dim (int, optional): Dimension of the model. Defaults to 4096. Returns: Dict[str, torch.Tensor]: State dict in Hugging Face's format. ci|]\}}|| Sr').0kvs r z%reward_tune_to_hf..ls>>>daQ>>>rc||dzddd|zSrrrs rrz#reward_tune_to_hf.._permuteosB FF7HM1c 2 2 Yq!__ Wh(3 / / rrr)rrr) rrr r r inverted_mapping_dictrr!r"r#r s ` @rreward_tune_to_hfr.Ts.>>gmmoo>>>iH      !&&((.. U &;<< s??HUI..EE __HUL11E(-W%% r)rrrN)rrr) typingrtorch torchtune.models.convert_weightsrrstrTensorintr$r.r'rrr5sR ;;;;;;"9/M/M/M/R,E*C,E.G7Q%#  $ 4 4 S%,&'4 4 4  4  4  #u|  4 4 4 4 r * * S%,&'* * *  *  #u|  * * * * * * r