.`i!ddlZddlZddlZddlmZddlmZddlmZddl m Z m Z m Z ddl mZddlmZddlmZeeZGd d ZdS) N) init_logger)LoRALayerWeights) PEFTHelper) get_lora_idis_base_embeddding_weightsparse_fine_tuned_lora_name)TensorizerConfig) WeightsMapper)is_pin_memory_availablecleZdZdZdededeeefddfdZdeddfdZ d ededzfd Z d ede fd Z e ddedeeejfdededejdzdedzdedzddfdZe dd ddddddedeedededzdedejdzdedzdedzdedzddfdZdS) LoRAModelzA LoRA fine-tuned model. lora_model_idranklorasreturnNc`||_|dksJd|j||_||_dS)z Args: lora_model_id: The integer id for the lora model. rank: lora rank. loras: module name -> weights for lora-replaced layers. rz.a valid lora id should be greater than 0, got N)idrr)selfrrrs h/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/lora/lora_model.py__init__zLoRAModel.__init__sE q   FTW F F!   27 cj|||j|jS)z[Return a copy of the object with different ids. Will share the underlying tensors.)rr) __class__rrcopy)rrs rclonezLoRAModel.clone0s6~~ *//##   r module_namec8|j|dS)z#Get LoRA for a given module by nameN)rget)rrs rget_lorazLoRAModel.get_lora:sz~~k4000r lora_namec||jvS)N)r)rr s rcheck_lora_namezLoRAModel.check_lora_name>sDJ&&rcudatensors peft_helperdevicedtypemodel_vocab_sizeweights_mapperct|dko t}i} |D]\} } t| rt | |\} } | | vrt j| || | <| rd| vr4|2|| jdkr!td| jdd|d| ||| | _ |r*| | j | | _ | ||| | _ |r*| | j | | _ |||j | S) z0Create a LoRAModel from a dictionary of tensors.cpulora_embedding_ANzThe embedding LoRA size(z;) must be consistent with the base model's vocabulary size(z).)r&r')strr itemsrrr from_configshape RuntimeErrortolora_a pin_memorylora_br)clsrr$r%r&r'r(r)r5r tensor_nametensorr is_lora_as rfrom_lora_tensorszLoRAModel.from_lora_tensorsAs[[E)G.E.G.G -/#*==?? W W K)+66 %?^&& "K%''%5%A&&k" W&+55(4(FLO;;&W6<?WWBRWWW-3IIV5I,Q,Qk")W05k0B0I0T0T0V0VE+&-,2IIV5I,Q,Qk")W05k0B0I0T0T0V0VE+&-s=+-777r)rr&r'r(r)tensorizer_config_dictlora_direxpected_lora_modulesr=c jtjd} tjd} tjd} i} gdtffd }| rmddlm}t di| }tj|jd} |}|| fd |j i|j } || n!tj | rlgtj | d 5}|||D]}||| |< d d d n #1swxYwYntj | stj | rFtj | r| n| }t!j||d } || nt%d||t)n|| ||||S)a8Create a LoRAModel from a local checkpoint. Args: lora_dir: The local path that has lora data. expected_lora_modules: Name of modules that are expected to be replaced by lora. peft_helper: Loaded lora configuration information. lora_model_id: LoRA model id. If not given, automatically set by a global counter. device: Device where the lora model is loaded. dtype: dtype of the lora model weights. Returns: Loaded LoRA Model. zadapter_model.safetensorszadapter_model.binzadapter_model.ptmodulesc |D]}t|rd|vrt| \}}d|vr<|d}||dzd}|vr|j|dddvr|rt dddd dS) N base_layerz.expertsr-.zWhile loading z, expected target modules in z but received z6. Please verify that the loaded LoRA module is correct)keysrrfindappendrsplit ValueError) rA lora_moduler_ expert_idx expert_suffixr?r>unexpected_modulesr)s rcheck_unexpected_moduleszALoRAModel.from_local_checkpoint..check_unexpected_moduless5&||~~ ; ; -k:: ;..!;K!X!X Q,,!,!1!1*!=!=J$/ Q0@0@$AM$,AAA*11+>>> ''Q//3;PPP&--k:::!  MXMM*?MM%7MMM  rr)TensorDeserializerzadapter_model.tensorsr'pt) frameworkNT) map_location weights_onlyz doesn't contain tensors)rr$r%r&r'r(r))ospathjoindict tensorizerrQr tensorizer_dir_construct_tensorizer_argsr'deserialization_kwargsisfile safetensors safe_openrF get_tensortorchloadrJr<r)r8r>r?r%rr&r'r(r)r=lora_tensor_pathlora_bin_file_pathlora_pt_file_pathr$rPrQtensorizer_configtensorizer_argsfmodulelora_file_pathrOs `` ` @rfrom_local_checkpointzLoRAModel.from_local_checkpointos:7<<2MNNW\\(4GHHGLL3EFF+-46 d         6 "% D 5 5 5 5 5 5 0 J J3I J J !w||!02I   0JJLLO(( '-"8G % $W - - - - W^^, - - D"$ &'74HHH ;A((+++ffhh;;F&'ll6&:&:GFOO; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; W^^. / / D27>>BS3T3T D7>>"455'""&  jfSWXXXG $ $W - - - -BBBCC C$$+8+@+---m#-)%   s;E''E+.E+)r#NNN)__name__ __module__ __qualname____doc__intrZr.rrrrboolr" classmethodrcTensorrr'r r<setrmrVrrr r s7""888C))* 8  8888* 3 ;    1C1,rs  ######333333,,,,,, IHHHHH::::::====== X  E E E E E E E E E E r