.`i+ddlZddlmZddlmZddlmZddlmZm Z ddl m Z ddl m Z ddlmZdd lmZmZGd d eZGd d eZdS)N)PretrainedConfig) LoRAConfig)split_tensor_along_last_dim tensor_model_parallel_all_reduce)RowParallelLinear)current_platform)BaseLinearLayerWithLoRA)_fully_sharded_can_replace_not_fully_sharded_can_replacec eZdZdeddffd ZdejdejfdZdejdejfdZd ejdeje ejejdzfzfd Z e e dd e jd ed ededzdef dZxZS)RowParallelLinearWithLoRA base_layerreturnNct||jj|_|jj|_d|_dSNr )super__init__rinput_size_per_partition input_size output_sizen_slices)selfr __class__s x/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/lora/layers/row_parallel_linear.pyrz"RowParallelLinearWithLoRA.__init__sA $$$/B?6 lora_ac^|j}|j|z}|jdz|z}|dd||f}|Sr)rtp_rank)rr shard_size start_idxend_idxs r slice_lora_az&RowParallelLinearWithLoRA.slice_lora_asC_ L:- .>5oFFFF$F.2o.KUdo**QU * M{""r source_layer lora_configpacked_modules_list model_configc.t|tuSr&)typer)clsr;r<r=r>s rcan_replace_layerz+RowParallelLinearWithLoRA.can_replace_layerSsL!!%666rr&)__name__ __module__ __qualname__rrtorchTensorr#r(tupler: classmethodr nnModulerlistrboolrB __classcell__rs@rrrsE#45<EL5<EL(#l(# elEL4,??@ @(#(#(#(#T# 15 77i7 7" 7 '- 7  777$#[77777rrc eZdZdZdejdejfdZddejdejdzdejfdZee dd e j d e d e d edzdef fd ZxZS) RowParallelLinearWithShardedLoRAa Differs from RowParallelLinearWithLoRA by slicing the LoRA B's also. Based on S-LoRA, slicing happens along the output dim. This yields a combined partial sum from the row parallel base layer and column partitioned output from the LoRA. r$rc|jdjd}|j|z}|jdz|z}|||ddf}|S)Nrr )lora_b_stackedshaper)rr$r r!r"s rr(z-RowParallelLinearWithShardedLoRA.slice_lora_bnsQ(+1!4 L:- (DQ(G H-8    .2-@-K-K At*C. .  244 #"F +I+I      % ,J, ,  244 ! Fn- rr;r<r=r>cPt||||dS)NF)r;r<r=r>decorate)rrB)rAr;r<r=r>rs rrBz2RowParallelLinearWithShardedLoRA.can_replace_layers5ww((%# 3% )   rr&)rCrDrE__doc__rFrGr(r1rIr rJrKrrLrrMrBrNrOs@rrQrQds5<EL))u|)5<$+>)%,))))V 15   i   "  '-         [     rrQ)rFtorch.nnrJ transformersrvllm.config.lorarvllm.distributedrr!vllm.model_executor.layers.linearrvllm.platformsr base_linearr utilsr r rrQr'rrrvs2  ))))))''''''@?????++++++000000MMMMMMMMG7G7G7G7G7 7G7G7G7^L L L L L '@L L L L L r