.`iddlmZddlZerddlmZdejdeejejejeeeeffdZ ddd e edzd ed ed ed ej deejejejeje effdZ dS)) TYPE_CHECKINGN) LoRAMappingtoken_lora_tensorreturnctj|d\}}tj|d}tj|}|dd|dd|}|}|d}d}|dkr|dkrd}|||||||fS) at Get the information required for the sgmv kernel. With the features: 1. If consecutive requests in the batch use the same LoRA, this function will combine them into a single request, improving sgmv kernel inference performance. 2. At the beginning of each prefill stage inference, recalculations are needed based on the input, but only once. T) return_countsr)dimNF) torchunique_consecutivecumsum zeros_likecopy_maxitemsumsize) rlora_indices_tensorseq_length_tensor cum_resultb_seq_start_tensor max_length token_nums batch_sizeno_loras r/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/lora/punica_wrapper/utils.py compute_metar s.3-E...**/Q777J)*;<<qrr  CRC111"&&((--//J"&&((--//J$))!,,JGQ."44 mappingrlora_index_to_id max_loras vocab_sizeextra_vocab_sizedevicect|j}|}|}fd|jD} d} t t |D]C} || dkr|| nd} || dkr| nd|| <| || <D|||g} tj| tj |} tj| tj |}tj | d|z| d||zzg}tj |dk|dz |}| d}|}| }tj |dk|dz |}tj dt ||tj |t |zz}|jd|jd|jd|jdg}|||||fS) axConverts LoRAMapping to index tensors. Args: mapping: LoRAMapping mapping rows in a batch to LoRA ids. lora_index_to_id: List mapping LoRA ids to LoRA indices. max_loras: Maximum number of LoRAs. vocab_size: Model vocab size. extra_vocab_size: Extra vocab size each LoRA can have. Returns: A tuple of tensors: base_indices: Tensor of shape [batch_size] mapping batch rows to LoRA indices. sampler_indices: Tensor of shape [batch_size] mapping requests to LoRA indices for sampler. For generation, this will be the same as base_indices. For prefill, this will map requests to LoRA indices. sampler_indices_padded: Tensor of shape [batch_size] mapping requests to LoRA indices for sampler with padding. Same as sampler_indices, but -1 is replaced with max_loras. embeddings_indices: Tensor of shape [2, batch_size] mapping requests to embedding indices. First row is for embeddings added by the LoRAs, second row is for the LoRA.lora_a embeddings. indices_len: List of lengths of the above tensors. It contains (base_indices, sampler_indices, sampler_indices_padded, embeddings_indices). cJg|]}|dkr|nd S)rr )index).0xr!s r z#convert_mapping..]sB!!!78QUUq!!!!!!rNrr )dtyper%r )r%r,)list index_mappingcopyprompt_mappingrangelenr(r tensorlongstackwhereclonearangeshape)r r!r"r#r$r%index_mapping_indicesembedding_indices lora_indicesr1lora_idxi indices_listindicesprompt_mapping_tensorembeddings_indices base_indicessampler_indicessampler_indices_padded indices_lens ` rconvert_mappingrH4s{J(,G,A'B'B'G'G'I'I-2244(--//L!!!!