-`iS2tddlmZddlmZddlmZddlmZmZddl Z ddl Z ddl m Z ddlmZddlmZmZdd lmZdd lmZdd lmZeeZeGd d ZeGddZGddZeddddgdZe deZGddeeZ eGddZ!Gdde e!Z"eGddZ#Gdd e e#Z$eGd!d"Z%Gd#d$e e%Z&dS)%)MutableSequence)Sequence) dataclass)AnyGenericN)TypeVar) init_logger)PromptLogprobsSampleLogprobs) LoRARequest)MultiModalPlaceholderDict)RequestStateStatsceZdZUdZeed<eed<eeed<edzed<e dzed<dZ e j dzed<dZ edzed <dZeezdzed <dZedzed <d efd Zd efdZdS)CompletionOutputa!The output data of one completion output of a request. Args: index: The index of the output in the request. text: The generated output text. token_ids: The token IDs of the generated output text. cumulative_logprob: The cumulative log probability of the generated output text. logprobs: The log probabilities of the top probability words at each position if the logprobs are requested. finish_reason: The reason why the sequence is finished. stop_reason: The stop string or token id that caused the completion to stop, None if the completion finished for some other reason including encountering the EOS token. lora_request: The LoRA request that was used to generate the output. indextext token_idsNcumulative_logproblogprobsrouted_experts finish_reason stop_reason lora_requestreturnc|jduSN)rselfs `/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/outputs.pyfinishedzCompletionOutput.finished3s!--cd|jd|jd|jd|jd|jd|jd|jd|jd S) NzCompletionOutput(index=z, text=z , token_ids=z, routed_experts=z, cumulative_logprob=z , logprobs=z, finish_reason=z, stop_reason=))rrrrrrrrrs r__repr__zCompletionOutput.__repr__6s /dj / /I / / / /#1 / /#'"9  / /   / / "/  / / + / / / r!)__name__ __module__ __qualname____doc__int__annotations__strGenericSequencefloatr rnpndarrayrrrr boolr r$r!rrrs" JJJ IIIs#### $$$t####(,NBJ%,,, $M3:$$$$(KsT!((('+L+$+++.$....  #       r!rcDeZdZUdZejed<defdZde de fdZ dS) PoolingOutputznThe output data of one pooling output of a request. Args: data: The extracted hidden states. datarcd|jdS)NzPoolingOutput(data=r#)r4rs rr$zPoolingOutput.__repr__Ms1TY1111r!otherct||jo.t|j|jkSr) isinstance __class__r0r4all)rr6s r__eq__zPoolingOutput.__eq__Ps?%00 T Y%* $ ) ) + +6 6  r!N) r%r&r'r(torchTensorr*r+r$objectr0r;r1r!rr3r3Csm ,2#2222 F t      r!r3c eZdZdZ dddddededzdeedzdedzdeed e d e dzd e dzd edzd eedzdedzde dzde eefdzdeddfdZddde ddfdZdefdZdS) RequestOutputakThe output data of a completion request to the LLM. Args: request_id: The unique ID of the request. prompt: The prompt string of the request. For encoder/decoder models, this is the decoder input prompt. prompt_token_ids: The token IDs of the prompt. For encoder/decoder models, this is the decoder input prompt token ids. prompt_logprobs: The log probabilities to return per prompt token. outputs: The output sequences of the request. finished: Whether the whole request is finished. metrics: Metrics associated with the request. lora_request: The LoRA request that was used to generate the output. encoder_prompt: The encoder prompt string of the request. None if decoder-only. encoder_prompt_token_ids: The token IDs of the encoder prompt. None if decoder-only. num_cached_tokens: The number of tokens with prefix cache hit. kv_transfer_params: The params for remote K/V transfer. N)multi_modal_placeholderskv_transfer_params request_idpromptprompt_token_idsprompt_logprobsoutputsr metricsrencoder_promptencoder_prompt_token_idsnum_cached_tokensrArBkwargsrc  |r(tdt|||_||_||_| pi|_||_||_||_ ||_ ||_ | |_ | |_ | |_| |_dS)Nz+RequestOutput: Ignoring extra arguments: %s)logger warning_oncer+rCrDrErArFrGr rHrrIrJrKrB)rrCrDrErFrGr rHrrIrJrKrArBrLs r__init__zRequestOutput.__init__ns(     =s6{{   % 0(@(FB%.    (,(@%!2"4r! next_output aggregatecz|xj|jzc_|j|_|jD]}t|jD]\}}|j|jkr|r|xj|jz c_t |jtst|j|_|j |j|j r(|j J|j |j |j |_ |j |_ |j|_n ||j|<n|j|dS)z,Merge subsequent RequestOutput into this oneN)r rBrG enumeraterrr8rrlistextendrrrrappend)rrQrRnext_completioni completions raddzRequestOutput.addsY -- "-"@*2 5 5O!*4PPN37 8L3M3MJ0",33O4MNNN*3Q#-#6#B#B#B&/667OPPP+>#54C3P 01@1L ..+: QE%=( ##O444- 5 5r!cd|jd|jd|jd|jd|jd|jd|jd|jd |jd |j d |j d |j d S)NzRequestOutput(request_id=z , prompt=, prompt_token_ids=z, encoder_prompt=z, encoder_prompt_token_ids=z, prompt_logprobs= , outputs= , finished=z , metrics=z, lora_request=, num_cached_tokens=z, multi_modal_placeholders=r#) rCrDrErIrJrFrGr rHrrKrArs rr$zRequestOutput.__repr__s I I Ik I I $ 5 I I#1 I I)-(E I I $3 I I | I I  I I| I I!- I I"&!7 I I)-(E I I I r!)NNNNN)r%r&r'r(r+rUr)r rr0rr r dictrrPr[r$r1r!rr@r@Vsz>-1+/%)59(,$5FJ48$5$5$5$5d $5s)d* $5 ($. $5 &' $5$5#T)$5"D($5d $5#'s)d"2$5:$5#