-`ietdZddlZddlmZddlmZmZddlmZddl m Z m Z ddl Z ddl mZddlmZdd lmZdd lmZdd lmZdd lmZeeZd ZdZGddeZeGddZGddeZGddee j ddZ!Gdde j ddZ"dS)z(Sampling parameters for text generation.N)field)EnumIntEnum)cached_property) AnnotatedAny) dataclass)VLLMValidationError) init_logger)LogitsProcessor) TokenizerLike)PydanticMsgspecMixingh㈵>g{Gz?ceZdZdZdZdZdS) SamplingTyperN)__name__ __module__ __qualname__GREEDYRANDOM RANDOM_SEEDh/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/sampling_params.pyrrs F FKKKrrcHeZdZUdZeezdzed<dZedzed<dZe edzed<dZ edzed<dZ e dzed<dZ e ed<dZe ed <dZe ed <dZedzed <dZedzed <edd Zedzed< edd Ze ed< dZde fdZde fdZdS)StructuredOutputsParamsNjsonregexchoicegrammar json_objectFdisable_fallbackdisable_any_whitespacedisable_additional_propertieswhitespace_patternstructural_tag)defaultinit_backend_backend_was_autoc t|jdu|jdu|jdu|jdu|jdu|jdug}|dkrtd|j|dkrtd|jdS)z1Validate that some fields are mutually exclusive.NrzWYou can only use one kind of structured outputs constraint but multiple are specified: zOYou must use one kind of structured outputs constraint but none are specified: ) sumrrr r!r"r' ValueError__dict__)selfcounts r __post_init__z%StructuredOutputsParams.__post_init__5s % $& 4' D( ,#4/      199?/3}??  199;+/=;;  9rreturnc:tfddDS)S Returns True if all structured-output constraint fields are None. c3<K|]}t|duVdSNgetattr.0rr0s r z?StructuredOutputsParams.all_constraints_none..PsF    D% D (       r)rrr r!r"r'allr0s`rall_constraints_nonez,StructuredOutputsParams.all_constraints_noneLs?           rc:tfddDS)r5c3<K|]}t|duVdSr7r8r:s rr<zRStructuredOutputsParams.all_non_structural_tag_constraints_none..`sF    D% D (       r)rrr r!r"r=r?s`r'all_non_structural_tag_constraints_nonez?StructuredOutputsParams.all_non_structural_tag_constraints_none\s?           r)rrrrstrdict__annotations__rr listr!r"boolr#r$r%r&r'rr*r+r2r@rCrrrrr!sp#D#*t """E3:#FDI ###GS4Z#K###"d"""#(D(((*/!4///%)d )))!%NC$J%%% 5E:::HcDj:::N#eE>>>t>>>N. d            rrceZdZdZdZdZdS)RequestOutputKindrrrN)rrr CUMULATIVEDELTA FINAL_ONLYrrrrJrJlsJ EJJJrrJc>*eZdZUdZdZeed< dZeed< dZ eed< dZ eed< dZ eed < dZ eed < d Z eed < dZeed < dZedzed< dZeeezdzed< dZeedzed< dZeed< dZedzed< d Zeed< dZedzed< dZedzed< dZeed< dZeed< dZeed< dZeed< dZedzed< dZ eed< dZ!e"ee#j$d !fdzed"< e%j&Z'e%ed#<dZ(eed$< d Z)eed%<e#j*e+&Z,e+eed'<dZ-e.dzed(< dZ/e0eefdzed)< dZ1eedzed*< dZ2e0eefdzed+< dZ3eedzed,< dZ4eeedzed-<dZ5edzed.<e6ddddddd ddddddddd ddddddde%j&dddddfdedzdedzdedzdedzd edzd edzd ed ededzdeeezdzdeedzd,eedzdedededzdededzdedzdedededee7dzd"e"ee#j$d !fdzd#e%d(e.dzd)e0eefe0eefzdzd*eedzd+e0eefdzd$ed/dfd1Z9d>d2Z:d>d3Z; d?d4e0eefd5edzd/dfd6Ze?d/e@fd9ZAeBd/e+efd:ZCeBd/eeedzfd;ZDd@d<ZEd/efd=ZFdS)ASamplingParamsaSampling parameters for text generation. Overall, we follow the sampling parameters from the OpenAI text completion API (https://platform.openai.com/docs/api-reference/completions/create). In addition, we support beam search, which is not supported by OpenAI. rnpresence_penaltyfrequency_penalty?repetition_penalty temperaturetop_prtop_kmin_pNseedstopstop_token_idsF ignore_eos max_tokens min_tokenslogprobsprompt_logprobs flat_logprobsT detokenizeskip_special_tokensspaces_between_special_tokenslogits_processorsinclude_stop_str_in_output)getruncate_prompt_tokens output_kind skip_cloneoutput_text_buffer_length)default_factory_all_stop_token_idsstructured_outputs logit_biasallowed_token_ids extra_args bad_words_bad_words_token_idsskip_reading_prefix_cacher3c4|d|D}td"id|dn|d|dn|d|dn|d|dn|d |dn|d |dn|d |d |d |d| d| d| d| d| d|d|d|d|d|d|d|d|d|d|d|d|d|d |d!|S)#Nc li|]1\}}t|tdtd|2S)gY@gY)intminmax)r;tokenbiass r z0SamplingParams.from_optional.."sHE4E Cs64'8'899rrPrrRrQrSrUrTrVrWrXrYrZr[r\rurhr]r_r`rarbrdrerfrgrkrlrqrrrsrtrmr)itemsrO)rPrRrSrUrVrWrXrYrZr[r\rurhr]r_r`rarbrdrerfrgrkrlrqrrrsrtrms r from_optionalzSamplingParams.from_optionals@  !#-#3#3#5#5J     9aa!  $4$ccDU  ") #s#   +2   ##E  %  %      *>   i  (B'A  "z!  ""z#  $"z%  &X'  (,O)  *"z+  ,!4 3-  .+H*G/  00/1  2$:#93  4$ 5  6 217  8"z9  :0/;  <"z=  >"z?  rcFd|jcxkr tkrNnnKtd|jttt |jt|_|jdkrd|_|jg|_n't|jtr |jg|_|j g|_ |j g|_ |j durd|_ |j durd|_ |jr-|j s&t d|jDdz |_||jt kr)d|_d|_d|_||j|j |j|j du|_dSdS) Nrzstemperature %s is less than %s, which may cause numerical errors nan or inf in tensors. We have maxed it out to %s.riTrc34K|]}t|VdSr7)len)r;ss rr<z/SamplingParams.__post_init__..ks(0K0KAQ0K0K0K0K0K0KrrTrQ)rV _MAX_TEMPloggerwarningr|rZr[ isinstancerDr\rurarbrhrn _verify_args _SAMPLING_EPSrWrXrY_verify_greedy_samplingrpupdaterwr?s rr2zSamplingParams.__post_init__Is t + + + +) + + + + + NNL      #4#3Y??D  9??DI 9 DII  3 ' ' $ DI   &"$D  > !DN =D DM  4 ' '#$D  9 PT< P-00K0K0K0K0K-K-Ka-OD *   m + +DJDJDJ  ( ( * * *  ''(;<<<  ) 1.2-A-MD * * * 2 1rct|jts$tdt |j|jdkrtd|jdd|jcxkrdksntd|jdd|jcxkrdksntd|jd|jd krtd |jd|jd kr td |jdd |j d |j cxkrdks"ntd|j dd|j |j dkrtd|j dt|j ts)tdt |j j d |jcxkrdksntd|jd|j+|jdkr td|jdd|j |jdkrtd|jd|j0|j|jkr td|jd|jd|j6|jdkr+|jdkr td|jdd|j |j6|jdkr+|jdkr td|jdd|j |j5|jdks |jdkrtd|jd |j t|jt*sJt-d!|jDstd"|jdt|jt*sJt1d#|jDrtd$|jr|jstd%dSdS)&Nz!n must be an int, but is of type rzn must be at least 1, got .gg@z)presence_penalty must be in [-2, 2], got z*frequency_penalty must be in [-2, 2], got rQz2repetition_penalty must be greater than zero, got z&temperature must be non-negative, got rV parametervaluerTztop_p must be in (0, 1], got rWriz.top_k must be 0 (disable), or at least 1, got ztop_k must be an integer, got zmin_p must be in [0, 1], got z#max_tokens must be at least 1, got r_rz3min_tokens must be greater than or equal to 0, got z4min_tokens must be less than or equal to max_tokens=z, got z)logprobs must be non-negative or -1, got raz0prompt_logprobs must be non-negative or -1, got rbz:truncate_prompt_tokens must be an integer >= 1 or -1, got rkc3@K|]}t|tVdSr7)rrz)r;st_ids rr<z.SamplingParams._verify_args..s,KKe:eS))KKKKKKrz/stop_token_ids must contain only integers, got c3K|]}| VdSr7r)r;stop_strs rr<z.SamplingParams._verify_args..s$668|666666rz$stop cannot contain an empty string.zYstop strings are only supported when detokenize is True. Set detokenize=True to use stop.)rrPrzr.typerRrSrUrVr rWrX TypeErrorrrYr_r`rarbrkr\rGr>r[anyrdr?s rrzSamplingParams._verify_argss)$&#&& QOdfOOPP P 6A::C$&CCCDD Dt,33333333TD+>%HdoHHH&o  ?Q  XdoXXX  ? &4?T_+L+LH"oHH59_HHH  = $")<)<QRARAR%LDMLLL$m   ,$**$q((%+'++++*    & 2  '1 , ,0Kb0P0P%525521   $-t44444KKt7JKKKKK X$BUXXX $)T***** 66DI666 6 6 ECDD D 9 T_ 3     rcL|jdkrtd|jddS)Nrz,n must be 1 when using greedy sampling, got r)rPr.r?s rrz&SamplingParams._verify_greedy_samplings0 6A::UDFUUUVV V :rgeneration_configmodel_eos_token_idc||j||dx}t|tr|hnt |}||||rS|j||js4||j t||_ dSdSdSdS)z=Update if there are non-default values from generation_configN eos_token_id) rpaddgetrrzsetdiscardrr]r\rG)r0rreos_idss rupdate_from_generation_configz,SamplingParams.update_from_generation_configs  )  $ ( (); < < <),,^<< z8SamplingParams.update_from_tokenizer..sQ   #/  !||x)*@@@  A@@@rzThe model vocabulary size is rz2, but the following tokens were specified as bad: zF. All token id values should be integers satisfying: 0 <= token_id <= rrur)rurvlstripencoderappendr r)r0rbad_wordadd_prefix_spaceprefixpromptprompt_token_idsinvalid_token_idss ` rupdate_from_tokenizerz$SamplingParams.update_from_tokenizers~  F$&! G GH%2 G G 08b(//"3"33#,#3#3E$4$$ )G$G(+t/H/LQ/OOO,--T5Nr5R1S1SSS-445EFFF G     '+'@     ! !A % %%? 0F0J??+<??&/%; ??? &n  & %rcx|jtkr tjS|j tjStjSr7)rVrrrrZrrr?s r sampling_typezSamplingParams.sampling_type's3  m + +& & 9 + +""rc|jSr7)rpr?s rall_stop_token_idsz!SamplingParams.all_stop_token_ids/s ''rc|jSr7)rvr?s rrz"SamplingParams.bad_words_token_ids3s ((rc|jrtj|S|jdnd|jD}tj||S)aDeep copy, but maybe not the LogitsProcessor objects. LogitsProcessor objects may contain an arbitrary, nontrivial amount of data that is expensive to copy. However, if not copied, the processor needs to support parallel decoding for multiple sequences See https://github.com/vllm-project/vllm/issues/3087 If skip_clone is True, uses shallow copy instead of deep copy. Ncvi|]6}t|t|dr|n|7S)clone)idhasattrr)r;lps rrz(SamplingParams.clone..IsK2gb'&:&:B r)memo)rmcopyrgdeepcopy)r0logit_processor_refss rrzSamplingParams.clone8sj ? #9T?? "%- D0 }T(<====rcdgd|jd|jd|jd|jd|jd|jd|jd |jd |j d |j d |j d |j d|j d|jd|jd|jd|jd|jd|jd|jd|jd|jd|jdS)NrzSamplingParams(n=z, presence_penalty=z, frequency_penalty=z, repetition_penalty=z, temperature=z, top_p=z, top_k=z, min_p=z, seed=z, stop=z, stop_token_ids=z , bad_words=z, include_stop_str_in_output=z , ignore_eos=z , max_tokens=z , min_tokens=z , logprobs=z, prompt_logprobs=z, skip_special_tokens=z , spaces_between_special_tokens=z, truncate_prompt_tokens=z, structured_outputs=z , extra_args=))joinrPrRrSrUrVrWrXrYrZr[r\rurhr]r_r`rarbrerfrkrqrtr?s r__repr__zSamplingParams.__repr__Ps - - - - - - - - - - $ 5 - - - -!%!7 - - - -#'"9 - - - - +  - - - - Z  - - - - Z  - - - -Z - - - -I - - - -I - - - -#1 - - - - - - - -+/*I - - - -/ - - - -/ - - - -/ - - - -  ! - - - -" $3# - - - -$$(#;% - - - -(1) - - - -*'+&A+ - - - -,#'"9- - - - -.// - - - - - - r)r3Nr7)r3rO)Grrr__doc__rPrzrFrRfloatrSrUrVrWrXrYrZr[rDrGr\r]rHr_r`rarbrcrdrerfrgrrhrkrmsgspecMetarJrKrlrmrnrrrprqrrrrErsrtrurvrw staticmethodr rr2rrrr rrrrpropertyrrrrrrrrOrOushAsJJJ "e!!!- #u"""-!$###>KE56E3NNNE5!D#*0#'D#S /D '''1'+NDI$+++J0Jd CJ)HcDjL#'OS4Z&&&BM47J+ $$$$7*.!4...E%)sTz(((5',,,,=IMIc<7<2+>+>+>&>?$FMMMJ&7%AK"AAAJ-&'s&&&$1GM#$F$F$FSFFF:>/$6===8*.JS%Z 4'...*.tCy4'.../(,JS#X%,,, #'ItCy4&&&248$tCy/D0777-1td{111),*-+.$'!'++/&*+0 !##&*$(.2:>MQ):)E=AAE.2,0 ;G G :G $,G !4<G "DL G T\ G t| G G G DjG DIo$G S D(G 9t#G %)G G $JG !G "*#G $t%G &'G (")G *(,+G , 047-G .!*#|w|r/B/B/B*B Cd J/G 0'1G 24d:3G 4e$tCJ'77$>5G 6 9t+7G 8cNT)9G :;G < =G G G \G R4N4N4N4Nl^^^^@WWWW*.88S>8 $J8  88886'}'''''R#|###_#(CH(((X()T$s)_t%;)))X)>>>>0 #      rrOT) omit_defaultsrEc`eZdZUdZeed<eed<dZeed<dZe ed<dZ e ed <dZ eed <d S) BeamSearchParamsz+Beam search parameters for text generation. beam_widthr_Fr]rQrVrTlength_penaltyrhN) rrrrrzrFr]rHrVrrrhrrrrrmsm 65OOOOOOJKNE',,,,,,rr)#rr dataclassesrenumrr functoolsrtypingrrrpydantic.dataclassesr vllm.exceptionsr vllm.loggerr vllm.logits_processr vllm.tokenizersr vllm.v1.serial_utilsrrrrrrrrJStructrOrrrrrs/. %%%%%%!!!!!!!!******//////######//////))))))555555 X    7 G G G G G G G  G Tu u u u u  N u u u u p - - - - - N - - - - - -r