.`icHUddlmZddlmZddlmZddlmZddlm Z ddl m Z ddl m Z mZddlmZdd lmZdd lmZmZmZerdd lmZmZeeZGd d ZGddeZGddeZGddeZGddeZ GddeZ!GddeZ"GddeZ#GddeZ$GddeZ%Gd d!e%Z&Gd"d#eZ'Gd$d%eZ(Gd&d'eZ)Gd(d)eZ*Gd*d+eZ+Gd,d-eZ,Gd.d/eZ-id0e(d1ed2ed3ed4e!d5e!d6e"d7e#d8e$d9e%d:e&d;e de)d?e*d@e*e*e,e-dAZ.e/e0e1efe2dB<dCS)D)deepcopy)lcm) TYPE_CHECKING) init_logger) ModelRegistry)current_platform)cdivround_up)STR_DTYPE_TO_TORCH_DTYPE)AttentionBackendEnum)FullAttentionSpec MambaSpecMLAAttentionSpec) ModelConfig VllmConfigc>eZdZed dZed dZdS) VerifyAndUpdateConfig vllm_configrreturnNcdSN)rs u/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/model_executor/models/config.pyverify_and_update_configz.VerifyAndUpdateConfig.verify_and_update_config model_configrcdSrrrs rverify_and_update_model_configz4VerifyAndUpdateConfig.verify_and_update_model_configrrrrrNrrrN)__name__ __module__ __qualname__ staticmethodrr rrrrrsR\\rrc&eZdZeddZdS)Gemma3TextModelConfigrrrNc.|j}|j |_dSr) hf_configuse_bidirectional_attention is_causal)rr*s rr z4Gemma3TextModelConfig.verify_and_update_model_config s * "+"GG rr"r#r$r%r&r rrrr(r(s8HHH\HHHrr(c&eZdZeddZdS)GteNewModelConfigrrrNc|j}|jjdksJ|jdksJd|_|j|jz}t |d|}||z |jd<||j|jd|_ dS)N NewConfiggelugeglurotary_emb_dimpartial_rotary_factor head_size max_positionrope_parameters r* __class__r# hidden_act hidden_sizenum_attention_headsgetattrr9max_position_embeddings rotary_kwargsrconfighead_dim rotary_dims rr z0GteNewModelConfig.verify_and_update_model_config''(K7777 F****#%)CCV%5x@@ :Dx:O67!":%5  rr"r-rrrr/r/&2   \   rr/c&eZdZeddZdS)$JambaForSequenceClassificationConfigrrrNc4|j}|j d|_dSdSNF) pooler_configuse_activationrrLs rr zCJambaForSequenceClassificationConfig.verify_and_update_model_config;s)$2  ' /+0M ( ( ( 0 /rr"r-rrrrIrI:s2111\111rrIc&eZdZeddZdS)JinaRobertaModelConfigrrrNc|j}|jdkro|jjdksJ|j|jz}|j}|jst|d}t|d|}||z |j d<|||j d|_ dSdS)NrotaryXLMRobertaFlashConfigr4r5r6) r*position_embedding_typer;r#r=r>r@ enforce_eagerr r?r9rA)rrCrDr8rEs rr z5JinaRobertaModelConfig.verify_and_update_model_configCs'  )X 5 5#,0GGGGG)V-GGH!9L  - 9' a88  )98DDJ>H8>SF "#: ;& ,#)#9$$F ! 6 5rr"r-rrrrPrPBs2\rrPc&eZdZeddZdS)LlamaBidirectionalConfigrrrNcddlm}|j}d|_dddd}||jd}|t d|jd ||j_dS) Nr)SequencePoolingTypeFMEANCLSLAST)avgclslastz pool_type z not supported) vllm.config.poolerrZr*r,getpooling ValueErrorrLseq_pooling_type)rrZr*pooling_type_map pooling_types rr z7LlamaBidirectionalConfig.verify_and_update_model_config_s:::::: * # < <  (++I,=tDD  M)*;MMMNN N6B "333rr"r-rrrrXrX^s8CCC\CCCrrXc&eZdZeddZdS)NomicBertModelConfigrrrNc|j}|jjdksJ|jdvsJt |dd|_|jdkrd|_n |j|_|j|jcxkr |j ksnJ|j |_ |j J|j rJ|j |_|j|_|j|_|j|_|j|j_|j|j_|j|jz}t |dd}|||jd |_|jsm|jf|j}t;|j|}|||_|j|kr#t> d ||jdSdS|j!}tE|jtFr!|j$d |j}n|j}tK|d rtM|d ||_'|jd |_tQ|d f|j_)tU|j+}|,dd||_+|||_dS)NNomicBertConfig)swiglur2rUroperlsilumax_trained_positionsir6zNomic context extension is disabled. Changing max_model_len from %s to %s. To enable context extension, see: https://github.com/vllm-project/vllm/tree/main/examples/offline_inference/context_extension.html max_model_lenr9r@max_seq_length)-r*r;r#activation_functionr?rUr< mlp_fc1_bias mlp_fc2_bias qkv_proj_biasbiasrotary_emb_scale_baserotary_emb_interleavedlayer_norm_epsilonlayer_norm_epsn_innerintermediate_sizen_embdr=n_layernum_hidden_layersmodel_arch_configtotal_num_hidden_layersr>r9rA hf_overridesoriginal_max_model_lenrpminget_and_verify_max_lenloggerwarninghf_text_config isinstancedictrbhasattrdelattrr@floatderived_max_model_len_and_keyrencoder_configpop)rrCrDromax_model_len_beforerprrs rr z3NomicBertModelConfig.verify_and_update_model_configts'(,=====)-?????)0 -v* * &  % 1 1 &F   & :F "f&9QQQQV=QQQQQQQ* +3330000 & 9#)> #]#)> 5;5G &2  $ &>%)CC '0G N N"1%5  ); 3; $0#=  :"=E<@  9 KKOQU      ? >EErr!r#r$r%r&rrrrrr(s2\rrc&eZdZeddZdS)MambaModelConfigrrrNcl|j}|j}|jr|jdkr6|jrdnd|_t d|j|j|jdkr(|js!d|_t d|jdkr'|jj s Jd|j r Jdt d|j|j |j |_ d Sd S|jdkr!d|_t d |j |j|_ d Sd S) z Enable FULL_AND_PIECEWISE cuda graph mode by default (required to get good performance for mamba layers in V1). Args: vllm_config: vLLM Config noneallalignzPMamba cache mode is set to '%s' for %s by default when prefix caching is enabledzHybrid or mamba-based model detected without support for prefix caching with Mamba cache 'all' mode: falling back to 'align' mode.z9Chunked prefill is required for mamba cache mode 'align'.zOMamba cache mode 'align' is currently not compatible with speculative decoding.zWarning: Prefix caching in Mamba cache '%s' mode is currently enabled. Its support for Mamba layers is experimental. Please report any issues you may observe.NzAMamba cache mode is set to 'none' when prefix caching is disabled)r cache_configenable_prefix_cachingmamba_cache_modesupports_mamba_prefix_cachingrr architecturescheduler_configenable_chunked_prefillspeculative_configrmamba_block_size block_sizerp)r_rrrs rrz)MambaModelConfig.verify_and_update_config@s#/ "/  -0 K,66)GTEEW-5 1 - -66$B718 -4 ,77"3JOJ'919 KK<-    ,40<0G ---54,6606 -W,40<0J ---54rr!r#r$r% classmethodrrrrrr?s8;K;K;K[;K;K;Krrc&eZdZeddZdS)HybridAttentionMambaModelConfigrrrNc|jj}t||j}|j}|j}|j}|jdkr|j}nt|j}|j rY|j tj k}|rdnd} td||||j} nd} t%jdr6|dkr|j |j tjkrd } t+d||||j} t-j|j| \} } t3| || |d j} | dkrd S|jdkrQ|p|}t=| | }t?|| }|t=||z}||_n| t=| | | zz}|j |j |kr"||_ tB"d||jdkr |j |_|j | z}|| ksJ|| krd S|j# |j#|kr/||_#d|| z z| z }tB"d|d Sd S)a~ Ensure that page size of attention layers is greater than or equal to the mamba layers. If not, automatically set the attention block size to ensure that it is. If the attention page size is strictly greater than the mamba page size, we pad the mamba page size to make them equal. Args: vllm_config: vLLM Config auto@r)r num_kv_headsr7dtypedN r)shapesdtypesrrrzcSetting attention block size to %d tokens to ensure that attention page size is >= mamba page size.rzkPadding mamba page size by %.2f%% to ensure that mamba page size and attention page size are exactly equal.)$rrrrattention_configrparallel_config cache_dtyperr use_mlabackendr CUTLASS_MLArget_num_kv_heads get_head_sizepage_size_bytesris_device_capability_family FLASHINFERr rresolve_model_clsrr!get_mamba_state_shape_from_config!get_mamba_state_dtype_from_configrget_mamba_chunk_sizer rrrrmamba_page_size_padded)r_rrrrrrkv_cache_dtypeuse_cutlass_mlakernel_block_alignment_sizeattn_page_size_1_token model_cls_mamba_page_sizebase_chunk_sizeattn_tokens_per_mamba_state chunk_sizeattn_block_sizeattn_page_sizemamba_padding_pcts rrz8HybridAttentionMambaModelConfig.verify_and_update_configsw'3D11+>>>&7"/ "/ %5  #v - -)/NN5l6NON    (,@,LL 2A*H##b '%5)::?KK&4466$ &&&  # "+- ' >{KK>>{KK      a   F  (E 1 1"/U,2S2S2U2UO*.@V*W*W '_.IJJJ(40KZ+X+XXO,;L ) ):D!r,sc ######444444++++++00000000;;;;;;DDDDDDUUUUUUUUUU433333333 X  HHHHH1HHH     -   (11111+@11128CCCCC4CCC*hhhhh0hhhV/////'<///***** 5***33333+@333,     -Q   ,,,,,,A,,,     !6   (3.=K=K=K=K=K,=K=K=K@]]]]]&;]]]@DDDDD2DDD,GGGGG!6GGG(=*=$=&'8=, = 23K = 7 =*=!"B=4=%&J='(N=-===%&J=0= (!=")#=$/46)===4T"7889r