.`i2QUddlZddlZddlZddlmZddlmZmZddlm Z ddl m Z m Z ddl mZmZmZddlZddlmZddlmZdd lmZdd lmZdd lmZdd lmZmZerdd lm Z ddl!m"Z"ddl#m$Z$ddl%m&Z&m'Z'n e(Z e(Z"e(Z$e(Z&e(Z'ee)Z*ej+ddZ,ej+e-dze.d<de-dzfdZ/e de-dedfdZ0e GddZ1dede2e-e2e-e3fffdZ4e ddde-fdZ5ed Z6ed!e e "Z7ed#e$e$"Z8e d$%Gd&dZ9Gd'd(Z:dS))N)abstractmethod) GeneratorMapping)contextmanager) dataclassfield) TYPE_CHECKINGAnyoverload)TypeVar) init_logger) TokenizerLike)cached_processor_from_config) get_allowed_kwarg_only_overrides)JSONTreejson_map_leavesPretrainedConfig BatchFeatureProcessorMixin) ModelConfigObservabilityConfig_request_id_context)defaultreturnc4tS)z:Get the current request_id from the context, if available.)rgetv/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/multimodal/processing/context.pyget_current_request_idr#/s  " " $ $$r! request_id)NNNc#Kt|} dVt|dS#t|wxYw)z>Context manager to set the request_id for the current context.N)rsetreset)r$tokens r"set_request_idr)4s^ # #J / /E) !!%(((((!!%((((s >AceZdZUdZdZeed< dZeed< dZeed< dZ eed< dZ eed< de e effd Z d S) MultiModalProcessorTimingStatsz>Per-request timing statistics for multimodal processor stages.hf_processor_time hashing_timecache_lookup_timeprompt_update_timepreprocessor_total_timercD|j|j|j|j|jdS)z5Convert stats to a dictionary for JSON serialization.r-r.r/r0r1r3selfs r"to_dictz&MultiModalProcessorTimingStats.to_dictQs1"&!7 -!%!7"&"9'+'C    r!N)__name__ __module__ __qualname____doc__r-float__annotations__r.r/r0r1dictstrr6r r!r"r+r+>sHH"u""">L%@"u"""; ####P%(U(((- c5j)      r!r+ engine_clientc |jjjsiSn#ttf$ricYSwxYwi} |j}|j}t|drF|}|0t|dr |j j }| }n#ttf$rYnwxYwi} t|dr| d}|rt|dkr|D]}|s|D]\} } | |vrt| || <|| dd} | dd} t#| | || d<|| d d} | d d}t#| ||| d <n#ttf$rYnwxYwi}|D]\} }t||| <|D]t\} }| |vr|| |%| d d}|r ||vr|||bt||| <u|S) a5 Get all multimodal timing stats from the engine client. Collects both preprocessing stats (HF processor, hashing, cache lookup, prompt update) and encoder forward pass timing, merged by request_id. Args: engine_client: The engine client (has input_processor and workers). Returns: Dictionary mapping request_id to merged stats dict containing both preprocessing and encoder timing metrics. Example: { 'request-123': { 'hf_processor_time': 0.45, 'hashing_time': 0.02, 'cache_lookup_time': 0.01, 'prompt_update_time': 0.03, 'preprocessor_total_time': 0.51, 'encoder_forward_time': 0.23, 'num_encoder_calls': 1 } } _get_mm_processorNinfocollective_rpcget_encoder_timing_statsrencoder_forward_timer,num_encoder_calls-) vllm_configobservability_configenable_mm_processor_statsAttributeError RuntimeErrorinput_processorinput_preprocessorhasattrrArBctxget_all_timing_statsrClenitemsr=rmaxupdate rpartition)r?preprocessing_statsrMrN mm_processorrP encoder_statsencoder_stats_results worker_statsr$ stats_dict current_timenew_time current_calls new_calls merged_stats prep_dictenc_dictpossible_original_ids r"#get_timing_stats_from_engine_clientre\s[:(=W I  L )   '7,? %': ; ; A-??AAL'GL&,I,I'"'+&)&>&>&@&@# L )     M ="2 3 3 $1$@$@*%% !% -B)C)Ca)G)G$9L'! 2>2D2D2F2F. J%]::8 ) )  # #w . # # # ? * *  $ $ / $ $ %%0%%%%#%% 2  ' '  # #w . # # # 9 $ $   ' )    > ) )  # #w . # # # ? * *  $ $ / $ $ %%0%%%%%%s CBE_T_C)boundr_PT)frozencLeZdZUdZeed< edzed< edddZded< ee dd Z e e e fed < ee jdd Ze jed < d efd Zed efdZedeeeeedfzd efdZ d)deeeeedfzdzd efdZd e e effdZdZeded efdZedeeeeedfzded efdZ d)deeeeedfzdzded efdZdeeded efdZde d e fdZ!ifdddded e"e efde"e efd!e#d"e#d e$e zf d#Z%d$e d e dzfd%Z&d$e d e fd&Z'd e#fd'Z(d e e e e e)fffd(Z*dS)*rfzZ Contains information about the model which may be used to modify the inputs. model_configN tokenizerF)rcomparereprzObservabilityConfig | NonerI)default_factoryr|r}timing_stats_registry_timing_stats_registry_lockrc<|jtd|jS)Nz !N ~r!cdSNr r4s r" get_hf_configz$InputProcessingContext.get_hf_configs47Cr!typ.cdSrr )r5rs r"rz$InputProcessingContext.get_hf_configs Sr!c|ddlm}|}|jj}t ||s"t d|dt ||S)z Get the HuggingFace configuration (`transformers.PretrainedConfig`) of the model, additionally checking its type. Raises: TypeError: If the configuration is not of the specified type. Nrrz3Invalid type of HuggingFace config. Expected type: z, but found type: ) transformers.configuration_utilsrrz hf_config isinstance TypeErrortype)r5rrrs r"rz$InputProcessingContext.get_hf_configs| ; I I I I I I"C%/ )S)) 1"%11#I11  r!c|jjS)zQ Get the HuggingFace image processor configuration of the model. )rzhf_image_processor_configr4s r"get_hf_image_processor_configz4InputProcessingContext.get_hf_image_processor_config9s ::r!c@|jj}|td|S)z Get the multimodal config of the model. Raises: RuntimeError: If the model is not a multimodal model. NzNot a multimodal model)rzmultimodal_configrL)r5 mm_configs r" get_mm_configz$InputProcessingContext.get_mm_config?s*%7  788 8r!kwargsc dSrr r5rs r"get_hf_processorz'InputProcessingContext.get_hf_processorLsGJsr!c dSrr )r5rrs r"rz'InputProcessingContext.get_hf_processorOs Sr!c |ddlm}|}ddlm}|j}t ||r|j}t|jf||d|S)z Get the HuggingFace processor (`transformers.ProcessorMixin`) of the model, additionally checking its type. Raises: TypeError: If the processor is not of the specified type. Nrr)MistralTokenizer) processor_clsr{) transformers.processing_utilsrvllm.tokenizers.mistralrr{rtransformers_tokenizerrrz)r5rrrrr{s r"rz'InputProcessingContext.get_hf_processorWs ; D D D D D D C<<<<<<N i!1 2 2 9!8I+         r!c f|j}|j}|i}i||}|di|S)z Initialize a HuggingFace-like processor class, merging the keyword arguments with those in the model's configuration. Nr )rzget_multimodal_configmm_processor_kwargs)r5rrr base_kwargs merged_kwargss r"init_processorz%InputProcessingContext.init_processorwsP%;;== 3  K1;1&1 s##]###r!outputc>dtffd }t||S)Nxct|tjr4|r |jj}|S)N)dtype)rtorchTensoris_floating_pointtorzr)rr5s r"_postprocess_onezDInputProcessingContext._postprocess_output.._postprocess_onesI!U\** <&&((<4#4#:;;AHr!)objectr)r5rrs` r"_postprocess_outputz*InputProcessingContext._postprocess_outputs;       /888r! num_tries max_triesridatarrc>t|sJ|j}||}t ||dd} |di||ddi} n#t $r} t | trk| ri| jddkrX||krRt d||tj d | ||||d z| cYd } ~ Sd t|jd|d|} t!| | d } ~ wwxYwddlm} t | | r%|| j} | | Stdt|j|| S)z~ Call `hf_processor` on the prompt `data` (text, image, audio...) with configurable options `kwargs`. FT)requires_kw_onlyallow_var_kwargsreturn_tensorsptrzAlready borrowedzBFailed to acquire tokenizer in current thread. Retrying (%d/%d)...g?rrNzFailed to apply z on data=z with kwargs=rz{%s did not return `BatchFeature`. Make sure to match the behaviour of `ProcessorMixin` when implementing custom processors.r )callablerzrmerge_mm_processor_kwargsr ExceptionrrLargsloggerwarningrnsleepcall_hf_processorrr7r%transformers.feature_extraction_utilsrrr warning_once)r5rirrrrrrallowed_kwargsrexcmsgroutput_s r"rz(InputProcessingContext.call_hf_processors8 %%%%%%;;== !;;FCC 9  "!     +!\PPDPNPP4PPPFF + + +3 --  HQK#555 ))*   3-- '!m' .?4 #5#5#>????.<??  S//s *9 +> GFFFFF fl + + )..v{;;G<(( ( .    '    ''///s%A## D-A:D'D--DDr$c|j |jjsdS|j5|j|cdddS#1swxYwYdS)z1 Get timing stats for a request. N)rIrJrrr)r5r$s r"rmz'InputProcessingContext.get_timing_statss  % -,F .4  - > >-11*== > > > > > > > > > > > > > > > > > >sAA Ac|j |jjstS|j5||jvrt d|t}||j|<|cdddS#1swxYwYdS)z Create and store timing stats in the registry for a request. This should be called at the start of processing for a request. The stats object is created immediately and stored in the registry. Nz+Timing stats already exist for request_id: )rIrJr+rrr)r5r$rps r"create_timing_statsz*InputProcessingContext.create_timing_statss  % -,F .233 3  -  T777 N*NN344E5:D &z 2                  s5A++A/2A/c|j |jjsdS|j5t|j}|j|cdddS#1swxYwYdS)zY Clear all stats from the registry. Returns the number of stats cleared. Nr)rIrJrrRrclear)r5counts r"clear_timing_stats_registryz2InputProcessingContext.clear_timing_stats_registrys  % -,F .1  -  233E  & , , . . .                  s/AA Ac|j |jjsiS|j5d|jDcdddS#1swxYwYdS)zI Get all timing stats as a dictionary for API endpoints. Nc>i|]\}}||Sr )r6).0ridrps r" z?InputProcessingContext.get_all_timing_stats..s6CU]]__r!)rIrJrrrSr4s r"rQz+InputProcessingContext.get_all_timing_statss  % -,F .I  -  "&"<"B"B"D"D                  s#A  AAr)+r7r8r9r:rr<rrrIr=rr>r+ threadingLockrrr rrrrutupler rrrrrrwrtrrrrintrrrmrrr;rQr r!r"rfrfs8 )t####49>e%:::64GLue%HHH4%C CDA27%!E333@}7"2777X7  "Xd2hm, , X9= #YtCy#~. . 5 8;tCH~;;;;   JFJ~JJJXJ  "Xd2hm, ,  X9=  #YtCy#~. . 5       @$ "X$ $ $$$$& 9 9  9 9 9 9$(* G0 G0G0G0$G0c6k"G0V $ G0  G0G0  G0G0G0G0R > > '$ . > > > >c6T, S     d3S%Z0@+@&A      r!ceZdZdZdeddffd ZedefdZde fdZ de fdZ d e defd Zedefd Zedeeedzffd Zdeeeffd ZdedeeefdeeefdzfdZxZS)BaseProcessingInfozDBase class to provide the information necessary for data processing.rPrNcVt||_dSr)super__init__rP)r5rP __class__s r"rzBaseProcessingInfo.__init__&s$ r!c$|jjjSr)rPrzmodelr4s r"model_idzBaseProcessingInfo.model_id+sx$**r!c4|jSr)rPrr4s r"rz BaseProcessingInfo.get_tokenizer/x%%'''r!c4|jSr)rPrr4s r"rz BaseProcessingInfo.get_hf_config2rr!rc &|jjdi|S)zy Subclasses can override this method to handle specific kwargs from model config or user inputs. r )rPrrs r"rz#BaseProcessingInfo.get_hf_processor5s )tx(226222r!cdS)NFr r4s r"skip_prompt_length_checkz+BaseProcessingInfo.skip_prompt_length_check<sur!ct)z Return the maximum supported number of items for each modality. A value of `None` means unlimited number of items. Omitting a modality from the returned dictionary means that it is not supported at all. )NotImplementedErrorr4s r"get_supported_mm_limitsz*BaseProcessingInfo.get_supported_mm_limits@s "!r!c,|}|j}ttt f}|D]1\}}||}||nt||||<2|S)z=Return the maximum allowed number of items for each modality.) rrPrr=r>rrSget_limit_per_promptmin)r5supported_mm_limitsrallowed_limitsmodalitysupported_limit user_limits r"get_allowed_mm_limitsz(BaseProcessingInfo.get_allowed_mm_limitsLs"::<<H**,, c3h)))<)B)B)D)D   %Ho"77AAJ#* _55 8 $ $ r!seq_len mm_countscdS)a Return the maximum number of tokens per item of for each modality. When `None` (the default) is returned, vLLM will generate dummy inputs (images/videos) at maximum possible sizes and process them to determine the maximum token count per modality. This approach works but can be very slow for certain models (e.g., Qwen2.5-VL), leading to very long startup time. For better performance, each model can override this method to return pre-computed maximum token counts, avoiding the need for dummy input generation and processing. Note: The maximum number of tokens per item of each modality returned from this function should respect the model's maximum sequence length and the maximum number of items of each modality allowed, and agree with dummy inputs (images/videos) at maximum possible sizes. Nr )r5rrs r"get_mm_max_tokens_per_itemz-BaseProcessingInfo.get_mm_max_tokens_per_item]s 0tr!)r7r8r9r:rfrpropertyr>rrrrrrrrboolrrrrrrr __classcell__)rs@r"rr#sNN2t +#+++X+(}(((((/((((33N3333$X "cDj)A " " "^ "wsCx'8"38$ c T ! r!r); contextvarsrrnabcrcollections.abcrr contextlibr dataclassesrrtypingr r r rtyping_extensionsr vllm.loggerr vllm.tokenizersr!vllm.transformers_utils.processorrvllm.utils.func_utilsrvllm.utils.jsontreerrrrrrrr vllm.configrrrr7r ContextVarrr>r<r#r)r+r=r;rersrtrurwrfrr r!r"r s ........%%%%%%((((((((  %%%%%%######))))))JJJJJJBBBBBB99999999 !AAAAAABBBBBB<<<<<<<<<<<<<<<LNK  X  ;Q+:P4;;;[+C$J7 %d %%%% )s)y1A'B))))          :ee #tCJ  eeeeP"1&>"1C"1"1"1"1J WT]] WT)3CDDD WT@@@ $ppppppppf RRRRRRRRRRr!