.`i dZddlmZddlZddlmZddlmZddlm Z ddl m Z ddl m Z ddlmZer dd lmZdd lmZeeZdd ZddZdS)z Warmup kernels used during model execution. This is useful specifically for JIT'ed kernels as we don't want JIT'ing to happen during model execution. ) TYPE_CHECKINGN) init_logger)deep_gemm_warmup)current_platform)is_deep_gemm_supported)has_flashinfer)GPUModelRunner)Workerworkerr c*tjototjdk}|r0|}|jj}t||tr(tj drt|j d|j j sl|j jrbtfd|j jDr?t d|j ddddddSdSdSdS) NskipZcV |dkS#t$rYdSwxYw)N FLASHINFERF)get_nameNotImplementedErrorbackends |/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/model_executor/warmup/kernel_warmup.py_is_flashinfer_backendz-kernel_warmup.._is_flashinfer_backend.s@ ##%%5 5"   55 s  ((c3BK|]}|D]}|jVdS)Nr).0groupsgrouprs r z kernel_warmup..:s[     # "5= 1 1       z Warming up FlashInfer attention.T) num_tokens skip_eplb is_profileforce_attentioncreate_mixed_batch)envsVLLM_USE_DEEP_GEMMrVLLM_DEEP_GEMM_WARMUP get_modelscheduler_configmax_num_batched_tokensrrrhas_device_capabilityflashinfer_autotune model_runneris_pooling_model attn_groupsallloggerinfo _dummy_run)r do_deep_gemm_warmupmodel max_tokensrs @r kernel_warmupr5ss  1 " $ $ 1  && 0 ,  "",C  +++1,B2FF1F/000    0   +      -9        6777 && # '           rrunnerr returncddlm}tj5|5||jjdddddn #1swxYwYddddS#1swxYwYdS)a Autotune FlashInfer operations. FlashInfer have many implementations for the same operation, autotuning runs benchmarks for each implementation and stores the results. The results are cached transparently and future calls to FlashInfer will use the best implementation. Without autotuning, FlashInfer will rely on heuristics, which may be significantly slower. r)autotuneT)rr N)vllm.utils.flashinferr9torchinference_moder1r'r()r6r9s rr*r*Ls/.....            # :                                     s4 A,#A A,A A,A A,,A03A0)r r )r6r r7N)__doc__typingrr; vllm.envsr# vllm.loggerr+vllm.model_executor.warmup.deep_gemm_warmuprvllm.platformsrvllm.utils.deep_gemmrr:rvllm.v1.worker.gpu_model_runnerr vllm.v1.worker.gpu_workerr __name__r/r5r*rrrHs ! ######HHHHHH++++++7777770000001>>>>>>000000 X  . . . . b      r