.`i1 ddlmZmZddlmZddlmZddlZddlm Z ddl m Z ddl m Z mZddlmZer dd lmZdd lmZe eZGd d eZdS) )ABCabstractmethod)Iterator) TYPE_CHECKINGN) init_logger)AttentionBackend) LoadStoreSpecOffloadingManager)OffloadingHandler) VllmConfig) KVCacheConfigc eZdZdZ d dZedefdZed ee e j fd ee e e fdeee ee eeffd Zd S)OffloadingSpecz Spec for an offloading connector vllm_configr kv_cache_configKVCacheConfig | Nonec4td||_||_|j}|J|j|_|jj|_ t|j d|j |_ |j |j zdksJdS)NzsInitializing OffloadingSpec. This API is experimental and subject to change in the future as we iterate the design. block_sizer) loggerwarningrrkv_transfer_configkv_connector_extra_config extra_config cache_configrgpu_block_sizeintgetoffloaded_block_size)selfrrrs k/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/v1/kv_offload/spec.py__init__zOffloadingSpec.__init__s  H   '.(;!---.H)6A$'   ! !,0C D D% % !(4+>>!CCCCCCreturncdS)z Get an OffloadingManager that will be used by the scheduler-side offloading connector to track offloaded blocks and manage evictions. N)rs r get_managerzOffloadingSpec.get_manager-s  r" kv_caches attn_backendscdS)aM Get offloading handlers along with their respective src and dst types. Args: kv_caches: A dictionary of layer_name -> gpu_kv_cache tensor. attn_backends: A dictionary of layer_name -> AttentionBackend. Yields: Tuples of (src_type, dst_type, offloading_handler). Nr%)rr'r(s r get_handlerszOffloadingSpec.get_handlers6s r"N)rr rr)__name__ __module__ __qualname____doc__r!rr r&dictstrtorchTensortyperrtupler r r*r%r"r rrs**D'D:PDDDD* .   ^  U\)* C&6!778  %]+T--@BSST U    ^   r"r)abcrrcollections.abcrtypingrr1 vllm.loggerrvllm.v1.attention.backendrvllm.v1.kv_offload.abstractr r vllm.v1.kv_offload.worker.workerr vllm.configr vllm.v1.kv_cache_interfacer r+rrr%r"r r>s $#######$$$$$$ ######666666HHHHHHHH>>>>>>9&&&&&&888888 X  1 1 1 1 1 S1 1 1 1 1 r"