&`i+PUdZddlZddlmZmZddlmcmZddlm Z ddl m Z m Z mZmZmZddlmZeeZdZdeded efd Zd ed egeffd ZGd dZiZeeefed<ded efdZdZeej vrej !edSdS)apGeneric registry for LLM serving components using Ray's internal KV store. This module provides a reusable registry mechanism that enables components to be registered in the driver process and accessed across all Ray processes in the cluster, including Ray Serve child processes. Similar to RLlib/Tune's registry but with a fixed global prefix for cross-job access. N)AnyCallable)_internal_kv_del_internal_kv_exists_internal_kv_get_internal_kv_initialized_internal_kv_put) get_logger serve_globalcategorynamereturncdtdzdz|dzdz|dzS)zGenerate a binary key for the KV store. Args: category: The component category (e.g., "kv_connector_backend") name: The component name Returns: The key to use for storing the value sLLMServeRegistry:ascii:/)_SERVE_REGISTRY_PREFIXencode)r r s z/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/llm/_internal/serve/utils/registry.py _make_keyrs`  ' ' 0 0 1   //' " " #    ++g    valuecttr8dvrtdddd\fd}|SfdS)aCreate a loader callable for a value. Handles both direct objects/classes and string paths for lazy loading. Args: value: Either: - A class, object, or callable (returns lambda: value) - A string in format "module_path:class_name" (creates import loader) Returns: A callable that returns the value when called Raises: ValueError: If value is a string but doesn't have the correct format :z"Invalid format for string value: 'z?'. Expected format: 'module_path:class_name' or a class/object.cLtj}t|SN) importlib import_modulegetattr)module class_name module_paths rloaderz_create_loader..loaderKs#,[99F6:.. .rcSr)rsrz _create_loader..Rsur) isinstancestr ValueErrorrsplit)rr$r"r#s` @@r_create_loaderr,3s % e  PUPPP #(,,sA"6"6 Z / / / / / / }}}rcjeZdZdZdefdZdededdfdZdedefd Zdede fd Z deddfd Z d d Z dS)ComponentRegistrya}Generic registry for LLM serving components using Ray's internal KV store. This registry enables components to be registered in the driver process and accessed across all Ray processes in the cluster, including Ray Serve child processes. Similar to RLlib/Tune's registry but with a fixed global prefix for cross-job access. **Usage Pattern:** This registry is designed for a "register once, read many" pattern: - Components are typically registered in the driver process before deployment - Ray Serve replicas read from the KV store during initialization - Once a component is resolved and cached in a process, subsequent `get()` calls return the cached value without checking the KV store for updates Example: # Create a registry for a component category registry = ComponentRegistry("my_component") # Register a component registry.register("my_component", MyComponentClass) # Get a registered component component = registry.get("my_component") # Check if registered if registry.contains("my_component"): ... r c>||_i|_i|_i|_dS)zInitialize a registry for a specific component category. Args: category: The category name (e.g., "kv_connector_backend") N)r _loader_cache_resolved_cache_pending)selfr s r__init__zComponentRegistry.__init__rs& ! ;=/1*, rr rrNc &||rt|jd|dt|}t j|}||j|<tr t|j|}t||dt d|jd|dd S#t$r?}t d|jd|d|d ||j|<Yd }~d Sd }~wwxYw||j|<d S) aRegister a component. Args: name: The name to register under value: The component to register. Can be: - A class, object, or callable (serialized directly) - A string in format "module_path:class_name" (lazy-loaded via import) Raises: ValueError: If the component is already registered. Use unregister() first if you need to change the registration. Examples: # Register a class directly registry.register("MyClass", MyClass) # Register via module path (lazy loading) registry.register("MyClass", "my.module:MyClass")  'zW' is already registered. Use unregister() first if you need to change the registration.T overwritez Registered z ' in KV storezFailed to register z' in KV store: exc_infoN)containsr*r r,pickledumpsr0rrr loggerdebug Exceptionwarningr2)r3r rr$ serializedkeyes rregisterzComponentRegistry.register}sv( ==   =RRDRRR   &&\&)) $*4  $ % % - 1 t44 jDAAAA O4=OODOOOPPPPP 1 1 1S$-SS4SSPQSS!'1 d#######  1#-DM$   s,A B;; D4C??Dc ||jvr |j|S|j|}|tr t |j|}t |}|Dtj|}||j|<t d|jd|dnA#t$r4}t d|jd|d|dYd}~nd}~wwxYw||}||j|<|St|jd|d t|j) atGet a registered component. Args: name: The name of the component Returns: The registered component. If registered with a string path, returns the imported class/object. If registered directly, returns the original value. Raises: ValueError: If the component is not registered NzLoaded r6' from KV storezFailed to load ' from KV store: Tr9z' not found. Registered: )r1r0getrrr rr<loadsr>r?r@rAr*listkeys)r3r r$rCrBrDrs rrIzComponentRegistry.gets 4' ' ''- -#''-- >688>  t44-c22 )#\*55F/5D&t,LL!Q4=!Q!QD!Q!Q!QRRR   QdmQQtQQaQQ!   FHHE).D  &L} = = = = 2 7 7 9 9:: = =   sA*B-- C+7*C&&C+c ||jvrdStrg t|j|}t |S#t $r5}t d|jd|d|dYd}~dSd}~wwxYwdS)zCheck if a component is registered. Args: name: The name to check Returns: True if registered, False otherwise TzFailed to check if r6z' exists in KV store: r9NF)r0rrr rr@r>rAr3r rCrDs rr;zComponentRegistry.containss 4% % %4 # % %   t44*3///   Z$-ZZ4ZZWXZZ!uuuuu  us#? A> *A99A>c ||jvr|j|=||jvr|j|=||jvr|j|=tr t |j|}t |td|jd|ddS#t$r5}t d|jd|d|dYd}~dSd}~wwxYwdS) zUnregister a component. Removes the component from local cache, pending registrations, and KV store. Args: name: The name of the component to unregister z Unregistered r6rGzFailed to unregister rHTr9N) r0r1r2rrr rr>r?r@rArNs r unregisterzComponentRegistry.unregisters4 4% % %"4( 4' ' '$T* 4=  d# $ % %   t44 %%% ST]SSdSSSTTTTT   WDMWWTWWTUWW!   sA B C*C  Cc tr|jsdS|jD]\}} t|j|}t ||dt d|jd|dT#t$r4}t d|jd|d|d Yd}~d}~wwxYw|j dS) zFlush pending registrations to KV store. This is called automatically when Ray initializes via _post_init_hooks. NTr7z!Flushed pending registration for r6'zFailed to flush z': r9) rr2itemsrr r r>r?r@rAclear)r3r rBrCrDs r flush_pendingzComponentRegistry.flush_pendings0 ())   F $ 3 3 5 5   D*  t44 jDAAAA P PPPPP   Dt}DDDDDDt  sA B C*B>>C)rN) __name__ __module__ __qualname____doc__r)r4rrErIboolr;rPrUr&rrr.r.Us8 - - - - -0-S0-0-0-0-0-0-d, , , , , , \ST2st<rr. _registriesc\|tvrt|t|<t|S)zGet or create a registry for a component category. Args: category: The component category name Returns: The ComponentRegistry instance for this category )r[r.)r s r get_registryr]/s,{"" 1( ; ; H x  rcftD]}|dS)zFlush all pending registrations to KV store. This is registered as a Ray post-init hook to ensure registrations made before Ray initialization are available across processes. N)r[valuesrU)registrys r_flush_all_registriesra=s>  &&((!!    !!r)"rYrtypingrrray._private.worker_privateworkerray.cloudpickle cloudpickler<ray.experimental.internal_kvrrrrr -ray.llm._internal.serve.observability.loggingr rVr>rr)bytesrr,r.r[dict__annotations__r]ra_post_init_hooksappendr&rrros $$$$$$$$$ EDDDDD H  (35(#(2s7"3DSSSSSSSSn-/ T#(( )... !3 !#4 ! ! ! !!!! 777 ""#89999987r