&`i<ddlZddlZddlZddlmZmZmZmZmZm Z ddl Z ddl m Z ddl mZdZ d dedede eeegeffd e eeefeegeeefffd eeeegefd ef d ZdS)N)AnyCallableDictListOptionalUnion)RequestMetadata)DeploymentHandlehandle method_nameargskwargscombinereturnc*|d}|i}|js|dtj}|sVtj|z tkrt dtjd|V|j}|tdd}t|dr |j j }nt|d r|j }|td |j }g} t|} | D]P} | } tj| d } n#t"$rY;wxYw|}|}t%|r || }t%|r || }t'|tt(fst#d t+|t'|t,s$|i}nt#dt+|dt/j}t3|||}t5j|}| | jj|g|Ri|Rg}| rtj| }|r ||S|S)al Dispatches a method call to all replicas of the given handle. This is useful for dispatching a control plane message such as kv-cache reset or weight update to all replicas of the given handle. NOTE: This API is experimental and may later be promoted to a public API in Ray Serve directly. For now, it is only available in Ray LLM and is intended to enable control plane operations during RL training which is required when orchestrating trianing and inference loops. Args: handle: The DeploymentHandle to dispatch to. method_name: The name of the method to call on the deployment. args: The arguments to pass to the method. Can be a list/tuple of args, or a callable that takes the replica object and returns args. kwargs: The keyword arguments to pass to the method. Can be a dict, or a callable that takes the replica object and returns kwargs. combine: An optional callable that takes the list of results from all replicas and returns an aggregated result. If not provided, returns the list of results. The default combine function is to return the list of results. Returns: The result of the method call to all replicas. If combine is provided, returns the aggregated result. Otherwise, returns the list of results. NT)_run_router_in_separate_loopz:Timed out waiting for deployment replicas to be populated.g?z DeploymentHandle router is None._asyncio_router_request_routerz7Request router not initialized. No replicas accessible.serve) namespacez"args must be a list or tuple, got zkwargs must be a dict, got z dispatch-) request_idinternal_request_id call_method) is_initialized_inittimerunning_replicas_populated%DISPATCH_REPLICA_POPULATION_TIMEOUT_S TimeoutErrorsleep_router RuntimeErrorhasattrrr_replica_id_setlistto_full_id_strray get_actor ValueErrorcallable isinstancetupletypedictuuiduuid4r pickledumpsappendhandle_requestremoteget)r r rrr start_timerouterrequest_router replica_setfuturesreplicasreplica actor_name actor_handle call_args call_kwargsrdummy_rm pickled_rmresultss z/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/llm/_internal/serve/utils/dispatch.pydispatchrHsD | ~  8  $ 777J//11 9;; #&K K KL  3 //11^F ~=>>>Nv())0/? * + +0/TUUU 0KGK  H* * ++--  =wGGGLL    H    D>> &W I F   * &//K)dE]33 US$y//SSTT T+t,, T"  !RtK?P?P!R!RSSS0// "! *#    \(++   .L ' .z UI U U U U U    G#''"" ww Ns4E  EE)NNN)r3rr1typingrrrrrrr)ray.serve._private.commonr ray.serve.handler r strrHrrGrNsH ================ 555555------)+% .2EI48 BB BB XseSj)) *B $sCx.(C5$sCx.+@"AA B B hS {C/0 1 B  BBBBBBrM