&`ijddlZddlZddlmZmZmZmZmZmZm Z m Z ddl m Z ddl mZddlmZddlmZddlmZmZmZmZmZmZmZddlmZdd lmZdd lm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(m)Z)dd l*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0dd l1m,Z2m3Z4m5Z5m6Z7m8Z9m:Z;m0Z<ddl=m>Z>de5de?fdZ@dedefdZAGddeZ3de eBe(fdee ee,e(ffdZCGddZ:de deDe/fde deEe.fde.fdZFdS)N)AnyCallableDictListOptionalSetTupleUnion)FieldDescriptor)Message) cloudpickle)ray_option_utils) BaseModelFieldNonNegativeFloatNonNegativeInt PositiveFloat PositiveInt validator) pickle_dumps)resources_from_ray_options)DEFAULT_CONSTRUCTOR_RETRY_COUNT#DEFAULT_GRACEFUL_SHUTDOWN_TIMEOUT_S%DEFAULT_GRACEFUL_SHUTDOWN_WAIT_LOOP_SDEFAULT_HEALTH_CHECK_PERIOD_SDEFAULT_HEALTH_CHECK_TIMEOUT_SDEFAULT_MAX_ONGOING_REQUESTSMAX_REPLICAS_PER_NODE_MAX_VALUE)DEFAULTDeploymentOptionUpdateType)AggregationFunctionAutoscalingConfigDeploymentMode HTTPOptions ProxyLocationRequestRouterConfig)r"DeploymentConfigDeploymentLanguage EncodingType LoggingConfig ReplicaConfigr&)validate_placement_groupdeployment_languageis_cross_languagecV|tjkr|sdS|tjkr|rdSdS)zGFrom Serve client API's perspective, decide whether pickling is needed.TF)r(PYTHONJAVA)r-r.s m/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/serve/_private/config.py _needs_pickler33s=0777@Q7t  2 7 7 7z"_proto_to_dict..Ps-$$$*+N1%%$$$r4) ListFieldslabelr LABEL_REPEATEDtype TYPE_MESSAGEnamelistr: DESCRIPTORfieldscontaining_oneof default_value)r5datafieldvalues r2r:r:@s D((**%% u ;/8 8 8z_999$$/4$$$UZ  $(;;UZ  Z?7 7 7-e44D  $D  !(33 Jd " " o:::*; %2D  Kr4ceZdZUdZedejZee e d<ee ej Z ee d<edejZee d<edej Zee d <eeejZee d <eeej Zee d <eeejZee d <eeejZee d <edej Zeee d<ee ej Z!e e d<dZ"e#e d<e$j%Z&ee d<edej'Z(ee)e d<edej Z*ee+e d<ee,ejZ-ee d<e.Z/e0e)e d<GddZ1e2d ddZ3e2dddZ4e2dddZ5dZ6d Z7d!Z8e9d"e:fd#Z;e9d$edS)'r'aInternal datastructure wrapping config options for a deployment. Args: num_replicas: The number of processes to start up that handles requests to this deployment. Defaults to 1. max_ongoing_requests: The maximum number of queries that is sent to a replica of this deployment without receiving a response. Defaults to 5. max_queued_requests: Maximum number of requests to this deployment that will be queued at each *caller* (proxy or DeploymentHandle). Once this limit is reached, subsequent requests will raise a BackPressureError (for handles) or return an HTTP 503 status code (for HTTP requests). Defaults to -1 (no limit). user_config: Arguments to pass to the reconfigure method of the deployment. The reconfigure method is called if user_config is not None. Must be JSON-serializable. graceful_shutdown_wait_loop_s: Duration that deployment replicas wait until there is no more work to be done before shutting down. graceful_shutdown_timeout_s: Controller waits for this duration to forcefully kill the replica for shutdown. health_check_period_s: Frequency at which the controller health checks replicas. health_check_timeout_s: Timeout that the controller waits for a response from the replica's health check before marking it unhealthy. autoscaling_config: Autoscaling configuration. logging_config: Configuration for deployment logs. user_configured_option_names: The names of options manually configured by the user. request_router_config: Configuration for deployment request router. max_constructor_retry_count: Maximum number of times to retry the deployment constructor. Defaults to 20. )default update_type num_replicasmax_ongoing_requestsmax_queued_requestsN user_configgraceful_shutdown_timeout_sgraceful_shutdown_wait_loop_shealth_check_period_shealth_check_timeout_sautoscaling_config)default_factoryrOrequest_router_configFr.r-versionlogging_configmax_constructor_retry_countuser_configured_option_namesceZdZdZdZdS)DeploymentConfig.ConfigTN)__name__ __module__ __qualname__validate_assignmentarbitrary_types_allowedr9r4r2Configras""&r4rgT)alwaysct|tr|S|H tj|n2#t$r%}t dt |dd}~wwxYw|S)Nz&user_config is not JSON-serializable: .) isinstancebytesjsondumps TypeError ValueErrorstr)clsr<es r2user_config_json_serializablez.DeploymentConfig.user_config_json_serializables} a   H = U 1  U U U !S#a&&!S!S!STTT Us0 A AAc||St|ts tdt|dddlm}|di|}|S)NzGot invalid type 'z,' for logging_config. Expected a dictionary.r)r*r9)rkdictrorAray.serve.schemar*)rrr<r*s r2logging_config_validz%DeploymentConfig.logging_config_valids 9H!T"" )T!WW)))  322222 M  A   # # % %r4ct|tstd|dkr|dkrtd|S)Nz'max_queued_requests must be an integer.rMrRz@max_queued_requests must be -1 (no limit) or a positive integer.)rkintrorp)rrr<s r2validate_max_queued_requestsz-DeploymentConfig.validate_max_queued_requestssQ!S!! GEFF F q55Q"WWR r4c6t|j|jSN)r3r-r.selfs r2 needs_picklezDeploymentConfig.needs_picklesT5t7MNNNr4c|}|d1|rtj|d|d<|dr5|jjj|ddd<td i|d|d<|dr|dd}|R|s d|dd<nD|rtj||dd<ntd||j j |dd <td i|d|d<|d rHd |d vr)tj|d d |d d <td i|d |d <t!|d |d <t#d i|S)NrTrYpolicy_serialized_policy_defr[request_router_kwargsr4zRNon-empty request_router_kwargs not supportedfor cross-language deployments. Got: _serialized_request_router_clsr]encodingr_r9)rvgetrr rnrYrrAutoscalingConfigProtorpr[rRequestRouterConfigProtoEncodingTypeProtoValueLoggingConfigProtorDDeploymentConfigProto)rrI router_kwargss r2to_protozDeploymentConfig.to_protos`yy{{ 88M " " .  "" M&1&7]8K&L&L]# 88( ) )  '.E % &x 0( *@**+,**D% & 88+ , ,  !89==>UVVM($ MPD012IJJ&&(( $)-8801/%P@MPP*I ( )0 -E--./--D( ) 88$ % % RT"23335F5L)*:666%&z2&8%Q%Q$?O:P%Q%QD! "/3 / 00 0  +,%,,t,,,r4cN|Sr}rSerializeToStringr~s r2to_proto_byteszDeploymentConfig.to_proto_bytes,}}00222r4r5ct|}d|vr|dn tj}d|vr|dnd}t||}d|vr;|ddkr*|rt j|j|d<n|j|d<nd|d<d|vr~d|dvr_|dd}|dkr@|r(t j|jj|dd<n!|jj|dd<n i|dd<tdi|d|d<d|vr|d d s d|dd <|d d s d|dd <|d d s d|dd <|d d s d|dd <|d d s d|dd <|d dstj |dd<tdi|d|d<d|vr|ddkrd|d<d|vrt|d|d<d|vr3d|dvr)tj|dd|dd<|di|S)Nr-r.FrTr4r[rrYupscale_smoothing_factordownscale_smoothing_factorupscaling_factordownscaling_factortarget_ongoing_requestsaggregation_functionr\r_r]rr9)r:r(r0r3r loadsrTr[rr&rr!MEANr"setrName)rrr5rIr-r.rrs r2 from_protozDeploymentConfig.from_proto/se$$%,, & ' '#*  *=)D)DD$ % %% %%8:KLL D M"c))<*5*;E,-112FGG HCG)*+?@,-112KLL MHL)*+DE,-112HII -(,)***;)X)XTBV=W)X)XD% &   I"$$"&Y )T 1 1363444D/ 0 t # #T"23335F5K)*:666%&z2s{{T{{r4 proto_bytescTtj|}||Sr})r FromStringr)rrrr5s r2from_proto_bytesz!DeploymentConfig.from_proto_bytesxs$%0==~~e$$$r4c |}t|}|D]/\}}||vr&t d|d|dt |d0d|D}|D]\}}||||S)zCreates a default DeploymentConfig and overrides it with kwargs. Ignores any kwargs set to DEFAULT.VALUE. Raises: TypeError: when a keyword that's not an argument to the class is passed in. z&Got invalid Deployment config option "z" (with value zO) as keyword argument. All Deployment config options must come from this list: rjc:i|]\}}|tjk||Sr9)rVALUE)r;keyvals r2 z1DeploymentConfig.from_default..s*RRRxsCSGM=Q=Q#s=Q=Q=Qr4)rrvkeysitemsrorD __setattr__)rrkwargsconfigvalid_config_optionsrrs r2 from_defaultzDeploymentConfig.from_default}s"6;;==#5#5#7#788   HC...5S55#&55011555/SR6<<>>RRR  ) )HC   sC ( ( ( ( r4)?rbrcrd__doc__rr LightWeightrPrr__annotations__rNeedsActorReconfigurerQrrSrzrTrrNeedsReconfigurerUrrrVrrWrrrXrYr"r&r[r.boolr(r0r- HeavyWeightr\rqr]rvrr^rr_rrgrrtrxr{rrr classmethodrrrlrrr9r4r2r'r'fsZ!!F.3U9E...L(>*).,.D)))+ %u.:   u"<"RK5:E3.?555!17Internal datastructure wrapping config options for a deployment's replicas. Provides five main properties (see property docstrings for more info): deployment_def: the code, or a reference to the code, that this replica should run. init_args: the deployment_def's init_args. init_kwargs: the deployment_def's init_kwargs. ray_actor_options: the Ray actor options to pass into the replica's actor. resource_dict: contains info on this replica's actor's resource needs. Offers a serialized equivalent (e.g. serialized_deployment_def) for deployment_def, init_args, and init_kwargs. Deserializes these properties when they're first accessed, if they were not passed in directly through create(). Use the classmethod create() to make a ReplicaConfig with the deserialized properties. Note: overwriting or setting any property after the ReplicaConfig has been constructed is currently undefined behavior. The config's fields should not be modified externally after it is created. NTdeployment_def_nameserialized_deployment_defserialized_init_argsserialized_init_kwargsray_actor_optionsplacement_group_bundlesplacement_group_strategymax_replicas_per_noderc ||_||_||_||_d|_d|_d|_||_||_||_ ||_ | t|j|_ | |_dS)zConstruct a ReplicaConfig with serialized properties. All parameters are required. See classmethod create() for defaults. N)rrrr_deployment_def _init_args _init_kwargsrrrr _validater resource_dictr) rrrrrrrrrrs r2__init__zReplicaConfig.__init__s $7 *C&$8!&<# $ "3'>$(@%%:"  88NOO(r4c||||j|jt ddSdS)NzVSetting max_replicas_per_node is not allowed when placement_group_bundles is provided.)_validate_ray_actor_options!_validate_placement_group_options_validate_max_replicas_per_noderrrpr~s r2rzReplicaConfig._validatesq ((*** ..000 ,,...  & 2,87  3 288r4c||_||_||_||_|t |j|_dSr})rrrrrrr)rrrrrs r2rzReplicaConfig.updatesM"3'>$(@%%:" 78NOOr4deployment_def init_args init_kwargsc t|s$t|tstd|+t|tt fstd|$t|t stdtj|r"|rtd|rtdt|ttfs tdt|d|d }|i}|i}|t|tr|}n|j }||t|d t|t|d t|d ||||} || _|| _|| _| S) z4Create a ReplicaConfig from deserialized parameters.z8@serve.deployment must be called on a class or function.Nzinit_args must be a tuple.zinit_kwargs must be a dict.z1init_args not supported for function deployments.z3init_kwargs not supported for function deployments.Got invalid type "zQ" for deployment_def. Expected deployment_def to be a class, function, or string.r9z#Could not serialize the deployment z,Could not serialize the deployment init argsz.Could not serialize the deployment init kwargs)callablerkrqrotuplerDrvinspect isfunctionrprrArbrreprrrr) rrrrrrrrrrrs r2createzReplicaConfig.create#s'' X >30O0O XVWW W!Z E4=%I%I!899 9#z+t'D'D#9:: :  n - - X X !TUUU X !VWWW.8S/:: .T.%9%9...   I  K  $ "   &.#.. >&4##&4&=#  Ld>6J6JLL   $R S S &V W W  # $ !   "0%) r4cTt|jts%tdt |jdhd}|jD]}||vrt d|d|t j|jd|jd d |jd<dSdS) Nrz/" for ray_actor_options. Expected a dictionary.>memorynum_cpusnum_gpus resources runtime_envaccelerator_typez Specifying 'z8' in ray_actor_options is not allowed. Allowed options: T) in_optionsrrM) rkrrvrorArprvalidate_actor_optionsr)rallowed_ray_actor_optionsoptions r2rz)ReplicaConfig._validate_ray_actor_optionsgs$0$77 EI26/3!+)+) +)$)+)$ +) !& +)  +)"*$tCJ/?*@!A+)#+3-+) (}+)+)+)+)+)Z   "EI26/3 PPP"*$tCJ/?*@!AP#+3- P (} PPPP$+/04,0DH26/3-1AAhm,AE#J'Ad38n- A $D> A "*$tCJ/?*@!A A#+3-A (}A&c]AAA[AF333>&3333j$hm 4$$$X$* 8E%*e*;$<=   X  !XeCj1 ! ! !X !  1    [ "3353333[3   33333r4r+proxy_location http_optionsc|d}t}nt|trd|v}tdi|}n]t|tr#d}tdi|d}n%tdt |jd||st j|_ntj ||_|S) aPrepare `HTTPOptions` with a resolved `location` based on `proxy_location` and `http_options`. Precedence: - If `proxy_location` is provided, it overrides any `location` in `http_options`. - Else if `http_options` specifies a `location` explicitly (HTTPOptions(...) or dict with 'location'), keep it. - Else (no `proxy_location` and no explicit `location`) set `location` to `DeploymentMode.EveryNode`. A bare `HTTPOptions()` counts as an explicit default (`HeadOnly`). Args: proxy_location: Optional ProxyLocation (or its string representation). http_options: Optional HTTPOptions instance or dict. If None, a new HTTPOptions() is created. Returns: HTTPOptions: New instance with resolved location. Note: 1. Default ProxyLocation (when unspecified) resolves to DeploymentMode.EveryNode. 2. Default HTTPOptions() location is DeploymentMode.HeadOnly. 3. `HTTPOptions` is used in `imperative` mode (Python API) cluster set-up. `Declarative` mode (CLI / REST) uses `HTTPOptionsSchema`. Raises: ValueError: If http_options is not None, dict, or HTTPOptions. NFlocationTrz#Unexpected type for http_options: ``r9) r$rkrvrprArbr# EveryNoder r%_to_deployment_mode)r r location_set_explicitlys r2prepare_imperative_http_optionsr+s8"'"}} L$ ' '  ", "<"22\22 L+ . . "&"KK\%6%6T%6%J%JKK  P$|2D2D2M P P P   & =$2$r s IIIIIIIIIIIIIIIIIIII666666++++++((((((322222888888IHHHHHHH>===== '9 d    #'#d####Ltttttytttn 4W -4 t->'G!HI4444@k3k3k3k3k3k3k3k3\ 1$]231dK/01111111r4