&`ilddlZddlZddlZddlZddlZddlZddlZddlmZm Z ddl m Z m Z m Z mZmZmZmZddlZddlZddlZddlmZddlmZddlZddlmZddlmZddlmZm Z m!Z!m"Z"m#Z#m$Z$dd l%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4dd l5m6Z6dd l7m8Z8m9Z9m:Z:dd l;mZ>m?Z?m@Z@mAZAmBZBdd lCmDZDmEZEmFZFmGZGddlHmIZImJZJddlKmLZLmMZMmNZNmOZOmPZPmQZQmRZRddlSmTZTddlUmVZVddlWmXZXddlYmZZZm[Z[m\Z\ddl]m^Z^m_Z_ddl`maZambZbddlcmdZdddlemfZfmgZgddlhmiZiejje2ZkejlmdddkZnejlmdekoddZpdZqdZrGd d!eZsGd"d#esZtGd$d%esZuGd&d'eZvejwd(Gd)d*evZxd+ZydS),N)ABCabstractmethod)AnyCallableDict GeneratorOptionalSetTuple)version)Receive)CoreContextFilter)get_or_create_event_loop) DeploymentID EndpointInfoNodeId ReplicaIDRequestMetadataRequestProtocol)HEALTHY_MESSAGEPROXY_MIN_DRAINING_PERIOD_S'RAY_SERVE_ENABLE_PROXY_GC_OPTIMIZATIONSRAY_SERVE_PROXY_GC_THRESHOLD&RAY_SERVE_REQUEST_PATH_LOG_BUFFER_SIZEREQUEST_LATENCY_BUCKETS_MSSERVE_CONTROLLER_NAMESERVE_HTTP_REQUEST_ID_HEADERSERVE_LOG_COMPONENTSERVE_LOG_COMPONENT_IDSERVE_LOG_REQUEST_IDSERVE_LOG_ROUTESERVE_LOGGER_NAMESERVE_MULTIPLEXED_MODEL_IDSERVE_NAMESPACE)get_proxy_handle)get_grpc_response_statusset_grpc_code_and_detailsstart_grpc_server) MessageQueueconfigure_http_middlewaresconvert_object_to_asgi_messagesget_http_response_statusreceive_http_bodysend_http_response_on_exceptionstart_asgi_http_server)access_log_msgconfigure_component_logger#configure_component_memory_profilerget_component_logger_file_path)LongPollClientLongPollNamespace)ASGIProxyRequestHandlerMetadata ProxyRequestResponseGeneratorResponseHandlerInfoResponseStatusgRPCProxyRequest)ProxyResponseGenerator) ProxyRouter) ServeUsageTag)generate_request_idget_head_node_idis_grpc_enabled) HTTPOptions gRPCOptions)HealthzResponseListApplicationsResponse)DeploymentHandle) EncodingType LoggingConfig)metricsSERVE_SOCKET_REUSE_PORT_ENABLED1"SERVE_REQUEST_PROCESSING_TIMEOUT_Sa*The `SERVE_REQUEST_PROCESSING_TIMEOUT_S` environment variable has been deprecated. Please set `request_timeout_s` in your Serve config's `http_options` or `grpc_options` field instead. `SERVE_REQUEST_PROCESSING_TIMEOUT_S` will be ignored in future versions. See: https://docs.ray.io/en/releases-2.5.1/serve/api/doc/ray.serve.schema.HTTPOptionsSchema.html#ray.serve.schema.HTTPOptionsSchema.request_timeout_s and https://docs.ray.io/en/latest/serve/api/doc/ray.serve.config.gRPCOptions.request_timeout_s.html#ray.serve.config.gRPCOptions.request_timeout_sg?zThis node is being drained.ceZdZdZ d$dededededee de ee ff d Z e ed efd Zd efd Zd ZdefdZeded efdZededed efdZededed efdZdZdZded efdZded efdZded efdZedededededed e eeff dZ!e d%d!edededed"ed ef d#Z"dS)& GenericProxyaThis class is served as the base class for different types of proxies. It contains all the common setup and methods required for running a proxy. The proxy subclass need to implement the following methods: - `protocol()` - `not_found_response()` - `routes_response()` - `health_response()` - `setup_request_context_and_handle()` - `send_request_to_replica()` Nnode_idnode_ip_addressis_head proxy_routerrequest_timeout_saccess_log_contextc.||_|j|jdkrd|_||_||_||_t jd|jdd|jdd|_t jd|jdd |jd d |_ t jd |jdd |jd d|_ t dtt jd|jdd|jd|jdtd|_t jd|jdd|jdd||d|_d|_d|_|pi|_t-t.|jdddS)Nr serve_num_ _requestszThe number of z requests processed.routemethod application status_code) descriptiontag_keys_error_requestszThe number of errored z responses.)r[ error_coder\r]serve_num_deployment_z' responses returned by each deployment.) deploymentrbr\r[r]zREQUEST_LATENCY_BUCKET_MS: serve__request_latency_mszThe end-to-end latency of z# requests (measured from the Serve z proxy).)r\r[r]r^)r_ boundariesr`serve_num_ongoing_z'The number of ongoing requests in this z proxy.)rQrR)namer_r` _PROXY_USEDrL)rU_node_id_is_headrTrJCounterprotocollowerrequest_counterrequest_error_counter deployment_request_error_counterloggerdebugr Histogramprocessing_latency_trackerGaugeset_default_tagsnum_ongoing_requests_gauge_ongoing_requests_draining_start_time_access_log_contextgetattrr?upperrecord)selfrQrRrSrTrUrVs l/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/serve/_private/proxy.py__init__zGenericProxy.__init__s"3  ! -$2H12L2L%)D "  (& 9,,.. 9 9 9LLLLF    &-_ ?,,.. ? ? ?KKKK & & & "18 JDM$7$7$9$9 J J J9999 1 1 1 -  O3MOOPPP*1*; ?T]((** ? ? ?DT]DD,0MDDD2 + + + '+2-Fdm&9&9&;&;FFF$-3 + + +  "#2    '"#6:!#5#;  $-"5"5"7"7DDDEELLSQQQQQreturnct)zvProtocol used in the proxy. Each proxy needs to implement its own logic for setting up the protocol. NotImplementedErrorrs rrnzGenericProxy.protocols "!rc|jduS)z8Whether is proxy actor is in the draining status or not.N)r{rs r _is_drainingzGenericProxy._is_drainings(44rc|sdS|j o#tj|jz tkS)zCheck whether the proxy actor is drained or not. A proxy actor is drained if it has no ongoing requests AND it has been draining for more than `PROXY_MIN_DRAINING_PERIOD_S` seconds. F)rrztimer{rrs r is_drainedzGenericProxy.is_drainedsG  "" 5** Y[[44 48S S rdrainingc@|rS|s?td|jdddit j|_|sD|r2td|jdddid|_dSdSdS)zUpdate the draining status of the proxy. This is called by the proxy state manager to drain or un-drain the proxy actor. z'Start to drain the proxy actor on node . log_to_stderrFextraz&Stop draining the proxy actor on node N)rrsinforkrr{)rrs rupdate_drainingzGenericProxy.update_drainings  4T..00 4 KKJ$-JJJ&.    )- D % -d//11 - KKIIII&.    )-D % % %  - - - -r proxy_requestcKtNr)rrs rnot_found_responsezGenericProxy.not_found_response"!rhealthymessagecKtrrrrrs rroutes_responsezGenericProxy.routes_response rrcKtrrrs rhealth_responsezGenericProxy.health_responserrcd|xjdz c_|j|jdS)aqOngoing requests start. The current autoscale logic can downscale nodes with ongoing requests if the node doesn't have replicas and has no primary copies of objects in the object store. The counter and the dummy object reference will help to keep the node alive while draining requests, so they are not dropped unintentionally. Nrzrysetrs r_ongoing_requests_startz$GenericProxy._ongoing_requests_starts8 !# '++D,BCCCCCrcd|xjdzc_|j|jdS)zOngoing requests end. Decrement the ongoing request counter and drop the dummy object reference signaling that the node can be downscaled safely. rNrrs r_ongoing_requests_endz"GenericProxy._ongoing_requests_end$s8 !# '++D,BCCCCCrcl|j|j\}}|r d}t}n|sd}|}n d}t }|jr|||}n |jsJ| ||}t|t|j ddS)zGet the response handler for system health and route endpoints. If the proxy is draining or has not yet received a route table update from the controller, both will return a non-OK status. FT)rrr[response_generatormetadatashould_record_access_log!should_increment_ongoing_requests) rTready_for_trafficrlrDRAINING_MESSAGEris_health_requestris_route_requestrr:r7 route_path)rrrouter_ready_for_traffic router_msgrrrs r_get_health_or_routes_reponsez*GenericProxy._get_health_or_routes_reponse-s04/@/R/R M0 0 , *      &G&GG) &G GGG%G  * X!%!5!5gw!5!W!W   1 1 11!%!5!5gw!5!W!W "1$#.&+.3     rc|js|jr||Sd}|jtjkr |j|j}n4|jtj kr|j |j}|4t| |tdddS|\}}}|j}|dkr|jtjkr|drJ||j|zt#jt&jt#jdkr*|||dd|jtjkr!|j||j}n |jj}t7}||jj|||| \}} || |||| } t| t|jj|jj| ddS) NrTFr/z0.33.0r)app_namehandler[rinternal_request_id) request_idrrrapp_is_cross_language)application_namedeployment_namer[)rrrrnrHTTPrT match_routerGRPCget_handle_for_endpointr:rr7endswith set_root_path root_pathr parse starlette __version__set_pathreplacematch_route_patternscope deployment_idrr@ setup_request_context_and_handlesend_request_to_replicari) rr matched_route route_prefixrrrlogs_and_metrics_routerrrs r_get_response_handler_infoz'GenericProxy._get_response_handler_infoQs  * Em.L E55mDD D =O0 0 0 -99-:RSSMM ]o2 2 2 -EE(M  &#'#:#:=#I#I(  *.27    ;H 7L&"7'1Js""t}8L'L'L'0055555++M,Cl,RSSS=!677'-:Q:QQQ!**:+=+=lBPQ+R+RSSS} 444)-):)N)N -"5**&&*0)=)F&"5"7"7 !%!F!F-6,+$7 "G"" FJ"&!=!=%$7+&; ">"" '#5(%+%9%B$*$8$=0 *.26    rc K|jdvsJ||}tj}|jr| d}|j23d{V}t |tr|}|WV$6|t |tsJ |jr|n!#|jr|wwxYwtj|z dz}|j rtj j }|j|jt <|j|jt$<t&t+|j|jt/|j||j|j|jj|j|jjt/|jd|j||jj|j|jjt/|jd|jr|j |jj|j|jjt/|jd|j!|jj|j|jjt/|j|jj"d dSdS) zWrapper for proxy request. This method is served as common entry point by the proxy. It handles the routing, including routes and health checks, ongoing request counter, and metrics. >grpchttp websocketNg@@)r\r[status latency_msrrZ)tags)r[r\r]rb)r[r\r]rbrd)# request_typerrrrr isinstancer;rrrayservecontext_get_serve_request_contextr[r|r!rr rsrr0r\strcoderpincrrrvobserveis_errorrqrrr)rrresponse_handler_info start_timerrrrequest_contexts rrzGenericProxy.proxy_requests;)-JJJJJ $ ? ? N NY[[ B +  ( ( * * * -/3F!6!I       gg~66%$F "J %*V^*L*L%%L%%F -**,,,%F -**,,,, -ikkJ.&8 9 !i/JJLLO8G8MD $_ 5=L=WD $%9 : KK(/)/v{++)  .       .7='.4=N"6;//  !    '// .7='.4=N"6;//  0    ?   & * *2;A+2#8#A#R"%fk"2"2  +     1 5 52;A+2#8#A#R"%fk"2"2"7"@"P  6       s B6A?!8B66Crrr[rct)zSetup the request context and handle for the request. Each proxy needs to implement its own logic for setting up the request context and handle. r)rrrr[rrs rrz-GenericProxy.setup_request_context_and_handles "!rFrrcKt)zSend the request to the replica and handle streaming response. Each proxy needs to implement its own logic for sending the request and handling the streaming response. r)rrrrrrs rrz$GenericProxy.send_request_to_replicas"!rNNF)#__name__ __module__ __qualname____doc__rrboolr>r floatrrrpropertyrrrnrrrr8r9rrrrrr:rrrrGr rrrrrPrPss  $.2-1VRVRVRVR VR " VR $E? VR!cNVRVRVRVRp"/"""^X"5d5555    -----(")" """^" "")," """^" "")," """^" D D DDDD" )" " " " " HN)N NNNN`QQBSQQQQf " "! " " $ " ! " $ % " " "^ "', " " "! "! " $ " $ "  " " "^ " " "rrPceZdZdZedefdZdedefdZ de de defdZ de defd Z d e d e defd Zd e dede dede deee ff dZ dde de dedede def dZdS) gRPCProxya This class is meant to be instantiated and run by an gRPC server. This is the servicer class for the gRPC server. It implements `unary_unary` as the entry point for unary gRPC request and `unary_stream` as the entry point for streaming gRPC request. rctjSr)rrrs rrnzgRPCProxy.protocol ##rrcK|jsd}n d|jd}|d}ttjj|dWVdS)NzApplication metadata not set.z Application 'z ' not found.zP Ping /ray.serve.RayServeAPIService/ListApplications for available applications.Trrr)rr;r StatusCode NOT_FOUND)rrapplication_messagenot_found_messages rrzgRPCProxy.not_found_response!s% W"A  "V-2H"V"V"V " Y Y Y  *%         rrrcKtd|jjDWVt |rt jjnt jj|| WVdS)Ncg|] }|j Sr)r).0endpoints r z-gRPCProxy.routes_response..7s(&.!r)application_namesr) rFrT endpointsSerializeToStringr;rrOK UNAVAILABLErs rrzgRPCProxy.routes_response3s'262C2M             '.O##DO4O [         rcKt|WVt|rtjjntjj|| WVdS)Nrr)rEr r;rrr r rs rrzgRPCProxy.health_responseBs{g...@@BBBBBB'.O##DO4O [         rservice_methodstreamc dtdtjjjdt ffd }dtdtjjjdt t ddfffd }|r|n|S)N request_protorrcKt||d}d}d}|23d{V}t|tr|}|}"6t |||S)a,Entry point of the gRPC proxy unary request. This method is called by the gRPC server when a unary request is received. It wraps the request in a ProxyRequest object and calls proxy_request. The return value is serialized user defined protobuf bytes. FrrrrNrr<rrr;r')rrrrresponserrrs r unary_unaryz6gRPCProxy.service_handler_factory..unary_unaryKs-+- MFH!%!3!3-!3!P!P ' ' ' ' ' ' 'gg~66'$FF&HH "Q &gv 6 6 6OsAcKt||d}d}|23d{V}t|tr|}|WV%6t ||dS)aOEntry point of the gRPC proxy streaming request. This method is called by the gRPC server when a streaming request is received. It wraps the request in a ProxyRequest object and calls proxy_request. The return value is a generator of serialized user defined protobuf bytes. TrNrr)rrrrrrrs r unary_streamz7gRPCProxy.service_handler_factory..unary_streamgs-+- MF!%!3!3-!3!P!P " " " " " " "gg~66"$FF!MMMMM "Q &gv 6 6 6 6 6sA)rr_cythoncygrpc_ServicerContextbytesr)rrrrrs`` rservice_handler_factoryz!gRPCProxy.service_handler_factoryJs  )-)<)M         8 7 7)-)<)M 7 udD( ) 7 7 7 7 7 7 74 &6||;6rrrr[rcd|j}|j}|st}||_||j||j}||||||jd}tjj j tjj j di|| |||fS)zSetup request context and handle for the request. Unpack gRPC request metadata and extract info to set up request context and handle. )rmultiplexed_model_id method_name)r[r_internal_request_idrr  grpc_context)rr)r rr@optionsrr!ray_serve_grpc_contextrrr_serve_request_contextr_RequestContextsend_request_id) rrrr[rrr rrequest_context_infos rrz*gRPCProxy.setup_request_context_and_handles -A"-  2,..J'1M $ '!5%1   $$7 $8)@    044 I  - E E0D E E    %%%<<<z!!rFrrcKt|||j} |23d{V\}}||j|WV*6t tjj } n-#t$r } t| |j|} Yd} ~ nd} ~ wwxYw| J| WVdS)N) timeout_s)r) r=remoteserialized_replica_argrU_set_on_grpc_contextrr;rrr  BaseExceptionr&) rrrrrrrrresultres rrz!gRPCProxy.send_request_to_replicas4 MM->>@@ A A,     U);       ogv,,]-BCCC *<$);<<> $ $ $ $ $ $ $'g~66 $d7mm#######?>>sArNrKcK |d{V}||d{V|ddkr |dS|ddkr|d|Sm#|wxYw)aYProxies the `receive` interface, placing its messages into the queue. Once a disconnect message is received, the call exits and `receive` is no longer called. For HTTP messages, `None` is always returned. For websocket messages, the disconnect code is returned if a disconnect code is received. TNtypezhttp.disconnectwebsocket.disconnectr)close)rrNrKmsgs rproxy_asgi_receivezHTTPProxy.proxy_asgi_receive/s  '#GIIooooooeCjj       v;"333 KKMMMMM v;"888v; KKMMMM ' KKMMMMs.A2 A21A22Brrr[rc|||dd}|jD]\}}|tkr/|} || }| |d<|tkr||d<t jjj t jjj di|||dfS)zSetup request context and handle for the request. Unpack HTTP request headers and extract info to set up request context and handle. T)r[rr"is_http_request)r r rr) headersdecoder#r$rrrrr&rr') rrrr[rrr)keyvaluer s rrz*HTTPProxy.setup_request_context_and_handleJs $7#    (/ D DJCzz||999',||~~$=QRR?S$%;<zz||;;;5:\\^^$\2 044 I  - E E0D E E   +L999rcK|jd}t|j|j|jd{V}|r/|ddd}n|}|S)zIConvert an HTTP request to the Java-accepted format (single byte string). query_stringN=r)rrGr-rNrOrZsplit)rrr^http_body_bytesargs r_format_handle_arg_for_javaz%HTTPProxy._format_handle_arg_for_javais %*..~>> 1  !6 8J! !         +%%''--c155a8CC!((**C rFrrcK|r#||d{V}t}n'||j}tj}t }||j|<t | |j |} t| ||j| |} d} d} d} | 23d{V}|D]=}|ddkrPt|d}t!||d } |d d} n|dd kr| n|dd kr-|d ds| s| n}|ddkr+|dds| nF|ddvr,>. ,. ,. ,. ,. ,. ,. ,(%7+,+,L#F+/DDD'*,x*@&A&A !/!,%0%;%;J%G%G""".:-=-=j%-P-P**%f-1CCC+GGIIII$V,0DDD , 0 0e D DE 2E +GGIIII%f-1III ,//GGN.KKMMM%f-2'*,v*>&?&? !/!,&18H%H """ +GGIII&&&&&'+$$W+,-?,>^ # # #-a1GTTF ?(!! # # #""""" # # # # # #-2 )*//11 5'..000004- ~-"< "K"K0 + !8!?!?!A!ABB!%FF,#!%F ()<==3-2 )*//11 5'..000004- ~-"< "K"K0 + !8!?!?!A!ABB!%FF,#!%F ()<= = = = =!!! s>H!HEH! K-! I#+.IK-I##K--BM5rr)#rrrrrrrr>r rrrrrrrnr8r9rrrrrLrPr r)intrVrGr rrrcr __classcell__r9s@rr3r3sPP.2-1CCCC C " C  C$E?C!cNCCCCCC*$/$$$X$ >) >  > > > >  ),     :02    ),     + /+ ++++ $ $ $'3 #6::!: : $ : ! : $ %::::># ,', BBB!B! B $ B $ B BBBBBBBBrr3c eZdZdZeddedededefdZ e defd Z e dd e dd fd Z e dde deedd fdZe ddeede fdZe de fdZe defdZe dedefdZe defdZe deefdZe dedefdZdefdZd S)ProxyActorInterfacezAbstract interface for proxy actors in Ray Serve. This interface defines the contract that all proxy actor implementations must follow, allowing for different proxy backends (Ray HTTP/gRPC proxies, HAProxy, etc.). )log_buffer_sizerQrRlogging_configrch||_||_||_||_||dS)a Initialize the proxy actor. Args: node_id: ID of the node this proxy is running on node_ip_address: IP address of the node logging_config: Logging configuration log_buffer_size: Size of the log buffer N)rk_node_ip_address_logging_config_log_buffer_size_update_logging_config)rrQrRrrs rrzProxyActorInterface.__init__s=   /- / ##N33333rrc KdS)zBlocks until the proxy is ready to serve requests. Returns: JSON-serialized metadata containing proxy information (worker ID, log file path, etc.) Nrrs rreadyzProxyActorInterface.ready  rTwait_for_applications_runningNc KdS)zWait for the proxy to be ready to serve requests. Args: wait_for_applications_running: Whether to wait for the applications to be running Returns: None Nrrrs rservingzProxyActorInterface.serving%  rr_afterc KdS)zUpdate the draining status of the proxy. Args: draining: Whether the proxy should be draining _after: Optional ObjectRef for scheduling dependency Nrrrrs rrz#ProxyActorInterface.update_draining1rrc KdS)zCheck whether the proxy is drained. Args: _after: Optional ObjectRef for scheduling dependency Returns: True if the proxy is drained, False otherwise Nrrrs rrzProxyActorInterface.is_drained=rrc KdS)zsCheck the health of the proxy. Returns: True if the proxy is healthy, False otherwise Nrrs r check_healthz ProxyActorInterface.check_healthIrrcdS)zWRespond to ping from replicas. Returns: A response string Nrrs rpongzProxyActorInterface.pongRs  rrEc KdS)zHandle ASGI messages for HTTP requests. Args: request_metadata: Metadata about the request Returns: Serialized ASGI messages NrrrEs rrLz)ProxyActorInterface.receive_asgi_messages[rrcdS)z#Get HTTP options used by the proxy.Nrrs r_get_http_optionsz%ProxyActorInterface._get_http_optionsh  rcdS)z8Get the file path for the logger (for testing purposes).Nrrs r_get_logging_configz'ProxyActorInterface._get_logging_configmrrr[cdS)z'Get replicas for a route (for testing).Nr)rr[s r"_dump_ingress_replicas_for_testingz6ProxyActorInterface._dump_ingress_replicas_for_testingrrrc@td|j||jdS)Nproxy)component_name component_idr buffer_size)r1rr)rrs rrz*ProxyActorInterface._update_logging_configws5"".)-       rTr)rrrrrrrrIr}rrrrrr rrrrrrrrLrCrrr rrrrrrrsW F 4444 4 & 4  4444. S   ^    4  4    ^  6:     &.sm      ^     x}      ^   D   ^  c   ^   O  PU    ^   ;   ^  Xc]   ^      ^  ]      rr)num_cpusceZdZdddedededededee f fd Z d e e e ffd Zd efd Zded eefdZd efdZdded dfdZddedeefdZddeefdZd efdZdZded efdZd efdZxZ S) ProxyActorN)long_poll_client http_options grpc_optionsrQrRrrc Zt|||||_t||_t |j}t }|pQttj tttj |jtj|ji||_d|jd|jj} |r| d|jjdz } n| dz } t(| t(dtjd |t3d | |jt6jkrXt;j} t:jD]} | | di| tBd tD|j#d d ddddi} nd ddd} |jtIk} tKtL|_'tQ|j|j#| tj)|j'|jj*| |_+|r.tY|j|j#| |j'|jj*| nd|_-|.t_|j+|j|t`|_1d|_2d|_3|r?|.ti|j-j5|j|t`|_3d|_6todS)N)rQrRr) namespace)call_in_event_loopzProxy starting on node z (HTTP port: z , gRPC port: z).zConfigure Proxy actor z logger with logging config: r)rrrFskip_context_filterTserve_access_log)rrr)rQrRrSr4rTrUrV)rQrRrSrTrUrV) event_loopenable_so_reuseport)8r6r _grpc_optionsr* _http_optionsrBrr4r get_actorrr$r5GLOBAL_LOGGING_CONFIGr ROUTE_TABLE_update_routes_in_proxiesrrkportrsrrtget_runtime_context get_actor_idr2encodingrHJSONrget_ray_core_logging_contextTASK_LEVEL_LOG_KEYSpoprrrrAr>r%rTr3get_actor_namerU http_proxyr grpc_proxyrqr/SOCKET_REUSE_PORT_ENABLED_start_http_server_task_running_http_server_task_start_grpc_server_taskr(r_running_grpc_server_task_configure_gc_options)rrrrQrRrr grpc_enabledr startup_msgray_core_logging_contextr[rVrSr9s rrzProxyActor.__init__s +)    *7 EE&t'9:: -// 0! N M/? K K K!79T!-t/M  * 5 5 5 f eeDL^Lcee  F4+=+BFFF FKK 4 K K     %>%K%K%M%M < <+9 < <   ,"      "l&7 7 7 (9'U'W'W $ )< 8 8(,,S$7777"*"#W&(=%t"D ""  "''+$(""  -#3#5#55'(899#M 1355DDFF*"0B1   $ I $ 5!."&"4"F#5      (2'='= ""%$=    ( ( $BF&@D$  +5+A+A!O;&)(A ,,D (BF&rrc:|j|dSr)rT update_routes)rrs rrz$ProxyActor._update_routes_in_proxiess '' 22222rrc~d}tjD]-}t|tjjr |jj}.|S)z5Get the logging configuration (for testing purposes).N)rshandlersrlogging MemoryHandlertarget baseFilename)r log_file_pathhandlers rrzProxyActor._get_logging_configsA  < AA!! B +BB Trc KdS)z1Wait for the proxy to be ready to serve requests.Nrrs rrzProxyActor.serving.s rrrcK|j||jr|j|dSdS)zUpdate the draining status of the HTTP and gRPC proxies. Unused `_after` argument is for scheduling: passing an ObjectRef allows delaying this call until after the `_after` call has returned. N)rrrrs rrzProxyActor.update_draining2sN ''111 ? 6 O + +H 5 5 5 5 5 6 6rc|K|jo!|jdup|jS)zCheck whether both HTTP and gRPC proxies are drained or not. Unused `_after` argument is for scheduling: passing an ObjectRef allows delaying this call until after the `_after` call has returned. N)rrrrs rrzProxyActor.is_drained=s@))++ Ot # Ct'A'A'C'C rcFKtdddidS)zwNo-op method to check on the health of the HTTP Proxy. Make sure the async event loop is not blocked. zReceived health check.rFrT)rsrtrs rrzProxyActor.check_healthHs)  -ou5M NNNtrcdS)z2| | | | | | |34 l8R3S3333UNNINNNN S    >44 6 6d 6HSM 6 6 6 6   x}     D     O  PU     ";""""""""rrctsdStjdtjtjt dS)N)rgccollectfreeze set_thresholdrrrrrrfsC 2JqMMMIKKK122222r)zasynciorrrosrorabcrrtypingrrrrr r r rrstarlette.routing packagingr starlette.typesr rray._common.filtersrray._common.utilsrray.serve._private.commonrrrrrrray.serve._private.constantsrrrrrrrrrrr r!r"r#r$ray.serve._private.default_implr%ray.serve._private.grpc_utilr&r'r(ray.serve._private.http_utilr)r*r+r,r-r.r/ ray.serve._private.logging_utilsr0r1r2r3ray.serve._private.long_pollr4r5)ray.serve._private.proxy_request_responser6r7r8r9r:r;r<+ray.serve._private.proxy_response_generatorr=ray.serve._private.proxy_routerr>ray.serve._private.usager?ray.serve._private.utilsr@rArBray.serve.configrCrDray.serve.generated.serve_pb2rErFray.serve.handlerGray.serve.schemarHrIray.utilrJ getLoggerrsenvironrGrwarningINITIAL_BACKOFF_PERIOD_SECMAX_BACKOFF_PERIOD_SECrrPrr3rr,rrrrrrs  ########GGGGGGGGGGGGGGGGGG ###### 111111666666"=<<<<<  KJJJJJJJONNNNN777777222222 65555555SSSSSSSS------88888888  , - -JNN4c::cA:>>677C NN 9   "0_"_"_"_"_"3_"_"_"D mmmmm mmm`vvvvv vvvr      #   D Qb"b"b"b"b"$b"b"b"J 3 3 3 3 3r