&`iLddlZddlZddlZddlZddlmZmZmZddlZddl m cm Z ddl m Z mZddlmZddlmZddlmZejeZGddZdS) N)DictOptionalTuple)HEAD_NODE_RESOURCE_NAMENODE_ID_PREFIX)RESOURCE_CONSTRAINT_PREFIX) accelerators)AcceleratorManagercZeZdZdZ ddeedeedeedeedeeeefdeeeeff d Z d e fd Z d e fd Z d Z dde deed dfdZed eeeffdZedeeefdeeeffdZ dde deed dfdZed eeeffdZedeed eeeffdZdeed dfdZdZdZedeedeeefd eeeffdZdS)ResourceAndLabelSpeca(Represents the resource and label configuration passed to a raylet. All fields can be None. Before starting services, resolve() should be called to return a ResourceAndLabelSpec with unknown values filled in with merged values based on the local machine and user specifications. Nnum_cpusnum_gpusmemoryobject_store_memory resourceslabelsch||_||_||_||_||_||_d|_dS)a  Initialize a ResourceAndLabelSpec Args: num_cpus: The CPUs allocated for this raylet. num_gpus: The GPUs allocated for this raylet. memory: The memory allocated for this raylet. object_store_memory: The object store memory allocated for this raylet. resources: The custom resources allocated for this raylet. labels: The labels associated with this node. Labels can be used along with resources for scheduling. FN)r rrrrr _is_resolved)selfr rrrrrs x/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/_private/resource_and_label_spec.py__init__zResourceAndLabelSpec.__init__s=*!    #6 " !returnc|jS)zmReturns if resolve() has been called for this ResourceAndLabelSpec and default values are filled out.)rrs rresolvedzResourceAndLabelSpec.resolved6s   rc |td|j|j|j|j|j|jfDS)zEReturns whether all fields in this ResourceAndLabelSpec are not None.c3K|]}|duV dSN).0vs r z7ResourceAndLabelSpec._all_fields_set..=s:    TM       r)allr rrrrrrs r_all_fields_setz$ResourceAndLabelSpec._all_fields_set;sS      (       rc ,|sJt|j|j|jt |jt |j}d|D}|D]\}}t|t s4t|tsJ|dt|d|t|tr7| s#td|||dkr#td|||tjkr.tdtj|||S) zReturns a dict suitable to pass to raylet initialization. This renames num_cpus / num_gpus to "CPU" / "GPU", and check types and values. )CPUGPUrrc&i|]\}}|dk ||S)rr )r!resource_labelresource_quantitys r z9ResourceAndLabelSpec.to_resource_dict..Ys5   1 1 A%% -%%%rz (z): zOResource quantities must all be whole numbers. Violated by resource '{}' in {}.rzIResource quantities must be nonnegative. Violated by resource '{}' in {}.zHResource quantities must be at most {}. Violated by resource '{}' in {}.)rdictrr rintrritems isinstancefloattype is_integer ValueErrorformat ray_constantsMAX_RESOURCE_QUANTITY)rrr*r+s rto_resource_dictz%ResourceAndLabelSpec.to_resource_dictIs }} N  t{## #D$< = =      5>__5F5F   2;1B1B   -N-/55 !5::  "XXT*;%<%<XXEVXX   ,e44 )4466 !77=vni7X7X!1$$ 77=vni7X7X!=#FFF 77=v%;^Y88Gris_headnode_ip_addresscX|||t|j|j\}}||||jd|_|||d|_| sJ|S)aFills out this ResourceAndLabelSpec instance with merged values from system defaults and user specification. Args: is_head: Whether this is the head node. node_ip_address: The IP address of the node that we are on. This is used to automatically create a node id resource. Returns: ResourceAndLabelSpec: This instance with all fields resolved. )r9r:NrT) _resolve_resourcesr _get_current_node_acceleratorrr_resolve_accelerator_resources_resolve_labels_resolve_memory_resourcesrr%)rr9r:accelerator_managernum_acceleratorss rresolvezResourceAndLabelSpec.resolve}s QQQ ! > > M4>     ++,?AQRRR = DM 0111 &&((( ##%%%%% rc i}tjtj}|r` t j|}n,#t $rtd|wxYwt d|d|S)z9Load resource overrides from the environment, if present.Failed to load z!Autoscaler overriding resources: .) osgetenvr6RESOURCES_ENVIRONMENT_VARIABLEjsonloads Exceptionlogger exceptiondebug) env_resources env_strings r_load_env_resourcesz(ResourceAndLabelSpec._load_env_resourcess Y}KLL  O  $ : 6 6      !?:!?!?@@@  LLM]MMM N N N 9)A"env_dict params_dictc |dd}|dd}|dd}|dd}|}||t||piD]D}||||kr0t d|d||d||d E|||||fS) zMerge environment and Ray param-provided resources, with env values taking precedence. Returns separated special case params (CPU/GPU/memory) and the merged resource dict. r'Nr(rrz(Autoscaler is overriding your resource: : z with rF)popcopyupdatesetkeys intersectionrMwarning)rTrUr rrrresultkeys r_merge_resourcesz%ResourceAndLabelSpec._merge_resourcess' <<t,,<<t,,h--&ll+@$GG!!## hx}}''44[5FBGG  C38C=00@s@@"3'@@/7}@@@ 6+>FFrcHt}t||jpi\}}}}}||jn||_||jn||_||jn||_||jn||_||_|tj }d|jt|z<t|jvrttd|rd|jt<|j*tjj|_dSdS)zResolve CPU, GPU, and custom resources. Merges resources from environment, Ray params, and defaults in that order of precedence.Ng?z7 is a reserved resource name, use another name instead.)r rRrarr rrrrayutilget_node_ip_addressrrr4_privateutils get_num_cpus) rr9r:rPr rrrmerged_resourcess rr<z'ResourceAndLabelSpec._resolve_resourcessK-@@BB  ! 1 1-AUSU V V      *2)9 x )1)9 x %+^dkk #*  $ $$  *  "!h::<QN3;: ;  62TTVV  %%K= #4466%??0TTVV 6"))*555rc$t|}|jpiD]U\}}||vrG|||kr;td|d|d||d|d|j|d |||<Vt}|piD]M\}}||vr?|||kr3td|d||d|d||d |||<N||_dS)z{Resolve and merge environment override, user-input from params, and Ray default labels in that order of precedence.z,User label is overriding Ray default label: rWz to rFz$Autoscaler is overriding your label:N)r rrr/rMr^ro)rrAmergedr`val env_labelss rr?z$ResourceAndLabelSpec._resolve_labels.s&99:MNN*1133  HCf}}!3!3232222$Sk2222"k#.222 F3KK*::<< #)r0022  HCf}}!3!3111$Sk1111(o111 F3KK rc |sdS|}|}|C|A|t|kr.td|d|d|d|d |dkr||_n ||j|<|}|rd|jt|<| }|r|j |dSdS) z2Detect and update accelerator resources on a node.Nz Attempting to start raylet with  z, but z contains rFr() r+get_current_process_visible_accelerator_idslenr4#get_visible_accelerator_ids_env_varrrr~r%get_current_node_additional_resourcesrZ)rrArBaccelerator_resource_namevisible_accelerator_idsradditional_resourcess rr>z3ResourceAndLabelSpec._resolve_accelerator_resourcesNsQ"  F$7$I$I$K$K!  K K M M   ('3 3'>#?#???73C77,77*NNPP774777  % - -,DMM8HDN4 5.PPRR  RPQDN8L:JLL M  E E G G   8 N ! !"6 7 7 7 7 7 8 8rc tjj}tjj}|j}|t|tj z}tj dkrt|tj }tj}tj dkstj dkrPtjjdz}t!tj|}t||}|r;||kr5t$d|dzdz|}|j}|[||z }|dkrP|d |zkrGt-d t/|dz d td ||z z||_||_dS) Ndarwinlinuxlinux2gffffff?z.Warning: Capping object memory store to {}GB. geAz]To increase this further, specify `object_store_memory` when calling ray.init() or ray start.gחAg?zAfter taking into account object store and redis memory usage, the amount of memory on this node available for tasks and actors ({} GB) is less than {}% of total. You can adjust these settings with ray.init(memory=, object_store_memory=).d)rc_commonrgget_system_memoryrfestimate_available_memoryrr.r6&DEFAULT_OBJECT_STORE_MEMORY_PROPORTIONsysplatformmin!MAC_DEGRADED_PERF_MMAP_SIZE_LIMIT%DEFAULT_OBJECT_STORE_MAX_MEMORY_BYTESget_shared_memory_bytesmaxREQUIRE_SHM_SIZE_THRESHOLDrMrOr5rr4round)r system_memory avail_memoryrobject_store_memory_cap shm_availshm_caprs rr@z.ResourceAndLabelSpec._resolve_memory_resourcests );;== |)CCEE "6  &"%}SS## |x''&)')X''# C $ |w&&#,(*B*B L.FFHH4O mF RR*-.Ew*O*O'( >'*AAA DKK/36<<'># >!$77F~~&4-+?"?"? 4 5;FfslA..C6M;Q4R0S0S55   $7  rcFtjD]}tj|}||dkr|}n||}|G|}|}|t |t|}|dkr||fcSdS)a2 Returns the AcceleratorManager and accelerator count for the accelerator associated with this node. This assumes each node has at most one accelerator type. If no accelerators are present, returns None. The resolved accelerator count uses num_gpus (for GPUs) or resources if set, and otherwise falls back to the count auto-detected by the AcceleratorManager. The resolved accelerator count is capped by the number of visible accelerators. Args: num_gpus: GPU count (if provided by user). resources: Resource dictionary containing custom resource keys. Returns: Tuple[Optional[AcceleratorManager], int]: A tuple containing the accelerator manager (or None) the final resolved accelerator count. Nr(r)Nr)r "get_all_accelerator_resource_names$get_accelerator_manager_for_resourcert!get_current_node_num_acceleratorsrrr)rr resource_namerArBrs rr=z2ResourceAndLabelSpec._get_current_node_accelerators**LNN = =M"."S## #*%%#+  #,==#?#? ''IIKK!(SSUU(+6'*(#.E*F*F(($ !##*,<<<<<$wr)NNNNNNr)__name__ __module__ __qualname____doc__rr.r1rstrrboolrr%r8rC staticmethodrRrar<ror rr?r>r@rr=r rrr r s#'"&"&/304+/""3-"3-" " &e_ " De,- "c3h("""":!$!!!!       222j?C%%%.6sm% %%%%N c5j!1   \ G4U #3G$sEzBRGGG\G,?C.>.>.>.6sm.> .>.>.>.>` #d38n # # #\ ##%&89# c3h###\#J#+,>#? @$8$8$8L???B.3-.,0e,<. !3& '...\...rr )rJloggingrGrtypingrrrrcray._private.ray_constantsrfr6ray._common.constantsrrray._common.utilsr ray._privater ray._private.acceleratorsr getLoggerrrMr r rrrs  (((((((((( 222222222IIIIIIII888888%%%%%%888888  8 $ $SSSSSSSSSSr