&`i9dZddlZddlZddlZddlmZddlmZmZm Z m Z m Z m Z m Z ddlmZddlmZddlmZddlmZmZdd lmZed d d d d e eefd ededede eeff dZe d:d e eefdededdfdZeddd d d dd dd e eefde ededededede ejdede efdZedd d ddd e eefd e ed!e ed"ed#e ed$eded%efd&Z ed e eefdefd'Z!ed e eefde efd(Z"e d;d)e e#d*e e ed+e e eddfd,Z$e d;d-e ed.e ed/e e#fd0Z%eed e eefde efd1Z&e d local. ip_address: Address of node. use_internal_ip: Whether the provided ip_address is public or private. no_config_cache: Whether to disable the config cache and fully resolve all environment settings from the Cloud provider again. should_bootstrap: whether to bootstrap cluster config before syncing Raises: RuntimeError: If the cluster head node is not found. NF) rr5r6rr7r2r3r all_nodesr4)rr rsync) rr5r6r7r2r3rr4rs r r:r:s<  ( (  K~#"&!++-                       r!c|t|5}tj|cdddS#1swxYwYdS)a]Returns head node IP for given configuration file if exists. Args: cluster_config (Union[str, dict]): Either the config dict of the cluster, or a path pointing to a file containing the config. Returns: The ip address of the cluster head node. Raises: RuntimeError: If the cluster is not found. N)rr get_head_node_iprrs r r<r<s  ( (6K(55666666666666666666 155c|t|5}tj|cdddS#1swxYwYdS)aOReturns worker node IPs for given configuration file. Args: cluster_config (Union[str, dict]): Either the config dict of the cluster, or a path pointing to a file containing the config. Returns: List of worker node ip addresses. Raises: RuntimeError: If the cluster is not found. N)rr get_worker_node_ipsr=s r r@r@s  ( (9K+K88999999999999999999r>num_cpusbundlesbundle_label_selectorsc|$t|tstd|t|tr|D]|}t|trW|D]A}t|t rt||tstdBntdntd||tdt|t|krtd|D]}t|trVtd|Dr+td | Dstd t|}|rtd |d tj|||S) a Command the autoscaler to scale to accommodate the specified requests. The cluster will immediately attempt to scale to accommodate the requested resources, bypassing normal upscaling speed constraints. This takes into account existing resource usage. For example, suppose you call ``request_resources(num_cpus=100)`` and there are 45 currently running tasks, each requiring 1 CPU. Then, enough nodes will be added so up to 100 tasks can run concurrently. It does **not** add enough nodes so that 145 tasks can run. This call is only a hint to the autoscaler. The actual resulting cluster size may be slightly larger or smaller than expected depending on the internal bin packing algorithm and max worker count restrictions. Args: num_cpus: Scale the cluster to ensure this number of CPUs are available. This request is persistent until another call to request_resources() is made to override. bundles (List[ResourceDict]): Scale the cluster to ensure this set of resource shapes can fit. This request is persistent until another call to request_resources() is made to override. bundle_label_selectors: A list of label selectors, applied per-bundle to the same index in the `bundles` list. For bundles without a label requirement, the corresponding item in the list is an empty dictionary. For each bundle. Label selectors consist of zero or more key-value pairs where the key is a label and the value is a operator (in, !in, etc.) and label value. Examples: >>> from ray.autoscaler.sdk import request_resources >>> # Request 1000 CPUs. >>> request_resources(num_cpus=1000) # doctest: +SKIP >>> # Request 64 CPUs and also fit a 1-GPU/4-CPU task. >>> request_resources( # doctest: +SKIP ... num_cpus=64, bundles=[{"GPU": 1, "CPU": 4}]) >>> # Same as requesting num_cpus=3. >>> request_resources( # doctest: +SKIP ... bundles=[{"CPU": 1}, {"CPU": 1}, {"CPU": 1}]) >>> # Requests 2 num_cpus=1 bundles, the first with >>> # label_selector={"accelerator-type": "in(A100)"} and second with >>> # label_selector={"market-type": "spot"}. >>> request_resources( # doctest: +SKIP ... bundles=[{"CPU": 1}, {"CPU": 1}]), ... bundle_label_selectors=[{"accelerator-type": "in(A100)"}, ... {"market-type": "spot"}]) Nznum_cpus should be of type int.z/each bundle key should be str and value as int.zeach bundle should be a Dict.zbundles should be of type ListzF`bundles` must be provided when `bundle_label_selectors` is specified.zR`bundle_label_selector` must be a list with length equal to the number of bundles.c3@K|]}t|tVdSN isinstancestr).0ks r z$request_resources..s,MM!:a--MMMMMMc3@K|]}t|tVdSrFrG)rJvs r rLz$request_resources.. s,OO!:a--OOOOOOrMzBundle label selector must be a list of string dictionary label selectors. For example: `[{ray.io/market_type": "spot"}, {"ray.io/accelerator-type": "A100"}]`.zQInvalid label selector provided in bundle_label_selectors list. Detailed error: '')rHint TypeErrorrrkeysrI ValueErrorlendictallvaluesr r request_resources)rArBrCbundlekeylabel_selector error_messages r rYrYs.hJx$=$=9::: gt $ $ >! E Efd++E%{{}} *3 4 4F3KQT9U9U"+ Q## $$CDDD E<== =) ?X  % & &#g,, 6 6d 5  N~t44 MM~7J7J7L7LMMMMM OO~7L7L7N7NOOOOO  !^ 4NCCM  :)6:::   %h9O P PPrM log_style color_mode verbosityc4tj|||dS)aConfigures logging for cluster command calls. Args: log_style: If 'pretty', outputs with formatting and color. If 'record', outputs record-style without formatting. 'auto' defaults to 'pretty', and disables pretty logging if stdin is *not* a TTY. Defaults to "auto". color_mode (str): Can be "true", "false", or "auto". Enables or disables `colorful`. If `color_mode` is "auto", is set to `not stdout.isatty()` vebosity (int): Output verbosity (0, 1, 2, 3). Low verbosity will disable `verbose` and `very_verbose` messages. r^r_r`N)r configurerbs r configure_loggingrd1s/2 irMc#nKt|trXtjdd}|t j|||j}tj |s"td ||VdS)Nwzautoscaler-sdk-tmp-)prefixzCluster config not found {})rHrVtempfileNamedTemporaryFilewritejsondumpsflushnameospathexistsrTformat)rtmps r rrOs.$''")#6KLLL $*^,,---  7>>. ) )O6==nMMNNN rMc,tj||S)zoValidate and add provider-specific fields to the config. For example, IAM/authentication may be added here.)r _bootstrap_config)rrs r bootstrap_configrv\s  %no F FFrMconfigc$ddlm}||S)zBFillout default values for a cluster_config based on the provider.r)fillout_defaults)ray.autoscaler._private.utilry)rwrys r ryryes'>=====  F # ##rM event_namecallbackc0tj||dS)aHRegisters a callback handler for autoscaler events. Args: event_name: Event that callback should be called on. See CreateClusterEvent for details on the events available to be registered against. callback: Callable object that is invoked when specified event occurs. N)radd_callback_handler)r{r|s r register_callback_handlerrms,ZBBBBBrM cluster_namec2d}||S)z.Return host path that Docker mounts attach to.z!/tmp/ray_tmp_mount/{cluster_name})r)rr)rdocker_mount_prefixs r get_docker_host_mount_locationr~s!>  % %< % @ @@rM)FF)NNN)F)+__doc__rkrorh contextlibrtypingrrrrrr r ray._private.label_utilsr ray.autoscaler._privater "ray.autoscaler._private.cli_loggerr $ray.autoscaler._private.event_systemrrray.util.annotationsrrVrIboolrr% Port_forwardr1r:r<r@rQrYrdrrvryrrrMr rsLL %%%%%%GGGGGGGGGGGGGGGGGG<<<<<<,,,,,,999999.-----!    $)$         #s(^      F"  $)$        2!48* * * $)$*  #*  *  *  * * 801* * c]* * * * Z!%!!!) ) ) $)$)  SM)  SM )  )  ) ) ) ) ) ) ) X6U49%56#6666"9dCi(89T#Y9999""$(37]Q]Qsm]Q d4j !]Q%T$Z0]Q ]Q]Q]Q]Q@# $#} }: E$)$4#