Pi ddlZddlmZddlmZmZddlZddlmZddl m Z e dZ erddl m Z nejZ deeeffd Zdefd ZeZdeefd Zd ejdejfd ZdefdZd ejddfdZdd eedejfdZded ejddfdZGddeZdefdZdefdZ dS)N)Enum)OptionalTuple)_SUPPORTS_FLEX_ATTENTION) get_loggerDEBUG) BlockMaskreturnctjrZtjrs!""'aJ'))N ,K ,K-//L |;jAAA||002222 T[ T T T    < . .  V{ V V V   F### Mrctjrd}n-trd}n#tjrd}nd}|S)zFunction that gets the torch.device based on the current machine. This currently only supports CPU, CUDA, NPU. Returns: device cudarxpucpu)r r5ris_npu_availabler6r$s r_get_device_type_from_envr:bsW z       ! ! Mrct}|jdkr"| |j|krtd|d| t jd|dS#t$r}td|d|d}~wwxYw) aFunction that validates the device is correct given the current machine. This will raise an error if the device is not available or doesn't match the assigned process device on distributed runs. Args: device (torch.device): The device to validate. Raises: RuntimeError: If the device is not available or doesn't match the assigned process device. Returns: device r7NzVYou can't specify a device index when using distributed training. Device specified is z but local rank is:rr9z The device z" is not available on this machine.)r#r&r'r.r empty)r$r"es r_validate_device_from_envr>us!""J{e 6 <: % %O'-OOBLOO   Af%%%%%%  D& D D D   sA A8A33A8c|t}tj|}|jdvrt |}t ||S)axFunction that takes an optional device string, verifies it's correct and available given the machine and distributed settings, and returns a :func:`~torch.device`. If device string is not provided, this function will infer the device based on the environment. If CUDA-like is available and being used, this function also sets the CUDA-like device. Args: device (Optional[str]): The name of the device to use, e.g. "cuda" or "cpu" or "npu" or "xpu". Example: >>> device = get_device("cuda") >>> device device(type='cuda', index=0) Returns: torch.device: Device N)r5rr6)r:r r$r&r3r>r9s r get_devicer@sQ$~*,, \& ! !F {,,,v&&f%%% Mrbatchc |D]\}}t|trt||+t|tjr||||<^tr.t|tr||||<td|dt|dS)aFunction that takes a dictionary (or nested dictionary) of tensors and sets them all to the same device. This utility is intended to be used for batches of data to be moved to device, the update is inplace. Args: batch (dict): dict of Tensors or more nested dicts of tensors. device (torch.device): torch device to move the tensors to. Raises: ValueError: if batch dict contains anything other than ``torch.Tensor``. zUTo use batch_to_device, all elements in the batch must be a dict or Tensor. Got key "z" with value of type N) items isinstancedictbatch_to_devicer Tensortorr ValueErrorr&)rAr$kvs rrFrFs  1 a    Av & & & & 5< ( ( ttF||E!HH % *Q *B*B ttF||E!HH. .."&q''..   rcReZdZdZdZdZdZdZdededefd Z e defd Z d S) DeviceSupporta This is a simple enum for compute devices, This currently only supports CPU, CUDA, NPU, and XPU. The following enumeration defines various device configurations with attributes: 1. `device_type` (str): The type of device (e.g., "cpu", "cuda", "npu", "xpu"). 2. `device_name` (str): A user-friendly name for the device (e.g., "CPU", "GPU", "NPU", "XPU"). 3. `communication_backend` (str): Specifies the backend used for communication on this device (e.g., "gloo", "nccl", "hccl", "ccl"). )r7CPUgloo)r5GPUnccl)rNPUhccl)r6XPUcclr*r+communication_backendc0||_||_||_dSN)r*r+rV)selfr*r+rVs r__init__zDeviceSupport.__init__s" '&%:"""rcZtD]}|j|kr|cStd|d)NzUnknown device type: .)rMr*rI)r*members r from_typezDeviceSupport.from_typesG#  F![00 1????@@@rN) __name__ __module__ __qualname____doc__rNCUDArRrTstrrZ staticmethodr^rrrrMrMs !C "D C C;;; # ;;;;AsAAA\AAArrMcRt}t|S)zfunction that gets the DeviceSupport with compute devices based on the current machine. This currently only supports CPU, CUDA, NPU, XPU. Returns: device_support: DeviceSupport )r:rMr^r*s rr)r)s#,--K  " "; / //rctj} tt|S#t$r-t d|dtjcYSwxYw)zReturn the corresponding torch attribute based on the device type string. Returns: module: The corresponding torch device namespace, or torch.cuda if not found. zDevice namespace 'z-' not found in torch, try to load torch.cuda.)r)r*getattrr r0loggerwarningr5rgs rr,r,sv %&&2Kuk***  [ [ [ [   z s*4A! A!rX)!renumrtypingrrr torchtune.utils._import_guardrtorchtune.utils._loggingrrj!torch.nn.attention.flex_attentionr rGr!rboolrr8r#r$r3rdr:r>r@rErFrMr)anyr,rrrrss: """""""" BBBBBB////// G  ;;;;;;; I sCx    *)++ #    !%,!5<!!!!H3&eltDx} 64$6AAAAADAAAD 0M 0 0 0 0 C      r