`idZddlmZddlmZddlmZddlmZddl m Z Gdd Z dd l Z Gd d Z dddZn#e$rdddZYnwxYwddZddZd S)zStream context.) annotations)c_void_p)Any)coredevicec*eZdZdZddZdd Zdd Zd S) StreamContexta1Represent a stream context in the FFI system. StreamContext helps setup ffi environment stream by python `with` statement. When entering `with` scope, it caches the current environment stream and setup the given new stream. When exiting `with` scope, it recovers the stream to the cached environment stream. Parameters ---------- device The device to which the stream belongs. stream The stream handle. See Also -------- :py:func:`tvm_ffi.use_raw_stream`, :py:func:`tvm_ffi.use_torch_stream` r core.Devicestreamint | c_void_preturnNonecT|j|_|j|_||_dS)z  rN)r r r rrr)rr rrrr__name__ __module__ __qualname____doc__rrr!rrr r sZ*       rr Nc*eZdZdZd dZd dZd d Zd S)TorchStreamContextz9Context manager that syncs Torch and FFI stream contexts.contextrrrc||_dS)z?Initialize with an optional Torch stream/graph context wrapper.N) torch_context)rr+s rrzTorchStreamContext.__init__Ms!(D   rc&|jr|jtj}t t t|j|j|_ |j |S)z)Enter both Torch and FFI stream contexts.) r-rtorchcudacurrent_streamr r str cuda_stream ffi_context)rr1s rrzTorchStreamContext.__enter__Qs~! /",,..."Z6688N,s>01122N4N  D    & & ( ( (KrrcP|jr|jj||jj|dS)z(Exit both Torch and FFI stream contexts.N)r-r!r4r s rr!zTorchStreamContext.__exit__\s9! 3+"+T22 %D  %t , , , ,rN)r+rrr)rr*r"r#r(rrr*r*JsVGG ) ) ) )     - - - - - -rr*r+rrc t|S)aCreate an FFI stream context with a Torch stream or graph. cuda graph or current stream if `None` provided. Parameters ---------- context The wrapped torch stream or cuda graph. Returns ------- context The ffi stream context wrapping torch stream context. Examples -------- .. code-block:: python s = torch.cuda.Stream() with tvm_ffi.use_torch_stream(torch.cuda.stream(s)): ... g = torch.cuda.CUDAGraph() with tvm_ffi.use_torch_stream(torch.cuda.graph(g)): ... Note ---- When working with a raw ``cudaStream_t`` handle, use :py:func:`tvm_ffi.use_raw_stream` instead. )r*r+s ruse_torch_streamr8bsB"'***rc td)z5Raise an informative error when Torch is unavailable.zCannot import torch) ImportErrorr7s rr8r8s/000rr r r rcxt|ttfstdt ||S)aCreate an FFI stream context with the given device and stream handle. Parameters ---------- device The device to which the stream belongs. stream The stream handle (for example, a CUDA ``cudaStream_t`` as an integer, or ``0``). Returns ------- context The FFI stream context. Examples -------- The example below uses a CPU device and a dummy stream handle. On CUDA, pass a real ``cudaStream_t`` integer. .. code-block:: python import tvm_ffi dev = tvm_ffi.device("cpu:0") with tvm_ffi.use_raw_stream(dev, 0): # Within the context, the current stream for this device is set assert tvm_ffi.get_raw_stream(dev) == 0 See Also -------- :py:func:`tvm_ffi.use_torch_stream` Use a Torch stream or CUDA graph as the source of truth. :py:func:`tvm_ffi.get_raw_stream` Query the current FFI stream for a device. zuse_raw_stream only accepts int or c_void_p as stream input, try use_torch_stream when using torch.cuda.Stream or torch.cuda.graph) isinstanceintr ValueErrorr )r r s ruse_raw_streamr?sEL fsHo . .  T     ( ((rr=cPtj|j|jS)apGet the current FFI stream of a given device. Parameters ---------- device The device to which the stream belongs. Returns ------- stream The current FFI stream as an integer handle. Examples -------- .. code-block:: python import tvm_ffi dev = tvm_ffi.device("cpu:0") # Default stream is implementation-defined; set it explicitly with tvm_ffi.use_raw_stream(dev, 0): assert tvm_ffi.get_raw_stream(dev) == 0 See Also -------- :py:func:`tvm_ffi.use_raw_stream` Set the current stream for a device. )r_env_get_current_streamrrrs rget_raw_streamrBs%<  '(A(A(C(CV\ R RRr)N)r+rrr*)r r r rrr )r r rr=)r' __future__rctypesrtypingrr_tensorr r r/r*r8r:r?rBr(rrrHs_$""""""' ' ' ' ' ' ' ' TB1LLL--------0!+!+!+!+!+!+F11111111111+)+)+)+)\SSSSSSsA AA