.`i#@UddlZddlZddlmZddlmZddlmZddlZddl m Z ddl m Z ddl mZddlmZe eZdeed fd ejd efd Zd ZdZdaeded<GddZd efdZddZ ddejdedzd dfdZ ddZ!ddZ"dS)N) accumulate)prod)Optional) init_loggerround_up)dbo_current_ubatch_idshape.dtypereturnc0t||jzSN)ritemsize)r r s l/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/v1/worker/workspace.py_compute_bytesrs ;; ''ii@WorkspaceManager_managerceZdZdZddejdedzfdZedej dzdefdZ dd Z de fd Z d eeed fejfdeej fd Zdedej fdZdS)rzManager for workspace allocation. Manages workspace buffers for DBO (Dual Batch Overlap) execution. Can be locked to prevent further growth during execution. Ndevice num_ubatchescJ||_||nd|_ddg|_d|_dS)NF)_device _num_ubatches_current_workspaces_locked)selfrrs r__init__zWorkspaceManager.__init__'s1 -9-E\\1?CTl " r workspacer c\|dS||zS)zGet size of workspace in bytes.Nr)numel element_size)r s r_workspace_size_bytesz&WorkspaceManager._workspace_size_bytes.s0  1  9#9#9#;#;;;rcd_tjr.tdfdjDdSdS)zLock the workspace to prevent further growth. After locking, any attempt to allocate a larger workspace will raise an assertion error. This ensures workspace size is fixed during execution. Tz5[WORKSPACE DEBUG] Workspace locked. Current sizes: %scNg|]!}||tz "Sr)r$_MB).0wsrs r z)WorkspaceManager.lock..?s9~..r22S8%~~rN)renvsVLLM_DEBUG_WORKSPACEloggerinforrs`rlockzWorkspaceManager.lock5sm   $  KKG"6       rc|jS)zCheck if workspace is locked.)rr/s r is_lockedzWorkspaceManager.is_lockedFs |rshapes_and_dtypes.c(dDdD}t|}ttdg|ddz||fdt t DS)a Get multiple workspace tensors simultaneously from a single allocation. Args: *shapes_and_dtypes: One or more (shape, dtype) tuples. Returns: List of tensor views into the workspace buffer, one per shape/dtype pair. c4g|]\}}t||S)r)r(sds rr*z5WorkspaceManager.get_simultaneous..Us&KKKAq!,,KKKrc.g|]}t|dS)r)r(actuals rr*z5WorkspaceManager.get_simultaneous..Vs"JJJ6&#..JJJrrNcg|]_}|||z|d|d`S)rr)viewreshape)r(i actual_bytescurrent_workspaceoffsetsr3s rr*z5WorkspaceManager.get_simultaneous..^sv    gaj71: Q+GG H T#A&q) * * W&q)!, - -   r)sumlistr_ensure_workspace_sizerangelen)rr3 aligned_bytes total_bytesrArBrCs ` @@@rget_simultaneousz!WorkspaceManager.get_simultaneousJsLK9JKKK JJ\JJJ -(( z1# crc(:":;;<< 77 DD       301122     rrequired_bytesc t}|j|}||}||kr/dtfd}|jr3t d|d|t z dd|t z ddt|jD]d}|j|}||||kr:d|j|<~tj |ftj |j |j|<etjrLtd ||t z |t z |j||jzt z |jt}|S) zEnsure workspace is allocated and large enough, return current workspace. Args: required_bytes: The number of bytes required. Returns: The current workspace tensor. r c4tj}|dS|j}|wt|jdt r|j}7tj |j j }|d|j d|j j SdS)z*Find first frame outside WorkspaceManager.Nunknownr:)inspect currentframef_back isinstancef_localsgetrospathbasenamef_code co_filenamef_linenoco_name) curr_framefilenames rget_caller_infoz@WorkspaceManager._ensure_workspace_size..get_caller_infots$133 %$9'.  , "*"5"9"9&"A"ACSTT!%/%6  !w// 0A0MNNH#WWj&9WWJ %.2f MB (%d ubatches, total memory %.2f MB))r rr$strrAssertionErrorr'rGrtorchemptyuint8rr+r,r-r.)rrL ubatch_idrB current_sizer`s rrFz'WorkspaceManager._ensure_workspace_sizees*++  4Y?112CDD . ( ( !S ! ! ! !,| $E@Q@QEE . 4OEE#c)4EEE#4#566   $($rs  ######******:::::: X  (%S/(%+(#(((( *.(% &---P!P!P!P!P!P!P!P!f $        6:66 L6(+d 6 6666.'''',r