Pi6ddlZddlZddlmZddlmZmZddlZddl Z ddl m Z ddl m Z dZ dZde jfd Zd ZGd d Zd dde jdfdZd dde jdfdZd de jdfdZdZdZdeeje jffdZdeeje jfdeeeeeffdZdS)N)Thread)TupleUnion)Image)tqdmc tjrtjdjdk}tjdjdk}|st jdtdtdtj dddD}|d kr*t jd tj d td|d kp| }nd }d }d }|||fS)NrzSFlash Attention is disabled as it requires a GPU with Ampere (8.0) CUDA capability.category stacklevelc34K|]}t|VdSN)int).0vs s/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/torchao/_models/sam2/utils/misc.py z$get_sdpa_settings..s(QQ1AQQQQQQ.)r r zYou are using PyTorch zw without Flash Attention v2 support. Consider upgrading to PyTorch 2.2+ for Flash Attention v2 (which could be faster).TF) torchcuda is_availableget_device_propertiesmajorwarningswarn UserWarningtuple __version__split)old_gpuuse_flash_attnpytorch_versionmath_kernel_ons rget_sdpa_settingsr's- z  *22155;a?99!<<BaG  Me$     QQ0A0G0G0L0LRaR0PQQQQQ V # # Me):eee$      )61G5G NN 22rcddlm}||tjS)a Get the connected components (8-connectivity) of binary masks of shape (N, 1, H, W). Inputs: - mask: A binary mask tensor of shape (N, 1, H, W), where 1 is foreground and 0 is background. Outputs: - labels: A tensor of shape (N, 1, H, W) containing the connected component labels for foreground pixels and 0 for background pixels. - counts: A tensor of shape (N, 1, H, W) containing the area of the connected components for foreground pixels and 0 for background pixels. r)_C)sam2r)get_connected_componnetstoruint8 contiguous)maskr)s rget_connected_componentsr00sD & &twwu{';';'F'F'H'H I IIrmasksc|j\}}}}|j}tj||tj}tj||tj}tj||d\}} |d|d||}| d|d||} tjtj||| dd\} }tj tj||d dd\} }tjtj|| | dd\} }tj tj|| d dd\} }tj | | | | fd}|S) z compute bounding box given an input mask Inputs: - masks: [B, 1, H, W] masks, dtype=torch.Tensor Returns: - box_coords: [B, 1, 4], contains (x, y) coordinates of top left and bottom right box corners, dtype=torch.Tensor )devicedtypexy)indexing)NN.dim) shaper3rarangeint32meshgridexpandminwhereflattenmaxstack)r1B_hwr3xsysgrid_xsgrid_ysmin_xsmax_xsmin_ysmax_ys bbox_coordss r mask_to_boxrSCsJAq!Q \F aek : : :B aek : : :B~b"t<<<GWo&--aAq99Go&--aAq99G %+eWa88@@DD"MMMIFA %+eWb99AA"EE2NNNIFA %+eWa88@@DD"MMMIFA %+eWb99AA"EE2NNNIFA+vvvv>BGGGK rc|tj|}tj|d||f}|jtjkr|dz }ntd|jd|tj | ddd}|j \}}|||fS)NRGBo@zUnknown image dtype: z on r rr7) ropennparrayconvertresizer4r- RuntimeErrorr from_numpypermutesize)img_path image_sizeimg_pilimg_npimg video_width video_heights r_load_img_as_tensorrg]sj""G Xgooe,,33Z4LMM N NF |rx%O6<OOXOOPPP  6 " " * *1a 3 3C ' K  k ))rc$eZdZdZdZdZdZdS)AsyncVideoFrameLoaderzZ A list of video frames to be load asynchronously without blocking session start. cH|_|_|_|_|_dgt |z_d_d_d_ |_ dfd}t|d_ j dS)Nrc tttjdD]}|dS#t $r}|_Yd}~dSd}~wwxYw)Nframe loading (JPEG)desc)rrangelenimages __getitem__ Exception exception)neselfs r _load_framesz4AsyncVideoFrameLoader.__init__.._load_framess #eC $4$455>>  rcF|jtd|j|j|}||St|j||j\}}}||_||_||jz}||j z}|j s| |j d}||j|<|S)NzFailure in frame loading threadT) non_blocking) rtr\rqrgr{rarfrer}r~r|r,r)rwindexrdrfres rrrz!AsyncVideoFrameLoader.__getitem__s > %@AAt~ Uk%  ?J)< N5 !4?* * &\;)& t} t|( A&&,4&@@C  E rc*t|jSr)rprq)rws r__len__zAsyncVideoFrameLoader.__len__s4;rN)__name__ __module__ __qualname____doc__rrrrrrririisL$$$L*     rri)g ףp= ?gv/?gCl?)gZd;O?gy&1?g?Frc ft|t}t|t}|o&tj|ddv} |s| rt ||||||S|r5tj|rt|||||||Std)z Load the video frames from video_path. The frames are resized to image_size as in the model and are loaded to GPU if offload_video_to_cpu=False. This is used by the demo. r9)z.mp4z.MP4) video_pathrar|r}r~r)rrar|r}r~async_loading_framesrz;Only MP4 video and JPEG folder are supported at this moment) isinstancebytesstrospathsplitext!load_video_frames_from_video_fileisdir!load_video_frames_from_jpg_imagesNotImplementedError) rrar|r}r~rris_bytesis_str is_mp4_paths rload_video_framesrs*e,,H  C ( (FQRW--j99"=AQQK ; 0!!!5)        BGMM*--  0!!!5!5)    " I   rct|tr"tj|r|nt ddtjD}|dt|}|dkrtdfd|D} tj |tj d d d d f}tj |tj d d d d f}|r#t| |||||} | | j| jfStj|d ||tj } t#t%| d D]\} } t'| |\| | <}}|s?| |} ||}||}| |z} | |z} | ||fS) aX Load the video frames from a directory of JPEG files (".jpg" format). The frames are resized to image_size x image_size and are loaded to GPU if `offload_video_to_cpu` is `False` and to CPU if `offload_video_to_cpu` is `True`. You can load a frame asynchronously by setting `async_loading_frames` to `True`. akOnly JPEG frames are supported at this moment. For video files, you may use ffmpeg (https://ffmpeg.org/) to extract frames into a folder of JPEG files, such as ``` ffmpeg -i .mp4 -q:v 2 -start_number 0 /'%05d.jpg' ``` where `-q:v` generates high-quality JPEG frames and `-start_number 0` asks ffmpeg to start the JPEG file from 00000.jpg.c`g|]+}tj|ddv)|,S)r9)z.jpgz.jpegz.JPGz.JPEG)rrr)rps r z5load_video_frames_from_jpg_images..sC 7  A  r "&H H H H H Hrcfttj|dS)Nr)rrrr)rs rz3load_video_frames_from_jpg_images..s#3rw'7'7':':1'=#>#>r)keyrzno images found in cPg|]"}tj|#Sr)rrjoin)r frame_name jpg_folders rrz5load_video_frames_from_jpg_images..s)TTT*j*55TTTrr4Nrlrm)rrrrrrlistdirsortrpr\rtensorfloat32rirfrezeros enumeraterrgr,)rrar|r}r~rr frame_names num_framesr{ lazy_imagesrqrur`rfrers @rrrs"*c""  rw}}Z'@'@   ! <   J''K >>???[!!JQ===>>>TTTT TTTI|HEM:::111dD=IHl7%-888D$GG N+        K4k6MMM [Q Jem T T TF i6L!M!M!MNNYY 8/B8Z/X/X,q < ->**;;~..**^,, hF gF < ,,rcddl}tj|tjddddf}tj|tjddddf}|jd||j\}}} g} ||||D],} | | ddd-tj | d d z } |s?| |} | |}| |}| |z} | |z} | ||fS) z(Load the video frames from a video file.rNrr)widthheightr r7r:rV)decordrrrbridge set_bridge VideoReadernextr<appendr^rEfloatr,) rrar|r}r~rrrfrerGrqframes rrrseMMM|HEM:::111dD=IHl7%-888D$GG MW%%%#)#5#5j#A#A#F#F#H#H#N L+q F##Jj#TT.. emmAq!,,---- [Q ' ' ' - - / /% 7F ->**;;~..**^,, hF gF < ,,rc|dks Jd|} t|dk\}}|dk||kz}tj|d|}n8#t$r+}t j|dt d|}Yd}~nd}~wwxYw|S)zY A post processor to fill small holes in mask scores with area under `max_area`. rzmax_area must be positiveg?a* Skipping the post-processing step due to the error above. You can still use SAM 2 and it's OK to ignore the error above, although some post-processing functionality may be limited (which doesn't affect the results in most cases; see https://github.com/facebookresearch/sam2/blob/main/INSTALL.md).r r N)r0rrBrsrrr)r/max_area input_masklabelsareasis_holervs rfill_holes_in_mask_scoresr9s a<<<4<<<J0;; A:%8"34{7C..      N N N!       Ks9A B!A>>Bc|||}}nrs  333<JJJ&u|4 * * *A A A A A A A A P# !5<''& & & & Z# !5<''@-@-@-@-N# !5<'' ----@:<<<  % EL 89      U\) * 6;Cc3