`ilddlmZddlmZmZddlZddlmZm Z m Z ddl m Z m Z mZddlmZmZmZmZmZmZmZmZmZmZmZddlmZmZmZmZerddl Z ej!e"Z#ed d e$d e$d e%e&e$e$ffdZ'ed de&e$e$fde&e$e$fd e$d e$d e&e$e$ff dZ(de$de$de$de$de$d e)f dZ*de$de$de&e$e$fd e%e&e$e$e$e$ffdZ+ed  d$de&e$e$fde$d e$d e)d e&e$e$ff d!Z,Gd"d#eZ-d#gZ.dS)%) lru_cache)OptionalUnionN)BaseImageProcessor BatchFeature get_size_dict)convert_to_rgbresizeto_channel_dimension_format) OPENAI_CLIP_MEANOPENAI_CLIP_STDChannelDimension ImageInputPILImageResamplinginfer_channel_dimension_formatis_scaled_imagemake_flat_list_of_imagesto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypefilter_out_non_signature_kwargsis_vision_availablelogging )maxsizemin_image_tilesmax_image_tilesreturncg}td|dzD]A}td|dzD]+}||z|kr ||z|kr|||f,Bt|d}|S)aK Computes all allowed aspect ratios for a given minimum and maximum number of input tiles. This function calculates all possible arrangements of tiles that can be formed within the constraint of the minimum and maximum number of tiles. Each arrangement is represented by its aspect ratio (width/height) and the corresponding tile configuration. Args: min_image_tiles (`int`): The minimum number of tiles allowed. max_image_tiles (`int`): The maximum number of tiles allowed. Returns: `List[Tuple[int, int]]`: A list of tuples, each tuple representing a valid (width, height) configuration in terms of number of tiles. Example: >>> get_all_supported_aspect_ratios(1, 4) [(1, 1), (1, 2), (2, 1), (1, 3), (3, 1), (1, 4), (2, 2), (4, 1)] c$|d|dzSNrr"xs /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/ovis2/image_processing_ovis2.pyz1get_all_supported_aspect_ratios..Ms!qt key)rangeappendsorted)rr aspect_ratioswidthheights r(get_all_supported_aspect_ratiosr3/s0Mq/A-..66A233 6 6Fv~00UV^5V5V$$eV_555 6=.C.CDDDM r*doriginal_image_sizetarget_tile_sizec"t||}|\}}|\}}||z } ||z} td} d} |D]V} | d| dz }t| |z }|| kr|} | } 0|| kr | d|z|z| dz| dzkr| } W| S)a Given a minimum and maximum number of tiles, find the canvas with the closest aspect ratio to the original image aspect ratio. In case of tie-breaking condition when two canvases have the same aspect ratio difference, we favor the canvas with more tiles, until the area covered by the tiles is more than twice the target area, in order to avoid unnecessarily excessive tiling. infr"r"rr"g?)r3floatabs)r5r6rrpossible_tile_arrangementsoriginal_heightoriginal_widthtarget_tile_heighttarget_tile_width aspect_ratioareabest_ratio_diff best_gridgridgrid_aspect_ratio ratio_diffs r(get_optimal_tiled_canvasrHRs"ARa!b!b&9#O^,<))!O3L O +DEllOI* ! ! Gd1g-(99::  ' '(OII ? * *c..1BBT!WLtTUwVVV r*leftupperrightlowersidec||z }||z }t||t||}}||kr ||z |z}|}||zS)N)maxmin)rIrJrKrLrMwhs r(compute_patch_covering_arearSysQ  A  A q!99c!QiiqA4xx EDL  q5Lr*rRrQrEcdzdzfdtdDS)Nrr"c g|]T}tdD]<}|z|z|ddz krn|dzz|ddz krn|dzzf=US)r"rr-).0rowcol col_widthrErR row_heightrQs r( z)split_image_into_grid..s    a>>    )O * Q! ##AA#'Y)>Q! ##AA#'Z)?      r*rV)rRrQrErZr[s```@@r(split_image_into_gridr]sld1gJT!W I        a>>    r*? image_sizetarget_patch_sizecovering_thresholdct|\}}||z}td|}g}g} |D]e} t||| } tfd| D|z } || | f| |kr| | | ff| rt | ddSt |ddS)Nr"c,g|]}tg|RSr%)rS)rWregionr`s r(r\z.get_min_tile_covering_grid..s.dddV,HfH6GHHHdddr*cN|dd|ddz|d fSr$r%r&s r(r)z,get_min_tile_covering_grid..s*QqT!WqtAw=NQRSTQUPU.s&AaD5!A$q'AaDG:K2Lr*)r3r]sumr.rP) r_r`rra image_height image_width image_areacandidate_tile_gridsevaluated_gridssufficient_covering_grids tile_grid tile_regionstile_covering_ratios ` r(get_min_tile_covering_gridrqs !+L+|+J:1oNNO ")OO ,\; RR ddddWcddd e ehr r   +>?@@@ !3 3 3 % , ,i9L-M N N N Q,2V2VWWWXYZZ?(L(LMMMaPPr*c&eZdZdZdgZdddddejdddddddf d ed ee e e fd ed e d e dedede e e fdedee e ee fdee e ee fdededdffd Zejddfdejd e e e fdedee e efdee e efdejf dZedddddddddddddejddfded eed ee e e fd eed ee d ee deedeedee deedee e ee fdee e ee fdee e efdeededee e efdedejjf$dZ d#dejd e d e ded ee ee e fdeed!e fd"ZxZS)$Ovis2ImageProcessora Constructs a Ovis2 image processor. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image's (height, width) dimensions to the specified `size`. Can be overridden by the `do_resize` parameter in the `preprocess` method. size (`dict`, *optional*, defaults to `{"height": 384, "width": 384}`): Size of the output image after resizing. Can be overridden by the `size` parameter in the `preprocess` method. crop_to_patches (`bool`, *optional*, defaults to `False`): Whether to crop the image to patches. Can be overridden by the `crop_to_patches` parameter in the `preprocess` method. min_patches (`int`, *optional*, defaults to 1): The minimum number of patches to be extracted from the image. Only has an effect if `crop_to_patches` is set to `True`. Can be overridden by the `min_patches` parameter in the `preprocess` method. max_patches (`int`, *optional*, defaults to 12): The maximum number of patches to be extracted from the image. Only has an effect if `crop_to_patches` is set to `True`. Can be overridden by the `max_patches` parameter in the `preprocess` method. resample (`PILImageResampling`, *optional*, defaults to `Resampling.BICUBIC`): Resampling filter to use if resizing the image. Only has an effect if `do_resize` is set to `True`. Can be overridden by the `resample` parameter in the `preprocess` method. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image by the specified scale `rescale_factor`. Can be overridden by the `do_rescale` parameter in the `preprocess` method. rescale_factor (`int` or `float`, *optional*, defaults to `1/255`): Scale factor to use if rescaling the image. Only has an effect if `do_rescale` is set to `True`. Can be overridden by the `rescale_factor` parameter in the `preprocess` method. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. Can be overridden by the `do_normalize` parameter in the `preprocess` method. Can be overridden by the `do_normalize` parameter in the `preprocess` method. image_mean (`float` or `List[float]`, *optional*, defaults to `IMAGENET_STANDARD_MEAN`): Mean to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_mean` parameter in the `preprocess` method. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`float` or `List[float]`, *optional*, defaults to `IMAGENET_STANDARD_STD`): Standard deviation to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_std` parameter in the `preprocess` method. Can be overridden by the `image_std` parameter in the `preprocess` method. do_convert_rgb (`bool`, *optional*, defaults to `True`): Whether to convert the image to RGB. use_covering_area_grid (`bool`, *optional*, defaults to `True`): Whether to use the covering area grid to determine the number of patches. Only has an effect if `crop_to_patches` is set to `True`. Can be overridden by the `use_covering_area_grid` parameter in the `preprocess` method. pixel_valuesTNFr" gp? do_resizesizecrop_to_patches min_patches max_patchesresample do_rescalerescale_factor do_normalize image_mean image_stddo_convert_rgbuse_covering_area_gridr c :tjdi|||nddd}t|d}||_||_||_||_||_||_||_ ||_ | |_ | | nt|_ | | nt|_| |_dS)Nir2r1Tdefault_to_squarer%)super__init__r rvrwrxryrzr{r|r}r~r rrrr)selfrvrwrxryrzr{r|r}r~rrrrkwargs __class__s r(rzOvis2ImageProcessor.__init__s" ""6"""'ttc-J-JTT:::" .&&  $,((2(>**DT&/&;,r*image data_formatinput_data_formatc t|}d|vsd|vr$td||d|df}t|f||||d|S)a Resize an image to `(size["height"], size["width"])`. Args: image (`np.ndarray`): Image to resize. size (`Dict[str, int]`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BICUBIC`): `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BICUBIC`. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. Returns: `np.ndarray`: The resized image. r2r1zFThe `size` dictionary must contain the keys `height` and `width`. Got )rwr{rr)r ValueErrorkeysr )rrrwr{rrr output_sizes r(r zOvis2ImageProcessor.resizesFT"" 4  7$#6#6sfjfofofqfqsstt tH~tG}5   #/       r*imagesreturn_tensorsc ||nj}||nj}njnj||nj}||nj}| | nj} | | nj} | | nj} | | nj } ||nj }nj td |}t|}t|st!dt#|| | | | |||r d|D}d|D}|r/t%|drt&d t+|d|r2d kr,fd |D}d |D}d |D}ndgt-|z}t/|D]\}}|r||||<|r!||| ||<| r"||| | ||<t7|||||<t9||d| }|S)a Preprocess an image or batch of images. Args: images (`ImageInput`): Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`Dict[str, int]`, *optional*, defaults to `self.size`): Controls the size of the image after `resize`. The shortest edge of the image is resized to `size["shortest_edge"]` whilst preserving the aspect ratio. If the longest edge of this resized image is > `int(size["shortest_edge"] * (1333 / 800))`, then the image is resized again to make the longest edge equal to `int(size["shortest_edge"] * (1333 / 800))`. crop_to_patches (`bool`, *optional*, defaults to `self.crop_to_patches`): Whether to crop the image to patches. min_patches (`int`, *optional*, defaults to `self.min_patches`): The minimum number of patches to be extracted from the image. Only has an effect if `crop_to_patches` is set to `True`. max_patches (`int`, *optional*, defaults to `self.max_patches`): The maximum number of patches to be extracted from the image. Only has an effect if `crop_to_patches` is set to `True`. resample (`PILImageResampling`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image values between [0 - 1]. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*, defaults to `self.image_mean`): Image mean to normalize the image by if `do_normalize` is set to `True`. image_std (`float` or `List[float]`, *optional*, defaults to `self.image_std`): Image standard deviation to normalize the image by if `do_normalize` is set to `True`. do_convert_rgb (`bool`, *optional*, defaults to `self.do_convert_rgb`): Whether to convert the image to RGB. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. use_covering_area_grid (`bool`, *optional*, defaults to `True`): Whether to use the covering area grid to determine the number of patches. Only has an effect if `crop_to_patches` is set to `True`. NFrzkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.)r|r}r~rrrvrwr{c,g|]}t|Sr%)r rWrs r(r\z2Ovis2ImageProcessor.preprocess..s @@@nU++@@@r*c,g|]}t|Sr%)rrs r(r\z2Ovis2ImageProcessor.preprocess..s <<.sR   ** + +# 1+A +   r*cg|]\}}|Sr%r%)rW_rEs r(r\z2Ovis2ImageProcessor.preprocess..s000gaT000r*c"g|] \}}|D]}| Sr%r%)rW images_listrrs r(r\z2Ovis2ImageProcessor.preprocess..s)RRR QkRRUeRRRRr*r9)rwr{r)rscaler)rmeanstdr)input_channel_dim)rtgrids)data tensor_type)rvrxryrzr{r|r}r~rrrrwr fetch_imagesrrrrrlogger warning_oncerlen enumerater rescale normalizer r)rrrvrwrxryrzr{r|r}r~rrrrrrrrirencoded_outputss` ` `` `` r( preprocesszOvis2ImageProcessor.preprocess6sS^"+!6IIDN -<-H//dNb%0%vay I I   +{Q         $   F10000ERR&RRRFFHs6{{*E!&)) q qHAu r KKD8_pKqqq  u LLvaybsLttq   NN )#!&7 +q 4F1I{^opppF1II&Ve,T,Tbpqqqr*r^rracB|t|}t|tj|}|d|d} }|jdd\} } |rt | | f|||\} } nt | | f|| f||\} } | | z}|| z}| | z}||||dtjtj}g}t|D]{}|| z}|| z}|| z||z|dz| z|dz|zf}|d |d|d |d |d f}t||tj}| ||t|dkr9||||tj}| d ||| | ffS) a Crop the image to patches and return a list of cropped images. The number of patches and their grid arrangement are determined by the original image size, the target patch size and the minimum and maximum number of patches. The aspect ratio of the patches grid is chosen to be the closest to the original image aspect ratio. Args: images (`np.ndarray`): The image to be cropped. min_patches (`int`): The minimum number of patches to be extracted from the image. max_patches (`int`): The maximum number of patches to be extracted from the image. use_covering_area_grid (`bool`, *optional*, defaults to `True`): Whether to use the covering area grid to determine the number of patches. patch_size (`int`, `Tuple[int, int]`, `dict`, *optional*): The size of the output patches. data_format (`ChannelDimension`, *optional*): The format of the image data. If `None`, the format is inferred from the input image. covering_threshold (`float`, *optional*, defaults to `0.9`): The threshold for the covering area grid. If the covering area is less than this value, the grid is considered invalid. Returns: List[`PIL.Image.Image`] or List[np.ndarray]: The list of cropped images. Nr2r1)r`rrar)rrr".rr) rr rFIRSTshaperqrHr r-r.rinsert)rrryrzrrrrapatch_size_heightpatch_size_widthr=r> num_columnsnum_rows target_width target_height num_blocks resized_imageprocessed_imagesrcolumnrXbox patch_image thumbnail_imgs r(rz)Ovis2ImageProcessor.crop_image_to_patchess)H  8@@K,V5E5K[YY.8.BJwDW+*0,rss*;' ! $> .1"3 +#5 %%% !K%= .1"$45 %% !K(+5 )H4  8+  $| < <(..4 $  z"" 1 1A_F{"C))''!//q-- C(SVc!f_c!fs1vo(MNK5k;P`PfggK  # #K 0 0 0 0  A % % KK O_Oe(M  # #A} 5 5 5(K!888r*)TNNr^)__name__ __module__ __qualname____doc__model_input_namesrBICUBICboolrdictstrintrr:listrnpndarrayrr rrrrPILImagertupler __classcell__)rs@r(rsrss--^(()- %'9'A,3!:>9=#'+ - - -tCH~& - -  -  -% - -c5j) - -U5$u+#567 -E%e"456 - -!% - ! - - - - - -L(:'A>BDH . . z. 38n. % . eC)9$9:; . $E#/?*?$@A .  . . . . `%$&&%))-*.%)%)15%)*.'+:>9=;?)-(8(>DH'+%dddD>dtCH~& d "$ d c] dc]d-.dTNd!dtndU5$u+#567dE%e"456d!sJ!78d!d &!d"$E#/?*?$@A#d$!%%d& 'ddd'&dV(,8<26$'^9^9 ^9^9 ^9 !% ^9 U5#t#345 ^9./^9"^9^9^9^9^9^9^9^9r*rs)r^)/ functoolsrtypingrrnumpyrimage_processing_utilsrrr image_transformsr r r image_utilsr rrrrrrrrrrutilsrrrrr get_loggerrrrrrr3rHr:rSr]rqrs__all__r%r*r(rsn  """"""""UUUUUUUUUUSSSSSSSSSS                          _^^^^^^^^^^^JJJ  H % % 2S3SWX]^acf^fXgShD 3#sCx#CHo## #  38_ ####Lc#c#UX]b S S c3h DsTWY\^aOaIbDc     3 !$ QQc3hQQQ Q  38_ QQQQ>I9I9I9I9I9,I9I9I9X ! !r*