`iH dZddlmZddlmZmZddlZddlm Z m Z m Z ddl m Z mZmZddlmZmZmZmZmZmZmZmZmZmZmZdd lmZmZmZm Z m!Z!m"Z"dd l#m$Z$e rddl%Z%erddl&Z&e!j'e(Z)d Z* dd ej+d ee,dee,deeee,ffdZ-e$dGdde Z.dgZ/dS)z%Image processor class for LayoutLMv3.)Iterable)OptionalUnionN)BaseImageProcessor BatchFeature get_size_dict)resizeto_channel_dimension_format to_pil_image) IMAGENET_STANDARD_MEANIMAGENET_STANDARD_STDChannelDimension ImageInputPILImageResamplinginfer_channel_dimension_formatis_scaled_imagemake_flat_list_of_imagesto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypefilter_out_non_signature_kwargsis_pytesseract_availableis_vision_availableloggingrequires_backends)requiresctd|d|z ztd|d|z ztd|d|z ztd|d|z zgS)Nirr)int)boxwidthheights /home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/transformers/models/layoutlmv3/image_processing_layoutlmv3.py normalize_boxr':sl DCFUN #$$ DCFVO $%% DCFUN #$$ DCFVO $%%  imagelangtesseract_configinput_data_formatc,t||}|j\}}tj||d|}|d|d|d|d|df\}} } } } d t |Dfd t |D}fd t | D} fd t | D} fd t | D} fdt | D} g} t | | | | D](\}}}}||||z||zg}| |)g}| D]&}|t|||'t|t|ks Jd||fS)zdApplies Tesseract OCR on a document image, and returns recognized words + normalized bounding boxes.r,dict)r* output_typeconfigtextlefttopr$r%c@g|]\}}||S)strip).0idxwords r& z#apply_tesseract..Rs)TTT)#ttzz||T#TTTr(c"g|] \}}|v | Sr6r6)r8r9r:irrelevant_indicess r&r;z#apply_tesseract..Ss( U U Uic4sBT7T7TT7T7T7Tr(c"g|] \}}|v | Sr6r6r8r9coordr=s r&r;z#apply_tesseract..Ts( U U Ujc5sBT7T7TE7T7T7Tr(c"g|] \}}|v | Sr6r6r?s r&r;z#apply_tesseract..Us( S S SZS%S@R5R5R55R5R5Rr(c"g|] \}}|v | Sr6r6r?s r&r;z#apply_tesseract..Vs( W W WzsEDV9V9VU9V9V9Vr(c"g|] \}}|v | Sr6r6r?s r&r;z#apply_tesseract..Ws( Y Y Y U3FX;X;Xe;X;X;Xr(z-Not as many words as there are bounding boxes) r size pytesseract image_to_data enumeratezipappendr'len)r)r*r+r, pil_image image_width image_heightdatawordsr3r4r$r% actual_boxesxywh actual_boxnormalized_boxesr#r=s @r&apply_tesseractrWCsU6GHHHI )K  $YTvVf g g gD&*6lDL$u+tT[}^bck^l&l#E4eVUTy/?/?TTT U U U U9U#3#3 U U UE U U U UIdOO U U UD S S S S9S>> S S SC W W W WYu%5%5 W W WE Y Y Y Yi&7&7 Y Y YFL$UF33(( 1aAE1q5) J''''OO c; M MNNNN u::-.. . . .0_ . . . " ""r()vision)backendsc!&eZdZdZdgZddejddddddddf dedee e e fd ed ed e d ed ee e ee fdee e ee fdedee dee ddffd Zejddfdejde e e fd edee e efdee e efdejf dZeddddddddddddejdfdedeedee e e fd eedee d eed ee e ee fdee e ee fdeedee dee dee e efdedee e efdejjfdZxZS)LayoutLMv3ImageProcessora Constructs a LayoutLMv3 image processor. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image's (height, width) dimensions to `(size["height"], size["width"])`. Can be overridden by `do_resize` in `preprocess`. size (`dict[str, int]` *optional*, defaults to `{"height": 224, "width": 224}`): Size of the image after resizing. Can be overridden by `size` in `preprocess`. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`): Resampling filter to use if resizing the image. Can be overridden by `resample` in `preprocess`. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image's pixel values by the specified `rescale_value`. Can be overridden by `do_rescale` in `preprocess`. rescale_factor (`float`, *optional*, defaults to 1 / 255): Value by which the image's pixel values are rescaled. Can be overridden by `rescale_factor` in `preprocess`. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. Can be overridden by the `do_normalize` parameter in the `preprocess` method. image_mean (`Iterable[float]` or `float`, *optional*, defaults to `IMAGENET_STANDARD_MEAN`): Mean to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`Iterable[float]` or `float`, *optional*, defaults to `IMAGENET_STANDARD_STD`): Standard deviation to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_std` parameter in the `preprocess` method. apply_ocr (`bool`, *optional*, defaults to `True`): Whether to apply the Tesseract OCR engine to get words + normalized bounding boxes. Can be overridden by the `apply_ocr` parameter in the `preprocess` method. ocr_lang (`str`, *optional*): The language, specified by its ISO code, to be used by the Tesseract OCR engine. By default, English is used. Can be overridden by the `ocr_lang` parameter in the `preprocess` method. tesseract_config (`str`, *optional*): Any additional custom configuration flags that are forwarded to the `config` parameter when calling Tesseract. For example: '--psm 6'. Can be overridden by the `tesseract_config` parameter in the `preprocess` method. pixel_valuesTNgp? do_resizerDresample do_rescale rescale_value do_normalize image_mean image_std apply_ocrocr_langr+returnc  (tjdi| ||nddd}t|}||_||_||_||_||_||_||nt|_ ||nt|_ | |_ | |_| |_dS)N)r%r$r6)super__init__r r^rDr_r`rescale_factorrbr rcrrdrerfr+)selfr^rDr_r`rarbrcrdrerfr+kwargs __class__s r&rkz!LayoutLMv3ImageProcessor.__init__s ""6"""'ttc-J-JT"""   $+((2(>**DZ&/&;AV"  0r(r) data_formatr,c t|}d|vsd|vr$td||d|df}t|f||||d|S)a Resize an image to `(size["height"], size["width"])`. Args: image (`np.ndarray`): Image to resize. size (`dict[str, int]`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`): `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BILINEAR`. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. Returns: `np.ndarray`: The resized image. r%r$zFThe `size` dictionary must contain the keys `height` and `width`. Got )rDr_rpr,)r ValueErrorkeysr )rmr)rDr_rpr,rn output_sizes r&r zLayoutLMv3ImageProcessor.resizesFT"" 4  7$#6#6sfjfofofqfqsstt tH~tG}5   #/       r(imagesrlreturn_tensorsc  ||nj}njtnj||nj}nj||nj}nj  nj | | nj } | | nj } | | nj } t|}t|stdt|| |d|D}|r/t!|drt"dt'|d| rYt)dg}g}|D]B}t+|| | \}}||||C|rfd |D}|rfd |D}|r fd |D}fd |D}t/d |i| }| r ||d<||d<|S)a% Preprocess an image or batch of images. Args: images (`ImageInput`): Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`dict[str, int]`, *optional*, defaults to `self.size`): Desired size of the output image after applying `resize`. resample (`int`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of the `PILImageResampling` filters. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image pixel values between [0, 1]. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to apply to the image pixel values. Only has an effect if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `Iterable[float]`, *optional*, defaults to `self.image_mean`): Mean values to be used for normalization. Only has an effect if `do_normalize` is set to `True`. image_std (`float` or `Iterable[float]`, *optional*, defaults to `self.image_std`): Standard deviation values to be used for normalization. Only has an effect if `do_normalize` is set to `True`. apply_ocr (`bool`, *optional*, defaults to `self.apply_ocr`): Whether to apply the Tesseract OCR engine to get words + normalized bounding boxes. ocr_lang (`str`, *optional*, defaults to `self.ocr_lang`): The language, specified by its ISO code, to be used by the Tesseract OCR engine. By default, English is used. tesseract_config (`str`, *optional*, defaults to `self.tesseract_config`): Any additional custom configuration flags that are forwarded to the `config` parameter when calling Tesseract. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. NzkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.)r`rlrbrcrdr^rDr_c,g|]}t|Sr6)r)r8r)s r&r;z7LayoutLMv3ImageProcessor.preprocess..Hs <<._s> %dXYj kkr(c@g|]}|S))r)scaler,)rescale)r8r)r,rlrms r&r;z7LayoutLMv3ImageProcessor.preprocess..es< 5Rc ddr(cBg|]}|S))r)meanstdr,) normalize)r8r)rcrdr,rms r&r;z7LayoutLMv3ImageProcessor.preprocess..ks>U^oppr(c4g|]}t|S))input_channel_dim)r )r8r)rpr,s r&r;z7LayoutLMv3ImageProcessor.preprocess..ps7   ej '{N_ ` ` `   r(r\)rN tensor_typerOboxes)r^rDr r_r`rlrbrcrdrerfr+rrrrrrlogger warning_oncerrrWrIr)rmrur^rDr_r`rlrbrcrdrerfr+rvrpr, words_batch boxes_batchr)rOrrNs` `` ` `` `` r& preprocessz#LayoutLMv3ImageProcessor.preprocesss9L"+!6IIDN 'ttTYT""'388#-#9ZZt +9+E4K^'3'?||TEV #-#9ZZt !*!6IIDN !*!6IIDN '388/?/K++QUQf)&11F## :  &!)%! =vay I I   * dM 2 2 2KK * *.uh@Pduvvv u""5)))""5))))  #F  #F  #F      nt   .&!9~VVV  ('DM'DM r()__name__ __module__ __qualname____doc__model_input_namesrBILINEARboolrr/strr"floatrrrknpndarrayrr rFIRSTrrPILImager __classcell__)ros@r&r[r[is9$$L(()-'9'B&!>B=A"&*,111tCH~&1% 1  1  11U5(5/#9:;1E%%"89:113-1#3-1 111111H(:'B>BDH . . z. 38n. % . eC)9$9:; . $E#/?*?$@A .  . . . . `%$&&%))-%)*.'+>B=A$("&*.;?(8(>DH!UUUD>UtCH~& U TN U!UtnUU5(5/#9:;UE%%"89:UD>U3-U#3-U!sJ!78U&U $E#/?*?$@A!U" #UUU'&UUUUUr(r[)N)0rcollections.abcrtypingrrnumpyrimage_processing_utilsrrr image_transformsr r r image_utilsr rrrrrrrrrrutilsrrrrrrutils.import_utilsrrrE get_loggerrrr'rrrWr[__all__r6r(r&rs|,+$$$$$$""""""""UUUUUUUUUUQQQQQQQQQQ                          +*****JJJ  H % %AE #### :## 3-##sm## &6&; <= ########L ;OOOOO1OO Od & &r(