Pi,BUddlZddlZddlZddlmZmZddlmZddlm Z ddl m Z m Z m Z mZmZddlZddlZddlmZddlmZdd lmZdd lmZmZdd lmZmZm Z e r ddl!Z"d d l#m$Z$da%ee&e'e(d<ej)dkrdndZ*ej+dej+dej+dej+dej+dej+dej+dej+dej+dej+dej+dej+dej+dej+dgZ,eGd d!Z-d"e&e'fd#Z.d$d%d"e/fd&Z0d$d%d"e1fd'Z2d(ej3d"e1fd)Z4d*ee&e'e&e1e&ej3e&d%fd"e&e1fd+Z5dS),N) dataclassfield)BytesIO)Path) TYPE_CHECKINGAnyClassVarOptionalUnion)config)DownloadConfig) array_cast) is_local_pathxopen)first_non_null_valueno_op_if_value_is_nullstring_to_dict) FeatureType_IMAGE_COMPRESSION_FORMATSlittle<>z|b1|u1zu2zi2zu4zi4zf4zf8ceZdZUdZdZeeed<dZe ed<e ddZ eeed<d Z e eed <ejejejd Ze eed <e ddd Zeed<dZdeeeeeejd fdefdZddedd fdZdedeedfffdZdeejej ej!fdej fdZ"ddej dej fdZ#dS)Imagea=Image [`Feature`] to read image data from an image file. Input: The Image feature accepts as input: - A `str`: Absolute path to the image file (i.e. random access is allowed). - A `pathlib.Path`: path to the image file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the image file to the archive file. - `bytes`: Bytes of the image file. This is useful for parquet or webdataset files which embed image files. - An `np.ndarray`: NumPy array representing an image. - A `PIL.Image.Image`: PIL image object. Output: The Image features output data as `PIL.Image.Image` objects. Args: mode (`str`, *optional*): The mode to convert the image to. If `None`, the native mode of the image is used. decode (`bool`, defaults to `True`): Whether to decode the image data. If `False`, returns the underlying dictionary in the format `{"path": image_path, "bytes": image_bytes}`. Examples: ```py >>> from datasets import load_dataset, Image >>> ds = load_dataset("AI-Lab-Makerere/beans", split="train") >>> ds.features["image"] Image(decode=True, id=None) >>> ds[0]["image"] >>> ds = ds.cast_column('image', Image(decode=False)) {'bytes': None, 'path': '/root/.cache/huggingface/datasets/downloads/extracted/b0a21163f78769a2cf11f58dfc767fb458fc7cea5c05dccc0144a2c0f0bc1292/train/healthy/healthy_train.85.jpg'} ``` NmodeTdecodeF)defaultrepridPIL.Image.Imagedtypebytespathpa_type)r initr!_typec|jSN)r()selfs k/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/datasets/features/image.py__call__zImage.__call___s |valuereturnctjrddl}ntdt |t rt j|}t |tr|ddSt |tr$t| ddSt |ttfrd|dSt |t j rt|St ||jjrt!|S|d=t$j|drd|ddS|d|d+|d|ddSt+d|d ) aEncode example into a format for Arrow. Args: value (`str`, `np.ndarray`, `PIL.Image.Image` or `dict`): Data passed as input to Image feature. Returns: `dict` with "path" and "bytes" fields rN4To support encoding images, please install 'Pillow'.r'r&r'r%r&zUAn image sample should have one of 'path' or 'bytes' but they are missing or None in .)r PIL_AVAILABLE PIL.Image ImportError isinstancelistnparraystrrabsoluter& bytearrayndarrayencode_np_arrayrencode_pil_imagegetosr'isfile ValueError)r-r1PILs r.encode_examplezImage.encode_examplebs   V     TUU U eT " " $HUOOE eS ! ! !D11 1 t $ $  0 011DAA A y1 2 2  511 1 rz * * "5)) ) sy / / #E** * YYv   *rw~~eFm/L/L *!599V+<+<== = YYw   +uyy/@/@/L"YYw//69J9JKK Kphmppp r0c|jstdtjr ddl}ddl}nt d|i}|d|d}}|7|td|dt|r|j |}n| d d }| tj r tjn tj}t!||} | || d nd} t%| } t'|d | 5} t)| }dddn #1swxYwY|j |}n'|j t)|}|||j jjj|j|}|jr*|j|jkr||j}|S)aqDecode example image file into image data. Args: value (`str` or `dict`): A string with the absolute image file path, a dictionary with keys: - `path`: String with absolute or relative image file path. - `bytes`: The bytes of the image file. token_per_repo_id (`dict`, *optional*): To access and decode image files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`). Returns: `PIL.Image.Image` zMDecoding is disabled for this feature. Please use Image(decode=True) instead.rNz4To support decoding images, please install 'Pillow'.r'r&zCAn image should have one of 'path' or 'bytes' but both are None in r6::repo_idtokenrbdownload_config)r RuntimeErrorr r7r8 PIL.ImageOpsr9rGrropensplit startswith HF_ENDPOINTHUB_DATASETS_URLHUB_DATASETS_HFFS_URLrrDrrrreadloadgetexifExifTagsBase OrientationImageOpsexif_transposerconvert) r-r1token_per_repo_idrHr'bytes_image source_urlpatternsource_url_fieldsrOrRfs r.decode_examplezImage.decode_examplesn${ pnoo o   V         TUU U  $ " V}eGnf >| !ogl!o!o!oppp &&3INN400EE!%D!1!1"!5J&001CDD://#9 )7z7(K(K%O`Ol)--.? .JKKKrv'55&A&A&AOtT?KKK3q!(!2!2333333333333333INN622EEINN76??33E ==??  sy16B C C OL//66E 9 -ej00MM$),,E s5"E##E'*E'rcNddlm}|jr|n|d|ddS)zfIf in the decodable state, return the feature itself, otherwise flatten the feature into a dictionary.r)Valuebinarystringr%)featuresrmr)r-rms r.flattenz Image.flattensK######{ DDxh r0storagectj|jrR |tj}n*#tj$r}td||d}~wwxYwtj|jrrtj dgt|ztj }tj ||gddg|}ntj|jrrtj dgt|ztj}tj ||gddg|}n tj|jr|jddkr|d}n8tj dgt|ztj }|jddkr|d}n8tj dgt|ztj}tj ||gddg|}ntj|jrtj d|Dtj }tj dgt|ztj}tj ||gddg|}t+||jS) aCast an Arrow array to the Image arrow storage type. The Arrow types that can be converted to the Image pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.large_string()` - it must contain the "path" data (will be cast to string if possible) - `pa.binary()` - it must contain the image bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter - `pa.list(*)` - it must contain the image array data Args: storage (`Union[pa.StringArray, pa.StructArray, pa.ListArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Image arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. zvFailed to cast large_string to string for Image feature. This can happen if string values exceed 2GB. Original error: Ntyper&r'maskrcdg|]-}|'ttj|dnd.S)Nr&)rBr<r=).0arrs r. z&Image.cast_storage..s8uuuZ]CO#//88QUuuur0)patypesis_large_stringrucastro ArrowInvalidrG is_stringr=lenrn StructArray from_arraysis_null is_binary is_structget_field_indexris_list to_pylistrr()r-rre bytes_array path_arrays r. cast_storagezImage.cast_storages=( 8 # #GL 1 1  !,,ry{{33?    +'(++   8  gl + + (D6CLL#8ry{{KKKKn00+w1G'SYIZahapaparar0ssGG X   - - 4&3w<<"7bikkJJJJn00':1FRXHY`g`o`o`q`q0rrGG X   - - |++G4499%mmG44  hvG '<29;;OOO |++F33q88$]]622 Xtfs7||&;")++NNN n00+z1JWV\L]dkdsdsdudu0vvGG X  gl + + (uuahararatatuuuY[[K4&3w<<"7bikkJJJJn00j)GV+<;CVCVCXCX1G'4<000s&A A4A//A4citfdtjfd|Dtj}tjd|dDtj}tj||gddg| }t||j S) a8Embed image files into the Arrow array. Args: storage (`pa.StructArray`): PyArrow array to embed. Returns: `pa.StructArray`: Array in the Image arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. Nc|dd}|tjr tjn tj}t ||}||dnd}t|}t|d|5}| cdddS#1swxYwYdS)NrKrLrMrNrPrQ) rVrWr rXrYrZrrDrrr[)r'rgrhrirOrRrjrds r. path_to_bytesz*Image.embed_storage..path_to_bytess D))"-J+5+@+@AS+T+Tv''Z`Zv !/z7 C C K\Kh%))*;I*FGGGnrE,5999OtT?CCC qvvxx                  s%CC  C cZg|]'}|!|d|dn |dnd(S)Nr&r')ryxrs r.r{z'Image.embed_storage..(sU   UVTaQwZ-?qy)))QwZZgk   r0rtcTg|]%}|tj|nd&Sr,)rEr'basename)ryr's r.r{z'Image.embed_storage../s2 p p pdt'7RW  d # # #T p p pr0r'r&rv) rr|r=rrnrrorrrrr()r-rrrdrrrs ` @r. embed_storagezImage.embed_storage s  $ "     h     **,,       X p pgmm\bNcNcNmNmNoNo p p p   .,,k:-FRXHY`k`s`s`u`u,vv'4<000r0r,)$__name__ __module__ __qualname____doc__rr r>__annotations__rboolrr"r$r r|structrnror(rr*r/r r&r@dictr<rArIrkrq StringArrayr ListArrayrrrr0r.rr.s%%ND(3-FDd777B 777,E8C=,,,&RYibikk'R'RSSGXc]SSSwU???E3???'E#uirzSd*d$e'jn''''R;;D;EV;;;;z  }d3 3E.FFG     61E".".",*V$W61\^\j61616161p&1&1R^&1PRP^&1&1&1&1&1&1r0rr2cXtjrddl}ntdt}|jtt|jj t|jj zatS)Nrr4) r r7r8r9rrr)r;setOPENkeysSAVE)rHs r.list_image_compression_formatsr6s RPQQQ") %)#cin.A.A.C.C*D*Ds39>K^K^K`K`GaGa*a%b%b" %%r0rfr#ct}|jtvr|j}n |jdvrdnd}||||S)zmConvert a PIL Image object to bytes using native compression if possible, otherwise use PNG/TIFF compression.)1LLARGBRGBAPNGTIFF)format)rrrrsavegetvalue)rfbufferrs r.image_to_bytesrCsd YYF |57777*(GGGV JJvfJ%%% ??  r0cpt|dr|jdkr |jddSdt|dS)Nfilenamer5)hasattrrr)rfs r.rCrCNsFuj!!>en&:&:666~e'<'<===r0r=ctjrddl}ntd|j}|jdkr|jnt }|j}|j}d}|j ddrP|dvrtd|d|dtjd }||krtj d |d |d n|tvr|}n|d krk||zt|z}tj|tvr0tj|}tj d |d |d n |dz}|d kk|td|dt|j||}dt'|dS)Nrr4=r )uizUnsupported array dtype z for image encoding. Only z' is supported for multi-channel arrays.rzDowncasting array dtype z to z to be compatible with 'Pillow'rzCannot downcast dtype z- to a valid image dtype. Valid image dtypes: r5)r r7r8r9r$ byteorder_NATIVE_BYTEORDERkinditemsizeshape TypeErrorr<warningswarn_VALID_IMAGE_ARRAY_DTPYESr>r fromarrayastyper) r=rHr$dtype_byteorder dtype_kinddtype_itemsize dest_dtype dtype_strrfs r.rBrBUs RPQQQ KE).C)?)?eooEVOJ^NJ {122 Z ' '5J Xe__ J   MkUkk kkk l l l + + + !!'*4s>7J7JJIx ""&???Xi00  oooJoooppp1$!!  xxx]vxx  I   Z 8 8 9 9E>%#8#8 9 99r0objsctjrddl}ntd|rt |\}}t |t r d|DSt |tjr"ttfd|DSt ||j j r"ttfd|DS|S|S)zmEncode a list of objects into a format suitable for creating an extension array of type `ImageExtensionType`.rNr4c g|] }||ddnd S)Nr5r)ryobjs r.r{z2objects_to_list_of_image_dicts..s*^^^RUCOS4000^^^r0c&g|] }|Srrryrobj_to_image_dict_funcs r.r{z2objects_to_list_of_image_dicts..%@@@C**3//@@@r0c&g|] }|Srrrs r.r{z2objects_to_list_of_image_dicts..rr0) r r7r8r9rr:r>r<rArrBrrC)rrH_rrs @r.objects_to_list_of_image_dictsrsRPQQQ %d++3 c3   _^^Y]^^^ ^ c2: & & %;O%L%L "@@@@4@@@ @ SY_ - - %;rrrr$rrrr&rrrCrArBrrr0r.rsB ((((((((@@@@@@@@@@@@@@55555533333333YYYYYYYYYY&%%%%%%37HT#Y/666=H44CC# BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO BHUOO$ D1D1D1D1D1D1D1 D1N &S & & & &+>->$>>>>(:2:(:$(:(:(:(:V S 4:tBJ'7>O9PP Q $Zr0