&`i0 2ddlZddlmZddlmZmZmZmZddlZ ddl m Z ddl m Z ddlmZmZerddlZ ddlZn #e$rdZYnwxYwdadZeGdd eeZeGd d eeZ dd e deddfdZ dd ddedede fdZe dd e defdZe dd ddedefdZd e dee jeee jfffdZ de jdedee jffdZ!de jde jfdZ"ddZ#ddZ$dS) N)Enum) TYPE_CHECKINGDictListUnion)TENSOR_COLUMN_NAME) DataBatchType) Deprecated DeveloperAPIc*tddl}|atS)Nr)_pandaspandas)rs v/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/air/util/data_batch_conversion.py_lazy_import_pandasrs  NceZdZdZdZdZdS) BatchFormatrarrownumpyN)__name__ __module__ __qualname__PANDASARROWNUMPYrrrr"s F E EEErrceZdZdZdZdZdZdS) BlockFormatz#Internal Dataset block format enum.rrsimpleN)rrr__doc__rrSIMPLErrrrr*s#-- F E FFFrrFdatacast_tensor_columnsreturn pd.DataFramect}t|tjr&|jt t |i}nt|tr|i}|D]T\}}t|tjs#tdt|d|dt |||<U|j|}nrt/t|tj r| }n> {!!! " " " t|   ! !9QQQT?++-- -K > >&*6l&;&;&=&= F##  " " " #  }((... QT Q Q- Q Q   rcXtjdtt||S)a5Convert the provided data to a Pandas DataFrame. This API is deprecated from Ray 2.4. Args: data: Data of type DataBatchType cast_tensor_columns: Whether tensor columns should be cast to NumPy ndarrays. Returns: A pandas Dataframe representation of the input data. z`convert_batch_type_to_pandas` is deprecated as a developer API starting from Ray 2.4. All batch format conversions should be done manually instead of relying on this API.r"r#)warningswarnPendingDeprecationWarningr=rKs rconvert_batch_type_to_pandasrOsA" M 8 "  ) ':   rcZtjdtt|||S)r?z`convert_pandas_to_batch_type` is deprecated as a developer API starting from Ray 2.4. All batch format conversions should be done manually instead of relying on this API.r"r4r#)rLrMrNrIrQs rconvert_pandas_to_batch_typerRsC" M 8 "  ) 2E   rct}t|tjr|St|trY|D]B\}}t|tjs#t dt|d|dC|Stt|tj rddl m }ddl m }g}|jD]A}||}|||d B|}|jt&gkr(t|jjd|r|dSt t-|j|St||jrt1|t2jSt d t|d t6) zConvert the provided data to a NumPy ndarray or dict of ndarrays. Args: data: Data of type DataBatchType Returns: A numpy representation of the input data. r'r(r)Nr),get_arrow_extension_fixed_shape_tensor_types)transform_pyarrowF)zero_copy_onlyr*r+)rr,r-r.r1r2r3r4r5r6$ray.air.util.tensor_extensions.arrowrTray.data._internal.arrow_opsrUrCcombine_chunked_arrayappendrE column_namesrschematypeszipr/rIrrr ) r"r9r;r<rTrUcolumn_values_ndarrayscombined_arrayarrow_fixed_shape_tensor_typess r_convert_batch_type_to_numpyrbs#   B$ ##+  D$  ) !ZZ\\  MHcc2:..   .23ii  BJ       D'-!@!@        CBBBBB!#<  C/DDSIIN " ) )!**>%*PP    *V)U)W)W&  !3 4 4 4 t{(+-K L L 5*!, ,C)+ABBCCC D", ' ' ,T;3DEEE "d4jj " " " "   rarrz pd.Seriesc~t} |j|S#t$rt|cYSwxYw)zConvert a NumPy ndarray into an appropriate column format for insertion into a pandas DataFrame. If conversion to a pandas Series fails (e.g. if the ndarray is multi-dimensional), fall back to a list of NumPy ndarrays. )rSeriesr3list)rcr9s rr0r0sS   Bry~~ Cyys  <<c|jjtjur0 tjd|D}n#t $rYnwxYw|S)zUnwrap an object-dtyped NumPy ndarray containing ndarray pointers into a single contiguous ndarray, if needed/possible. c6g|]}tj|Sr)r-asarray).0vs r z9_unwrap_ndarray_object_type_if_needed..s 777aBJqMM777r)dtyper4r-object_array Exception)rcs r%_unwrap_ndarray_object_type_if_neededrqsa y~## (77377788CC    D  Js9 AAdfct} |jjj}n#t$r|jj}YnwxYwddlm}m}| D]\}}||r tj 5tj dttj d|||||<dddn #1swxYwY|#t$r}td|d|d}~wwxYw|S)zY Cast all NumPy ndarray columns in df to our tensor extension type, TensorArray. r) TensorArraycolumn_needs_tensor_extensionignorecategoryNzTried to cast column z to the TensorArray tensor extension type but the conversion failed. To disable automatic casting to this tensor extension, set ctx = DataContext.get_current(); ctx.enable_tensor_extension_casting = False.)rcorecommonSettingWithCopyWarningAttributeErrorerrors%ray.air.util.tensor_extensions.pandasrtrur2rLcatch_warnings simplefilter FutureWarningrpr3)rrr9r{rtrur;r<es rrArAs   BB!#!F BBB!#!AB # ( ( - -   ,..44)(]KKKK)(=STTTT#.;s#3#3BxL444444444444444    CHCCC    $ IsI";;+C>AC > C C CC C C9 C44C9ct} |jjj}n#t$r|jj}YnwxYwddlm}|D]\}}t|j |rtj 5tj dttj d|t|||<dddn #1swxYwY|S)z:Cast all tensor extension columns in df to NumPy ndarrays.r) TensorDtypervrwN)rryrzr{r|r}r~rr2r,rmrLrrrrfrE)rrr9r{rr;r<s rr8r8IsM   BB!#!F BBB!#!ABBAAAAA  4 4 # ci - - 4 (** 4 4%hGGGG%h9OPPPP#CLLNN338  4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Is";;AC''C+ .C+ )F)rrr%r$r%)%rLenumrtypingrrrrrr-ray.air.constantsrray.air.data_batch_typer ray.util.annotationsr r rr9r5 ImportErrorr rstrrrboolr=rIrOrRr.rbr0rqrAr8rrrrs33333333333300000011111199999999NNNNGGG #t#t!&&& &&&&&&X!&, , , , ,  , , , , ^ !&  6 !&   68 8  2:tCO, ,-8 8 8 8 v BJ 5d2:>N1N+O     rz bj    ****Zs;AA