Pi~VdZddlmZddlmZddlmZmZddlZ ddlm Z ddl m Z ddl mZdd lmZdd lmZmZmZmZdd lmZmZmZmZmZmZdd lmZmZdd l m!Z!m"Z"ddl#m$Z$ddl%m&Z&m'Z(ddl)m*Z*ddl+m,Z,ddl-m.Z/m0Z0m1Z1m2Z2ddl3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;ddlZ>m?Z?m@Z@ddlAmBZBmCZCerddlDmEZEmFZFddlGmHZHmIZImJZJmKZKddddd[d#ZLd\d'ZMd]d(ZNd^d-ZOddd.d_d0ZPd`d4ZQdad5ZRdbdcd6ZSddd7ZTded9ZUdfd:ZVdgdAZWdfdBZXdhdFZYdGZZ didjdHZ[dkdJZ\dldMZ]dmdOZ^dndQZ_dodSZ` dpdqdZZadS)rz~ Functions for preparing various inputs passed to the DataFrame or Series constructors before passing them to a BlockManager. ) annotations)abc) TYPE_CHECKINGAnyN)ma)using_string_dtype)lib)astype_is_view)"construct_1d_arraylike_from_scalar dict_compatmaybe_cast_to_datetimemaybe_convert_platform)is_1d_only_ea_dtypeis_integer_dtype is_list_likeis_named_tupleis_object_dtype is_scalar)BaseMaskedDtypeExtensionDtype) ABCDataFrame ABCSeries)isna) algorithmscommon)ExtensionArray) StringDtype)array extract_arrayrange_to_ndarraysanitize_array) DatetimeIndexIndexTimedeltaIndex default_index ensure_indexget_objs_combined_axismaybe_sequence_to_range union_indexes)BlockPlacementensure_block_shape new_block new_block_2d) create_block_manager_from_blocks'create_block_manager_from_column_arrays)HashableSequence) ArrayLikeDtypeObjManagernptT)dtypeverify_integrity consolidatecolumnsr#r6DtypeObj | Noner7boolr8returnr4c8|r6|t|}nt|}t|||\}}nt|}d|D}dgt|z}|D]]}t |t jtfr+|jdks t|t|krtd^t|}t|t|krtd||g}t||||S)zs Segregate Series based on type and coerce into matrices. Needs to handle a lot of exceptional cases. Nc0g|]}t|dS)T extract_numpy)r.0xs v/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/pandas/core/internals/construction.py z!arrays_to_mgr..~s%GGG1-666GGGzYArrays must be 1-dimensional np.ndarray or ExtensionArray with length matching len(index)z#len(arrays) must match len(columns))r8refs) _extract_indexr& _homogenizelen isinstancenpndarrayrndim ValueErrorr/) arraysr9indexr6r7r8rHarraxess rD arrays_to_mgrrU_sC ="6**EE ''E#65%88 U##GGGGGvF #  CsRZ$@AA 8q==s88s5zz)) 6*7##G 7||s6{{"">??? U D 2+D   rFdatanp.rec.recarray | np.ndarraycopycttj|}|tt|}nt |}|t |}t ||\}}t |||t|\}}||}t||||}|r|d}|S)zA Extract from a masked rec array and create the manager. Nr6Tdeep) rgetdatar%rKr& to_arraysreorder_arraysrUrX) rVrRr9r6rXfdatarQ arr_columnsmgrs rDrec_array_to_mgrrcs Jt  E }c%jj))U##w''#E733FK)gs5zzRRFK e < < tQtSt|} tU| |} | g} t|d krg} t_| ||gdS)N)rrGrZr6rGc(g|]}dd|fSN)rBnvaluess rDrEz"ndarray_to_mgr..s5qqq!t rFTr?F)rXorderrX)r6rXallow_2dr)rRr9c dg|]-}tj|dddtjd.S)FTM8[s]convert_numericconvert_non_numericconvert_to_nullable_dtypedtype_if_all_nat)r maybe_convert_objectsrMr6rAs rDrEz"ndarray_to_mgr..'sW      %!&$(*/!#'!2!2       rFc3$K|] \}}||uV dSrfrg)rBrCys rD z!ndarray_to_mgr..4s*TTdaqzTTTTTTrF)strictclg|]1\}}tt|dt|2S)) placement)r-r+r*)rBrhdvals rDrEz"ndarray_to_mgr..5sMAt/a88NSTDUDUVVVrF)r}rHU)na_valuec g|]\\}}t|tt ||dzd]S)rZrGr|)rO)r,construct_array_type_from_sequencer*slice)rBirVr6s rDrEz"ndarray_to_mgr..Asz   4 **,,;;D;NNuQA//      rFF)r7)0rLrr#rnamerRreindexrKrMemptyobjectgetattrrrNrOrangeshaper&rUrrrXreshaper r6 _references_values _ensure_2drasarray_prep_ndarrayliker! _get_axes!_check_values_indices_shape_matchTrlistanyzip enumerater*rr-kindrrnanr.) rirRr9r6rX infer_objectvdtyperH obj_columnsmaybe_datetime block_valuesbpnbs ` ` rDndarray_to_mgrrs "&9e^*LMMML&)$$ 4 ?{& .. =LEE^^E**F6{{ 4w2s7||2XfF333F VWd + +F D6""76&9%&@&@76 frz>: ; ;  a v|A//FF XF ?E#f++..//GG"7++GVWe5AAAA FN + +!6vT:::  #[[]]F ;!  ^^B**F FY. / /6 &.u*M*M%D  $^((**FF^FF## FRZ8 9 9 6  (U]nV\5&I&I]Xf4s;;;FFZ''FF## #6555 V\U22       QawNE7&feW=== XF  }}/&,*G*G}6ll    !     TTc+~d&S&S&STTT T T (88LL  c'll 3 344Bf>>>B4LL 6<,338J8L8L3RV,,,6ll     %[11    E#g,,// 0 0 &BT : : :t  7||q +w&   rFri np.ndarrayrRNonec`|jdt|ks|jdt|krq|jddcxkrt|krnntd|j}t|t|f}td|d|dS)z\ Check that the shape implied by our axes matches the actual shape of the data. rGrz)Empty data passed with indices specified.zShape of passed values is z, indices imply N)rrKrP)rirRr9passedimplieds rDrrXs|A#g,,&&&,q/SZZ*G*G <?a , , , ,#e** , , , , ,HII Iu::s7||,WfWWgWWXXX+H*GrFr6rXdictc|t|}|3t|tjs|jgt |z}ntjgt |z}tt}t }t|D]\}} | | } n%#t$r|Y=wxYw|| } | ||<t| r$t!| r||;r)t#fdt|D}nt#|}nt|}rSt%|sDD]A}t'||t |||ntjd} | ||<Bn\t)t} | rt+| nt-d}fd| D}|r d|D}t/|||||S)z Segregate Series based on type and coerce into matrices. Needs to handle a lot of exceptional cases. Used in DataFrame.__init__ Nc"g|] \}}|v | Srgrg)rBrrmidxss rDrEzdict_to_mgr..s"OOOxq%UrFrrcDg|]}tj|Srg)commaybe_iterable_to_listrBkrVs rDrEzdict_to_mgr..s(DDD!#,T!W55DDDrFc g|]}t|tr|nUt|ts)t|tr*t |jr|dn|S)Tr[)rLrrXr#rrr6rAs rDrEzdict_to_mgr..s   a00  #1e,,'q)44:MQW9U9UAFFF%%%    rF)r6r8)r&rLrMr6rrKrsetkeysrrirget_locKeyErroraddrrrIrr r(r#r%rU)rVrRr9r6rXrQ data_keys data_valuesrcolumnidxrrSrrs` @rD dict_to_mgrrjs w''  Zrx%@%@ n%G 4FFfXG ,F -- 4;;==)) "7++  IAv ''//    !   $EF1I DKK  ! = /&OOOO9V+<+<OOO'v.. ''E  )%00   81IJJ".EEBHX4F4F  q 'tDIIKK'8'899!%;%+++=+;+;DDDDtDDD        %u$ O O OOs C""DDr1 Index | None$tuple[list[ArrayLike], Index, Index]cHt|dr|t|dj}t|||\}}t|}|Gt |dt rt |}ntt|}|||fS)zA Convert a single sequence of arrays to multiple arrays. rNrZ) rr&_fieldsr^rLr_get_names_from_indexr%rK)rVr9rRr6rQs rDnested_data_to_arraysrsd1g07?tAw//gU;;;OFG7##G } d1gy ) ) -)$//EE!#d)),,E 7E !!rFct|dkoPt|do;t|ddddko t|to |jdk S)z7 Check if we should use nested_data_to_arrays. rrOrGr|)rKrrrLrrO)rVs rDtreat_as_nestedrsl D A  F a ! ! F DGVQ ' '1 , FD.11Ddi1n E rFct|dkrtjdtSt |t r#t |}|dtjfSdt|dr!tj fd|D}n]t |dtj r2|dj dkr!tj fd|D}n |}t|S)Nr)rrrZ.ct|rt|tr|St|d}t |}|S)NTr?)rrLrrr)vress rDconvertz"_prep_ndarraylike..convertsLA *Q "="= H !4 0 0 0$Q'' rFc&g|] }|SrgrgrBrrs rDrEz%_prep_ndarraylike..!666!771::666rFc&g|] }|Srgrgrs rDrEz%_prep_ndarraylike.. rrF) rKrMrrrLrr newaxisrrrNrOr)rirXrSrs @rDrrs 6{{axf---- FE " "$v&&3 ?##F1I!6666v66677 F1Irz * *!vay~/B/B6666v66677 f  rFc|jdkr#||jddf}n"|jdkrtd|j|S)zB Reshape 1D values, raise on anything else other than 2D. rGrr|zMust pass 2-d input. shape=)rOrrrP)ris rDrrsZ{aa! 455   Ev|EEFFF MrF!tuple[list[ArrayLike], list[Any]]cd}g}g}|D]j}t|ttfrl|||}t|tr|j|ur||}||j|j}nt|trx||d}t|ttfrt|}nt|}tj||jtj}t#|||d}t%j|||d||l||fS)NO)defaultFr)rLrr#astyperRrappendrrrr"r$r r fast_multigetrMrr!rrequire_length_match)rVrRr6oindex homogenizedrHvals rDrJrJstFKD   cIu- . .  jj''#y)) )ciu.D.Dkk%(( KK ( ( (+CC#t$$ M>"\\#..Fem^%DEE$%c**CCs))C'V^RVLLL e5uEEEC  $S% 0 0 0 KK   3  rFc$t|dkrtdSt}g}d}d}d}|D]}t|trd}||j4t|tr7d}|t| t|r:t|dddkr%d}| t|t|tjr|jdkrt!d|s|st!d|rt#|}n|rt#|d}|rt|dkrt!d |rt!d |}|r8|t|kr$d |d t|} t!| nt|}t'|S) zR Try to infer an Index from the passed data, raise ValueError on failure. rFTrOrGz,Per-column arrays must each be 1-dimensionalz2If using all scalar values, you must pass an indexsortz%All arrays must be of the same lengthz.s3KK6400<KKKKKKrFrrzUnnamed rG)rr%rKrrrrr#)rV has_some_namerRcountrrrhs rDrrsKKdKKKKKM (SYY''' s4yy!1!122E E$1 Avt $ $ =E!HH)%))E!H QJEE <<rFNKtuple[Index, Index]c|t|}nt|}|t|}nt|}||fSrf)r%r&)rrrRr9s rDrrsP  }a  U##""w'' '>rFcHddlm}tt||S)a Converts a list of dataclass instances to a list of dictionaries. Parameters ---------- data : List[Type[dataclass]] Returns -------- list_dict : List[dict] Examples -------- >>> from dataclasses import dataclass >>> @dataclass ... class Point: ... x: int ... y: int >>> dataclasses_to_dicts([Point(1, 2), Point(2, 3)]) [{'x': 1, 'y': 2}, {'x': 2, 'y': 3}] r)asdict) dataclassesrrmap)rVrs rDdataclasses_to_dictsrs.0#""""" FD!! " ""rFctsttjryjjmt jj}fd|D}tdkr/t|D]\}}|jdkr|dddf||< ||fSgt gfSttjr9jj-|tjj}fd|D}||fStdttfrt}n~tdtjrt|\}}nJtdt rt#|\}}ndDt}t%|||\}}||fS)a  Return list of arrays, columns. Returns ------- list[ArrayLike] These will become columns in a DataFrame. Index This will become frame.columns. Notes ----- Ensures that len(result_arrays) == len(result_index). Nc g|] }| Srgrg)rBrrVs rDrEzto_arrays..s999$t*999rFrr|c g|] }| Srgrgrs rDrEzto_arrays..s+++a$q'+++rFc,g|]}t|Srg)tuplerAs rDrEzto_arrays..s'''Qa'''rF)rKrLrMrNr6namesr&rrOr#rr_list_to_arraysrMapping_list_of_dict_to_arraysr_list_of_series_to_arrays_finalize_columns_and_data)rVr9r6rQrrScontents` rDr^r^s$ t99 dBJ ' ' 'z+&tz'7889999999t99>>#,F"3"32238q==(+AAAqD F1Iw&<#### D"* % %$**:*F ?DJ,--G++++7+++w$q'D%=)) $d## DGS[ ) )$.tW== WW DGY ' '$0w?? WW('$'''d##1#wFFGW G rFlist[tuple | list]ct|dtrtj|}ntj|}|S)Nr)rLrr to_object_array_tuplesto_object_array)rVr s rDrrsC$q'5!!,,T22%d++ NrFrtuple[np.ndarray, Index]c|d|D}t|d}i}g}|D]}t|dd}|tt|}t ||vr|t |}n'||x}|t |<t |d}|tj ||tj |} | |fS)NcJg|] }t|ttf|!Srg)rLrrrAs rDrEz-_list_of_series_to_arrays..,s,QQQ1 1y,6O(P(PQQQQQrFFrrRTr?) r'rr%rKidrrrrtake_ndrMvstack) rVr9 pass_data indexer_cachealigned_valuesrrRrrir s rDrr$s QQQQQ (???+-MN  C C7D)) =!#a&&))E e99 % %#BuII.GG161B1B71K1K KGmBuII.q555j0AABBBBi''G G rF list[dict]c|Kd|D}td|D }tj||}t|}d|D}tj|t |}||fS)a Convert list of dicts to numpy arrays if `columns` is not passed, column names are inferred from the records - for OrderedDict and dicts, the column names match the key insertion-order from the first record to the last. - For other kinds of dict-likes, the keys are lexically sorted. Parameters ---------- data : iterable collection of records (OrderedDict, dict) columns: iterables or None Returns ------- content : np.ndarray[object, ndim=2] columns : Index Nc3XK|]%}t|V&dSrf)rrrAs rDryz*_list_of_dict_to_arrays..[s0,,!tAFFHH~~,,,,,,rFc3@K|]}t|tVdSrf)rLrrBds rDryz*_list_of_dict_to_arrays..\s,99qz!T**999999rFrc\g|])}t|tur|nt|*Srg)typerrs rDrEz+_list_of_dict_to_arrays..bs0 < < >> W s' AAAlist[np.ndarray]cZ|tt|}n t|tot d|D}|sOt|t|kr/t t|dt|d|rtd|Ddkrt d|r[t|dt|kr5t t|ddt|d|S) a If columns is None, make numbers as column names; Otherwise, validate that columns have valid length. Parameters ---------- content : list of np.ndarrays columns : Index or None Returns ------- Index If columns is None, assign positional column index value as columns. Raises ------ 1. AssertionError when content is not composed of list of lists, and if length of columns is not equal to length of content. 2. ValueError when content is list of lists, but length of each sub-list is not equal 3. ValueError when content is list of lists, but length of sub-list is not equal to length of content Nc3@K|]}t|tVdSrf)rLrrBcols rDryz0_validate_or_indexify_columns..s=7 7 &)JsD ! !7 7 7 7 7 7 rFz! columns passed, passed data had z columnsc,h|]}t|Srg)rKr.s rD z0_validate_or_indexify_columns..s000CHH000rFrGz.converts BHSMM ! !(72Xj6X6XK+&*5C}9 --0=0H-3).,02K)+'):):C1KSY"(3--5O5O$/MM "+"@"@"B"B%44S 4JJ"g--*S"*2M2M-y~//&s777E>22 90022((E(FFt## -S%88 rFc&g|] }|Srgrg)rBrSrs rDrEz(convert_object_array..s! . . .sggcll . . .rFrg)r r6r6r8rQrs ``` @rDr(r(sJ,4444444l/ . . .g . . .F MrF) r9r#r6r:r7r;r8r;r<r4)rVrWr6r:rXr;r<r4)r6r:rXr;r<r4)rirrRr#r9r#r<r)rVrr6r:rXr;r<r4) rVr1r9rrRrr6r:r<r)r<r;)T)rXr;r<r)rirr<r)rRr#r6r:r<r)r<r#) rQrrar#r9rrrr<r) rrrrrRrr9rr<rrf)r9rr6r:r<r)rVr r<r)rVrr9rr<r)rVrr9rr<r)r rr9rr6r:r<r)r r+r9rr<r#)r4F) r r5r6r:r6r7r8r;r<r)b__doc__ __future__r collectionsrtypingrrr4rMrpandas._configr pandas._libsr pandas.core.dtypes.astyper pandas.core.dtypes.castr r r rpandas.core.dtypes.commonrrrrrrpandas.core.dtypes.dtypesrrpandas.core.dtypes.genericrrpandas.core.dtypes.missingr pandas.corerrrpandas.core.arraysrpandas.core.arrays.string_rpandas.core.constructionrr>rr r!pandas.core.indexes.apir"r#r$r%r&r'r(r)pandas.core.internals.blocksr*r+r,r-pandas.core.internals.managersr.r/collections.abcr0r1pandas._typingr2r3r4r5rUrcrrrrrrrrJrIr_rrrr^rrrr r%r(rgrFrDrYs #""""" ------444444 ,+++++.-----222222                        "!888888vLUUUUpYYYY." UPUPUPUPUPUPp""""4    #####L''''T7777t6$$###D;?66666r>""""J,4444t! NNNNNNNrF