&`iC$ddlmZmZmZmZmZmZmZddlm Z ddl m Z ddl m Z ddlmZddlmZddlmZddlmZdd lmZdd lmZmZmZdd lmZdd lmZdd l m!Z!ddl"m#Z#ddl$m%Z%ddl&m'Z'm(Z(m)Z)m*Z*ddl+m,Z,m-Z-ddl.m/Z/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6ddl7m8Z8ddl9m:Z:ddl;mZ>m?Z?m@Z@mAZAddlBmCZCddlDmEZEedeZFeeFee eEge fZGde#dee d eEd!e fd"ZHd#e!dee d eEd!e fd$ZId%ZJd&ZKd'ZLd(ZMde%dee d eEd!e fd)ZNde4dee d eEfd*ZOGd+d,ZPded-eeeeGfd!eeGfd.ZQd/S)0)CallableDictListOptionalTupleTypeTypeVar)PhysicalOperator)AggregateNumRows)InputDataBuffer) JoinOperator) LimitOperator)OutputSplitter) UnionOperator) ZipOperator)LogicalOperator LogicalPlan PhysicalPlan)AbstractAllToAll)Count) AbstractFrom) InputData)Join)AbstractUDFMapFilterProjectStreamingRepartition)UnionZip)DownloadLimit)Read)StreamingSplit)Write)plan_all_to_all_op)plan_download_op) plan_read_op)plan_filter_opplan_project_opplan_streaming_repartition_opplan_udf_map_op) plan_write_op) DataContextLogicalOperatorType)bound logical_opphysical_children data_contextreturncXt|dksJt||jS)z>Get the corresponding DAG of physical operators for InputData.r) input_datalenr r5r0r1r2s v/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/data/_internal/planner/planner.pyplan_input_data_opr:8s>  ! !Q & & & & (   opcVt|dksJt||jS)Nrr6)r<r1r2s r9 plan_from_opr>Fs0  ! !Q & & & & < 7 77r;cFt|dksJt|g|RSN)r7r_r1r2s r9 plan_zip_oprDOs3  ! !Q & & & & | 8&7 8 8 88r;cFt|dksJt|g|RSr@)r7rrBs r9 plan_union_oprFTs3  ! !Q & & & &  :(9 : : ::r;cdt|dksJt|j|d|S)Nr)r7r_limitr8s r9 plan_limit_oprJYs7  ! !Q & & & & *,=a,@, O OOr;crt|dksJt|dg|tjS)NrHr) column_name)r7r r COLUMN_NAMEr8s r9 plan_count_oprN^sE  ! !Q & & & &  1  %:K   r;c t|dksJt||d|d|j|j|j|j|j|j|j|j  S)NrArrH) r2 left_input_opright_input_op join_typeleft_key_columnsright_key_columnsleft_columns_suffixright_columns_suffixnum_partitionspartition_size_hint#aggregator_ray_remote_args_override) r7r _join_type_left_key_columns_right_key_columns_left_columns_suffix_right_columns_suffix _num_outputs_partition_size_hint_aggregator_ray_remote_argsr8s r9 plan_join_oprbesx  ! !Q & & & & !'*(+'#5$7&;'=!.&;,6,R    r;c~t|dksJt|d|j|j||jS)NrHr)nequalr2locality_hints)r7r _num_splits_equal_locality_hintsr8s r9plan_streaming_split_oprjzsR  ! !Q & & & & !  !!1    r;c eZdZdZieeeeee e e e e eeeeeeeeeeeeeeeeeee e!e"e#Z$de%de&fdZ'de(de)fdZ*de(de+de,e-e.e(e-fffdZ/d S) PlannerzThe planner to convert optimized logical to physical operators. Note that planner is only doing operators conversion. Physical optimization work is done by physical optimizer. logical_planr3cx||j|j\}}t|||j}|S)z@Convert logical to physical operators recursively in post-order.)_plan_recursivelydagcontextr)selfrm physical_dagop_map physical_plans r9planz Planner.plansC#55  l2   f%\6<;OPP r;r0cXt||j}||Std|)Nz0Found unknown logical operator during planning: ) find_plan_fn_DEFAULT_PLAN_FNS ValueError)rrr0plan_fns r9 get_plan_fnzPlanner.get_plan_fns<z4+ABB  N Kz K K   r;r2ci}g}|jD]E}|||\}}||||F||}||||} | g} | rR| } | jrn6| |||| <| | j| R||| <| |fS)a]Plan a logical operator and its input dependencies recursively. Args: logical_op: The logical operator to plan. data_context: The data context. Returns: A tuple of the physical operator corresponding to the logical operator, and a mapping from physical to logical operators. ) input_dependenciesroappendupdater|pop_logical_operatorsset_logical_operatorsextend) rrr0r2rtr1childphysical_child child_op_mapr{ physical_opqueuecurr_physical_ops r9rozPlanner._plan_recursivelys;=2 ( (E+/+A+A%+V+V (NL  $ $^ 4 4 4 MM, ' ' ' '"":..gj*;\JJ   >$yy{{ 2   2 2: > > >'1F# $ LL)< = = = >){F""r;N)0__name__ __module__ __qualname____doc__r"r'rr:r$r,rr>rr(rr+rr%rrFrrDr!rJrrNrr)rr*rrbr#rjr r&ryrrrvrPlanLogicalOpFnr|r-rr rror;r9rlrlsm  l% } l      , } [ } }  ; l / "!& o /    ,#),#9D,# o7G&G!HH I,#,#,#,#,#,#r;rlplan_fnscb|D]\}}t||r|cSdS)aFind the plan function for a logical operator. This function goes through the plan functions in order and returns the first one that is an instance of the logical operator type. Args: logical_op: The logical operator to find the plan function for. plan_fns: The dictionary of plan functions. Returns: The plan function for the logical operator, or None if no plan function is found. N)items isinstance)r0rop_typer{s r9rxrxsG$%NN,, j' * * NNN  4r;N)Rtypingrrrrrrr 'ray.data._internal.execution.interfacesr 9ray.data._internal.execution.operators.aggregate_num_rowsr 8ray.data._internal.execution.operators.input_data_bufferr +ray.data._internal.execution.operators.joinr 5ray.data._internal.execution.operators.limit_operatorr6ray.data._internal.execution.operators.output_splitterr5ray.data._internal.execution.operators.union_operatorr3ray.data._internal.execution.operators.zip_operatorr%ray.data._internal.logical.interfacesrrr8ray.data._internal.logical.operators.all_to_all_operatorr3ray.data._internal.logical.operators.count_operatorr3ray.data._internal.logical.operators.from_operatorsr8ray.data._internal.logical.operators.input_data_operatorr2ray.data._internal.logical.operators.join_operatorr1ray.data._internal.logical.operators.map_operatorrrrr3ray.data._internal.logical.operators.n_ary_operatorrr8ray.data._internal.logical.operators.one_to_one_operatorr r!2ray.data._internal.logical.operators.read_operatorr"=ray.data._internal.logical.operators.streaming_split_operatorr#3ray.data._internal.logical.operators.write_operatorr$-ray.data._internal.planner.plan_all_to_all_opr%+ray.data._internal.planner.plan_download_opr&'ray.data._internal.planner.plan_read_opr'*ray.data._internal.planner.plan_udf_map_opr(r)r*r+(ray.data._internal.planner.plan_write_opr,ray.data.contextr-r.rr:r>rDrFrJrNrbrjrlrxrr;r9rsGGGGGGGGGGGGGGGGGGDDDDDDEDDDDDOOOOOOQQQQQQOOOOOOKKKKKK FEEEEELLLLLLNNNNNNCCCCCC KJJJJJJJTTTTTTTTCCCCCCXXXXXXEEEEEELLLLLLHHHHHH@@@@@@ CBBBBB((((((g3?KKK$/0+>@PP  ,-      88,-88 8888999 ;;; PPP ,- *  ,-     W#W#W#W#W#W#W#W#t+/_0E0V+W or;