&`i?FvddlmZmZmZmZddlZddlZddl Z ddl m Z e Gddej Z e dejdejfdZe dejdefd Ze dejdeejfd Ze d Ze d"d dddddejdedeeeefdeedededej fdZe dZe dddeedeeeffdZe dZe dZe dZe dZe ded edefd!ZdS)#)AnyListOptionalUnionN) DeveloperAPIceZdZdZdZdS)BatchedNdArrayaA ndarray-wrapper the usage of which indicates that there a batch dim exists. This is such that our `batch()` utility can distinguish between having to stack n individual batch items (each one w/o any batch dim) vs having to concatenate n already batched items (each one possibly with a different batch dim, but definitely with some batch dim). TODO (sven): Maybe replace this by a list-override instead. cTtj||}|SN)npasarrayview)cls input_arrayobjs v/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/utils/spaces/space_utils.py__new__zBatchedNdArray.__new__s$j%%**3// N)__name__ __module__ __qualname____doc__rrrr r s-rr spacereturncNt|drt|jS|S)aRReturns the original space of a space, if any. This function recursively traverses the given space and returns the original space at the very end of the chain. Args: space: The space to get the original space for. Returns: The original space or the given space itself if no original space is found. original_space)hasattrget_original_spacerrs rrrs-u&''!%"6777 rct|tjjtjjtjjtjjfvrdSdS)aaReturns true, if the space is composite. Note, we follow here the glossary of `gymnasium` by which any spoace that holds other spaces is defined as being 'composite'. Args: space: The space to be checked for being composed of other spaces. Returns: True, if the space is composed of other spaces, otherwise False. TF)typegymspacesDictGraphSequenceTupler s ris_composite_spacer)/sG E{{      turc.fdg}|||S)aFlattens a gym.Space into its primitive components. Primitive components are any non Tuple/Dict spaces. Args: space: The gym.Space to flatten. This may be any supported type (including nested Tuples and Dicts). Returns: List[gym.Space]: The flattened list of primitive Spaces. This list does not contain Tuples or Dicts anymore. c:ddlm}t|tjjr|D]}||dSt|tjj|fr+t|jD]}|||dS||dS)Nr)FlexDict) ray.rllib.utils.spaces.flexdictr, isinstancer#r$r(r%sortedappend)space_ return_listr,sk_helper_flattens rr5z&flatten_space.._helper_flattenVs<<<<<< fcj. / / ' 0 0;//// 0 0 ( ; < < 'FM** 8 8q ;7777 8 8   v & & & & &rr)rretr5s @r flatten_spacer7Gs: ' ' ' ' ' COE3 Jrc$fd|S)aaReturns a Tuple/Dict Space as native (equally structured) py tuple/dict. Args: space: The Space to get the python struct for. Returns: Union[dict,tuple,gym.Space]: The struct equivalent to the given Space. Note that the returned struct still contains all original "primitive" Spaces (e.g. Box, Discrete). .. testcode:: :skipif: True get_base_struct_from_space(Dict({ "a": Box(), "b": Tuple([Discrete(2), Discrete(3)]) })) .. testoutput:: dict(a=Box(), b=tuple(Discrete(2), Discrete(3))) cttjjrt fdDSttjjrfdjDSS)Nc3.K|]}|VdSr r).0r3_helper_structs r zEget_base_struct_from_space.._helper_struct..s-;;q**;;;;;;rc4i|]}||Srr)r;r4r<r1s r zFget_base_struct_from_space.._helper_struct..s)HHHQA~~fQi00HHHr)r.r#r$r(tupler%)r1r<s`rr<z2get_base_struct_from_space.._helper_structsx fcj. / / ;;;;F;;;;; ;   0 0 HHHHH&-HHH HMrr)rr<s @rget_base_struct_from_spacerAgs.2 >%  r F) fill_value time_size time_majorone_hot_discrete batch_sizerDrErFrGcttjjtjjt t fr]}ttjjtjjfrt}tj fd|Srttjj r3tj ddj ftjncttjjrDtj ddtjjftjdkrʉzdkrdksJr5tjfdt'DjStjfd t'DjStjdkrfd t'DnjSdkrdksJrg}ng}n dkrgng}tj|t/jzj S) aReturns batched dummy data (using `batch_size`) for the given `space`. Note: The returned batch will not pass a `space.contains(batch)` test as an additional batch dimension has to be added at axis 0, unless `batch_size` is set to 0. Args: space: The space to get a dummy batch for. batch_size: The required batch size (B). Note that this can also be 0 (only if `time_size` is None!), which will result in a non-batched sample for the given space (no batch dim). fill_value: The value to fill the batch with or "random" for random values. time_size: If not None, add an optional time axis of `time_size` size to the returned batch. This time axis might either be inserted at axis=1 (default) or axis=0, if `time_major` is True. time_major: If True AND `time_size` is not None, return batch as shape [T x B x ...], otherwise as [B x T x ...]. If `time_size` if None, ignore this setting and return [B x ...]. one_hot_discrete: If True, will return one-hot vectors (instead of int-values) for those sub-components of a (possibly complex) `space` that are Discrete or MultiDiscrete. Note that in case `fill_value` is 0.0, this will result in zero-hot vectors (where all slots have a value of 0.0). Returns: The dummy batch of size `bqtch_size` matching the given space. c.t|S)N)rrHrDrErFrG)get_dummy_batch_for_space)r3rHrDrGrFrEs rz+get_dummy_batch_for_space..s)/%%#%!1 rrC?randomNrcFg|]}fdtDS)c8g|]}Srsampler;_rs r z8get_dummy_batch_for_space...s!CCCACCCrrange)r;trHrs rrUz-get_dummy_batch_for_space..sFDCCCz1B1BCCCrdtypecFg|]}fdtDS)c8g|]}SrrQ)r;rXrs rrUz8get_dummy_batch_for_space...s!BBBABBBrrV)r;rTrrEs rrUz-get_dummy_batch_for_space..sFCBBBy1A1ABBBrc8g|]}SrrQrSs rrUz-get_dummy_batch_for_space..s!;;;A;;;r)rDrZ)r.r#r$r%r(dictr@rAtree map_structureDiscreteBoxnr float32 MultiDiscretesumnvecarrayrWrZrRfulllistshape)rrHrDrErFrG base_structrks`````` rrKrKsL%#*/3:+;T5IJJ  ecjosz/?@ A A <4U;;K!            P eSZ0 1 1 PJNN3egZDDEE sz7 8 8 PJNN3bfUZ.@.@-BBJOOEX  >>i!mmm3 x!&y!1!1 + x!&z!2!2 + 8>><;;;z):):;;;;\\^^k    >>i!mmm3 0"J/#Y/$.NNZLLEw D%% %*EK    rct|tttfrjg}t j|D]+}|tj|dg,tj |d}|S)a:Returns a single np.ndarray given a list/tuple of np.ndarrays. Args: input_ (Union[List[np.ndarray], np.ndarray]): The list of ndarrays or a single ndarray. Returns: np.ndarray: The result after concatenating all single arrays in input_. .. testcode:: :skipif: True flatten_to_single_ndarray([ np.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]]), np.array([7, 8, 9]), ]) .. testoutput:: np.array([ 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0 ]) raxis) r.rjr@r^r_flattenr0r reshape concatenate)input_expandedin_s rflatten_to_single_ndarrayrws4&4-..<<'' 3 3C OOBJsRD11 2 2 2 2q11199;; Mr)'individual_items_already_have_batch_dimlist_of_structsrxc|std|dkr5tj|d}t|dt}|r t jn t jtjfdg|R}|S)aConverts input from a list of (nested) structs to a (nested) struct of batches. Input: List of structs (each of these structs representing a single batch item). [ {"a": 1, "b": (4, 7.0)}, <- batch item 1 {"a": 2, "b": (5, 8.0)}, <- batch item 2 {"a": 3, "b": (6, 9.0)}, <- batch item 3 ] Output: Struct of different batches (each batch has size=3 b/c there were 3 items in the original list): { "a": np.array([1, 2, 3]), "b": (np.array([4, 5, 6]), np.array([7.0, 8.0, 9.0])) } Args: list_of_structs: The list of (possibly nested) structs. Each item in this list represents a single batch item. individual_items_already_have_batch_dim: True, if the individual items in `list_of_structs` already have a batch dim. In this case, we will concatenate (instead of stack) at the end. In the example above, this would look like this: Input: [{"a": [1], "b": ([4], [7.0])}, ...] -> Output: same as in above example. If the special value "auto" is used, Returns: The struct of component batches. Each leaf item in this struct represents the batch for a single component (in case struct is tuple/dict). If the input is a simple list of primitive items, e.g. a list of floats, a np.array of floats will be returned. z3Input `list_of_structs` does not contain any items.autorcBtj|dS)Nrro)r ascontiguousarray)r3np_funcs rrLzbatch..Es!2'(:(:(:;;r) ValueErrorr_rqr.r r rsstackr`)ryrxflatr6r~s @rbatchrsL PNOOO/&88|OA.//2M   C Jrc tj|g}ttdD]Q|tj|fdttDR|S)aConverts input from (nested) struct of batches to batch of structs. Input: Struct of different batches (each batch has size=3): { "a": np.array([1, 2, 3]), "b": (np.array([4, 5, 6]), np.array([7.0, 8.0, 9.0])) } Output: Batch (list) of structs (each of these structs representing a single action): [ {"a": 1, "b": (4, 7.0)}, <- action 1 {"a": 2, "b": (5, 8.0)}, <- action 2 {"a": 3, "b": (6, 9.0)}, <- action 3 ] Args: batches_struct: The struct of component batches. Each leaf item in this struct represents the batch for a single component (in case struct is tuple/dict). Alternatively, `batches_struct` may also simply be a batch of primitives (non tuple/dict). Returns: The list of individual structs. Each item in the returned list represents a single (maybe complex) batch item. rc,g|]}|Srr)r;i batch_pos flat_batchess rrUzunbatch..ms"NNNa+NNNr)r_rqrWlenr0 unflatten_as)batches_structoutrrs @@runbatchrJs8<//L C3|A//00    NNNNNU3|;L;L5M5MNNN      Jrc4d}tj|||S)aClips all components in `action` according to the given Space. Only applies to Box components within the action space. Args: action: The action to be clipped. This could be any complex action, e.g. a dict or tuple. action_space: The action space struct, e.g. `{"a": Distrete(2)}` for a space: Dict({"a": Discrete(2)}). Returns: Any: The input action, but clipped by value according to the space's bounds. ct|tjjr t j||j|j}|Sr )r.r#r$rbr cliplowhighar3s rmap_zclip_action..map_s4 a ( ( *15!&))Arr_r`)action action_spacers r clip_actionrss("  dFL 9 99rc4d}tj|||S)aUnsquashes all components in `action` according to the given Space. Inverse of `normalize_action()`. Useful for mapping policy action outputs (normalized between -1.0 and 1.0) to an env's action space. Unsquashing results in cont. action component values between the given Space's bounds (`low` and `high`). This only applies to Box components within the action space, whose dtype is float32 or float64. Args: action: The action to be unsquashed. This could be any complex action, e.g. a dict or tuple. action_space_struct: The action space struct, e.g. `{"a": Box()}` for a space: Dict({"a": Box()}). Returns: Any: The input action, but unsquashed, according to the space's bounds. An unsquashed action is ready to be sent to the environment (`BaseEnv.send_actions([unsquashed actions])`). ct|tjjrt j|jrt j|jr|jtj ks|jtj krA|j |dz|j |j z zdz z}t j ||j |j }n.t j|jtjr |j |z}|S)NrM@)r.r#r$rbr all bounded_below bounded_aboverZrdfloat64rrr issubdtypeintegerrs rrzunsquash_action..map_s q#*. ) ) q'' q''  w"*$$2:(=(=EQW!%83>>GAquaf--qw 33 EAIrrraction_space_structrs runsquash_actionrs),&  dF,? @ @@rc4d}tj|||S)aNormalizes all (Box) components in `action` to be in [-1.0, 1.0]. Inverse of `unsquash_action()`. Useful for mapping an env's action (arbitrary bounded values) to a [-1.0, 1.0] interval. This only applies to Box components within the action space, whose dtype is float32 or float64. Args: action: The action to be normalized. This could be any complex action, e.g. a dict or tuple. action_space_struct: The action space struct, e.g. `{"a": Box()}` for a space: Dict({"a": Box()}). Returns: Any: The input action, but normalized, according to the space's bounds. ct|tjjrJ|jt jks|jt jkr ||jz dz|j |jz z dz }|S)NrrM) r.r#r$rbrZr rdrrrrs rrznormalize_action..map_s^ a ( ( = Grz ! !QW %:%:ae)s"qv~6.map_s_ a $ $ &dBJ// ,!5#,MMLTLLLMM7b==8D888DD$%%+V JJ&& DJ&&{{17++3   &:a#9#9 &$&& !4??+<+< !4yy$ ** &x~~ rF) check_typesr)rrrs rconvert_element_to_space_typers-8  dG_% P P PPr)rB)typingrrrr gymnasiumr#numpyr r_ray.rllib.utils.annotationsrrr Spacerboolr)r7rArrstrrKrwrrrrrrrrrrs------------ 444444RZ"ciCI$ciD.tCI> ! ! !Fc *-#"c c c 9c c eS#o& c } c  c c Zc c c c LBAF222#Y2.349-=2222j%%%P:::0(A(A(AVAAA<)Q3)Q)Q)Q)Q)Q)Q)Q)Qr