&`iO4ddlZddlmZmZmZmZmZmZddlZ ddl Z ddl m Z mZddlmZmZmZddlmZmZmZmZmZmZmZddlmZejeZeGddZ eGd d e Z!eGd d eZ"dS) N)CallableListOptionalSetTupleUnion)_DUMMY_AGENT_IDBaseEnv) Deprecated OldAPIStackoverride)AgentID EnvActionTypeEnvID EnvInfoDict EnvObsTypeEnvType MultiEnvDict)log_oncecVeZdZdZdejdejdefdZe d'd e e ege fd e e ej dede ejde ejd ed d fdZdddde e ede e ed ee ee effdZ d(dddde ede ede ed eeeefeffdZd(de ed dfdZde ed ee ee ee ee ee effdZd e e fdZd(de ed e ejfdZ d)d e e ege fdededed ed df d Z e!d!d"#d*d$Z"e!d%d"#d e e fd&Z#dS)+ VectorEnvzGAn environment that supports batch evaluation using clones of sub-envs.observation_space action_spacenum_envsc0||_||_||_dS)aInitializes a VectorEnv instance. Args: observation_space: The observation Space of a single sub-env. action_space: The action Space of a single sub-env. num_envs: The number of clones to make of the given sub-env. Nrrr)selfrrrs l/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/env/vector_env.py__init__zVectorEnv.__init__s"3(  NFmake_env existing_envsrestart_failed_sub_environmentsreturn_VectorizedGymEnvc0t||pg||||S)aTranslates any given gym.Env(s) into a VectorizedEnv object. Args: make_env: Factory that produces a new gym.Env taking the sub-env's vector index as only arg. Must be defined if the number of `existing_envs` is less than `num_envs`. existing_envs: Optional list of already instantiated sub environments. num_envs: Total number of sub environments in this VectorEnv. action_space: The action space. If None, use existing_envs[0]'s action space. observation_space: The observation space. If None, use existing_envs[0]'s observation space. restart_failed_sub_environments: If True and any sub-environment (within a vectorized env) throws any error during env stepping, the Sampler will try to restart the faulty sub-environment. This is done without disturbing the other (still intact) sub-environment and without the RolloutWorker crashing. Returns: The resulting _VectorizedGymEnv object (subclass of VectorEnv). )r"r#rrrr$)r&)r"r#rrrr$ env_config policy_configs rvectorize_gym_envszVectorEnv.vectorize_gym_envs*s3D!'-2/%,K     r seedsoptionsr,r-ct)a?Resets all sub-environments. Args: seed: The list of seeds to be passed to the sub-environments' when resetting them. If None, will not reset any existing PRNGs. If you pass integers, the PRNGs will be reset even if they already exists. options: The list of options dicts to be passed to the sub-environments' when resetting them. Returns: Tuple consitsing of a list of observations from each environment and a list of info dicts from each environment. NotImplementedError)rr,r-s r vector_resetzVectorEnv.vector_resetUs "!r seedr-indexr3ct)aResets a single sub-environment. Args: index: An optional sub-env index to reset. seed: The seed to be passed to the sub-environment at index `index` when resetting it. If None, will not reset any existing PRNG. If you pass an integer, the PRNG will be reset even if it already exists. options: An options dict to be passed to the sub-environment at index `index` when resetting it. Returns: Tuple consisting of observations from the reset sub environment and an info dict of the reset sub environment. Alternatively an Exception can be returned, indicating that the reset operation on the sub environment has failed (and why it failed). r/)rr4r3r-s rreset_atzVectorEnv.reset_atgs ."!r ct)zsRestarts a single sub-environment. Args: index: An optional sub-env index to restart. r/rr4s r restart_atzVectorEnv.restart_ats "!r actionsct)aPerforms a vectorized step on all sub environments using `actions`. Args: actions: List of actions (one for each sub-env). Returns: A tuple consisting of 1) New observations for each sub-env. 2) Reward values for each sub-env. 3) Terminated values for each sub-env. 4) Truncated values for each sub-env. 5) Info values for each sub-env. r/)rr:s r vector_stepzVectorEnv.vector_steps $"!r cgS)zxReturns the underlying sub environments. Returns: List of all underlying sub environments. rs rget_sub_environmentszVectorEnv.get_sub_environmentss  r cdS)aRenders a single environment. Args: index: An optional sub-env index to render. Returns: Either a numpy RGB image (shape=(w x h x 3) dtype=uint8) or None in case rendering is handled directly by this method. Nr>r8s r try_render_atzVectorEnv.try_render_ats  r r remote_envsremote_env_batch_wait_msr c$t|}|S)aConverts an RLlib MultiAgentEnv into a BaseEnv object. The resulting BaseEnv is always vectorized (contains n sub-environments) to support batched forward passes, where n may also be 1. BaseEnv also supports async execution via the `poll` and `send_actions` methods and thus supports external simulators. Args: make_env: A callable taking an int as input (which indicates the number of individual sub-environments within the final vectorized BaseEnv) and returning one individual sub-environment. num_envs: The number of sub-environments to create in the resulting (vectorized) BaseEnv. The already existing `env` will be one of the `num_envs`. remote_envs: Whether each sub-env should be a @ray.remote actor. You can set this behavior in your config via the `remote_worker_envs=True` option. remote_env_batch_wait_ms: The wait time (in ms) to poll remote sub-environments for, if applicable. Only used if `remote_envs` is True. Returns: The resulting BaseEnv object. )VectorEnvWrapper)rr"rrCrDr$envs r to_base_envzVectorEnv.to_base_envsBt$$ r r*T)newerrorcdSNr>)rargskwargss rwrapzVectorEnv.wrap r r@cdSrLr>r?s r get_unwrappedzVectorEnv.get_unwrappedrPr )NNr!NNFNNrL)Nr!FrF)r%r&)$__name__ __module__ __qualname____doc__gymSpaceintr staticmethodrrrrEnvboolr*dictrrrr1r Exceptionr6r9rfloatr<r@npndarrayrBrHr rOrRr>r rrrs}QQ !!$ !:=) !OR ! ! ! !7;15,01505( ( 8SE7N34( SW .( ( sy) ( $CI. ( *. (  ( ( ( \( V/3TX""" c+"=Ed4j=Q" tJk!22 3""""( $"#"& """}"sm " $ " uZ,-y8 9 """"2"" """"""M*"  Z$u+tDz4:tK?PP """"(d7m   8C=  HRZ"   /N,$)nnx'' I  XXc$)nn55 6 6 6$)nnx'' /Q49Q<3Q%B1)B      r r+r,r-r%c`|p dg|jz}|p dg|jz}g}g}tt|jD]k} ||||||\}}t |t sn>||||l||fS)NTr2)rrangererdr6 isinstancer^rf)rr,r- resetted_obsresetted_infosiobsinfoss rr1z_VectorizedGymEnv.vector_resets/$$-/3dVdm3 s49~~&& ) )A !]]158WQZ]PP U!#y11     $ $ $  ! !% ( ( ( (^++r r2r4r3c|d} |j|||}n_#t$rR}|jr?t|jd|||if}n|Yd}~nd}~wwxYw|S)Nrr2)rdresetr^r$logger exceptionrMr9)rr4r3r- obs_and_infoses rr6z_VectorizedGymEnv.reset_at#s =E  Ie,22g2NNMM   3   +++&&& !2   s") BABBc|d} |j|nF#t$r9}tdr td|d|Yd}~nd}~wwxYw|j|}d|j|<~td|d|||j|<td|ddS) Nr close_sub_envzBTrying to close old and replaced sub-environment (at vector index=z"), but closing resulted in error: z+Trying to restart sub-environment at index .zSub-environment at index z restarted successfully.)rdcloser^rrswarningr")rr4rv env_to_dels rr9z_VectorizedGymEnv.restart_at:s =E  Ie  " " $ $ $ $   (( K"KKGHKK  Yu%  %  MUMMMNNN==// %R5RRRSSSSSs& A)/A$$A)r:c gggggf\}}}}}t|jD]C} |j|||}nb#t$rU} |jrBt | jd| || dddif}n| Yd} ~ nd} ~ wwxYw|\} } } } }t|ts0td |t||| || || || ||E|||||fS)NrTz"Info should be a dict, got {} ({}))rjrrdstepr^r$rsrtrMr9rkr] ValueErrorformattyperf)rr: obs_batch reward_batchterminated_batchtruncated_batch info_batchrnresultsrvroreward terminated truncatedinfos rr<z_VectorizedGymEnv.vector_stepQs      R N '4 #" % &&&&&r r%ctddlm}|js-d|_|j\|_|_tt|j}tt|j }tt|j }tt|j }tt|j}g|_g|_ g|_ g|_ g|_||||||d||d||ifS)Nr)with_dummy_agent_idT__all__) ray.rllib.env.base_envrrrr1rrr] enumeraterrr)rrrrewards terminateds truncatedsrps rpollzVectorEnvWrapper.polls= ?>>>>>$ J$(D !+/?+G+G+I+I (DL$.y..//y!122339T%9::;; )D$78899 Yt~..// !   ( (   ( (   Y 7 7   I 6 6   & &    r action_dictNcddlm}dg|jz}t|jD]}|||||<|j|\|_|_|_|_ |_ dS)Nrr ) rr rrjrr<rrrrr)rrr action_vectorrns r send_actionszVectorEnvWrapper.send_actionss::::::. t}%% ? ?A*1~o>M!   O ' ' 6 6  L      NNNr r2env_idr3r-cddlm}|d}t|tsJ|j|||\}}t|t r||i||ifS|||ii|||iifS)Nrrr2)rr rkrYrr6r^)rrr3r-r rorps r try_resetzVectorEnvWrapper.try_resets ;::::: >F&#&&&&&_--f4-QQ U c9 % % XC=65/1 1_c23fPU>V5WW Wr c|t|tsJ|j|||dSrL)rkrYrr9rrrs r try_restartzVectorEnvWrapper.try_restartsJ~FC!8!8~~8 ""6*** V$$$$$r Fas_dictc|s|jStt|jSrL)rr@r]r)rrs rr@z%VectorEnvWrapper.get_sub_environmentssC K?7799 9 $/"F"F"H"HIIJJ Jr ch|t|tsJ|j|SrL)rkrYrrBrs r try_renderzVectorEnvWrapper.try_renders0~FC!8!8~~8,,V444r c|jSrL)rr?s rrz"VectorEnvWrapper.observation_spaces &&r c|jSrL)rr?s rrzVectorEnvWrapper.action_spaces !!r cthSrLrr?s r get_agent_idszVectorEnvWrapper.get_agent_idss   r rc|dt|jD|_dt|jD|_dt|jD|_dt|jD|_dt|jD|_dS|j|\|j|<|j|<d|j|<d|j|<d|j|<dS) zResets all or one particular sub-environment's state (by index). Args: idx: The index to reset at. If None, reset all the sub-environments' states. Ncg|]}dSrLr>.0_s r z4VectorEnvWrapper._init_env_state..s???QD???r cg|]}dS)r~r>rs rrz4VectorEnvWrapper._init_env_state..sBBBBBBr cg|]}dSFr>rs rrz4VectorEnvWrapper._init_env_state..s#H#H#HaE#H#H#Hr cg|]}dSrr>rs rrz4VectorEnvWrapper._init_env_state..s"G"G"GQ5"G"G"Gr cg|]}iSr>r>rs rrz4VectorEnvWrapper._init_env_state..s???Qb???r r~F) rjrrrrrrrr6)rrs rrz VectorEnvWrapper._init_env_state s ;??% *>*>???DLBBU4=-A-ABBBD #H#H53G3G#H#H#HD "G"G% 2F2F"G"G"GD ??% *>*>???DNNN6:_5M5Mc5R5R 2DL t~c2$'D S !(-D  %',D  $ $ $r rLr)!rSrTrUrVrrr r rrrrrrrYr]rrr\rrrr@rpropertyrWrXrrrrrrr>r rrFrFs'9''''"Xg$    $ $ $ $ LXg 7 7 7 7 7 7Xg#'X#"& XXXXsm X $ X |\) * XXXX.Xg%%(5/%T%%%%XgKKDKU4=RVCV=WKKKK Xg55%5D5555 Xg'39'''X' Xg"ci"""X"Xg!s7|!!!!--8C=-D------r rF)#loggingtypingrrrrrr gymnasiumrWnumpyr`rr r ray.rllib.utils.annotationsr r r ray.rllib.utils.typingrrrrrrrray.utilr getLoggerrSrsrr&rFr>r rrs>>>>>>>>>>>>>>>>;;;;;;;;IIIIIIIIII  8 $ $ C C C C C C C  C L _)_)_)_)_) _)_) _)D \-\-\-\-\-w\-\- \-\-\-r