&`idZddlZddlmZddlZddlZddlm Z m Z ddl m Z m Z ddlmZmZe ddd Zed ed d GddejZedkr`eZeejedejiZe eedSdS)a)Example of defining a custom gymnasium Env to be learned by an RLlib Algorithm. This example: - demonstrates how to write your own (single-agent) gymnasium Env class, define its physics and mechanics, the reward function used, the allowed actions (action space), and the type of observations (observation space), etc.. - shows how to configure and setup this environment class within an RLlib Algorithm config. - runs the experiment with the configured algo, trying to solve the environment. To see more details on which env we are building for this example, take a look at the `SimpleCorridor` class defined below. How to run this script ---------------------- `python [script file name].py` Use the `--corridor-length` option to set a custom length for the corridor. Note that for extremely long corridors, the algorithm should take longer to learn. For debugging, use the following additional command line options `--no-tune --num-env-runners=0` which should allow you to set breakpoints anywhere in the RLlib code and have the execution stop there for inspection and debugging. For logging to your WandB account, use: `--wandb-key=[your WandB API key] --wandb-project=[some project name] --wandb-run-name=[optional: WandB run name (within the defined project)]` Results to expect ----------------- You should see results similar to the following in your console output: +--------------------------------+------------+-----------------+--------+ | Trial name | status | loc | iter | |--------------------------------+------------+-----------------+--------+ | PPO_SimpleCorridor_78714_00000 | TERMINATED | 127.0.0.1:85794 | 7 | +--------------------------------+------------+-----------------+--------+ +------------------+-------+----------+--------------------+ | total time (s) | ts | reward | episode_len_mean | |------------------+-------+----------+--------------------| | 18.3034 | 28000 | 0.908918 | 12.9676 | +------------------+-------+----------+--------------------+ N)Optional)BoxDiscrete)add_rllib_example_script_args#run_rllib_example_script_experiment)get_trainable_cls register_envg?2i)default_reward default_itersdefault_timestepsz--corridor-length zcThe length of the corridor in fields. Note that this number includes the starting- and goal states.)typedefaulthelpc@eZdZdZddeefdZddddZdZdS) SimpleCorridoraExample of a custom env in which the agent has to walk down a corridor. ------------ |S........G| ------------ , where S is the starting position, G is the goal position, and fields with '.' mark free spaces, over which the agent may step. The length of the above example corridor is 10. Allowed actions are left (0) and right (1). The reward function is -0.01 per step taken and a uniform random value between 0.5 and 1.5 when reaching the goal state. You can configure the length of the corridor via the env's config. Thus, in your AlgorithmConfig, you can do: `config.environment(env_config={"corridor_length": ..})`. Nconfigc|pi}|dd|_d|_td|_t d|jdt j|_dS)Ncorridor_lengthrg))shapedtype) getend_poscur_posr action_spacernpfloat32observation_space)selfrs z/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/examples/envs/custom_gym_env.py__init__zSimpleCorridor.__init__bsY2zz"3Q77  $QKK!$S$,d"*!U!U!U)seedoptionsctj|d|_tj|jgtjddifS)Nr env_statereset)randomr'rr arrayr!)r#r'r(s r$r+zSimpleCorridor.resetis; D x 33k75KKKr&c<|dvs J||dkr|jdkr|xjdzc_n|dkr|xjdz c_|j|jk}d}|rtjddnd}i}t j|jgtj||||fS)N)rrrrFg?g?g{Gz)rrr,uniformr r-r!)r#action terminated truncatedrewardinfoss r$stepzSimpleCorridor.stepos Q;;4rOs..b ********>======= & &      ! 4 4 4 4 4 SW4 4 4 r z     D $)$$      )4+?@   (' T:::::3r&