&`ia pddlZddlZddlmZddlmZddlm Z m Z m Z m Z eGddZ dS)N)ModelV2) OldAPIStack)ListModelConfigDict TensorTypeUnionc eZdZdZdeedefdZdefdZdefdZ defdZ d edefd Z d ddefd Z defd Z d ddefdZdefdZeedejdedeeejffdZdS)ActionDistributionzThe policy action distribution of an agent. Attributes: inputs: input vector to compute samples from. model (ModelV2): reference to model producing the inputs. inputsmodelc"||_||_dS)aInitializes an ActionDist object. Args: inputs: input vector to compute samples from. model (ModelV2): reference to model producing the inputs. This is mainly useful if you want to use model variables to compute action outputs (i.e., for autoregressive action distributions, see examples/autoregressive_action_dist.py). N)r r )selfr r s p/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/ray/rllib/models/action_dist.py__init__zActionDistribution.__init__s  returnct)z+Draw a sample from the action distribution.NotImplementedErrorrs rsamplezActionDistribution.sample!!rct)z Get the deterministic "sampling" output from the distribution. This is usually the max likelihood output, i.e. mean for Normal, argmax for Categorical, etc.. rrs rdeterministic_samplez'ActionDistribution.deterministic_sample#s "!rct)z7Returns the log probability of the last sampled action.rrs rsampled_action_logpz&ActionDistribution.sampled_action_logp+rrxct)z.The log-likelihood of the action distribution.r)rrs rlogpzActionDistribution.logp/rrotherct)z3The KL-divergence between two action distributions.rrr s rklzActionDistribution.kl3rrct)z'The entropy of the action distribution.rrs rentropyzActionDistribution.entropy7rrc,||S)zThe KL-divergence between two action distributions. This differs from kl() in that it can return an array for MultiDiscrete. TODO(ekl) consider removing this. )r#r"s rmulti_klzActionDistribution.multi_kl;s wwu~~rc*|S)zThe entropy of the action distribution. This differs from entropy() in that it can return an array for MultiDiscrete. TODO(ekl) consider removing this. )r%rs r multi_entropyz ActionDistribution.multi_entropyCs ||~~r action_space model_configct)a|Returns the required shape of an input parameter tensor for a particular action space and an optional dict of distribution-specific options. Args: action_space (gym.Space): The action space this distribution will be used for, whose shape attributes will be used to determine the required shape of the input parameter tensor. model_config: Model's config dict (as defined in catalog.py) Returns: model_output_shape (int or np.ndarray of ints): size of the required input vector (minus leading batch dimension). r)r*r+s rrequired_model_output_shapez.ActionDistribution.required_model_output_shapeKs &"!rN)__name__ __module__ __qualname____doc__rrrrrrrrr#r%r'r) staticmethodrgymSpacerrintnpndarrayr-rrr r s tJ/     " """""j"""""Z"""""j"Z""""",""""""""""2zz"i"/>" sBJ """[\"""rr ) gymnasiumr3numpyr6ray.rllib.models.modelv2rray.rllib.utils.annotationsrray.rllib.utils.typingrrrrr r8rrr>s,,,,,,333333KKKKKKKKKKKK T"T"T"T"T"T"T" T"T"T"r