PiA) ddlZddlZddlmZddlmZddlmZm Z m Z m Z ej ddZGddZejdddfd e ejejfd e ed e ed e efd Zej jjdZGddejZGddejZGddejZdS)N)partial)current_module)AnyCallableLiteralOptionalhaikuhkceZdZGddejjjZGddejjjZGddejj Z dS) ParamFactoryceZdZdZdZdS)ParamFactory.Concretec||_||_|(t}t|dr|j}nd}||_||_dS)Ndtypefloat32)nameinitrhasattrr param_type)selfrrrrmodules a/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/einx/nn/haiku.py__init__zParamFactory.Concrete.__init__ sRDIDI}'))67++&"LEE%EDJ(DOOOchdt|j|j|j|jfSN)r CacheKeyrrrrrs rto_value_and_keyz&ParamFactory.Concrete.to_value_and_keys+..ty$)TZQUQ`aaa arN)__name__ __module__ __qualname__rrrrConcreter s7 ) ) ) b b b b brr$c&eZdZdZdZdZdZdS)ParamFactory.CacheKeyc>||_||_||_||_dSr)rrrr)rrrrrs rrzParamFactory.CacheKey.__init__s"DIDIDJ(DOOOrcPt|j|j|j|jfSr)hashrrrrrs r__hash__zParamFactory.CacheKey.__hash__$s!DItz4?KLL Lrct|tjo?|j|jko/|j|jko|j|jko|j|jkSr) isinstancer rrrrr)rothers r__eq__zParamFactory.CacheKey.__eq__'s`5,"7888I+8I+8J%+-8Ou'77  rcjdt|j|j|j|j|fSr)r Tracerrrrr)rbackend virtual_args r to_tracerzParamFactory.CacheKey.to_tracer0s1##DIty$*doWbcc rN)r r!r"rr*r.r3r#rrrr&sS ) ) )  M M M        rrceZdZdZdZdS)ParamFactory.TracercL||_||_||_||_||_dSr)rrrr depend_on)rrrrrr7s rrzParamFactory.Tracer.__init__7s)DIDIDJ(DO&DNNNrc,|j|jn|dd}|j|jn|dd}|j|jn|dd}|t d|t dt |t r|dvs|dkr!tj d }n|d kr tj d }n|d kr tj d }n~|dkr*tj d dd|d}nNt d|dt |ttfrtj |}|jdkr tj}n|jdkr tj}nJt"j|||||dt"j||jgS)Nrrrz=Must specify name for tensor factory hk.get_{parameter|state}z=Must specify init for tensor factory hk.get_{parameter|state}get_at rearrangeg{Gz?)stddevaddgmultiplyg?dotfan_intruncated_normalin_axis) fan_in_axesz3Don't know which initializer to use for operation '' parameterstateF)shaperrr)kwargsoutputr7)rgetrr ValueErrorr,strthk initializers RandomNormalConstantVarianceScalingintfloatr get_parameter get_stateeinxtracerapplyTensorr7)rrFrGrrrfuncs r__call__zParamFactory.Tracer.__call__>s$(I$54996::fd;S;SD$(I$54996::fd;S;SD&*j&8DJJfjjRV>W>WE| !`aaa| !`aaaD#&& 78##t{':':+888EEDDU]]+44S99DDZ''+44S99DDU]]+;;X'9viGX<DD%%b[_%b%b%bcccD3,// 7'0066+--(G++}u;$$!&udSS{))%00>* % rN)r r!r"rrZr#rrr0r56s2 ' ' '& & & & & rr0N) r r!r"rUrVinputInputr$r TensorFactoryr0r#rrr r sbbbbb4;$*bbb$4;$-2.....*.....rr rYrrrc|tjkrd}n&|tjkrd}ntd|dt||||S)aCreate a tensor factory for Haiku parameters. Args: func: Either ``hk.get_parameter`` or ``hk.get_state``. Defaults to ``hk.get_parameter``. name: Name of the parameter. If ``None``, uses a default name determined from the calling operation. Defaults to ``None``. init: Initializer for the parameter. If ``None``, uses a default init method determined from the calling operation. Defaults to ``None``. dtype: Data type of the parameter. If ``None``, uses the ``dtype`` member of the calling module or ``float32`` if it does not exist. Defaults to ``None``. Returns: A tensor factory with the given default parameters. rDrEzUnknown parameter function 'rC)r rSrTrJr r$)rYrrrrs rparamr_gsa( r    ????@@@  tUJ ? ??rct|ttjks*t|ttjkr!t |SdSr)idr rSrTr_r)xs rtensor_factoryrcsU !uu2#$$$$1BL1A1A(A(AQxx((***trceZdZdZ ddeded ed ed ed ed edededeedeedeffd Z ddZ xZ S)NormaNormalization layer. Args: stats: Einstein string determining the axes along which mean and variance are computed. Will be passed to ``einx.reduce``. params: Einstein string determining the axes along which learnable parameters are applied. Will be passed to ``einx.elementwise``. Defaults to ``"b... [c]"``. mean: Whether to apply mean normalization. Defaults to ``True``. var: Whether to apply variance normalization. Defaults to ``True``. scale: Whether to apply a learnable scale according to ``params``. Defaults to ``True``. bias: Whether to apply a learnable bias according to ``params``. Defaults to ``True``. epsilon: A small float added to the variance to avoid division by zero. Defaults to ``1e-5``. fastvar: Whether to use a fast variance computation. Defaults to ``True``. dtype: Data type of the weights. Defaults to ``"float32"``. decay_rate: Decay rate for exponential moving average of mean and variance. If ``None``, no moving average is applied. Defaults to ``None``. name: Name of the module. Defaults to ``None``. **kwargs: Additional parameters that specify values for single axes, e.g. ``a=4``. b... [c]Th㈵>rNstatsparamsmeanvarscalebiasepsilonfastvarr decay_raterrGc  t| ||_||_||_||_||_||_||_||_ | |_ | |_ | |_ dSNr) superrrhrirjrkrlrmrnrorrprG)rrhrirjrkrlrmrnrorrprrG __class__s rrz Norm.__init__sq d###        $ rc l|j|td|jduo| ptj}t jj||j|jf|r"|j rttj dn|j |r"|j rttj dn|j |j rttjdnd|jrttjdnd|j|jd|j\}}}|jduo|otj }|r|j r=tjdtj d|jz|d|jz zz|j r=tjdtj d|jz|d|jz zz|S) Nz2training must be specified when decay_rate is usedrjrsrkrlrm)rjrkrlrmrnro)rprJr running_initrUnnnormrhrirjr_rTrkrlrSrmrnrorG set_state)rrbtraininguse_emarjrk update_emas rrZz Norm.__call__s ? &8+;QRR R/-Ux<3T2?CTCTw| J K  6=YYr|&1111PTPY3:UtxUbl////TX;?:O%(w777749=Lr'f5555LL   k   4_D0WXWboFWFWBW  y  BL004?BTQQUQ`M`Eaax  2<..@3!doJ]C^^r) rfTTTTrgTrNNr) r r!r"__doc__rKboolrRrrrrZ __classcell__rus@rreres0!&*"     UOsm8rrec ReZdZdZ d dedededeed ef fd Zd Z xZ S) LinearaLinear layer. Args: expr: Einstein string determining the axes along which the weight matrix is multiplied. Will be passed to ``einx.dot``. bias: Whether to apply a learnable bias. Defaults to ``True``. dtype: Data type of the weights. Defaults to ``"float32"``. name: Name of the module. Defaults to ``None``. **kwargs: Additional parameters that specify values for single axes, e.g. ``a=4``. TrNexprrmrrrGc t|||_||_||_||_dSrr)rtrrrmrrG)rrrmrrrGrus rrzLinear.__init__s@ d###    rctjj||jf|jrt t jdndt t jdd|jS)Nrmrsweight)rmr) rUrylinearrrmr_r rSrG)rrbs rrZzLinear.__call__sjw~ I :>Lr'f5555)999   k    r)TrN) r r!r"rrKrrrrrZrrs@rrrs  "      sm               rrc HeZdZdZd dededeedeffd ZdZ xZ S) DropoutaVDropout layer. Args: expr: Einstein string determining the axes along which dropout is applied. Will be passed to ``einx.elementwise``. drop_rate: Drop rate. name: Name of the module. Defaults to ``None``. **kwargs: Additional parameters that specify values for single axes, e.g. ``a=4``. Nr drop_raterrGc vt|||_||_||_dSrr)rtrrrrG)rrrrrGrus rrzDropout.__init__s7 d### " rc|r;tjj||jf|jt jd|jS|S)N)rrng)rUrydropoutrrr next_rng_keyrG)rrbr|s rrZzDropout.__call__sW  7? .O%%  +  Hrr) r r!r"rrKrRrrrrZrrs@rrrs|SU(3-Z]       rr)r r rU functoolsrhaiku._src.basertypingrrrrrVimport_rLr rSrTrKr_r[register_tensor_factoryrcModulererrr#rrrs ******333333333333 k'4((ZZZZZZZZ|574D @@ ""BL0 1@ 3-@ 3-@ C= @@@@<*+*OOOOO29OOOd! ! ! ! ! RY! ! ! Hbir