PiX+ddlZddlZddlZddlmZddlZddlmZm Z m Z m Z e dej dddDZedkredej d Zejd ZGd d Z dd e ejjjejjjejjjfde e fdZejjjdZGddejjZ edZ!GddejjZ"edZ#GddejjZ$edZ%dS)N)partial)CallableUnionOptionalAnyc#4K|]}t|VdSN)int).0is a/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/einx/nn/torch.py rs(BBAQBBBBBB.)rrz7einx.nn.torch requires PyTorch version >= 2, but found cdttvrtj|Sddlm}||S)Ncompilerr)dirtorchrallow_in_graph torch._dynamo_dynamo)funcrs r _allow_in_graphr sOSZZ~,,T222''''''%%d+++rrceZdZGddejjjZGddejjjZGddejj Z dS) ParamFactoryceZdZdZdZdS)ParamFactory.Concretec"||_||_dSr )paraminit)selfr r!s r __init__zParamFactory.Concrete.__init__sDJDIIIrcN|jt|jfSr )r rCacheKeyr!r"s r to_value_and_keyz&ParamFactory.Concrete.to_value_and_keys:|44TY??? ?rN)__name__ __module__ __qualname__r#r'rr Concreters7    @ @ @ @ @rr,c&eZdZdZdZdZdZdS)ParamFactory.CacheKeyc||_dSr r!r"r!s r r#zParamFactory.CacheKey.__init__# DIIIrc*t|jSr )hashr!r&s r __hash__zParamFactory.CacheKey.__hash__&s ?? "rcVt|tjo|j|jkSr ) isinstancerr%r!)r"others r __eq__zParamFactory.CacheKey.__eq__)s#e\%:;;W UZ@W WrcHt|j}||fSr )rTracerr!)r"backend virtual_argxs r to_tracerzParamFactory.CacheKey.to_tracer,s ##DI..Aa4KrN)r(r)r*r#r5r9r?r+rr r%r."sS    # # # X X X     rr%ceZdZdZdZdS)ParamFactory.Tracerc||_dSr r0r1s r r#zParamFactory.Tracer.__init__1r2rc |j|jn|dd}|}tj}tj|jg|||fg}|tdt|tr|dks|dkr&ttj jj d}n|dkrtj jj}n|d krtj jj}n|d krit!jfd |d D}t!jd |z dz }ttj jjd|dd}nTtd|dt|t(t*fr%ttj jj|}tj}tj||g|||fg}|S)Nr!)argsoutputinplace_updateszFMust specify init for tensor factory torch.nn.parameter.Uninitialized*get_at rearrangeg{Gz?)stdaddmultiplydotc g|] }| Sr+r+)r r shapes r z0ParamFactory.Tracer.__call__..Ms%J%J%J1eAh%J%J%Jrin_axisg?g۶%?ggg@)meanrIabz3Don't know which initializer to use for operation '')val)r!geteinxtracerTensorapply materialize ValueErrorr7strrttorchnnnormal_zeros_ones_npprodsqrt trunc_normal_r float constant_)r"rNkwargsr!r>rEfan_inrIs ` r __call__zParamFactory.Tracer.__call__4s$(I$54996::fd;S;SDA[''..F !! W"#V "A| \D#&& C8##t{':':"69>#9tDDDDDU]]!9>0DDZ''!9>/DDU]]W%J%J%J%Jy8I%J%J%JKKF'#,//2EEC"69>#?csVZ^abbbDD$%b[_%b%b%bcccD3,// Cvy~7TBBB[''..F !!S"#V "AHrN)r(r)r*r#rkr+rr r;rA0s2   ) ) ) ) ) rr;N) r(r)r*rWrXinputInputr,r% TensorFactoryr;r+rr rrs@@@@@4;$*@@@     4;$-   -----*-----rrr>r!ct|tjjjtjjjfr:t|tjjst ||S|S)aCreate a tensor factory for an uninitialized PyTorch parameter or buffer. If the given parameter is not initialized, this returns a tensor factory that calls the ``materialize`` method of ``uninitialized`` with the given shape and returns ``uninitialized``. Otherwise, the parameter is returned as is. Args: x: An instance of ``torch.nn.parameter.UninitializedParameter``, ``torch.nn.parameter.UninitializedBuffer`` or ``torch.nn.parameter.Parameter``. init: Initializer for the parameter. If ``None``, uses a default init method determined from the calling operation. Defaults to ``None``. Returns: A tensor factory with the given default parameters, or the parameter itself if it is already materialized. ) r7rr_ parameterUninitializedParameterUninitializedBuffer _subclasses FakeTensorrr,)r>r!s r r r `sg0 EH  5ux7I7] ^E-899$$Q---rct|tjjjtjjjfr@t|tjjs!t| SdSr ) r7rr_rprqrrrsrtr r')r>s r tensor_factoryrvse EH  5ux7I7] ^E-899Qxx((***trceZdZdZ ddeded ed ed ed ed ededeej efde ede ffd Z dZ xZS)NormaNormalization layer. Args: stats: Einstein string determining the axes along which mean and variance are computed. Will be passed to ``einx.reduce``. params: Einstein string determining the axes along which learnable parameters are applied. Will be passed to ``einx.elementwise``. Defaults to ``"b... [c]"``. mean: Whether to apply mean normalization. Defaults to ``True``. var: Whether to apply variance normalization. Defaults to ``True``. scale: Whether to apply a learnable scale according to ``params``. Defaults to ``True``. bias: Whether to apply a learnable bias according to ``params``. Defaults to ``True``. epsilon: A small float added to the variance to avoid division by zero. Defaults to ``1e-5``. fastvar: Whether to use a fast variance computation. Defaults to ``True``. dtype: Data type of the weights. Defaults to ``"float32"``. decay_rate: Decay rate for exponential moving average of mean and variance. If ``None``, no moving average is applied. Defaults to ``None``. **kwargs: Additional parameters that specify values for single axes, e.g. ``a=4``. b... [c]Th㈵>float32NstatsparamsrQvarscalebiasepsilonfastvardtype decay_rateric  Rt||_||_||_||_||_||_| |_| |_ |rT| R| dtj j tt| nd|_|rT| R| dtj j tt| nd|_|r=tj j tt| nd|_|r=tj j tt| nd|_d|_dS)NrQrr~F)superr#r|r}use_meanuse_varrrrriregister_bufferrr_rprrvarsrQr~rqrr initialized) r"r|r}rQr~rrrrrrri __class__s r r#z Norm.__init__s       $  J*  *>>T%[[QVEW>XX    DI  :)  ux)==DKKPUDV=WW    DHSX bEH  5 5DKKuse_emarQr~ update_emas r forwardz Norm.forwards/-]t}3D3\DL\H\ J K%J$-JT]#F F$, $ 6$**D"i3LL;    4_D0BT]  ] ] ]#IOODOdi$?1tCVZ^B^$^___?HNN4?TX#=T_ATX[@[#[\\\  ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]  $#D s;A/D66D:=D:) ryTTTTrzTr{N)r(r)r*__doc__r]boolrgrrrrrr#r __classcell__rs@r rxrxs.!)2&*+!+!+!+! +!  +!  +!+!+!+!U[#%&+!UO+!+!+!+!+!+!+!Zrrxc |j5tjj|||f||||||dd| cdddS#1swxYwYdS)Nr)rQr~rrrrr<)devicerWr_norm) r>r|r}rQr~rrrrris r rrs    w|                             s !6::c ZeZdZdZ d dededeejefde ffd Z d Z xZ S) LinearaLinear layer. Args: expr: Einstein string determining the axes along which the weight matrix is multiplied. Will be passed to ``einx.dot``. bias: Whether to apply a learnable bias. Defaults to ``True``. dtype: Data type of the weights. Defaults to ``"float32"``. **kwargs: Additional parameters that specify values for single axes, e.g. ``a=4``. Tr{exprrrric ttjjt t||_|rCtjjt t||_nd|_||_ ||_ dS)Nr) rr#rr_rprqrweightrrri)r"rrrrirs r r#zLinear.__init__s h(??d5kkRWFX?YY  *AAU TYHZA[[DIIDI  rcPt||j|j|j|jSr )_linearrrrrir"r>s r rzLinear.forwards q$)T[$)T[IIIr)Tr{) r(r)r*rr]rrrrrr#rrrs@r rrs)2 U[#%&   $JJJJJJJrrc||j5tjj||||fddi|cdddS#1swxYwYdS)Nr<r)rrWr_linear)r>rrrris r rrs   w~                             s 155c6eZdZdZdededeffd ZdZxZ S)DropoutaDropout layer. Args: expr: Einstein string determining the axes along which dropout is applied. Will be passed to ``einx.elementwise``. drop_rate: Drop rate. **kwargs: Additional parameters that specify values for single axes, e.g. ``a=4``. r drop_rateric rt||_||_||_dSr )rr#rrri)r"rrrirs r r#zDropout.__init__3s2  " rcV|jr!t||j|j|jS|Sr )r_dropoutrrrirs r rzDropout.forward:s+ = Aty$.$+FF FHr) r(r)r*rr]rgrr#rrrs@r rr)skSUcrrcz|j5tjj||f|dd|cdddS#1swxYwYdS)Nr)rr<)rrWr_dropout)r>rrris r rrAs   w                           s 044r )&rrWmath functoolsrnumpyrctypingrrrrtuple __version__split_version ImportErrorrrXimport_r^rr_rprqrr Parameterr rlregister_tensor_factoryrvModulerxrrrrrr+rr rs 111111111111 5BB!2!8!8!=!=bqb!ABBB B B f +cPUPacc d dd,,,   W % %DDDDDDDDZ   1 . $ & 3- F*+*YYYYY58?YYYx   "JJJJJUX_JJJB     eho0     r