`iddlZddlZddlZddlZddlm Z ddlm Z ddl mZddlmZddlmZddlmZddlmZddlZGd d eZd Zd Zd Zidddddddddddddddddddddddfdddddfd dd!dd"d#d$d#d%d#d&d#d#ddd'd'd#d#d#d(d#d)d)d*d+d,Zidd-dej dfdd-dd-dd-dd-dd-dd-dd-dd.dd.d d-d!d-d"d-d$d-d%d-d&d-d-d-d-ej dfej dfd-d-d-d-d-d.d.ej dfd/d,Z!d0Z"ej#d1Z$d2e%fd3Z&dgd6Z'dhd7Z(did9Z)dgd:Z*djd=Z+dkd>Z,dkd?Z-dld@Z.dmdAZ/dhdBZ0dhdCZ1dDZ2dEZ3dndGZ4dHZ5dIZ6dJZ7dKZ8dLZ9dMZ:dNZ;dOZdRZ?dSZ@GdTdUeZAGdVdWeAZBGdXdYeAZCGdZd[eAZDGd\d]eAZEdgd^ZFdod_ZG dpd`ZHdqdaZIdrdcZJdhddZKdsdeZLdtdfZMdS)uN)driver)runtime)cusparse)_dtype)device)stream)_utilc\eZdZdZedZdZejfdZ dZ dZ dZ dZ d S) MatDescriptorc||_dSN) descriptor)selfrs b/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/cupyx/cusparse.py__init__zMatDescriptor.__init__s $cFtj}t|Sr ) _cusparsecreateMatDescrr )clsdescrs rcreatezMatDescriptor.creates(**U###rc|jdfS)N)r)rs r __reduce__zMatDescriptor.__reduce__s{Brcp|rdS|jr"tj|jd|_dSdSr )rrdestroyMatDescrris_shutting_downs r__del__zMatDescriptor.__del__sJ      F ? #  %do 6 6 6"DOOO # #rc:tj|j|dSr )r setMatTyper)rtyps r set_mat_typezMatDescriptor.set_mat_type%sT_c22222rc:tj|j|dSr )rsetMatIndexBaser)rbases rset_mat_index_basez MatDescriptor.set_mat_index_base(s!$/488888rc:tj|j|dSr )rsetMatFillModer)r fill_modes rset_mat_fill_modezMatDescriptor.set_mat_fill_mode+ )<<<<z%_cast_common_type..3s 3 3 3!Q]ag]]]rcXg|]&}| |jkr|n|'Sr )r9astype)r:r;r9s rr<z%_cast_common_type..5sG    !}E1A1AAHHUOOOq   r) _functoolsreduce_numpy promote_types)xsdtypesr9s @r_cast_common_typerE2sV 3 3r 3 3 3F  f2F ; ;E       rc6|r tjStjSr )rCUSPARSE_OPERATION_TRANSPOSE CUSPARSE_OPERATION_NON_TRANSPOSE)transs r_transpose_flagrJ9s :5599rc|dkrd}n"|dkrd}n|dkrd}n|dkrd}nttt||z}||S)NfsdFcDz) TypeErrorgetattrr)namer9argsprefixrLs r_call_cusparserX@se || # # # 6D=))A 1d8Orcsrmv)@*csrmvExcsrmmcsrmm2csrgeamcsrgeam2)<#Ncsrgemmcsrgemm2)rZ.gthrspmv'r[)LinuxWindowsspmmi=( csr2dense csc2densecsrsort)rZNcscsortcoosortcoo2csr)rgN)rard)i$,N)i\+N)iP-N)csr2coocsr2csccsc2csr csr2cscEx2 csc2csrEx2 dense2csc dense2csrcsr2csr_compresscsrsm2csrilu02 denseToSparse sparseToDensespgemmspsm)i1N)N)iNct|trCtj}||vr$d|}t |||S|S)Nz/No version information specified for the OS: {}) isinstancedict _platformsystemformat ValueError)r;os_namemsgs r_get_avail_version_from_specrs`!T"$$ !  CJJCS// !z HrcRtjst}tj}nt }t j}||vr$d|}t|||\}}t|}t|}|||krdS|||krdSdS)Nz1No available version information specified for {}FT) _runtimeis_hip_available_cusparse_versionrget_build_version_available_hipsparse_version_driverrrr)rUavailable_versionversionr version_addedversion_removeds rcheck_availabilityrs ?.7-//8+-- $$$AHHNNoo%6t%<"M?0??M2?CCO W}%<%<u"w/'A'Au 4rreturncLtjtjSr )r getVersion_deviceget_cusparse_handlerrrrrs   ; = = > >>rFctdstd||jjsJ|s|jn|jddd}|dt |krt dtj}|\}} t|||\}}}|j } |tj || }tj|| j}tj|| j}t!d| |t#||jd|jd|j|j|jj|jjj|jjj|jjj|jj|j|jj|S)a\Matrix-vector product for a CSR-matrix and a dense vector. .. math:: y = \alpha * o_a(A) x + \beta y, where :math:`o_a` is a transpose function when ``transa`` is ``True`` and is an identity function otherwise. Args: a (cupyx.cusparse.csr_matrix): Matrix A. x (cupy.ndarray): Vector x. y (cupy.ndarray or None): Vector y. It must be F-contiguous. alpha (float): Coefficient for x. beta (float): Coefficient for y. transa (bool): If ``True``, transpose of ``A`` is used. Returns: cupy.ndarray: Calculated ``y``. rYzcsrmv is not available.Nrdimension mismatchr)r RuntimeErrorflags f_contiguousshapelenrrrrEr9_cupyzerosrAarrayctypesrXrJnnzdata_descrrptrindptrindices) ar;yalphabetatransaa_shapehandlemnr9s rrYrYs`, g & &64555 9,99 ,#6agg2GqzSVV-...  ( * *F DAq1a((GAq! GEy K5 ! ! L & & -E <e $ $ +D''  AGAJuz183F  *AIN,>  DIqvz +++ Hrc|jjjdzdkrdS|jjjdzdkrdS|jjjdzdkrdS|jjdzdkrdS||jjdzdkrdSdS)aCheck if the pointers of arguments for csrmvEx are aligned or not Args: a (cupyx.cusparse.csr_matrix): Matrix A. x (cupy.ndarray): Vector x. y (cupy.ndarray or None): Vector y. Check if a, x, y pointers are aligned by 128 bytes as required by csrmvEx. Returns: bool: ``True`` if all pointers are aligned. ``False`` if otherwise. rFNT)rrrr)rr;rs rcsrmvExIsAlignedrs$ v{!!ux}3!##uy~C1$$uvzC1u}c)Q..u 4rTc$tdstd||jjsJ|jdt |krt dtj}|j\}}t|||\}}}|j } |tj || }tj| } |r tjn tj} tj} t'j|| j}t'j|| j}t-|||sJtj|| | |jd|jd|j|j| |jj|jjj| |jjj|jjj|jj| |j| |jj| | } tj| d}|jjdzdksJtj || | |jd|jd|j|j| |jj|jjj| |jjj|jjj|jj| |j| |jj| | |jj|S) aMatrix-vector product for a CSR-matrix and a dense vector. .. math:: y = \alpha * A x + \beta y, Args: a (cupyx.cusparse.csr_matrix): Matrix A. x (cupy.ndarray): Vector x. y (cupy.ndarray or None): Vector y. It must be F-contiguous. alpha (float): Coefficient for x. beta (float): Coefficient for y. merge_path (bool): If ``True``, merge path algorithm is used. All pointers must be aligned with 128 bytes. Returns: cupy.ndarray: Calculated ``y``. r\zcsrmvEx is not available.Nrrrbr)!rrrrrrrrrrEr9rrr to_cuda_dtyperCUSPARSE_ALG_MERGE_PATHCUSPARSE_ALG_NAIVErHrArrrcsrmvEx_bufferSizerrrrrrremptyr\)rr;rrr merge_pathrrrr9datatypealgmode transa_flag bufferSizebufs rr\r\s?* i ( (86777 9,99 ,wqzSVV-...  ( * *F 7DAq1a((GAq! GEy K5 ! !#E**H5i//!4 -  Hdi  Hh ((J +j# & &C 8<#  " " " "   AGAJuz8 QV[_h  19>-  Hdi  Hh 666 Hrctdstd|j|jcxkrdksnJ|jjsJ||jjsJ|s|jn|jddd}|d|jdkrt dtj}|\}} |jd} t|||\}}}|tj || f|j d }| } |} tj||j j}tj||j j}t!d|j |t#||jd| |jd|j|j|jj|jjj|jjj|jjj|jj| |j|jj| |S) a|Matrix-matrix product for a CSR-matrix and a dense matrix. .. math:: C = \alpha o_a(A) B + \beta C, where :math:`o_a` is a transpose function when ``transa`` is ``True`` and is an identity function otherwise. Args: a (cupyx.scipy.sparse.csr): Sparse matrix A. b (cupy.ndarray): Dense matrix B. It must be F-contiguous. c (cupy.ndarray or None): Dense matrix C. It must be F-contiguous. alpha (float): Coefficient for AB. beta (float): Coefficient for C. transa (bool): If ``True``, transpose of A is used. Returns: cupy.ndarray: Calculated C. r]zcsrmm is not available.NrrrrrO)rrndimrrrrrrrErrr9rArrrXrJrrrrrrr) rrrPrrrrrrkrldbldcs rr]r]Ts, g & &64555 6QV q 7   9,99 ,#6agg2GqzQWQZ-...  ( * *F DAq  A1a((GAq!y KA - - C C L ( ( /E <ag & & -D''  Aqwqz15 AH'  19>-  CAFJ 555 Hr?c8tdstd|j|jcxkrdksnJ|jsJ|jjsJ||jjsJ|r|rJ|s|jn|jddd}|s|jn|jddd}|d|dkrtdtj } |\} } |d} t|||\}}}|tj | | f|j d }|jd} |jd}t|}t|}tj||j j}tj||j j}t%d|j | |||jd| |jd|j|j|jj|jjj|jjj|jjj|jj| |j|jj||S) aVMatrix-matrix product for a CSR-matrix and a dense matrix. .. math:: C = \alpha o_a(A) o_b(B) + \beta C, where :math:`o_a` and :math:`o_b` are transpose functions when ``transa`` and ``tranb`` are ``True`` respectively. And they are identity functions otherwise. It is forbidden that both ``transa`` and ``transb`` are ``True`` in cuSPARSE specification. Args: a (cupyx.scipy.sparse.csr): Sparse matrix A. b (cupy.ndarray): Dense matrix B. It must be F-contiguous. c (cupy.ndarray or None): Dense matrix C. It must be F-contiguous. alpha (float): Coefficient for AB. beta (float): Coefficient for C. transa (bool): If ``True``, transpose of A is used. transb (bool): If ``True``, transpose of B is used. Returns: cupy.ndarray: Calculated C. r^zcsrmm2 is not available.rNrrrrrO)rrrhas_canonical_formatrrrrrrrErrr9rJrArrrXrrrrrrr)rrrPrrrtransbrb_shaperrrrrrop_aop_bs rr^r^s4 h ' '75666 6QV q !!! ! 7   9,99 ,"6"" "#6agg2G#6agg2GqzWQZ-...  ( * *F DAq A1a((GAq!y KA - - '!*C '!*C 6 " "D 6 " "D L ( ( /E <ag & & -D!'dAGAJ171:qu AH'  19>-  CAFJ 555 HrcTtdstdt|tjjjs/tdt|t|tjjjs/tdt||j sJ|j sJ|j |j krtdtj}|j \}}t||\}}t!jdd}t%j|t$jt*}t/j|dzd} t%j||||jj|j|jjj|jjj|jj|j|jjj|jjj|j| jj|j jt/jtC|d} t/jtC||j"} t!j#||j"j }t!j#||j"j }tId|j"||||j|jj|j|jjj|jjj|jjj|j|jj|j|jjj|jjj|jjj|j| jj| jj| jjtjj| | | f|j } d | _%| S) eMatrix-matrix addition. .. math:: C = \alpha A + \beta B Args: a (cupyx.scipy.sparse.csr_matrix): Sparse matrix A. b (cupyx.scipy.sparse.csr_matrix): Sparse matrix B. alpha (float): Coefficient for A. beta (float): Coefficient for B. Returns: cupyx.scipy.sparse.csr_matrix: Result matrix. r_zcsrgeam is not available.unsupported type (actual: {})inconsistent shapesrirrT)&rrrcupyxscipysparse csr_matrixrSrtyperrrrrrErArrsetPointerModeCUSPARSE_POINTER_MODE_HOSTr rr xcsrgeamNnzrrrrrrrrintr9rrX_has_canonical_format) rrrrrrrrc_descrc_indptr c_indicesc_datarPs rr_r_s i ( (86777 a+6 7 7I7>>tAwwGGHHH a+6 7 7I7>>tAwwGGHHH !!! ! !!! !w!'.///  ( * *F 7DAq Q " "DAq ,r3  C  4666""$$G{1q5#&&H 1 QUAHM$5qy~7I QUAHM$5qy~7IHM-sz @@@  CHHc**I [S17 + +F L ( ( /E <ag & & -D171ej QUAFKO  19>-ty QUAFKO  19>-FKOX]->  %% H%QW & 6 6A"A Hrc4 tdstdt|tjjjs/tdt|t|tjjjs/tdt||j sJ|j sJ|j |j krtdtj}|j \}}t||\}}t!jdd}t%j|t$jt!j||jj}t!j||jj}t0}t5j|dzd} d} t7d |j||||j|jj|j|jjj |j!jj |j"jj |j|jj|j|jjj |j!jj |j"jj |j| | jj | } t5j| t j#} t%j$||||jj|j|j!jj |j"jj |jj|j|j!jj |j"jj |j| jj |jj| jj t5jtK|d} t5jtK||j}t7d|j||||j|jj|j|jjj |j!jj |j"jj |j|jj|j|jjj |j!jj |j"jj |j|jj | jj | jj | jj tjj|| | f|j }d |_&|S) rr`zcsrgeam2 is not available.rrrrrrcsrgeam2_bufferSizeExtrT)'rrrrrrrrSrrrrrrrrErArrrrrr9rr rrrXrrrrrrrint8 xcsrgeam2Nnzrr)rrrrrrrrrrnull_ptr buff_sizebuffrrrPs rr`r` s j ) )97888 a+6 7 7I7>>tAwwGGHHH a+6 7 7I7>>tAwwGGHHH !!! ! !!! !w!'.///  ( * *F 7DAq Q " "DAq ,r3  C  4666 L ( ( /E <ag & & -D""$$G{1q5#&&HH !'1ej!("5quafko  19>-ty!(:M qv{ 119>3EHhm&7 CCI ;y&+ . .D 1ah)15!(-2C AH/ 8I G. 0A  (((  CHHc**I [S17 + +FAG1ej!("5quafko  19>-ty!(:M qv{ 119>3EFKOX]->DIM +++  %% H%QW & 6 6A"A Hrctdstd|j|jcxkrdksnJ|jsJ|jsJ|s|jn|jddd}|s|jn|jddd}|d|dkrt dt j}|\}}|d} t||\}}|j dks |j dkr-tj j || f|j St|} t|} t!jd d } t%j|t$jt*} t/j|dzd }t%j|| | || ||jj|j |jjj|jjj|jj|j |jjj|jjj| j|jj| jjt/jtA| d }t/jtA| |j}tCd|j|| | || ||jj|j |jjj|jjj|jjj|jj|j |jjj|jjj|jjj| j|jj|jj|jjtj j |||f|| f }d |_"|S)aMatrix-matrix product for CSR-matrix. math:: C = op(A) op(B), Args: a (cupyx.scipy.sparse.csr_matrix): Sparse matrix A. b (cupyx.scipy.sparse.csr_matrix): Sparse matrix B. transa (bool): If ``True``, transpose of A is used. transb (bool): If ``True``, transpose of B is used. Returns: cupyx.scipy.sparse.csr_matrix: Calculated C. rbzcsrgemm is not available.rNrrrrr8rrrT)#rrrrrrrrrErrrrrr9rJrArrrrr rr xcsrgemmNnzrrrrrrrrrXr)rrrrrrrrrrrrrrrrrrPs rrbrbPs i ( (86777 6QV q !!! ! !!! !#6agg2G#6agg2GqzWQZ-...  ( * *F DAq A Q " "DAquzzQUaZZ{!,,aV17,CCC 6 " "D 6 " "D ,r3  C  4666""$$G{1q5#&&H dAq!QX%8!%  19>-qx/BAE  19>-w/A 3:? ,,,  CHHc**I [S17 + +F17dAq!QX%8!%  *AIN,> QUAFKOQX]5F FKOX]->  %% H%aV & 5 5A"A Hrc| tdstd|j|jcxkrdksnJt|tjjjs/td t|t|tjjjs/td t||j sJ|j sJ|j d|j dkrtd||jdksJt|tjjjs/td t||j sJ|j d|j dks|j d|j dkrtdtjr&t!jd krtd t%j}|j \}}|j \}} |t)||\}}nt)|||\}}}t+j} t/j||jj}d} |$| } t6} d}| }| }| }ngt/j||jj}|j} |j} |j}|jjj }|j!jj }|j"jj }tGd |j||| ||j|jj$|j|j!jj |j"jj |jj$|j|j!jj |j"jj | | j$|||| }tKj&|t.j'}t/j&d d }t+j(|t*j)t6}tKj&|dzd }t+j*||| ||jj$|j|j!jj |j"jj |jj$|j|j!jj |j"jj | j$||||j$|jj |jj| |jj tKj&tW|d }tKj&tW||j}tGd|j||| ||j|jj$|j|jjj |j!jj |j"jj |jj$|j|jjj |j!jj |j"jj | | j$|||||j$|jj |jj |jj | |jj tjj|||f|| f}d|_,t+j-| |S)aMatrix-matrix product for CSR-matrix. math:: C = alpha * A * B + beta * D Args: a (cupyx.scipy.sparse.csr_matrix): Sparse matrix A. b (cupyx.scipy.sparse.csr_matrix): Sparse matrix B. d (cupyx.scipy.sparse.csr_matrix or None): Sparse matrix D. alpha (scalar): Coefficient beta (scalar): Coefficient Returns: cupyx.scipy.sparse.csr_matrix rczcsrgemm2 is not available.rrrrmismatched shapeNrz'd != None is supported since ROCm 4.2.0csrgemm2_bufferSizeExtrrrT).rrrrrrrrrSrrrrrrrrrrrrErcreateCsrgemm2InforArr9rr rrrrrrrrXrrrrrr xcsrgemm2NnzrrdestroyCsrgemm2Info)rrrNrrrrr_rinfor beta_datad_descrd_nnzd_datad_indptr d_indicesrrc_nnzrrrrrPs rrcrcsO" j ) )97888 6QV q a+6 7 7I7>>tAwwGGHHH a+6 7 7I7>>tAwwGGHHH !!! ! !!! !wqzQWQZ+,,,}v{{{{!U[/:;; M;BB477KKLL L%%%% 71: # #qwqzQWQZ'?'?/00 0 ? Jw8::S@@HII I  ( * *F 7DAq 7DAqy A&&11#Aq!,,1a  ' ) )D L ( ( /EHy &&(( |D!'**1I (8=$IN&  !'1aQX%8!%  19>-qx/BAE  19>-y':L xD **I ;y&+ . .D LS ! !E VY%IJJJ""$$G{1q5#&&H 1a,aeQX]5F AH/ 8I G.xHM-u|/@$  CJJ,,I [UQW - -FAG1aQX%8!%  *AIN,> QUAFKOQX]5F Iw'95&)W/ 9>-tTY]DDD  %% H%aV & 5 5A"A !$''' Hrctdstd|j}|jdvsJ|t j|j|d}n|jjsJtj }td|j||jd|jd|j j |jjj|jjj|jjj|jj|jd |S) a!Converts CSR-matrix to a dense matrix. Args: x (cupyx.scipy.sparse.csr_matrix): A sparse matrix to convert. out (cupy.ndarray or None): A dense metrix to store the result. It must be F-contiguous. Returns: cupy.ndarray: Converted result. rkzcsr2dense is not available.fdFDNrOr9orderrr)rrr9charrrrrrrrrXrrrrrrr;outr9rs rrkrks k * *:8999 GE :     {k!'c:::y%%%%  ( * *FQW AGAJ(;  *AIN,>  agaj """ Jrctdstd|j}|jdvsJ|t j|j|d}n|jjsJtj }td|j||jd|jd|j j |jjj|jjj|jjj|jj|jd |S) a!Converts CSC-matrix to a dense matrix. Args: x (cupyx.scipy.sparse.csc_matrix): A sparse matrix to convert. out (cupy.ndarray or None): A dense metrix to store the result. It must be F-contiguous. Returns: cupy.ndarray: Converted result. rlzcsc2dense is not available.rNrOrrr)rrr9rrrrrrrrrXrrrrrrrs rrlrls k * *:8999 GE :     {k!'c:::y%%%%  ( * *FQW AGAJ(;  +QX]->  agaj """ Jrc tdstd|j}|dkrdStj}|j\}}t j|||||jj j |j j j }tj |d}tj |d}|j }t j|||j j t j|||||jj|jj j |j j j |j j |j j tdrJt'd|j|||j j |j j j |j j t jdSt,||j } t0|} t j|| j| jdS)zzSorts indices of CSR-matrix in place. Args: x (cupyx.scipy.sparse.csr_matrix): A sparse matrix to sort. rmzcsrsort is not available.rNrrre)rrrrrrrxcsrsort_bufferSizeExtrrrrrrcopycreateIdentityPermutationxcsrsortrrrXr9CUSPARSE_INDEX_BASE_ZEROSpVecDescriptorrDnVecDescriptorgatherdesc r;rrrr buffer_sizerP data_origdesc_xdesc_ys rrmrm> i ( (86777 %C axx  ( * *F 7DAq21c18=, K +k3 ' 'C CA I 'QVZ@@@ 1c18. 0A AFJ 666&!!; AG C+QV[_ FJ : < < < < < !''1622 '' 22fk:::::rc tdstd|j}|dkrdStj}|j\}}t j|||||jj j |j j j }tj |d}tj |d}|j }t j|||j j t j|||||jj|jj j |j j j |j j |j j tdrJt'd|j|||j j |j j j |j j t jdSt,||j } t0|} t j|| j| jdS)zzSorts indices of CSC-matrix in place. Args: x (cupyx.scipy.sparse.csc_matrix): A sparse matrix to sort. rnzcscsort is not available.rNrrre)rrrrrrrxcscsort_bufferSizeExtrrrrrrr r xcscsortrrrXr9rrrrrrrs rrnrndrrrc tdstd|j}|dkrdStj}|j\}}t j|||||jj j |j j j }tj |d}tj |d}|j } t j|||j j |dkrNt j|||||jj j |j j j |j j |j j nc|dkrNt j|||||jj j |j j j |j j |j j nt%d |jjd krtd rIt+d |j||| j j |j j j |j j t jnZt.||j } t2| } t j|| j| j|dkr d |_dSdS) zSorts indices of COO-matrix in place. Args: x (cupyx.scipy.sparse.coo_matrix): A sparse matrix to sort. sort_by (str): Sort the indices by row ('r', default) or column ('c'). rozcoosort is not available.rNrrrrPz!sort_by must be either 'r' or 'c'?reF)rrrrrrrxcoosort_bufferSizeExtrowrrcolrrr r  xcoosortByRowxcoosortByColumnrr9rrXrrrrrrr) r;sort_byrrrrrrrrrrs rroros/ i ( (86777 %C axx  ( * *F 7DAq21c15:>15:>;;K +k3 ' 'C CA I 'QVZ@@@#~~ Aq#quz~quz~ FJ  & & & & C" Aq#quz~quz~ FJ  & & & &<===w|s f % % ? Y^/ I> @ @ @ @ %++Aqv66F$++I66F  VV[&+ > > >#~~"'~rctj}|jd}|j}|dkrt j|dzd}nTt j|dzd}tj||j j j |||j j tj tjj|j |j|f|jS)Nrrrr)rrrrrrrrxcoo2csrr!rrrrrrrr")r;rrrrs rrprp  ( * *F  A %C axxQUC((QUC(( AEJNC KOY? A A A ;  ( ( qw ) 0 00rctj}|jd}|j}|dkrt j|dzd}nTt j|dzd}tj||j j j |||j j tj tjj|j |j|f|jS)Nrrrr)rrrrrrrrr'r"rrrrrr csc_matrixr!)r;rrrrs rcoo2cscr+r(rctdstdtj}|jd}|j}t j|d}tj ||j j j |||j j tj tjj|||ff|jS)aDConverts a CSR-matrix to COO format. Args: x (cupyx.scipy.sparse.csr_matrix): A matrix to be converted. data (cupy.ndarray): A data array for converted data. indices (cupy.ndarray): An index array for converted data. Returns: cupyx.scipy.sparse.coo_matrix: A converted matrix. rqzcsr2coo is not available.rrr)rrrrrrrrrxcsr2coorrrrrrr coo_matrix)r;rrrrrr!s rrqrqs i ( (86777  ( * *F  A %C +c3  C  !338<*,,, ;  ( ( W~ag ) / //rctdstdtj}|j\}}|j}t j||j}t j|d}|dkrt j |dzd}nt j|dzd}td|j|||||j j j |j j j |jj j |j j |j j |j j tjtjt$jj|||f|jS)Nrrcsr2csc is not available.rrrr)rrrrrrrrr9rrXrrrrrCUSPARSE_ACTION_NUMERICrrrrr*r;rrrrrrrs rrrrr) i ( (86777  ( * *F 7DAq %C ;sAG $ $Dk#s##G axxQUC((QUC(( qw Aq#qv{ HM qy~1 IM7<+V[_  -  .  0 0 0 ;  ( ( wqw ) 0 00rc.tdstdtj}|j\}}|j}t j||j}t j|d}|dkrt j |dzd}nWt j|dzd}tj |j}tj } tj} tj} tj|||||jjj|jjj|jjj|jj|jj|jj|| | | } t j| t*j} tj|||||jjj|jjj|jjj|jj|jj|jj|| | | | jjt0jj|||f|jS)Nrtzcsr2cscEx2 is not available.rrrr)rrrrrrrrr9rrrrr1rCUSPARSE_CSR2CSC_ALG1csr2cscEx2_bufferSizerrrrrArrtrrrr*r;rrrrrrrx_dtypeactionibasealgorbuffers rrtrt  l + +;9:::  ( * *F 7DAq %C ;sAG $ $Dk#s##G axxQUC((QUC((&qw//22.5 Aq#qv{ 0A IN  v{ L gvud<< [&+66 Aq#qv{ 0A IN  v{ L gvudFKO M M M ;  ( ( wqw ) 0 00rcPtj}|jd}|j}t j|d}t j||jj j |||j j t j tj j|||ff|jS)aDConverts a CSC-matrix to COO format. Args: x (cupyx.scipy.sparse.csc_matrix): A matrix to be converted. data (cupy.ndarray): A data array for converted data. indices (cupy.ndarray): An index array for converted data. Returns: cupyx.scipy.sparse.coo_matrix: A converted matrix. rrr)rrrrrrrr-rrrrrrrr.)r;rrrrrr"s rcsc2coor?*s ( * *F  A %C +c3  C  !338<*,,, ;  ( ( ~ag ) / //rctdstdtj}|j\}}|j}t j||j}t j|d}|dkrt j |dzd}nt j|dzd}td|j|||||j j j |j j j |jj j |j j |j j |j j tjtjt$jj|||f|jS)Nrsr0rrrrrr)rrrrrrrrr9rrXrrrrrr1rrrrrr2s rrsrsBr3rc.tdstdtj}|j\}}|j}t j||j}t j|d}|dkrt j |dzd}nWt j|dzd}tj |j}tj } tj} tj} tj|||||jjj|jjj|jjj|jj|jj|jj|| | | } t j| t*j} tj|||||jjj|jjj|jjj|jj|jj|jj|| | | | jjt0jj|||f|jS)Nruzcsc2csrEx2 is not available.rrrr)rrrrrrrrr9rrrrr1rr5r6rrrrrArrtrrrrr7s rruruZr=rctdstd|jdksJtj|}t jdd}tj}|j \}}t }tj|d}td|j |tj|||j|jj||jj|jj t)|}tj||j }tj|dzd}tj|d} td|j ||||j|jj||jj|jj| jj|jj t*jj|| |f|j } d | _| S) zConverts a dense matrix in CSC format. Args: x (cupy.ndarray): A matrix to be converted. Returns: cupyx.scipy.sparse.csc_matrix: A converted matrix. rvzdense2csc is not available.rrrr8rrrT)rrrrasfortranarrayrArrrrr rrXr9rCUSPARSE_DIRECTION_COLUMNrrrrrrrrr*r) r;rrrrr nnz_per_colrrrcscs rrvrvxs k * *:8999 6Q;;;; QA ,r % % %C  ( * *F 7DAq  " "E+a%%K qw 3Q5;K  A{'+SZ_>>> c((C ;sAG $ $D [Q $ $Fk#s##GQW1e&  A{'+ w|' ::: +  ' 'w(?qw ' O OC $C Jrctdstd|jdksJtj|}t jdd}tj}|j \}}t }tj|d}td|j |tj|||j|jj||jj|jj t)|}t*jr|dkrt/d tj||j }tj|d zd}tj|d} td|j ||||j|jj||jj|jj|jj| jj t0jj|| |f|j } d | _| S) zConverts a dense matrix in CSR format. Args: x (cupy.ndarray): A matrix to be converted. Returns: cupyx.scipy.sparse.csr_matrix: A converted matrix. rwzdense2csr is not available.rrrr8rr@hipSPARSE currently cannot handle sparse matrices with null ptrsrrT)rrrrrCrArrrrr rrXr9rCUSPARSE_DIRECTION_ROWrrrrrrrrrrrrr) r;rrrrr nnz_per_rowrrrcsrs rrwrws k * *:8999 6Q;;;; QA ,r % % %C  ( * *F 7DAq  " "E+a%%K qw 0!Q8H  A{'+SZ_>>> c((C? !88>?? ? ;sAG $ $D [Q $ $Fk#s##GQW1e&  A{'+ v{ (8 ::: +  ' 'w(?qw ' O OC $C JrcTtdstd|jjdvsJt j}|j\}}tj|d}td|j|||j j |j j j |jj j |j j | }tj||j}tj|dzd}tj|d} td|j||||j j |j j j |jj j |jj j |j|j j |j j | j j |j j |t$jj|| |f|jS)Nrxz"csr2csr_compress is not available.rr nnz_compressrr)rrr9rrrrrrrXrrrrrrrrrrrr) r;tolrrrrJrrrrs rrxrxst 0 1 1A?@@@ 7<6 ! ! ! !  ( * *F 7DAq+a%%K 18&  *K,<,@# G GC ;sAG $ $D [Q $ $Fk#s##GAG1ah)  +QX]-> {#TY]GL4D   ;  ( ( wqw ) 0 00rc||dkr tjS|dkr tjS|dkr tjSt)Nuint16int32int64)rCUSPARSE_INDEX_16UCUSPARSE_INDEX_32ICUSPARSE_INDEX_64IrSr8s r_dtype_to_IndexTyperVsC ++ '  ++ '  ++rc0eZdZddZejfdZdZdS)BaseDescriptorNc0||_||_||_dSr )rgetdestroy)rrrZ destroyers rrzBaseDescriptor.__init__s   rc|rdS|j d|_dS|j#||jd|_dSdSr )r[rrs rr zBaseDescriptor.__del__sZ      F < DIII Y " LL # # #DIII# "rcn|j(t||j|Str )rZrTrAttributeError)rrUs r __getattr__zBaseDescriptor.__getattr__s. 8 488DI..55 5r)NN)r2r3r4rr rr r`rrrrXrXsS!!!! (-'=rrXc*eZdZedZdZdS)SpMatDescriptorc ptjj|sJ|j\}}t j}tj|j }|j dkrt j |||j |j jj|jjj|jjjt#|j j t#|jj || }t j}n/|j dkrrt j|||j |jjj|jjj|jjjt#|jj || }t j}n|j dkrt j|||j |j jj|jjj|jjjt#|j j t#|jj || }d}n't1d |j t j}t5|||S)NrKcoorFz3csr, csc and coo format are supported (actual: {}).)rrrissparserrrrrr9r createCsrrrrrrrVcsrGet createCoor!r"cooGet createCscr destroySpMatrb) rrrowscolsidx_base cuda_dtyperrZr[s rrzSpMatDescriptor.creates{!**1-----W d5)!'22 8u  &dAE18=#4ain6H !4QX^!D!D#AIO44h LLD"CC X  &dAE15:>15:> !4QU[!A!A*&&D"CC X  &dAE18=#4ain6H !4QX^!D!D#AIO44h LLDCC--3VAH-=-=?? ?(tS'222rc<tj|j||dSr )rspMatSetAttributer)r attributers r set_attributezSpMatDescriptor.set_attribute-s#DIy$?????rN)r2r3r4r5rrsrrrrbrb sA33[3:@@@@@rrbc$eZdZedZdS)rc (|j}tj|j}t j|||jj|jjt|jtj |}tj }tj }t|||Sr ) sizerrr9r createSpVecrrrVrspVecGet destroySpVecr)ridxr;rrorrZr[s rrzSpVecDescriptor.create3swf)!'22 $S#sx|QVZ%8%C%C%.%G%/11 (tS'222rNr2r3r4r5rrrrrr1s- 3 3[ 3 3 3rrc$eZdZedZdS)rctj|j}tj|j|jj|}tj}tj }t|||Sr ) rrr9r createDnVecrvrrdnVecGet destroyDnVecr)rr;rorrZr[s rrzDnVecDescriptor.createBsP)!'22 $QVQVZDD (tS'222rNr{rrrrr@s-33[333rrc$eZdZedZdS)DnMatDescriptorc.|jdksJ|jjsJ|j\}}|}t j|j}tj||||j j |tj }tj }tj }t|||S)Nr)rrrrrrr9r createDnMatrrCUSPARSE_ORDER_COLdnMatGet destroyDnMatr) rrrlrmldrorrZr[s rrzDnMatDescriptor.createMsv{{{{w####W d )!'22 $T4QVZ%.%ACC (tS'222rNr{rrrrrKs- 3 3[ 3 3 3rrc vtdstdt|tjjjra|j}t|tjjjs1d t|}t||}| }t|tjjjtjjj fs/td t||s|j n|j ddd}|dt|krtd|jsJ|\} } t#|||\}}}|t%j| |j}n"t|| krtd|jd kr|d |St.|} t2|} t2|} t5j}t9|}t;j||jj}t;j||jj}tAj!|j}tDj#}tEj$|||j%| j&| j&|j%| j&|| }t%j'|t$j(}tEj)|||j%| j&| j&|j%| j&|||j%j* |S) aMultiplication of sparse matrix and dense vector. .. math:: y = \alpha * op(A) x + \beta * y Args: a (cupyx.scipy.sparse.csr_matrix, csc_matrix or coo_matrix): Sparse matrix A x (cupy.ndarray): Dense vector x y (cupy.ndarray or None): Dense vector y alpha (scalar): Coefficient beta (scalar): Coefficient transa (bool): If ``True``, op(A) = transpose of A. Returns: cupy.ndarray rfzspmv is not available."aT must be csr_matrix (actual: {})rNrrrr)+rrrrrrr*TrrrrSr.rrrrrErrr9rfillrbrrrrrJrArrrrrCUSPARSE_MV_ALG_DEFAULTspMV_bufferSizerrrrspMVr)rr;rrrraTrrrrdesc_arrrrroalgrrs rrfrf[s& f % %53444!U['233 S"ek0;<< !6==d2hhGGCC..  a%+,7+,79 : :I7>>tAwwGGHHH#6agg2GqzSVV-... !!! ! DAq1a((GAq!y K17 # # Q1-...uzz q   # #A & &F  # #A & &F  # #A & &F  ( * *F 6 " "D L ( ( /E <ag & & -D%ag..J  +C)&$ *0+v{DI*0+z3HHI ;y%* - -D N64V[&+9fk:sDIMKKK Hrc \tdstd|j|jcxkrdksnJ|jjsJ||jjsJt |t jjj ra|j }t |t jjj s1d t|}t||}| }t |t jjj t jjjfs/td t||s|jn|jddd} |s|jn|jddd} | d| d krt#d |jsJ| \} } | \} }t'|||\}}}|t)j| |f|jd }n1|jd | ks|jd|krt#d |jd kr|d |St2|}t6|}t6|}t9j}t=|}t=|}t?j ||jj!}t?j ||jj!}tEj#|j}tHj%}tIj&||||j'|j(|j(|j'|j(|| }t)j)|t(j*}tIj+||||j'|j(|j(|j'|j(|||j'j, }|S) aMultiplication of sparse matrix and dense matrix. .. math:: C = \alpha * op(A) op(B) + \beta * C Args: a (cupyx.scipy.sparse.csr_matrix, csc_matrix or coo_matrix): Sparse matrix A b (cupy.ndarray): Dense matrix B c (cupy.ndarray or None): Dense matrix C alpha (scalar): Coefficient beta (scalar): Coefficient transa (bool): If ``True``, op(A) = transpose of A. transb (bool): If ``True``, op(B) = transpose of B. Returns: cupy.ndarray rjzspmm is not available.rNrrrrrrrO)-rrrrrrrrrr*rrrrrSr.rrrrErrr9rrrbrrrrrJrArrrrrCUSPARSE_MM_ALG_DEFAULTspMM_bufferSizerrrrspMMr)rrrPrrrrrrrrrrrrrdesc_bdesc_crrrrorrrs rrjrjsY( f % %53444 6QV q 7   9,99 ,!U['233 S"ek0;<< !6==d2hhGGCC..  a%+,7+,79 : :I7>>tAwwGGHHH#6agg2G#6agg2GqzWQZ-... !!! ! DAq DAq1a((GAq!y KA - - qAGAJ!OO-...uzz q   # #A & &F  # #A & &F  # #A & &F  ( * *F 6 " "D 6 " "D L ( ( /E <ag & & -D%ag..J  +C)&$ej*0+v{DI*0+z3HHI ;y%* - -DvtT5:v{%{DIv{)3 ??I Hrc tdstdtjj|s3tjj|stdt|tj std|j dvrtd|j d|j dcxkr|j dksntd |j |j krtd |d ur tj}n3|d ur tj}n"td ||d ur tj} n3|d ur tj} n"td||d urd} n)|d urd} n"td||d ur tj} n3|d ur tj} n"td||j } | jdkrd} nL| jdkrd} n>| jdkrd} n0| jdkrd} n"td| t1t| dz}t1t| dz}t1t| dz}|d us|dkr tj}nb|d us|dkr tj}nK|dkr#| jdvr tj}n/tj}n"td |tjj|rV|tjkrtd!|j}tjj|sJd|z }d|z }|j d}|j dkrdn |j d}|jrtj}|}n%|jrtj}|}ntd"t?j }tCj"|| #}tF$}|%tj&|'tj(|)||*| tj+}||| |||||j,|j-j.|j/|j.j.j0|j1j.j0|j2j.j0|j.j0||| }tj3|ftBj4#}||| |||||j,|j-j.|j/|j.j.j0|j1j.j0|j2j.j0|j.j0||| |j.j0||| |||||j,|j-j.|j/|j.j.j0|j1j.j0|j2j.j0|j.j0||| |j.j0tkj67tj8|d$S)%aVSolves a sparse triangular linear system op(a) * x = alpha * b. Args: a (cupyx.scipy.sparse.csr_matrix or cupyx.scipy.sparse.csc_matrix): Sparse matrix with dimension ``(M, M)``. b (cupy.ndarray): Dense vector or matrix with dimension ``(M)`` or ``(M, K)``. alpha (float or complex): Coefficient. lower (bool): True: ``a`` is lower triangle matrix. False: ``a`` is upper triangle matrix. unit_diag (bool): True: diagonal part of ``a`` has unit elements. False: diagonal part of ``a`` has non-unit elements. transa (bool or str): True, False, 'N', 'T' or 'H'. 'N' or False: op(a) == ``a``. 'T' or True: op(a) == ``a.T``. 'H': op(a) == ``a.conj().T``. blocking (bool): True: blocking algorithm is used. False: non-blocking algorithm is used. level_info (bool): True: solves it with level information. False: solves it without level information. Note: ``b`` will be overwritten. ryzcsrsm2 is not available.z"a must be CSR or CSC sparse matrixzb must be cupy.ndarray)rrb.ndim must be 1 or 2rrz invalid shapedtype mismatchTFUnknown lower (actual: {})Unknown unit_diag (actual: {})zUnknown blocking (actual: {})Unknown level_info (actual: {})rLrMrNrOrPrQrRInvalid dtype (actual: {})csrsm2_bufferSizeExtcsrsm2_analysis csrsm2_solveNrHfdzUnknown transa (actual: {})z@If matrix is CSC format and complex dtype,transa must not be 'H''b must be F-contiguous or C-contiguous.r8N)9rrrrrisspmatrix_csrisspmatrix_cscrrrndarrayrrr9rSrCUSPARSE_FILL_MODE_LOWERCUSPARSE_FILL_MODE_UPPERrCUSPARSE_DIAG_TYPE_NON_UNITCUSPARSE_DIAG_TYPE_UNITCUSPARSE_SOLVE_POLICY_NO_LEVELCUSPARSE_SOLVE_POLICY_USE_LEVELrrTrHrG&CUSPARSE_OPERATION_CONJUGATE_TRANSPOSEr _f_contiguous _c_contiguousrrrArr rr$CUSPARSE_MATRIX_TYPE_GENERALr(rr,r1createCsrsm2Inforrrrrrrrr_streamget_current_stream synchronizedestroyCsrsm2Info)rrrlower unit_diagrblocking level_infor+r0r;policyr9thelperanalysissolvernrhsrrra_descrws_sizewss rryrys#: h ' '75666 K  - -a 0 0? K  - -a 0 0?=>>> a ' '31222vV0111 GAJ!'!* 2 2 2 2 2 2 2 2)))w!'())) }}6 %6 5<1 2 2E &C--; 46S==7 3 :  ;FFEFF6==fEEFFF {((++" YE E E899 9 C{!0033333V M   A! 11DD; D7BCCC  ( * *F Le , , ,E  ! ! # #F  >??? i@AAA Y''' Y'''  % ' 'DfVT661dAE\&(916;?X]& (:AFJ6##G gZv{ 3 3 3B HVT661dAE5<;L   1B Y^ S$ MMM E&$4 8I  QV[_ahm.? ). afj#tVRW[JJJ    ,,... %%%%%rc tdstdtjj|st d|jd|jdkr'td |j|dur tj }n3|dur tj }n"td ||j }|jd krd }nL|jd krd }n>|jd krd}n0|jdkrd}n"t d |tt|dz}tt|dz}tt|dz}ttd}t!j} |jd} |j} t&} | tj| tjtj} || | | | j|jjj|jjj|jjj| }t?j |ftBj"}tCj dtBj#}|| | | | j|jjj|jjj|jjj| ||jj  || | |j$jn6#tJ$r)td |dwxYw|| | | | j|jjj|jjj|jjj| ||jj  || | |j$jdS#tJ$r)td |dwxYw)aComputes incomplete LU decomposition for a sparse square matrix. Args: a (cupyx.scipy.sparse.csr_matrix): Sparse matrix with dimension ``(M, M)``. level_info (bool): True: solves it with level information. False: solves it without level information. Note: ``a`` will be overwritten. This function does not support fill-in (only ILU(0) is supported) nor pivoting. rzzcsrilu02 is not available.za must be CSR sparse matrixrrzinvalid shape (a.shape: {})FTrrLrMrNrOrPrQrRrcsrilu02_bufferSizecsrilu02_analysisxcsrilu02_zeroPivotr8rza({0},{0}) is missingzu({0},{0}) is zeroN)&rrrrrrrSrrrrrrr9rrTrrrr rr$rr(rcreateCsrilu02InforrrrrrrrArrQr Exception)rrrr9rrrrcheckrrrrrrrpositions rrzrz|s j ) )97888 ;  , ,Q / /75666wqzQWQZ6==agFFGGGU9 t  ::AA*MMNNN GE zS  s    s    s   4;;EBBCCC Y$9 9 : :Fy!&9"9::H Iq:~ . .E I4 5 5E  ( * *F  A %C    ! !Di<===I>???  ' ) )DfVQT_afkoX]& (:DBBG gZv{ 3 3 3B|D 555H HVQT_afko X]   2D&"'+OOOF fdHO01111 FFF077 DDEEEF E&!S$/16;? (- QY^/vrw{LLLC fdHO011111 CCC-44Xa[AABBBCs>L3M N663O)rKc@tdstd|jdksJ|jjdvsJt j|}t|}|dkr1tj j |j |j}n|dkr1tj j |j |j}nY|dkr1tj j |j |j}n"t!d |t$|}t&j}t+j}t'j||j|j|}t j|t j}t'j||j|j||jjt=jd d } t=jd d } t=jd d } t'j |j| j!j| j!j| j!jtE| } tFj$r| d krtKd |dkre|j&} t j| d } t j| |j}tj j || | f|j }n|dkre|j&} t j| d } t j| |j}tj j || | f|j }ny|dkrst j'| d }t j'| d }t j| |j}tj j |||ff|j }t$|}t'j(||j|j||jjd|_)|S)a0Converts a dense matrix into a CSR, CSC or COO format. Args: x (cupy.ndarray): A matrix to be converted. format (str): Format of converted matrix. It must be either 'csr', 'csc' or 'coo'. Returns: cupyx.scipy.sparse.spmatrix: A converted sparse matrix. r{zdenseToSparse is not available.rrrKr8rFrdzunsupported format (actual: {})rrRrHrrT)*rrrr9rrrCrrrrrrrr*r.rSrrbr"CUSPARSE_DENSETOSPARSE_ALG_DEFAULTrrdenseToSparse_bufferSizerrrdenseToSparse_analysisrrrAr spMatGetSizerrrrrrrdenseToSparse_convertr)r;rrrrr;rrr num_rows_tmp num_cols_tmprrrrr!r"s rr{r{s o . .><=== 6Q;;;; 7<6 ! ! ! ! QA  # #A & &F  K  ) )!' ) A A 5 K  ) )!' ) A A 5 K  ) )!' ) A A9@@HHIII  # #A & &F  7D  ( * *F266;39;FFI ;y%* - -D $VV[%+[$ GGG<111L<111L ,q ( ( (C 6; (;(@'.3SZ_FFF c((C? !88>?? ? +c3''{3(( K  ) )4&*A01 * 9 9 5+c3''{3(( K  ) )4&*A01 * 9 9 5k#s##k#s##{3(( K  ) )4#s*<=== GE :     {k!'c:::y%%%%yE!!!!  # #A & &F%%c**H  7D  ( * *F266;3;=$HHI ;y%* - -D? 5A::>?? ? FFK$M4@@@ Jrc  tdstd|durd}n|durd}n|dvrtd|d tjj|rntjj|rE|dkr |j}d}n1|dkr |j}d}n!|d kr| j}d}| }n4tjj |rntd |j sJ|j d krd}| d d }n|j dkrd}ntd|jd|jd cxkr|jdksntd|j}|jdvr"t#d|||jkrt#d|dur t&j}n3|dur t&j}n"td||dur t&j} n3|dur t&j} n"td||dkr t&j} n5|dkr t&j} n"|jdvr t&j} n t&j} |jr t&j} nP|jr:t'jdkrtd|j}t&j} ntd|j\} } | t&jkr |j\} }n |j\}} | |f}t=j||jd}tAj!}tD#|}tH#|}tH#|}t'j%}tMj'||jj(}tSj*|j}t&j+} |,t&j-||,t&j.| t'j/|| | |j0|j1|j1|j1||| }t=j2|t<j3}t'j4|| | |j0|j1|j1|j1||||j0j5 t'j6|| | |j0|j1|j1|j1||||j0j5 |r| d }|t'j7|S#t'j7|wxYw)aSolves a sparse triangular linear system op(a) * x = alpha * op(b). Args: a (cupyx.scipy.sparse.csr_matrix or cupyx.scipy.sparse.coo_matrix): Sparse matrix with dimension ``(M, M)``. b (cupy.ndarray): Dense matrix with dimension ``(M, K)``. alpha (float or complex): Coefficient. lower (bool): True: ``a`` is lower triangle matrix. False: ``a`` is upper triangle matrix. unit_diag (bool): True: diagonal part of ``a`` has unit elements. False: diagonal part of ``a`` has non-unit elements. transa (bool or str): True, False, 'N', 'T' or 'H'. 'N' or False: op(a) == ``a``. 'T' or True: op(a) == ``a.T``. 'H': op(a) == ``a.conj().T``. r~zspsm is not available.FrTrNTHzUnknown transa (actual: )rz'a must be CSR, CSC or COO sparse matrixrrrrrrrrrrrri-zb must be F-contiguous.rrLrr8)8rrrrrrrrrconjisspmatrix_coorrreshaperr9rrSrrrrrrrHrGrrrrrrrrrbrrspSM_createDescrrArrrrCUSPARSE_SPSM_ALG_DEFAULTrsCUSPARSE_SPMAT_FILL_MODECUSPARSE_SPMAT_DIAG_TYPEspSM_bufferSizerrrr spSM_analysisr spSM_solvespSM_destroyDescr)rrrrrr is_b_vectorr9r+r0rrrrrc_shaperPrmat_amat_bmat_c spsm_descrror;rrs rr~r~3sq& f % %53444 4 u  =F===>>> {((++D   * *1 - -D S==AFF s]]AFF s]] AF    * *1 - -D BCCC !!! ! v{{ IIb!   1 0111 GAJ!'!* 2 2 2 2 2 2 2 2+,,, GE z4;;EBBCCC ())) }}6 %6 5< JJJ I> JJJ- D$ EJ J D*66 {9EJ777  D$ EJ J D*dim E E E  D$ EJ J D*dim E E E   " A #J//// #J////s D!UU$c tdstd|j|jcxkrdksnJt|tjjjs/td t|t|tjjjs/td t||j sJ|j sJ|j d|j dkrtd|j \}}|j \}}t||\}}||f}tjj||j}t!j} t$|} t$|} t$|} t)j} t(j}t(j}t/j||jj}t/jd|jj}t5j|j}t(j}d}t)j| |||j| j| j|j| j||| d| }tAj!|t@j"}t)j| |||j| j| j|j| j||| ||jj# t)j$| |||j| j| j|j| j||| d| }tAj!|t@j"}t)j$| |||j| j| j|j| j||| ||jj# t/jdd }t/jdd }t/jdd }t)j%| j|jj|jj|jj|dtM|ksJ|dtM|ksJtM|}|j'}tAj!|d }tAj!||j}t)j(| j|jj#|jj#|jj#t)j)| |||j| j| j|j| j||| tjj|||f| }t)j*| |S) a+Matrix-matrix product for CSR-matrix. math:: C = alpha * A * B Args: a (cupyx.scipy.sparse.csr_matrix): Sparse matrix A. b (cupyx.scipy.sparse.csr_matrix): Sparse matrix B. alpha (scalar): Coefficient Returns: cupyx.scipy.sparse.csr_matrix r}zspgemm is not available.rrrrrr8rRrr)+rrrrrrrrrSrrrrrrEr9rrrbrrspGEMM_createDescrrHrArrrrCUSPARSE_SPGEMM_DEFAULTspGEMM_workEstimationrrrrrrspGEMM_computerrrcsrSetPointers spGEMM_copyspGEMM_destroyDescr)rrrrrrrrrPrrrr spgemm_descrrrrror;r buff1_sizebuff1 buff2_sizebuff2 c_num_rows c_num_colsrrrrs rr}r}s h ' '75666 6QV q a+6 7 7I7>>tAwwGGHHH a+6 7 7I7>>tAwwGGHHH !!! ! !!! !wqzQWQZ+,,, 7DAq 7DAq Q " "DAq!fG %%wqw%??A  ( * *F  " "1 % %E  " "1 % %E  " "1 % %E/11L  5D  5D Lag . . . 5E < ) ) ) 0D%ag..J  ,DH0dEJ EJ  JlAxAAJ K EJ / /E #dEJ EJ  JlJ PPP )dEJ EJ  JlAxAAJ K EJ / /E dEJ EJ  JlJ PPP aw///Jaw///J L' * * *E 5:z'8'=%,15<3DFFF 1:Z ( ( ( ( 1:Z ( ( ( ( JJExH E3''I [ ( (F UZ):INr sQ444444666666999999''''''''''''=====F===B:::   $ ]$ }$ ]$ m $ } $  $}$ $ M$ u / / 6$ u / / 6$ !$"#$$|%$&|'$(|)$*|+$,$"" G$$$N"  ["  D! "   [ "   k " {"  " {"  "  K"  K"  K" " "  {!" "{#" ${%" &{'" (:t$:t$#  z4 C" " " J   (?C????/ / / / d>A A A A H5 5 5 5 p< < < < ~= = = = @B B B B JB B B B Jf f f f R@@#;#;#;L#;#;#;L.(.(.(.(b 0 0 0 0 0 0///60000000<///00000000<'''T+++\0008V,!@!@!@!@!@n!@!@!@H 3 3 3 3 3n 3 3 333333n333 3 3 3 3 3n 3 3 3 @ @ @ @ FI I I I XAF%*O&O&O&O&dDCDCDCDCNE E E E P%%%%P_0_0_0_0DX X X X X X r