.`i ddlZddlmZmZddlmZerddlmZ ddlZn#e $redZYnwxYwde e de e de e fd Z de e de e de e fd Z dd ed de de e dzde e fdZ dd d de de e dzde e dzdee e e e ff dZdS)N) TYPE_CHECKINGOptional)PlaceholderModule) BaseClientboto3pathspatternsreturnc fd|DS)NcLg|]tfdD S)c3BK|]}tj|VdSNfnmatch.0patternpaths t/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/transformers_utils/s3_utils.py z+_filter_allow...s/FF'wtW--FFFFFFanyrrr s @r z!_filter_allow..sM     FFFFXFFF F F     rrr s `r _filter_allowr.       rc fd|DS)NcLg|]tfdD S)c3BK|]}tj|VdSrrrs rrz,_filter_ignore...s/JJg7?411JJJJJJrrrs @rrz"_filter_ignore..sM    JJJJJJJJJ     rrrs `r_filter_ignorer#rrs3rr allow_patternc|tjd}|ds|dz}t|||\}}fd|DS)a) List full file names from S3 path and filter by allow pattern. Args: s3: S3 client to use. path: The S3 path to list from. allow_pattern: A list of patterns of which files to pull. Returns: list[str]: List of full S3 paths allowed by the pattern Nr%/)rr&c g|] }dd| S)s3://r(r)rr bucket_names rrzglob..7s* ; ; ;T (K ( ($ ( ( ; ; ;r)rclientendswith list_files)r%rr&_rr+s @rglobr0"sk  z \$   ==  cz&rMRRRKE ; ; ; ;U ; ; ;;rignore_patternc~|dd}d|dd}|d}|||}d|dgD}t |d g}|t ||}|t ||}|||fS) aZ List files from S3 path and filter by pattern. Args: s3: S3 client to use. path: The S3 path to list from. allow_pattern: A list of patterns of which files to pull. ignore_pattern: A list of patterns of which files not to pull. Returns: tuple[str, str, list[str]]: A tuple where: - The first element is the bucket name - The second element is string represent the bucket and the prefix as a dir like string - The third element is a list of files allowed or disallowed by pattern r*r(Nr)BucketPrefixcg|] }|d S)Keyr)robjs rrzlist_files..Vs ? ? ?CSZ ? ? ?rContentsz*/) removeprefixsplitjoinlist_objects_v2getr#r) r%rr&r1partsprefixr+objectsrs rr.r.:s.   g & & , ,S 1 1E XXeABBi F(K  F CCG ? ?7;;z2#>#> ? ? ?E 54& ) )E e]33!un55  %%r)Nr$N)NN)rtypingrrvllm.utils.import_utilsrbotocore.clientrr ImportErrorliststrrr#r0tupler.rrrrIs********555555+******'LLLL'''  g & &EEE'cd3iDI$s)tCyT#Y"&&*<<< <9t#< #Y <<<<6'+'+ %&%&%& %&9t#%&I$ %&  3T#Y  %&%&%&%&%&%&s !44