.`i^% dZddlmZddlmZddlmZddlZddlZ ddl m Z m Z ddl mZddlmZmZmZdd lmZd d lmZeeZed eezd efdZed eezd efdZded efdZd eezd eeeffdZd eezd efdZ d ed edzfdZ!ded dfdZ"d eded efdZ# ddeezdededzd efdZ$dS)zGGUF utility functions.)cache)PathLike)PathN)KeysVisionProjectorType)GGMLQuantizationType) Gemma3ConfigPretrainedConfigSiglipVisionConfig) init_logger)list_filtered_repo_filesmodelreturnc`t|}|sdS|jdkrdS |d5}|d}dddn #1swxYwY|dkS#t $r'}t d||Yd}~dSd}~wwxYw) z"Check if the file is a GGUF model.F.ggufTrbNsGGUFzError reading file %s: %s)ris_filesuffixopenread Exceptionloggerdebug)rfheaderes v/home/jaya/work/projects/VOICE-AGENT/VIET/agent-env/lib/python3.11/site-packages/vllm/transformers_utils/gguf_utils.pycheck_gguf_filer s KKE ==??u  t ZZ   VVAYYF                  0%;;;uuuuus;A< A+ A<+A//A<2A/3A<< B-B((B-cd}t|}tj||r(|dd\}}t |SdS)z*Check if the model is a remote GGUF model.zG^[a-zA-Z0-9][a-zA-Z0-9._-]*/[a-zA-Z0-9][a-zA-Z0-9._-]*:[A-Za-z0-9_+-]+$:r F)strre fullmatchrsplitis_valid_gguf_quant_type)rpattern_ quant_types ris_remote_ggufr+)sSYG JJE |GU##4 S!,, :' 333 5gguf_quant_typec2tt|dduS)z3Check if the quant type is a valid GGUF quant type.N)getattrr)r-s rr'r'4s '$ ? ?t KKr,ct|}t|r&|dd}|d|dfStd|dtj)z,Split the model into repo_id and quant type.r"r rz-Wrong GGUF model or invalid GGUF quant type: zT. - It should be in repo_id:quant_type format. - Valid GGMLQuantizationType values: )r#r+r& ValueErrorr_member_names_)rpartss rsplit_remote_ggufr49s{ JJEe$ S!$$a%(##  V V V0D0S V V  r,c`t|}t|rdSt|S)zCheck if the model is a GGUF model. Args: model: Model name, path, or Path object to check. Returns: True if the model is a GGUF model, False otherwise. T)r#r r+)rs ris_ggufr6Fs6 JJEut %  r,c|dsdS t|}|sdS|j}gd}|D]0}t ||}|r |dcS1dS#t $rYdSwxYw)zCheck if GGUF model has multimodal projector file. Args: model: Model path string Returns: Path to mmproj file if found, None otherwise rN)z mmproj.ggufz mmproj-*.ggufz *mmproj*.ggufr)endswithrrparentlistglobr)r model_path model_dirmmproj_patternsr( mmproj_filess rdetect_gguf_multimodalr@Ys >>' " "t %[[ !!## 4% KKK& ' 'G w 7 788L '#A&&& 't tts#A>;A>> B  B  mmproj_pathzSiglipVisionConfig | Nonectjt|}d}|tjj}|rh t|jd d}n9#ttf$r%}t d|Yd}~nd}~wwxYwtjjdt ftjjdt ftjjdt ftjjjdt ftjjd t ftjjd t ftjjjd t0fi}i}|D]V\}\}} ||} | t d |dS| | jd||<W|t4jkrd |d<tdt;di|} |rtd|ntd| S)a[Extract vision config parameters from mmproj.gguf metadata. Reads vision encoder configuration from GGUF metadata fields using standardized GGUF constants. Automatically detects the projector type (e.g., gemma3, llama4) and applies model-specific parameters accordingly. The function extracts standard CLIP vision parameters from GGUF metadata and applies projector-type-specific customizations. For unknown projector types, it uses safe defaults from SiglipVisionConfig. Args: mmproj_path: Path to mmproj.gguf file (str or Path) Returns: SiglipVisionConfig if extraction succeeds, None if any required field is missing from the GGUF metadata Raises: Exception: Exceptions from GGUF reading (file not found, corrupted file, etc.) propagate directly from gguf.GGUFReader Nzutf-8z-Failed to decode projector type from GGUF: %s hidden_sizeintermediate_sizenum_hidden_layersnum_attention_heads image_size patch_sizelayer_norm_epsz8Missing required vision config field '%s' in mmproj.ggufFvision_use_headz8Detected Gemma3 projector, disabling vision pooling headz=Extracted vision config from mmproj.gguf (projector_type: %s)z1Extracted vision config from mmproj.gguf metadata)gguf GGUFReaderr# get_fieldrClipPROJECTOR_TYPEbytesr3decodeAttributeErrorUnicodeDecodeErrorrwarning ClipVisionEMBEDDING_LENGTHintFEED_FORWARD_LENGTH BLOCK_COUNT Attention HEAD_COUNT IMAGE_SIZE PATCH_SIZE LAYERNORM_EPSfloatitemsrGEMMA3infor ) rAreaderprojector_typeprojector_type_fieldrVISION_CONFIG_FIELDS config_paramsgguf_key param_namedtypefieldconfigs rextract_vision_config_from_ggufrousJ,_S-- . .FN!++DI,DEEO O"#7#=b#ABBII'RRNN 23 O O O NNJA N N N N N N N N O (=#*> +.A3-G #&93%? !,/Dc.J "\3$7 "\3$7 !/2BE1JM)=)C)C)E)E ; ;%%:u  ** = NNJ   44$)E%+b/$:$: j!!,333,1 '( NOOO 0 0- 0 0FI K      GHHH Ms -A99B/ B**B/ hf_configct|}|Wtt|}|}|jdv}||rt j||dg}|}|S)aCPatch HF config for GGUF models. Applies GGUF-specific patches to HuggingFace config: 1. For multimodal models: patches architecture and vision config 2. For all GGUF models: overrides vocab_size from embedding tensor This ensures compatibility with GGUF models that have extended vocabularies (e.g., Unsloth) where the GGUF file contains more tokens than the HuggingFace tokenizer config specifies. Args: model: Model path string hf_config: HuggingFace config to patch in-place Returns: Updated HuggingFace config N)gemma3 gemma3_textGemma3ForConditionalGeneration) text_config vision_config architectures)r@ror#get_text_config model_typer from_text_vision_configs)rrprArvru is_gemma3 new_hf_configs rmaybe_patch_hf_config_from_ggufr}s,)//K7K8H8HII  //11 (,EE  $ $(A'+?@M &I r,repo_idr*revisionct|}d|dd|dd|dd|dg}t|||}t|dkrtd|||d |d}|S) asGet the GGUF file path from HuggingFace Hub based on repo_id and quant_type. Args: repo_id: The HuggingFace repository ID (e.g., "Qwen/Qwen3-0.6B") quant_type: The quantization type (e.g., "Q4_K_M", "F16") revision: Optional revision/branch name Returns: The path to the GGUF file on HuggingFace Hub (e.g., "filename.gguf"), z*-rz-*.ggufz*/*-)allow_patternsrrz:Could not find GGUF file for repo %s with quantization %s.c0|d|fS)N-)count)xs rz,get_gguf_file_path_from_hf..sqwws||Q&7r,)key)r#rlenr1sort)r~r*r gguf_patternsmatching_files gguf_filenames rget_gguf_file_path_from_hfrs'llGZ Z    z   "z""" M .$N  >a H     77888"1%M r,)N)%__doc__ functoolsrosrpathlibrrMregexr$gguf.constantsrr gguf.quantsr transformersr r r vllm.loggerr repo_utilsr__name__rr#boolr r+r'tupler4r6r@ror}rrLr,rrs 44444444,,,,,,KKKKKKKKKK######000000 X  3>d$#*LcLdLLLL S4Z E#s(O    !3:!$!!!!&#$+8RR9TRRRRj% %%%%%%V && 4Z&&Dj& &&&&&&r,