All Supported Models

29681 AI models · 198875 quantization formats tracked

A

Afifit

1 models

Llama-3.2-1B-Instruct-Q4_K_M-GGUF

1B 8K ctx
Q4_0
0.56GB file 2.58GB
FP16
2GB file 4.02GB
A

Afkara L P

1 models

gemma-3-270M-coralmc-9500it-v06_F16

1B 8K ctx
FP16
2GB file 4.02GB
A

Afrideva

180 models

African-ultrachat-alpaca-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

ahxt_llama2_xs_460M_experimental_ptbr_instruct-GGUF

Unknown 8K ctx
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Aira-2-1B1-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB
FP16
2GB file 4.02GB

airoboros-3b-3p0-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Astrea-RP-v1-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Astridboros-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Astrohermes-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Astrorocketboros-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

beecoder-220M-python-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Bioinspired-Phi-3-mini-4k-GGUF

3.8B 8K ctx
Q2_0
1.19GB file 3.25GB
Q3_0
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q6_0
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB
FP16
7.6GB file 9.66GB

bling-sheared-llama-1.3b-0.1-GGUF

1.3B 8K ctx
Q2_0
0.41GB file 2.43GB
Q3_0
0.57GB file 2.59GB
Q4_0
0.73GB file 2.75GB
Q5_0
0.89GB file 2.91GB
Q6_0
1.06GB file 3.08GB
Q8_0
1.38GB file 3.4GB

bling-sheared-llama-2.7b-0.1-GGUF

2.7B 8K ctx
Q2_0
0.84GB file 2.88GB
Q3_0
1.18GB file 3.22GB
Q4_0
1.52GB file 3.56GB
Q5_0
1.86GB file 3.9GB
Q6_0
2.19GB file 4.23GB
Q8_0
2.87GB file 4.91GB

bloom-560m-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

bloomz-Alpaca-560M-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Buzz-8b-Large-v0.5-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_M
3.5GB file 5.63GB
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

DialoGPT-medium-Oshiwambo-Bot-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Dimensity-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

dolphin-2_6-phi-2_oasst2_chatML_V2-GGUF

2B 8K ctx
Q2_0
0.63GB file 2.66GB
Q3_0
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

dolphin-2.9.3-qwen2-0.5b-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q3_K_M
0.22GB file 2.23GB
Q4_K_M
0.28GB file 2.29GB
Q5_K_M
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB
FP16
1GB file 3.01GB

dopeyplats-1.1b-2T-v1-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

Echo-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

evolvedSeeker_1_3-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Ferret-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

gemma-1.1-2b-it_oasst_format_chatML_unsloth_V1_function_calling_V2-GGUF

2B 8K ctx
Q2_0
0.63GB file 2.66GB
Q3_0
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

gemma-1.1-2b-it_orpo_traslate_en_es_V1-GGUF

2B 8K ctx
Q2_0
0.63GB file 2.66GB
Q3_0
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

GIST-all-MiniLM-L6-v2-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_M
0.88GB file 2.91GB
Q4_K_M
1.13GB file 3.16GB
Q5_K_M
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

GPT-NeoX-1.3B-viet-final-GGUF

1.3B 8K ctx
Q2_0
0.41GB file 2.43GB
Q3_0
0.57GB file 2.59GB
Q4_0
0.73GB file 2.75GB
Q5_0
0.89GB file 2.91GB
Q6_0
1.06GB file 3.08GB
Q8_0
1.38GB file 3.4GB

gpt2-small-danish-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

gpt2-turkish-128-token-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Hercules-Mini-1.8B-GGUF

1.8B 8K ctx
Q2_K
0.56GB file 2.59GB
Q3_K_M
0.79GB file 2.82GB
Q4_K_M
1.01GB file 3.04GB
Q5_K_M
1.24GB file 3.27GB
Q6_K
1.46GB file 3.49GB
Q8_0
1.91GB file 3.94GB
FP16
3.6GB file 5.63GB

Hermes-Trismegistus-Mistral-7B-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
Q3_0
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

InstructWise-462M-GGUF

Unknown 8K ctx
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

japanese-mistral-300m-base-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

japanese-mistral-300m-instruction-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

kappa-3-phi-abliterated-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Llama-160M-Chat-v1-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB
FP16
2GB file 4.02GB

llama-160m-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Llama-2-7b-chat-igbo-yoruba-2024-04-12_23-35-37-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_K_M
3.94GB file 6.05GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
FP16
14GB file 16.11GB

Llama-3-8b-Cosmopedia-japanese-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_M
3.5GB file 5.63GB
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

Llama-3-8B-Synthia-v3.5-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_M
3.5GB file 5.63GB
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

Llama-3-Yggdrasil-8B-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_M
3.5GB file 5.63GB
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

Llama-68M-Chat-v1-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB
FP16
2GB file 4.02GB

llama2_xs_460M_experimental_evol_instruct-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

llama2_xs_460M_experimental_platypus-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

llama2_xs_460M_experimental-GGUF

Unknown 8K ctx
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

llama2_xs_460M_uncensored-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

llama3_8b_chat_brainstorm-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_M
3.5GB file 5.63GB
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

llama3-instrucTrans-enko-8b-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_M
3.5GB file 5.63GB
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

LMCocktail-phi-2-v1-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB
FP16
2GB file 4.02GB

LocutusqueXFelladrin-TinyMistral248M-Instruct_oasst2_chatML_V1_DPO_V3-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

LocutusqueXFelladrin-TinyMistral248M-Instruct-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

malaysian-mistral-1.1B-4096-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

malaysian-tinyllama-1.1b-16384-instructions-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

malaysian-tinyllama-1.1b-16k-instructions-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

Marx-3B-V2-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Mermaid-Llama-3-5B-Pruned-GGUF

5B 8K ctx
Q2_K
1.56GB file 3.64GB
Q3_K_M
2.19GB file 4.27GB
Q4_K_M
2.81GB file 4.89GB
Q5_K_M
3.44GB file 5.52GB
Q6_K
4.06GB file 6.14GB
Q8_0
5.31GB file 7.39GB
FP16
10GB file 12.08GB

Mermaid-Llama-6.7B-RAG-Code-Instruct-GGUF

6.7B 8K ctx
Q2_K
2.09GB file 4.2GB
Q3_K_M
2.93GB file 5.04GB
Q4_K_M
3.77GB file 5.88GB
Q5_K_M
4.61GB file 6.72GB
Q6_K
5.44GB file 7.55GB
Q8_0
7.12GB file 9.23GB
FP16
13.4GB file 15.51GB

Meta-Llama-3-8B-conceptnet_UsedFor_en_en-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_M
3.5GB file 5.63GB
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

Meta-Llama-3-8Bee-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_M
3.5GB file 5.63GB
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

MiniChat-1.5-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

MiniChat-2-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

MiniChat-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

MiniMA-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

minima-3b-layla-v2-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Mixtral_AI_MasterTron-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Mixtral_AI_TheAncient-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Mixtral-GQA-400m-v2-GGUF

2B 8K ctx
Q2_0
0.63GB file 2.66GB
Q3_0
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

Morph-Caller-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

mpt-3b-8k-instruct-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

NeuralReyna-Mini-1.8B-v0.3-GGUF

1.8B 8K ctx
Q2_0
0.56GB file 2.59GB
Q3_0
0.79GB file 2.82GB
Q4_0
1.01GB file 3.04GB
Q5_0
1.24GB file 3.27GB
Q6_0
1.46GB file 3.49GB
Q8_0
1.91GB file 3.94GB
FP16
3.6GB file 5.63GB

no-prompt-1.3b-GGUF

1.3B 8K ctx
Q2_0
0.41GB file 2.43GB
Q3_0
0.57GB file 2.59GB
Q4_0
0.73GB file 2.75GB
Q5_0
0.89GB file 2.91GB
Q6_0
1.06GB file 3.08GB
Q8_0
1.38GB file 3.4GB
FP16
2.6GB file 4.62GB

open_llama_3b_code_instruct_0.1-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

open-llama-3b-v2-instruct-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Orca-2.0-Tau-1.8B-GGUF

1.8B 8K ctx
Q2_K
0.56GB file 2.59GB
Q3_K_M
0.79GB file 2.82GB
Q4_K_M
1.01GB file 3.04GB
Q5_K_M
1.24GB file 3.27GB
Q6_K
1.46GB file 3.49GB
Q8_0
1.91GB file 3.94GB
FP16
3.6GB file 5.63GB

palmer-002-GGUF

002B 8K ctx
Q2_0
0.63GB file 2.66GB
Q3_0
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

palmer-x-002-GGUF

002B 8K ctx
Q2_0
0.63GB file 2.66GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

phi-1_5_dolly_instruction_polish-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phi-2_dolly_instruction_polish-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phi-2_LogiCoT_finetuned-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phi-2-arxiv-physics-instruct-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB

phi-2-chat-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phi-2-chat-turkish-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phi-2-code-instruct-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phi-2-GGUF

2B 8K ctx
Q2_0
0.63GB file 2.66GB
Q3_0
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

phi-2-meditron-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phi-2-psy-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phi-2-sft-alpaca_gpt4_en-ep1-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB
FP16
2GB file 4.02GB

phi-2-sft-dpo-gpt4_en-ep1-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB
FP16
2GB file 4.02GB

phi-2-uncensored-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Phi-3-Context-Obedient-RAG-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_M
1.66GB file 3.72GB
Q4_K_M
2.14GB file 4.2GB
Q5_K_M
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB
FP16
7.6GB file 9.66GB

Phi-3-mini-128k-instruct_function-GGUF

3.8B 8K ctx
Q2_0
1.19GB file 3.25GB
Q3_0
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q6_0
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB
FP16
7.6GB file 9.66GB

Phi-3-mini-4k-grammar-correction-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_M
1.66GB file 3.72GB
Q4_K_M
2.14GB file 4.2GB
Q5_K_M
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB
FP16
7.6GB file 9.66GB

Phi-3-mini-4k-instruct-function-calling-assistant-spanish-pofi-v2-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_M
0.88GB file 2.91GB
Q4_K_M
1.13GB file 3.16GB
Q5_K_M
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

phi-3-portuguese-tom-cat-4k-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_M
1.66GB file 3.72GB
Q4_K_M
2.14GB file 4.2GB
Q5_K_M
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB
FP16
7.6GB file 9.66GB

phi2-oasst-guanaco-bf16-custom-GGUF

Unknown 8K ctx
BF16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

phine-2-v0-GGUF

0B 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

pip-code-bandit-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

pip-library-etl-1.3b-GGUF

1.3B 8K ctx
Q2_K
0.41GB file 2.43GB
Q3_K_M
0.57GB file 2.59GB
Q4_K_M
0.73GB file 2.75GB
Q5_K_M
0.89GB file 2.91GB
Q6_K
1.06GB file 3.08GB
Q8_0
1.38GB file 3.4GB
FP16
2.6GB file 4.62GB

pip-sql-1.3b-GGUF

1.3B 8K ctx
Q2_K
0.41GB file 2.43GB
Q3_K_M
0.57GB file 2.59GB
Q4_K_M
0.73GB file 2.75GB
Q5_K_M
0.89GB file 2.91GB
Q6_K
1.06GB file 3.08GB
Q8_0
1.38GB file 3.4GB
FP16
2.6GB file 4.62GB

Qwen2-0.5B-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q3_K_M
0.22GB file 2.23GB
Q4_K_M
0.28GB file 2.29GB
Q5_K_M
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB
FP16
1GB file 3.01GB

Qwen2-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q3_K_M
0.22GB file 2.23GB
Q4_K_M
0.28GB file 2.29GB
Q5_K_M
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB
FP16
1GB file 3.01GB

Qwen2-1.5B-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_M
0.66GB file 2.68GB
Q4_K_M
0.84GB file 2.86GB
Q5_K_M
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB
FP16
3GB file 5.02GB

Qwen2-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_M
0.66GB file 2.68GB
Q4_K_M
0.84GB file 2.86GB
Q5_K_M
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB
FP16
3GB file 5.02GB

refusal-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

RocketZephyr-3b-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

smartyplats-1.1b-v1-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

smartyplats-1.1b-v2-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

smol_llama-101M-GQA-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

smol_llama-101M-GQA-python-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

smol_llama-101M-midjourney-messages-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

smol_llama-220M-bees-internal-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

smol_llama-220M-GQA-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

smol_llama-220M-open_instruct-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

smol_llama-220M-openhermes-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

smol-3b-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Smol-Llama-101M-Chat-v1-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB
FP16
2GB file 4.02GB

SmolLlamix-8x101M-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

stablelm-2-1_6b-GGUF

6B 8K ctx
Q2_0
1.88GB file 3.98GB
Q3_0
2.63GB file 4.73GB
Q4_0
3.38GB file 5.48GB
Q5_0
4.13GB file 6.23GB
Q6_0
4.88GB file 6.98GB
Q8_0
6.38GB file 8.48GB
FP16
12GB file 14.1GB

stablelm-2-zephyr-1_6b-GGUF

6B 8K ctx
Q2_0
1.88GB file 3.98GB
Q3_0
2.63GB file 4.73GB
Q4_0
3.38GB file 5.48GB
Q5_0
4.13GB file 6.23GB
Q6_0
4.88GB file 6.98GB
Q8_0
6.38GB file 8.48GB
FP16
12GB file 14.1GB

stablelm-3b-4e1t-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

tau-0.5B-GGUF

0.5B 8K ctx
Q2_0
0.16GB file 2.17GB
Q3_0
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q6_0
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB
FP16
1GB file 3.01GB

tau-0.5B-instruct-GGUF

0.5B 8K ctx
Q2_0
0.16GB file 2.17GB
Q3_0
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q6_0
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB
FP16
1GB file 3.01GB

TeenyTinyLlama-160m-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

TeenyTinyLlama-460m-Chat-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

TeenyTinyLlama-460m-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

tiny-llama-1.1b-chat-medical-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

tiny-llama-miniguanaco-1.5T-GGUF

Unknown 8K ctx
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Tiny-Llama-openhermes-1.1B-step-715k-1.5T-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

tiny-random-LlamaForCausalLM-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Tiny-Vicuna-1B-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB

TinyAlpaca-v0.1-GGUF

0.1B 8K ctx
Q2_0
0.03GB file 2.03GB
Q3_0
0.04GB file 2.04GB
Q4_0
0.06GB file 2.06GB
Q5_0
0.07GB file 2.07GB
Q6_0
0.08GB file 2.08GB
Q8_0
0.11GB file 2.11GB
FP16
0.2GB file 2.2GB

tinycodellama-1.3b-3k-GGUF

1.3B 8K ctx
Q2_0
0.41GB file 2.43GB
Q3_0
0.57GB file 2.59GB
Q4_0
0.73GB file 2.75GB
Q5_0
0.89GB file 2.91GB
Q6_0
1.06GB file 3.08GB
Q8_0
1.38GB file 3.4GB
FP16
2.6GB file 4.62GB

tinycodellama-1.3b-5k-GGUF

1.3B 8K ctx
Q2_0
0.41GB file 2.43GB
Q3_0
0.57GB file 2.59GB
Q4_0
0.73GB file 2.75GB
Q5_0
0.89GB file 2.91GB
Q6_0
1.06GB file 3.08GB
Q8_0
1.38GB file 3.4GB
FP16
2.6GB file 4.62GB

TinyKAI-3B-beta-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

TinyLlama-1.1B-2T-lr-2e-4-3ep-dolly-15k-instruct-v1-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-alpaca-chat-v1.5-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

TinyLlama-1.1B-Chat-v0.4-ENG-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-Chat-v0.4-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-Chat-v0.5-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-Chat-v0.6-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-DavidAttenborough-GGUF

1.1B 8K ctx
Q2_K
0.34GB file 2.36GB
Q3_K_M
0.48GB file 2.5GB
Q4_K_M
0.62GB file 2.64GB
Q5_K_M
0.76GB file 2.78GB
Q6_K
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-intermediate-1.5T-PTBR-Instruct-v0-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

TinyLlama-1.1B-intermediate-1.5T-PTBR-Instruct-v1-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

TinyLlama-1.1B-intermediate-1.5T-PTBR-Instruct-v2-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

TinyLlama-1.1B-intermediate-1.5T-PTBR-Instruct-v3-8k-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

TinyLlama-1.1B-intermediate-step-1195k-token-2.5T-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-intermediate-step-1431k-3T-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-intermediate-step-715k-1.5T-airoboros-3.1-instruct-V3-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

TinyLlama-1.1B-intermediate-step-715k-1.5T-airoboros-3.1-V2-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

TinyLlama-1.1B-intermediate-step-715k-1.5T-lr-5-4epochs-oasst1-top1-instruct-V1-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB

TinyLlama-1.1B-intermediate-step-955k-token-2T-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-intermediate-step-955k-token-2T-guanaco-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-1.1B-step-2T-lr-5-5ep-oasst1-top1-instruct-V1-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

TinyLlama-3T-1.1bee-GGUF

1.1B 8K ctx
Q2_0
0.34GB file 2.36GB
Q3_0
0.48GB file 2.5GB
Q4_0
0.62GB file 2.64GB
Q5_0
0.76GB file 2.78GB
Q6_0
0.89GB file 2.91GB
Q8_0
1.17GB file 3.19GB
FP16
2.2GB file 4.22GB

tinyllama-colorist-v2-GGUF

2B 8K ctx
Q2_0
0.63GB file 2.66GB
Q3_0
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

TinyLlama-con-brainstorming-v0.2-GGUF

0.2B 8K ctx
Q2_0
0.06GB file 2.06GB
Q3_0
0.09GB file 2.09GB
Q4_0
0.11GB file 2.11GB
Q5_0
0.14GB file 2.14GB
Q6_0
0.16GB file 2.16GB
Q8_0
0.21GB file 2.21GB
FP16
0.4GB file 2.4GB

TinyLlama-con-creative-writing-v0.2-GGUF

0.2B 8K ctx
Q2_0
0.06GB file 2.06GB
Q3_0
0.09GB file 2.09GB
Q4_0
0.11GB file 2.11GB
Q5_0
0.14GB file 2.14GB
Q6_0
0.16GB file 2.16GB
Q8_0
0.21GB file 2.21GB
FP16
0.4GB file 2.4GB

TinyLlama-haiku-dpo-v.0.1-GGUF

0.1B 8K ctx
Q2_0
0.03GB file 2.03GB
Q3_0
0.04GB file 2.04GB
Q4_0
0.06GB file 2.06GB
Q5_0
0.07GB file 2.07GB
Q6_0
0.08GB file 2.08GB
Q8_0
0.11GB file 2.11GB
FP16
0.2GB file 2.2GB

tinyllama-python-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

TinyLLama-v0-GGUF

0B 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

TinyMistral-248M-Alpaca-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

TinyMistral-248M-Evol-Instruct-GGUF

Unknown 8K ctx
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

TinyMistral-248M-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

TinyMistral-248M-SFT-v3-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

TinyMistral-248M-SFT-v4-GGUF

4B 8K ctx
Q2_0
1.25GB file 3.32GB
Q3_0
1.75GB file 3.82GB
Q4_0
2.25GB file 4.32GB
Q5_0
2.75GB file 4.82GB
Q6_0
3.25GB file 5.32GB
Q8_0
4.25GB file 6.32GB
FP16
8GB file 10.07GB

Tinystories-gpt-0.1-3m-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

verysmol_llama-v11-KIx2-GGUF

2B 8K ctx
Q2_0
0.63GB file 2.66GB
Q3_0
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q6_0
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB
FP16
4GB file 6.03GB

Walter-Llama-1B-GGUF

1B 8K ctx
Q2_0
0.31GB file 2.33GB
Q3_0
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q6_0
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB
FP16
2GB file 4.02GB

Walter-Phi-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Walter-StableLM-3B-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Yi-1.5-6B-Chat-GGUF

6B 8K ctx
Q2_K
1.88GB file 3.98GB
Q3_K_M
2.63GB file 4.73GB
Q4_K_M
3.38GB file 5.48GB
Q5_K_M
4.13GB file 6.23GB
Q6_K
4.88GB file 6.98GB
Q8_0
6.38GB file 8.48GB
FP16
12GB file 14.1GB

Yi-1.5-6B-GGUF

6B 8K ctx
Q2_K
1.88GB file 3.98GB
Q3_K_M
2.63GB file 4.73GB
Q4_K_M
3.38GB file 5.48GB
Q5_K_M
4.13GB file 6.23GB
Q6_K
4.88GB file 6.98GB
Q8_0
6.38GB file 8.48GB
FP16
12GB file 14.1GB

Yi-1.5-9B-Chat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K_M
3.94GB file 6.09GB
Q4_K_M
5.06GB file 7.21GB
Q5_K_M
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB
FP16
18GB file 20.15GB

yoruba-embedding-model-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_K
0GB file 2GB
Q3_K_M
0GB file 2GB
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

zephyr-220m-dpo-full-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

zephyr-220m-sft-full-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

Zephyr-3.43B-GGUF

3.43B 8K ctx
Q2_0
1.07GB file 3.13GB
Q3_0
1.5GB file 3.56GB
Q4_0
1.93GB file 3.99GB
Q5_0
2.36GB file 4.42GB
Q6_0
2.79GB file 4.85GB
Q8_0
3.64GB file 5.7GB
FP16
6.86GB file 8.92GB

zephyr-7b-gemma-sft-african-ultrachat-200k-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_K_M
3.94GB file 6.05GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
FP16
14GB file 16.11GB

zephyr-smol_llama-100m-dpo-full-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB

zephyr-smol_llama-100m-sft-full-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q2_0
0GB file 2GB
Q3_0
0GB file 2GB
Q4_0
0GB file 2GB
Q5_0
0GB file 2GB
Q6_0
0GB file 2GB
Q8_0
0GB file 2GB
A

Aftersoft

1 models

Meta-Llama-3.1-8B-Q4_K_M-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
FP16
16GB file 18.13GB
A

Aganonce

1 models

sleep_ai

Unknown 8K ctx
FP16
0GB file 2GB
A

Agasfsfsdfadfa

1 models

Qwen2.5-Coder-7B-Instruct-GPTQ-Int4-Q8_0-GGUF

7B 8K ctx
Q8_0
7.44GB file 9.55GB
A

Agency888

1 models

TaoGPT-v1-GGUF-GGUF

7B 8K ctx
Q4_K_M
3.94GB file 6.05GB
Q5_K_M
4.81GB file 6.92GB
A

Agent Anon

4 models

gemma-4-26B-A4B-it-uncensored-GGUF

26B 8K ctx
Q4_K_M
14.63GB file 17.06GB
Q8_0
27.63GB file 30.06GB

gemma-4-31B-it-uncensored-GGUF

31B 8K ctx
Q4_K_M
17.44GB file 19.95GB
Q8_0
32.94GB file 35.45GB

gemma-4-E2B-it-uncensored-GGUF

2B 8K ctx
Q4_K_M
1.13GB file 3.16GB
Q8_0
2.13GB file 4.16GB

gemma-4-E4B-it-uncensored-GGUF

4B 8K ctx
Q4_K_M
2.25GB file 4.32GB
Q8_0
4.25GB file 6.32GB
A

Agentlans

1 models

Qwen3-4B-multilingual-sft-GGUF

4B 8K ctx
Q4_K_M
2.25GB file 4.32GB
Q5_K_M
2.75GB file 4.82GB
Q8_0
4.25GB file 6.32GB
A

Agentv Online

1 models

RAGent_gen

Unknown 8K ctx
FP16
0GB file 2GB