All Supported Models

29681 AI models · 198875 quantization formats tracked

S

Samuellimabraz

2 models

granite-3.3-2b-instruct-Q4_K_M-GGUF

2B 8K ctx
Q4_0
1.13GB file 3.16GB
FP16
4GB file 6.03GB

Llama-Guard-3-1B-Q4_K_M-GGUF

1B 8K ctx
Q4_0
0.56GB file 2.58GB
FP16
2GB file 4.02GB
S

Samuelstolicny

3 models

claudie-expert-gemma4-26b-a4b-gguf

31B 8K ctx
Q4_K_M
17.44GB file 19.95GB
Q8_0
32.94GB file 35.45GB

claudie-expert-gemma4-31b

31B 8K ctx
Q4_K_M
17.44GB file 19.95GB
BF16
62GB file 64.51GB

claudie-expert-gemma4-31b-gguf

31B 8K ctx
Q4_K_M
17.44GB file 19.95GB
S

Samuioto

1 models

Qwen2.5-7B-Instruct-Q4_K_M-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB
S

Samunder12

7 models

llama-3.1-8b-OneLastStory-gguf

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB

llama-3.1-8b-roleplay-airtel-gguf

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB

llama-3.1-8b-roleplay-BSNL-gguf

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB

llama-3.1-8b-roleplay-jio-gguf

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB

llama-3.1-8b-roleplay-vi-gguf

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB

llama-3.1-8b-Rp-tadashinu-gguf

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB

Llama-3.2-3B-small_Shiro_roleplay-gguf

3B 8K ctx
Q4_K_M
1.69GB file 3.74GB
Q8_0
3.19GB file 5.24GB
S

Samurai Zero

1 models

Llama-3.2-3B-Instruct-Q8_0-GGUF

3B 8K ctx
Q8_0
3.19GB file 5.24GB
S

Samwell

1 models

ncd-gemma4-e4b-lora

4B 8K ctx
Q4_0
2.25GB file 4.32GB
FP16
8GB file 10.07GB
S

Samzito12

4 models

lora_model

Unknown 8K ctx
Q4_K_M
0GB file 2GB

lora_model2

Unknown 8K ctx
FP16
0GB file 2GB

lora_model4

Unknown 8K ctx
FP16
0GB file 2GB

lora_model5

Unknown 8K ctx
FP16
0GB file 2GB
S

San27079

3 models

Gemma-4-E2B-it-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q2_K_S
0.63GB file 2.66GB
Q2_M
0.63GB file 2.66GB
Q2_S
0.63GB file 2.66GB
Q2_XS
0.63GB file 2.66GB
Q2_XXS
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q3_M
0.88GB file 2.91GB
Q3_S
0.88GB file 2.91GB
Q3_XS
0.88GB file 2.91GB
Q3_XXS
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_1
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

Gemma-4-E2B-it-Imatrix-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q2_K_S
0.63GB file 2.66GB
Q2_M
0.63GB file 2.66GB
Q2_S
0.63GB file 2.66GB
Q2_XS
0.63GB file 2.66GB
Q2_XXS
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q3_M
0.88GB file 2.91GB
Q3_S
0.88GB file 2.91GB
Q3_XS
0.88GB file 2.91GB
Q3_XXS
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_1
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

Gemma-4-E4B-it-Imatrix-GGUF

4B 8K ctx
Q2_K
1.25GB file 3.32GB
Q2_K_S
1.25GB file 3.32GB
Q2_M
1.25GB file 3.32GB
Q2_S
1.25GB file 3.32GB
Q2_XS
1.25GB file 3.32GB
Q2_XXS
1.25GB file 3.32GB
Q3_K_L
1.75GB file 3.82GB
Q3_K_M
1.75GB file 3.82GB
Q3_K_S
1.75GB file 3.82GB
Q3_M
1.75GB file 3.82GB
Q3_S
1.75GB file 3.82GB
Q3_XS
1.75GB file 3.82GB
Q3_XXS
1.75GB file 3.82GB
Q4_0
2.25GB file 4.32GB
Q4_1
2.25GB file 4.32GB
Q4_K_M
2.25GB file 4.32GB
Q4_K_S
2.25GB file 4.32GB
Q5_K_M
2.75GB file 4.82GB
Q5_K_S
2.75GB file 4.82GB
Q6_K
3.25GB file 5.32GB
Q8_0
4.25GB file 6.32GB
S

Sanctum A I

11 models

gemma-2-2b-it-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_1
1.13GB file 3.16GB
Q4_K
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q5_1
1.38GB file 3.41GB
Q5_K
1.38GB file 3.41GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

gemma-2-9b-it-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_M
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_1
5.06GB file 7.21GB
Q4_K
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q5_1
6.19GB file 8.34GB
Q5_K
6.19GB file 8.34GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB

granite-3b-code-instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_1
1.69GB file 3.74GB
Q4_K
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_1
2.06GB file 4.11GB
Q5_K
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

granite-8b-code-instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_1
4.5GB file 6.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_1
5.5GB file 7.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Llama-3.2-1B-Instruct-GGUF

1B 8K ctx
Q2_K
0.31GB file 2.33GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_M
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q4_1
0.56GB file 2.58GB
Q4_K
0.56GB file 2.58GB
Q4_K_M
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q5_1
0.69GB file 2.71GB
Q5_K
0.69GB file 2.71GB
Q5_K_M
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB

Llama-3.2-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_1
1.69GB file 3.74GB
Q4_K
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_1
2.06GB file 4.11GB
Q5_K
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Meta-Llama-3-8B-Instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_1
4.5GB file 6.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_1
5.5GB file 7.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Meta-Llama-3.1-8B-Instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_1
4.5GB file 6.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_1
5.5GB file 7.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Mistral-7B-Instruct-v0.3-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_1
3.94GB file 6.05GB
Q4_K
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_1
4.81GB file 6.92GB
Q5_K
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Phi-3-mini-4k-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_M
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_1
2.14GB file 4.2GB
Q4_K
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q5_1
2.61GB file 4.67GB
Q5_K
2.61GB file 4.67GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB
FP16
7.6GB file 9.66GB

Phi-3.5-mini-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_M
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_1
2.14GB file 4.2GB
Q4_K
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q5_1
2.61GB file 4.67GB
Q5_K
2.61GB file 4.67GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB
S

Sand Logic Technologies

51 models

DeepSeek-Coder-V2-Lite-Instruct-GGUF

Unknown 8K ctx
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB

deepseek-llm-7b-chat-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
FP16
14GB file 16.11GB

DeepSeek-R1-Distill-Qwen-14B-GGUF

14B 8K ctx
Q4_K_M
7.88GB file 10.11GB
Q5_K_M
9.63GB file 11.86GB

Falcon3-7B-Base-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
FP16
14GB file 16.11GB

gemma-2-9b-GGUF

9B 8K ctx
Q4_0
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
FP16
18GB file 20.15GB

Gemma-2-9b-it-GGUF

9B 8K ctx
Q4_K_M
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K_M
6.19GB file 8.34GB

gemma-3-1b-it-GGUF

1B 8K ctx
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
FP16
2GB file 4.02GB

Gemma-3-270m-GGUF

1B 8K ctx
Q4_0
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
FP16
2GB file 4.02GB

gemma-4-E2B-GGUF

2B 8K ctx
Q3_M
0.88GB file 2.91GB
Q4_NL
1.13GB file 3.16GB
Q4_XS
1.13GB file 3.16GB

gemma-4-E4B-GGUF

4B 8K ctx
Q3_M
1.75GB file 3.82GB
Q4_NL
2.25GB file 4.32GB
Q4_XS
2.25GB file 4.32GB

gemma-7b-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
FP16
14GB file 16.11GB

Granite-3.3-8B-Instruct-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
FP16
16GB file 18.13GB

granite-4.1-8b-GGUF

8B 8K ctx
Q3_M
3.5GB file 5.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB

granite-guardian-4.1-8b-GGUF

8B 8K ctx
Q3_M
3.5GB file 5.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB

Hermes-2-Pro-Llama-3-8B-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
FP16
16GB file 18.13GB

Hermes-4-14B-GGUF

14B 8K ctx
Q4_0
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
FP16
28GB file 30.23GB

Internvl3-8b-instruct-GGUF

8B 8K ctx
Q3_S
3.5GB file 5.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
BF16
16GB file 18.13GB

llama-2-13b-chat-GGUF

13B 8K ctx
Q4_K_M
7.31GB file 9.52GB
Q5_K_M
8.94GB file 11.15GB

Llama-3-Sqlcoder-8B-GGUF

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB

Llama-3.2-1B-Instruct-Medical-GGUF

1B 8K ctx
Q4_K
0.56GB file 2.58GB
Q5_K
0.69GB file 2.71GB

Llama-3.2-3B-Instruct-GGUF

3B 8K ctx
Q4_K_M
1.69GB file 3.74GB
Q4_XS
1.69GB file 3.74GB
Q5_K_M
2.06GB file 4.11GB

LLama3-Gaja-Hindi-8B-GGUF

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB

Llama3-Med42-8B-GGUF

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB

MediPhi-Instruct-GGUF

Unknown 8K ctx
Q3_M
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB

MedScholar-1.5B-GGUF

1.5B 8K ctx
Q4_K_M
0.84GB file 2.86GB
Q5_K_M
1.03GB file 3.05GB

Meta-Llama-3-8B-Instruct-GGUF

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB

Ministral-3-8B-Instruct-2512-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
FP16
16GB file 18.13GB

Ministral-3-8B-Reasoning-2512-GGUF

8B 8K ctx
Q3_M
3.5GB file 5.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
BF16
16GB file 18.13GB

Mistral-7B-Instruct-v0.3-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
FP16
14GB file 16.11GB

Mistral-NeMo-12B-Instruct-GGUF

12B 8K ctx
Q4_0
6.75GB file 8.95GB
Q5_0
8.25GB file 10.45GB
FP16
24GB file 26.2GB

Mixtral-8x7B-v0.1-GGUF

7B 8K ctx
Q4_K_M
3.94GB file 6.05GB
Q5_K_M
4.81GB file 6.92GB

Nous-Hermes-13B-GGUF

13B 8K ctx
Q4_K_M
7.31GB file 9.52GB
Q5_K_M
8.94GB file 11.15GB

NVIDIA-Nemotron-Nano-9B-v2-GGUF

9B 8K ctx
Q4_0
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
FP16
18GB file 20.15GB

Nxcode-CQ-7B-orpo-GGUF

7B 8K ctx
Q4_K_M
3.94GB file 6.05GB
Q5_K_M
4.81GB file 6.92GB

openchat-3.6-8b-20240522-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
FP16
16GB file 18.13GB

OpenReasoning-Nemotron-1.5B-GGUF

1.5B 8K ctx
Q4_K_M
0.84GB file 2.86GB
Q5_K_M
1.03GB file 3.05GB

OpenReasoning-Nemotron-7B-GGUF

7B 8K ctx
Q4_K_M
3.94GB file 6.05GB
Q5_K_M
4.81GB file 6.92GB

Phi-3-medium-128k-instruct-GGUF

3.8B 8K ctx
Q4_0
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
FP16
7.6GB file 9.66GB

Phi-3-mini-4k-instruct-GGUF

3.8B 8K ctx
Q4_0
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
FP16
7.6GB file 9.66GB

Phi-3.1-mini-4k-instruct-GGUF

3.8B 8K ctx
Q4_K_M
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K_M
2.61GB file 4.67GB

Phi-4-mini-reasoning-GGUF

14B 8K ctx
Q4_K_M
7.88GB file 10.11GB

Phi-4-reasoning-plus-GGUF

14B 8K ctx
Q3_M
6.13GB file 8.36GB
Q4_NL
7.88GB file 10.11GB
Q4_XS
7.88GB file 10.11GB
Q6_K
11.38GB file 13.61GB

Qwen2.5-14B-Instruct-GGUF

14B 8K ctx
Q4_0
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
FP16
28GB file 30.23GB

Qwen2.5-Math-1.5B-GGUF

1.5B 8K ctx
Q4_K_M
0.84GB file 2.86GB
Q5_K_M
1.03GB file 3.05GB

Qwen2.5-VL-7B-Instruct-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
FP16
14GB file 16.11GB

Qwen3-4B-Thinking-2507-GGUF

4B 8K ctx
Q4_0
2.25GB file 4.32GB
Q5_0
2.75GB file 4.82GB
FP16
8GB file 10.07GB

Qwen3-GGUF

0.6B 8K ctx
Q4_K_M
0.34GB file 2.35GB

SmolLM-1.7B-Instruct-GGUF

1.7B 8K ctx
Q4_K_M
0.96GB file 2.99GB
Q4_XS
0.96GB file 2.99GB
Q5_K_M
1.17GB file 3.2GB

SOLAR-10.7B-v1.0-GGUF

10.7B 8K ctx
Q4_0
6.02GB file 8.2GB
Q5_0
7.36GB file 9.54GB
FP16
21.4GB file 23.58GB

Tesslate-UIGEN-T2-7B-GGUF

7B 8K ctx
Q4_K_M
3.94GB file 6.05GB

Yi-1.5-9B-Chat-GGUF

9B 8K ctx
Q4_0
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
FP16
18GB file 20.15GB