All Supported Models

29681 AI models · 198875 quantization formats tracked

G

Gagan271

1 models

test-llama

Unknown 8K ctx
Q4_K_M
0GB file 2GB
G

Gagan3012

1 models

llava_ar_pretrain_cc3_595k

Unknown 8K ctx
FP16
0GB file 2GB
G

Gai Labs

1 models

strela-GGUF

Unknown 8K ctx
Q2_0
0GB file 2GB
Q4_0
0GB file 2GB
Q8_0
0GB file 2GB
G

Gaianet

139 models

Bespoke-Stratos-32B-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

Bielik-1.5B-v3.0-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Bielik-11B-v2.3-Instruct-GGUF

11B 8K ctx
Q2_K
3.44GB file 5.62GB
Q3_K_L
4.81GB file 6.99GB
Q3_K_M
4.81GB file 6.99GB
Q3_K_S
4.81GB file 6.99GB
Q4_0
6.19GB file 8.37GB
Q4_K_M
6.19GB file 8.37GB
Q4_K_S
6.19GB file 8.37GB
Q5_0
7.56GB file 9.74GB
Q5_K_M
7.56GB file 9.74GB
Q5_K_S
7.56GB file 9.74GB
Q6_K
8.94GB file 11.12GB
Q8_0
11.69GB file 13.87GB

Bielik-4.5B-v3.0-Instruct-GGUF

4.5B 8K ctx
Q2_K
1.41GB file 3.48GB
Q3_K_L
1.97GB file 4.04GB
Q3_K_M
1.97GB file 4.04GB
Q3_K_S
1.97GB file 4.04GB
Q4_0
2.53GB file 4.6GB
Q4_K_M
2.53GB file 4.6GB
Q4_K_S
2.53GB file 4.6GB
Q5_0
3.09GB file 5.16GB
Q5_K_M
3.09GB file 5.16GB
Q5_K_S
3.09GB file 5.16GB
Q6_K
3.66GB file 5.73GB
Q8_0
4.78GB file 6.85GB

Breeze-7B-Instruct-v1_0-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

c4ai-command-r-08-2024-GGUF

2024B 8K ctx
Q2_K
632.5GB file 667.66GB
Q3_K_L
885.5GB file 920.66GB
Q3_K_M
885.5GB file 920.66GB
Q3_K_S
885.5GB file 920.66GB
Q4_0
1138.5GB file 1173.66GB
Q4_K_M
1138.5GB file 1173.66GB
Q4_K_S
1138.5GB file 1173.66GB
Q5_0
1391.5GB file 1426.66GB
Q5_K_M
1391.5GB file 1426.66GB
Q5_K_S
1391.5GB file 1426.66GB
Q6_K
1644.5GB file 1679.66GB
Q8_0
2150.5GB file 2185.66GB

Codestral-22B-v0.1-GGUF

22B 8K ctx
Q2_K
6.88GB file 9.24GB
Q3_K_L
9.63GB file 11.99GB
Q3_K_M
9.63GB file 11.99GB
Q3_K_S
9.63GB file 11.99GB
Q4_0
12.38GB file 14.74GB
Q4_K_M
12.38GB file 14.74GB
Q4_K_S
12.38GB file 14.74GB
Q5_0
15.13GB file 17.49GB
Q5_K_M
15.13GB file 17.49GB
Q5_K_S
15.13GB file 17.49GB
Q6_K
17.88GB file 20.24GB
Q8_0
23.38GB file 25.74GB

datagemma-rag-27b-it-GGUF

27B 8K ctx
Q2_K
8.44GB file 10.88GB
Q3_K_L
11.81GB file 14.25GB
Q3_K_M
11.81GB file 14.25GB
Q3_K_S
11.81GB file 14.25GB
Q4_0
15.19GB file 17.63GB
Q4_K_M
15.19GB file 17.63GB
Q4_K_S
15.19GB file 17.63GB
Q5_0
18.56GB file 21GB
Q5_K_M
18.56GB file 21GB
Q5_K_S
18.56GB file 21GB
Q6_K
21.94GB file 24.38GB
Q8_0
28.69GB file 31.13GB

datagemma-rig-27b-it-GGUF

27B 8K ctx
Q2_K
8.44GB file 10.88GB
Q3_K_L
11.81GB file 14.25GB
Q3_K_M
11.81GB file 14.25GB
Q3_K_S
11.81GB file 14.25GB
Q4_0
15.19GB file 17.63GB
Q4_K_M
15.19GB file 17.63GB
Q4_K_S
15.19GB file 17.63GB
Q5_0
18.56GB file 21GB
Q5_K_M
18.56GB file 21GB
Q5_K_S
18.56GB file 21GB
Q6_K
21.94GB file 24.38GB
Q8_0
28.69GB file 31.13GB

DeepHermes-3-Llama-3-8B-Preview-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

DeepSeek-Coder-V2-Lite-Instruct-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

DeepSeek-R1-0528-Qwen3-8B-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

DeepSeek-R1-Distill-Llama-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

DeepSeek-R1-Distill-Llama-8B-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

DeepSeek-R1-Distill-Qwen-1.5B-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

DeepSeek-R1-Distill-Qwen-32B-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

DeepSeek-R1-Distill-Qwen-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

DeepSeek-V2-Lite-Chat-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

EXAONE-3.0-7.8B-Instruct-GGUF

7.8B 8K ctx
Q2_K
2.44GB file 4.57GB
Q3_K_L
3.41GB file 5.54GB
Q3_K_M
3.41GB file 5.54GB
Q3_K_S
3.41GB file 5.54GB
Q4_0
4.39GB file 6.52GB
Q4_K_M
4.39GB file 6.52GB
Q4_K_S
4.39GB file 6.52GB
Q5_0
5.36GB file 7.49GB
Q5_K_M
5.36GB file 7.49GB
Q5_K_S
5.36GB file 7.49GB
Q6_K
6.34GB file 8.47GB
Q8_0
8.29GB file 10.42GB

EXAONE-3.5-2.4B-Instruct-GGUF

2.4B 8K ctx
Q2_K
0.75GB file 2.79GB
Q3_K_L
1.05GB file 3.09GB
Q3_K_M
1.05GB file 3.09GB
Q3_K_S
1.05GB file 3.09GB
Q4_0
1.35GB file 3.39GB
Q4_K_M
1.35GB file 3.39GB
Q4_K_S
1.35GB file 3.39GB
Q5_0
1.65GB file 3.69GB
Q5_K_M
1.65GB file 3.69GB
Q5_K_S
1.65GB file 3.69GB
Q6_K
1.95GB file 3.99GB
Q8_0
2.55GB file 4.59GB

EXAONE-3.5-32B-Instruct-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

EXAONE-3.5-7.8B-Instruct-GGUF

7.8B 8K ctx
Q2_K
2.44GB file 4.57GB
Q3_K_L
3.41GB file 5.54GB
Q3_K_M
3.41GB file 5.54GB
Q3_K_S
3.41GB file 5.54GB
Q4_0
4.39GB file 6.52GB
Q4_K_M
4.39GB file 6.52GB
Q4_K_S
4.39GB file 6.52GB
Q5_0
5.36GB file 7.49GB
Q5_K_M
5.36GB file 7.49GB
Q5_K_S
5.36GB file 7.49GB
Q6_K
6.34GB file 8.47GB
Q8_0
8.29GB file 10.42GB

EXAONE-Deep-2.4B-GGUF

2.4B 8K ctx
Q2_K
0.75GB file 2.79GB
Q3_K_L
1.05GB file 3.09GB
Q3_K_M
1.05GB file 3.09GB
Q3_K_S
1.05GB file 3.09GB
Q4_0
1.35GB file 3.39GB
Q4_K_M
1.35GB file 3.39GB
Q4_K_S
1.35GB file 3.39GB
Q5_0
1.65GB file 3.69GB
Q5_K_M
1.65GB file 3.69GB
Q5_K_S
1.65GB file 3.69GB
Q6_K
1.95GB file 3.99GB
Q8_0
2.55GB file 4.59GB

EXAONE-Deep-32B-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

EXAONE-Deep-7.8B-GGUF

7.8B 8K ctx
Q2_K
2.44GB file 4.57GB
Q3_K_L
3.41GB file 5.54GB
Q3_K_M
3.41GB file 5.54GB
Q3_K_S
3.41GB file 5.54GB
Q4_0
4.39GB file 6.52GB
Q4_K_M
4.39GB file 6.52GB
Q4_K_S
4.39GB file 6.52GB
Q5_0
5.36GB file 7.49GB
Q5_K_M
5.36GB file 7.49GB
Q5_K_S
5.36GB file 7.49GB
Q6_K
6.34GB file 8.47GB
Q8_0
8.29GB file 10.42GB

Falcon3-10B-Instruct-GGUF

10B 8K ctx
Q2_K
3.13GB file 5.29GB
Q3_K_L
4.38GB file 6.54GB
Q3_K_M
4.38GB file 6.54GB
Q3_K_S
4.38GB file 6.54GB
Q4_0
5.63GB file 7.79GB
Q4_K_M
5.63GB file 7.79GB
Q4_K_S
5.63GB file 7.79GB
Q5_0
6.88GB file 9.04GB
Q5_K_M
6.88GB file 9.04GB
Q5_K_S
6.88GB file 9.04GB
Q6_K
8.13GB file 10.29GB
Q8_0
10.63GB file 12.79GB

Falcon3-1B-Instruct-GGUF

1B 8K ctx
Q2_K
0.31GB file 2.33GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_M
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q4_K_M
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q5_K_M
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB

Falcon3-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Falcon3-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

FinGPT-MT-Llama-3-8B-LoRA-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_0
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

gemma-2-27b-it-GGUF

27B 8K ctx
Q2_K
8.44GB file 10.88GB
Q3_K_L
11.81GB file 14.25GB
Q3_K_M
11.81GB file 14.25GB
Q3_K_S
11.81GB file 14.25GB
Q4_0
15.19GB file 17.63GB
Q4_K_M
15.19GB file 17.63GB
Q4_K_S
15.19GB file 17.63GB
Q5_0
18.56GB file 21GB
Q5_K_M
18.56GB file 21GB
Q5_K_S
18.56GB file 21GB
Q6_K
21.94GB file 24.38GB
Q8_0
28.69GB file 31.13GB

gemma-2-2b-it-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

Gemma-2-9B-Chinese-Chat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_M
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB

gemma-2-9b-it-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_M
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB

gemma-3-1b-it-GGUF

1B 8K ctx
Q2_K
0.31GB file 2.33GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_M
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q4_K_M
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q5_K_M
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB

gemma-3-270m-it-GGUF

1B 8K ctx
Q2_K
0.31GB file 2.33GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_M
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q4_K_M
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q5_K_M
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB

gpt-oss-120b-GGUF

120B 8K ctx
FP4
120GB file 123.97GB

gpt-oss-20b-GGUF

20B 8K ctx
FP4
20GB file 22.33GB

Infinity-Instruct-7M-Gen-Llama3_1-8B-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

internlm2_5-1_8b-chat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

internlm2_5-20b-chat-GGUF

20B 8K ctx
Q2_K
6.25GB file 8.58GB
Q3_K_L
8.75GB file 11.08GB
Q3_K_M
8.75GB file 11.08GB
Q3_K_S
8.75GB file 11.08GB
Q4_0
11.25GB file 13.58GB
Q4_K_M
11.25GB file 13.58GB
Q4_K_S
11.25GB file 13.58GB
Q5_0
13.75GB file 16.08GB
Q5_K_M
13.75GB file 16.08GB
Q5_K_S
13.75GB file 16.08GB
Q6_K
16.25GB file 18.58GB
Q8_0
21.25GB file 23.58GB

internlm2_5-7b-chat-1m-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

internlm2_5-7b-chat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

internlm3-8b-instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Llama-3_1-Nemotron-51B-Instruct-GGUF

51B 8K ctx
Q2_K
15.94GB file 18.78GB
Q3_K_L
22.31GB file 25.15GB
Q3_K_M
22.31GB file 25.15GB
Q3_K_S
22.31GB file 25.15GB
Q4_0
28.69GB file 31.53GB
Q4_K_M
28.69GB file 31.53GB
Q4_K_S
28.69GB file 31.53GB
Q5_0
35.06GB file 37.9GB
Q5_K_M
35.06GB file 37.9GB
Q5_K_S
35.06GB file 37.9GB
Q6_K
41.44GB file 44.28GB
Q8_0
54.19GB file 57.03GB

Llama-3-8B-Japanese-Instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Llama-3-Groq-70B-Tool-Use-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Llama-3-Groq-8B-Tool-Use-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Llama-3-Instruct-8B-SimPO-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Llama-3-Taiwan-70B-Instruct-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Llama-3-Taiwan-8B-Instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Llama-3.1-Nemotron-70B-Instruct-HF-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Llama-3.1-Nemotron-70B-Reward-HF-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Llama-3.2-1B-Instruct-GGUF

1B 8K ctx
Q2_K
0.31GB file 2.33GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_M
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q4_K_M
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q5_K_M
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB

Llama-3.2-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Llama-3.2-3B-Instruct-Uncensored-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

llama-3.2-Korean-Bllossom-3B-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Llama-3.3-70B-Instruct-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Llama3-8B-Chinese-Chat-GGUF

8B 8K ctx
Q5_K_M
5.5GB file 7.63GB

Llama3.1-8B-Chinese-Chat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

llm-compiler-13b-ftd-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

llm-compiler-13b-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

llm-compiler-7b-ftd-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

llm-compiler-7b-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

mathstral-7B-v0.1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Megrez-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Meta-Llama-3.1-70B-Instruct-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Meta-Llama-3.1-8B-Instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Mistral-7B-Instruct-v0.3-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Mistral-Nemo-Instruct-2407-GGUF

2407B 8K ctx
Q2_K
752.19GB file 793.63GB
Q3_K_L
1053.06GB file 1094.5GB
Q3_K_M
1053.06GB file 1094.5GB
Q3_K_S
1053.06GB file 1094.5GB
Q4_0
1353.94GB file 1395.38GB
Q4_K_M
1353.94GB file 1395.38GB
Q4_K_S
1353.94GB file 1395.38GB
Q5_0
1654.81GB file 1696.25GB
Q5_K_M
1654.81GB file 1696.25GB
Q5_K_S
1654.81GB file 1696.25GB
Q6_K
1955.69GB file 1997.13GB
Q8_0
2557.44GB file 2598.88GB

Mistral-Small-24B-Instruct-2501-GGUF

24B 8K ctx
Q2_K
7.5GB file 9.89GB
Q3_K_L
10.5GB file 12.89GB
Q3_K_M
10.5GB file 12.89GB
Q3_K_S
10.5GB file 12.89GB
Q4_0
13.5GB file 15.89GB
Q4_K_M
13.5GB file 15.89GB
Q4_K_S
13.5GB file 15.89GB
Q5_0
16.5GB file 18.89GB
Q5_K_M
16.5GB file 18.89GB
Q5_K_S
16.5GB file 18.89GB
Q6_K
19.5GB file 21.89GB
Q8_0
25.5GB file 27.89GB

NVIDIA-Nemotron-Nano-9B-v2-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_M
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB

Palmyra-Fin-70B-32K-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Palmyra-Med-70B-32K-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Phi-3-medium-128k-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_M
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB

Phi-3-medium-4k-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_M
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB

Phi-3-mini-128k-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_M
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB

Phi-3-mini-4k-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_M
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB

Phi-3.5-mini-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_M
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB

Phi-3.5-MoE-instruct-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_M
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q5_0
2.61GB file 4.67GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q8_0
4.04GB file 6.1GB

phi-4-GGUF

4B 8K ctx
Q2_K
1.25GB file 3.32GB
Q3_K_L
1.75GB file 3.82GB
Q3_K_M
1.75GB file 3.82GB
Q3_K_S
1.75GB file 3.82GB
Q4_0
2.25GB file 4.32GB
Q4_K_M
2.25GB file 4.32GB
Q4_K_S
2.25GB file 4.32GB
Q5_0
2.75GB file 4.82GB
Q5_K_M
2.75GB file 4.82GB
Q5_K_S
2.75GB file 4.82GB
Q6_K
3.25GB file 5.32GB
Q8_0
4.25GB file 6.32GB

Phi-4-mini-instruct-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Phi-4-mini-reasoning-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Phi-4-reasoning-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Phi-4-reasoning-plus-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Qwen2-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_M
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q4_K_M
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q5_K_M
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB

Qwen2-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Qwen2-72B-Instruct-GGUF

72B 8K ctx
Q2_K
22.5GB file 25.68GB
Q3_K_L
31.5GB file 34.68GB
Q3_K_M
31.5GB file 34.68GB
Q3_K_S
31.5GB file 34.68GB
Q4_0
40.5GB file 43.68GB
Q4_K_M
40.5GB file 43.68GB
Q4_K_S
40.5GB file 43.68GB
Q5_0
49.5GB file 52.68GB
Q5_K_M
49.5GB file 52.68GB
Q5_K_S
49.5GB file 52.68GB
Q6_K
58.5GB file 61.68GB
Q8_0
76.5GB file 79.68GB

Qwen2-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Qwen2-Math-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Qwen2-Math-72B-Instruct-GGUF

72B 8K ctx
Q2_K
22.5GB file 25.68GB
Q3_K_L
31.5GB file 34.68GB
Q3_K_M
31.5GB file 34.68GB
Q3_K_S
31.5GB file 34.68GB
Q4_0
40.5GB file 43.68GB
Q4_K_M
40.5GB file 43.68GB
Q4_K_S
40.5GB file 43.68GB
Q5_0
49.5GB file 52.68GB
Q5_K_M
49.5GB file 52.68GB
Q5_K_S
49.5GB file 52.68GB
Q6_K
58.5GB file 61.68GB
Q8_0
76.5GB file 79.68GB

Qwen2-Math-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Qwen2.5-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_M
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q4_K_M
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q5_K_M
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB

Qwen2.5-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Qwen2.5-14B-Instruct-1M-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Qwen2.5-14B-Instruct-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Qwen2.5-32B-Instruct-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

Qwen2.5-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Qwen2.5-72B-Instruct-GGUF

72B 8K ctx
Q2_K
22.5GB file 25.68GB
Q3_K_L
31.5GB file 34.68GB
Q3_K_M
31.5GB file 34.68GB
Q3_K_S
31.5GB file 34.68GB
Q4_0
40.5GB file 43.68GB
Q4_K_M
40.5GB file 43.68GB
Q4_K_S
40.5GB file 43.68GB
Q5_0
49.5GB file 52.68GB
Q5_K_M
49.5GB file 52.68GB
Q5_K_S
49.5GB file 52.68GB
Q6_K
58.5GB file 61.68GB
Q8_0
76.5GB file 79.68GB

Qwen2.5-7B-Instruct-1M-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Qwen2.5-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Qwen2.5-Coder-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_M
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q4_K_M
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q5_K_M
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB

Qwen2.5-Coder-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Qwen2.5-Coder-14B-Instruct-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Qwen2.5-Coder-32B-Instruct-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

Qwen2.5-Coder-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Qwen2.5-Coder-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Qwen2.5-Math-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Qwen2.5-Math-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Qwen3-0.6B-GGUF

0.6B 8K ctx
Q2_K
0.19GB file 2.2GB
Q3_K_L
0.26GB file 2.27GB
Q3_K_M
0.26GB file 2.27GB
Q3_K_S
0.26GB file 2.27GB
Q4_0
0.34GB file 2.35GB
Q4_K_M
0.34GB file 2.35GB
Q4_K_S
0.34GB file 2.35GB
Q5_0
0.41GB file 2.42GB
Q5_K_M
0.41GB file 2.42GB
Q5_K_S
0.41GB file 2.42GB
Q6_K
0.49GB file 2.5GB
Q8_0
0.64GB file 2.65GB

Qwen3-1.7B-GGUF

1.7B 8K ctx
Q2_K
0.53GB file 2.56GB
Q3_K_L
0.74GB file 2.77GB
Q3_K_M
0.74GB file 2.77GB
Q3_K_S
0.74GB file 2.77GB
Q4_0
0.96GB file 2.99GB
Q4_K_M
0.96GB file 2.99GB
Q4_K_S
0.96GB file 2.99GB
Q5_0
1.17GB file 3.2GB
Q5_K_M
1.17GB file 3.2GB
Q5_K_S
1.17GB file 3.2GB
Q6_K
1.38GB file 3.41GB
Q8_0
1.81GB file 3.84GB

Qwen3-14B-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Qwen3-30B-A3B-Instruct-2507-GGUF

30B 8K ctx
Q2_K
9.38GB file 11.87GB
Q3_K_L
13.13GB file 15.62GB
Q3_K_M
13.13GB file 15.62GB
Q3_K_S
13.13GB file 15.62GB
Q4_0
16.88GB file 19.37GB
Q4_K_M
16.88GB file 19.37GB
Q4_K_S
16.88GB file 19.37GB
Q5_0
20.63GB file 23.12GB
Q5_K_M
20.63GB file 23.12GB
Q5_K_S
20.63GB file 23.12GB
Q6_K
24.38GB file 26.87GB
Q8_0
31.88GB file 34.37GB

Qwen3-30B-A3B-Thinking-2507-GGUF

30B 8K ctx
Q2_K
9.38GB file 11.87GB
Q3_K_L
13.13GB file 15.62GB
Q3_K_M
13.13GB file 15.62GB
Q3_K_S
13.13GB file 15.62GB
Q4_0
16.88GB file 19.37GB
Q4_K_M
16.88GB file 19.37GB
Q4_K_S
16.88GB file 19.37GB
Q5_0
20.63GB file 23.12GB
Q5_K_M
20.63GB file 23.12GB
Q5_K_S
20.63GB file 23.12GB
Q6_K
24.38GB file 26.87GB
Q8_0
31.88GB file 34.37GB

Qwen3-4B-GGUF

4B 8K ctx
Q2_K
1.25GB file 3.32GB
Q3_K_L
1.75GB file 3.82GB
Q3_K_M
1.75GB file 3.82GB
Q3_K_S
1.75GB file 3.82GB
Q4_0
2.25GB file 4.32GB
Q4_K_M
2.25GB file 4.32GB
Q4_K_S
2.25GB file 4.32GB
Q5_0
2.75GB file 4.82GB
Q5_K_M
2.75GB file 4.82GB
Q5_K_S
2.75GB file 4.82GB
Q6_K
3.25GB file 5.32GB
Q8_0
4.25GB file 6.32GB

Qwen3-Coder-30B-A3B-Instruct-GGUF

30B 8K ctx
Q2_K
9.38GB file 11.87GB
Q3_K_L
13.13GB file 15.62GB
Q3_K_M
13.13GB file 15.62GB
Q3_K_S
13.13GB file 15.62GB
Q4_0
16.88GB file 19.37GB
Q4_K_M
16.88GB file 19.37GB
Q4_K_S
16.88GB file 19.37GB
Q5_0
20.63GB file 23.12GB
Q5_K_M
20.63GB file 23.12GB
Q5_K_S
20.63GB file 23.12GB
Q6_K
24.38GB file 26.87GB
Q8_0
31.88GB file 34.37GB

QwQ-32B-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

QwQ-32B-Preview-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

Reflection-Llama-3.1-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q8_0
74.38GB file 77.53GB

Seed-OSS-36B-Instruct-GGUF

36B 8K ctx
Q2_K
11.25GB file 13.84GB
Q3_K_L
15.75GB file 18.34GB
Q3_K_M
15.75GB file 18.34GB
Q3_K_S
15.75GB file 18.34GB
Q4_0
20.25GB file 22.84GB
Q4_K_M
20.25GB file 22.84GB
Q4_K_S
20.25GB file 22.84GB
Q5_0
24.75GB file 27.34GB
Q5_K_M
24.75GB file 27.34GB
Q5_K_S
24.75GB file 27.34GB
Q6_K
29.25GB file 31.84GB
Q8_0
38.25GB file 40.84GB

Sky-T1-32B-Preview-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

SmolLM-135M-Instruct-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

SmolLM2-1.7B-Instruct-GGUF

1.7B 8K ctx
Q2_K
0.53GB file 2.56GB
Q3_K_L
0.74GB file 2.77GB
Q3_K_M
0.74GB file 2.77GB
Q3_K_S
0.74GB file 2.77GB
Q4_0
0.96GB file 2.99GB
Q4_K_M
0.96GB file 2.99GB
Q4_K_S
0.96GB file 2.99GB
Q5_0
1.17GB file 3.2GB
Q5_K_M
1.17GB file 3.2GB
Q5_K_S
1.17GB file 3.2GB
Q6_K
1.38GB file 3.41GB
Q8_0
1.81GB file 3.84GB

SmolLM3-3B-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

stablelm-2-12b-chat-GGUF

12B 8K ctx
Q2_K
3.75GB file 5.95GB
Q3_K_L
5.25GB file 7.45GB
Q3_K_M
5.25GB file 7.45GB
Q3_K_S
5.25GB file 7.45GB
Q4_0
6.75GB file 8.95GB
Q4_K_M
6.75GB file 8.95GB
Q4_K_S
6.75GB file 8.95GB
Q5_0
8.25GB file 10.45GB
Q5_K_M
8.25GB file 10.45GB
Q5_K_S
8.25GB file 10.45GB
Q6_K
9.75GB file 11.95GB
Q8_0
12.75GB file 14.95GB

stablelm-2-zephyr-1.6b-GGUF

6B 8K ctx
Q5_K_M
4.13GB file 6.23GB

StarCoder2-15B-Instruct-v0.1-GGUF

15B 8K ctx
Q2_K
4.69GB file 6.94GB
Q3_K_L
6.56GB file 8.81GB
Q3_K_M
6.56GB file 8.81GB
Q3_K_S
6.56GB file 8.81GB
Q4_0
8.44GB file 10.69GB
Q4_K_M
8.44GB file 10.69GB
Q4_K_S
8.44GB file 10.69GB
Q5_0
10.31GB file 12.56GB
Q5_K_M
10.31GB file 12.56GB
Q5_K_S
10.31GB file 12.56GB
Q6_K
12.19GB file 14.44GB
Q8_0
15.94GB file 18.19GB

Vitalik-ETH-Llama2-7B-Chat-GGUF

7B 8K ctx
Q5_K_M
4.81GB file 6.92GB

xLAM-1b-fc-r-GGUF

1B 8K ctx
Q2_K
0.31GB file 2.33GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_M
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q4_K_M
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q5_K_M
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB

xLAM-7b-fc-r-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

xLAM-8x22b-r-GGUF

22B 8K ctx
Q2_K
6.88GB file 9.24GB
Q3_K_L
9.63GB file 11.99GB
Q3_K_M
9.63GB file 11.99GB
Q3_K_S
9.63GB file 11.99GB
Q4_0
12.38GB file 14.74GB
Q4_K_M
12.38GB file 14.74GB
Q4_K_S
12.38GB file 14.74GB
Q5_0
15.13GB file 17.49GB
Q5_K_M
15.13GB file 17.49GB
Q5_K_S
15.13GB file 17.49GB
Q6_K
17.88GB file 20.24GB
Q8_0
23.38GB file 25.74GB

xLAM-8x7b-r-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Yi-1.5-34B-Chat-16K-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

Yi-1.5-34B-Chat-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

Yi-1.5-6B-Chat-GGUF

6B 8K ctx
Q2_K
1.88GB file 3.98GB
Q3_K_L
2.63GB file 4.73GB
Q3_K_M
2.63GB file 4.73GB
Q3_K_S
2.63GB file 4.73GB
Q4_0
3.38GB file 5.48GB
Q4_K_M
3.38GB file 5.48GB
Q4_K_S
3.38GB file 5.48GB
Q5_0
4.13GB file 6.23GB
Q5_K_M
4.13GB file 6.23GB
Q5_K_S
4.13GB file 6.23GB
Q6_K
4.88GB file 6.98GB
Q8_0
6.38GB file 8.48GB

Yi-1.5-9B-Chat-16K-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_M
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB

Yi-1.5-9B-Chat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_M
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB

Yi-Coder-1.5B-Chat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Yi-Coder-9B-Chat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_M
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB
G

Galatolo

1 models

cerbero-7b-openchat-gguf

7B 8K ctx
Q4_K
3.94GB file 6.05GB
Q8_0
7.44GB file 9.55GB
G

Gamallo

1 models

Llama-3.1-Carballo-Q4_K_M-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
FP16
16GB file 18.13GB
G

Ganchito

1 models

dante-7b.gguf

7B 8K ctx
FP16
14GB file 16.11GB
G

Gandhar

3 models

Llama-3.1-8B-Q4_K_M-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
FP16
16GB file 18.13GB

Llama-3.2-3B-Q4_K_M-GGUF

3B 8K ctx
Q4_0
1.69GB file 3.74GB
FP16
6GB file 8.05GB

Qwen2.5-32B-Instruct-Q4_K_M-GGUF

32B 8K ctx
Q4_0
18GB file 20.52GB
FP16
64GB file 66.52GB
G

Gannima

1 models

AutoGLM-Phone-9B-Multilingual-GGUF

9B 8K ctx
Q4_0
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q6_0
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB
G

Gaoqianshen

1 models

phi-4-Q4_K_M-GGUF

14B 8K ctx
Q4_0
7.88GB file 10.11GB
FP16
28GB file 30.23GB