All Supported Models

29681 AI models · 198875 quantization formats tracked

T

Thinktecture

4 models

gemma3-1b-ft-nextera-f16

1B 8K ctx
FP16
2GB file 4.02GB

gemma3-4b-ft-nextera-f16

4B 8K ctx
FP16
8GB file 10.07GB

gemma3-4b-ft-nextera-q4_k_m

4B 8K ctx
Q4_0
2.25GB file 4.32GB
FP16
8GB file 10.07GB

qwen3.5-4b-toolcalling-ft-nextera-q4_k_m

4B 8K ctx
Q4_0
2.25GB file 4.32GB
FP16
8GB file 10.07GB
T

Thisdood

1 models

transliteration-7b-32k

7B 8K ctx
Q8_0
7.44GB file 9.55GB
T

Thomas Baruzier

27 models

DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_1
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_1
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB

DeepSeek-R1-ReDistill-Qwen-7B-v1.1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_1
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_1
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB

EXAONE-3.5-2.4B-Instruct-GGUF

2.4B 8K ctx
Q2_K
0.75GB file 2.79GB
Q2_K_S
0.75GB file 2.79GB
Q2_M
0.75GB file 2.79GB
Q2_S
0.75GB file 2.79GB
Q2_XS
0.75GB file 2.79GB
Q2_XXS
0.75GB file 2.79GB
Q3_K_L
1.05GB file 3.09GB
Q3_K_M
1.05GB file 3.09GB
Q3_K_S
1.05GB file 3.09GB
Q3_M
1.05GB file 3.09GB
Q3_S
1.05GB file 3.09GB
Q3_XS
1.05GB file 3.09GB
Q3_XXS
1.05GB file 3.09GB
Q4_0
1.35GB file 3.39GB
Q4_1
1.35GB file 3.39GB
Q4_K_M
1.35GB file 3.39GB
Q4_K_S
1.35GB file 3.39GB
Q4_NL
1.35GB file 3.39GB
Q4_XS
1.35GB file 3.39GB
Q5_0
1.65GB file 3.69GB
Q5_1
1.65GB file 3.69GB
Q5_K_M
1.65GB file 3.69GB
Q5_K_S
1.65GB file 3.69GB
Q6_K
1.95GB file 3.99GB
Q1_M
2.4GB file 4.44GB
Q1_S
2.4GB file 4.44GB
Q8_0
2.55GB file 4.59GB

EXAONE-3.5-7.8B-Instruct-GGUF

7.8B 8K ctx
Q2_K
2.44GB file 4.57GB
Q2_K_S
2.44GB file 4.57GB
Q2_M
2.44GB file 4.57GB
Q2_S
2.44GB file 4.57GB
Q2_XS
2.44GB file 4.57GB
Q2_XXS
2.44GB file 4.57GB
Q3_K_L
3.41GB file 5.54GB
Q3_K_M
3.41GB file 5.54GB
Q3_K_S
3.41GB file 5.54GB
Q3_M
3.41GB file 5.54GB
Q3_S
3.41GB file 5.54GB
Q3_XS
3.41GB file 5.54GB
Q3_XXS
3.41GB file 5.54GB
Q4_0
4.39GB file 6.52GB
Q4_1
4.39GB file 6.52GB
Q4_K_M
4.39GB file 6.52GB
Q4_K_S
4.39GB file 6.52GB
Q4_NL
4.39GB file 6.52GB
Q4_XS
4.39GB file 6.52GB
Q5_0
5.36GB file 7.49GB
Q5_1
5.36GB file 7.49GB
Q5_K_M
5.36GB file 7.49GB
Q5_K_S
5.36GB file 7.49GB
Q6_K
6.34GB file 8.47GB
Q1_M
7.8GB file 9.93GB
Q1_S
7.8GB file 9.93GB
Q8_0
8.29GB file 10.42GB

gemma-2-27b-it-GGUF

27B 8K ctx
Q2_K
8.44GB file 10.88GB
Q2_K_S
8.44GB file 10.88GB
Q2_M
8.44GB file 10.88GB
Q2_S
8.44GB file 10.88GB
Q2_XS
8.44GB file 10.88GB
Q2_XXS
8.44GB file 10.88GB
Q3_K_L
11.81GB file 14.25GB
Q3_K_M
11.81GB file 14.25GB
Q3_K_S
11.81GB file 14.25GB
Q3_M
11.81GB file 14.25GB
Q3_S
11.81GB file 14.25GB
Q3_XS
11.81GB file 14.25GB
Q3_XXS
11.81GB file 14.25GB
Q4_0
15.19GB file 17.63GB
Q4_1
15.19GB file 17.63GB
Q4_K_M
15.19GB file 17.63GB
Q4_K_S
15.19GB file 17.63GB
Q4_NL
15.19GB file 17.63GB
Q4_XS
15.19GB file 17.63GB
Q5_0
18.56GB file 21GB
Q5_1
18.56GB file 21GB
Q5_K_M
18.56GB file 21GB
Q5_K_S
18.56GB file 21GB
Q6_K
21.94GB file 24.38GB
Q1_M
27GB file 29.44GB
Q1_S
27GB file 29.44GB
Q8_0
28.69GB file 31.13GB

gemma-2-2b-it-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q2_K_S
0.63GB file 2.66GB
Q2_M
0.63GB file 2.66GB
Q2_S
0.63GB file 2.66GB
Q2_XS
0.63GB file 2.66GB
Q2_XXS
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q3_M
0.88GB file 2.91GB
Q3_S
0.88GB file 2.91GB
Q3_XS
0.88GB file 2.91GB
Q3_XXS
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_1
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q4_NL
1.13GB file 3.16GB
Q4_XS
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q5_1
1.38GB file 3.41GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q1_M
2GB file 4.03GB
Q1_S
2GB file 4.03GB
Q8_0
2.13GB file 4.16GB

gemma-2-9b-it-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_M
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_1
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_0
6.19GB file 8.34GB
Q5_1
6.19GB file 8.34GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB

Llama-3.2-1B-Instruct-GGUF

1B 8K ctx
Q2_K
0.31GB file 2.33GB
Q2_K_S
0.31GB file 2.33GB
Q2_M
0.31GB file 2.33GB
Q2_S
0.31GB file 2.33GB
Q2_XS
0.31GB file 2.33GB
Q2_XXS
0.31GB file 2.33GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_M
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q3_M
0.44GB file 2.46GB
Q3_S
0.44GB file 2.46GB
Q3_XS
0.44GB file 2.46GB
Q3_XXS
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q4_1
0.56GB file 2.58GB
Q4_K_M
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q4_NL
0.56GB file 2.58GB
Q4_XS
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q5_1
0.69GB file 2.71GB
Q5_K_M
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q1_M
1GB file 3.02GB
Q1_S
1GB file 3.02GB
Q8_0
1.06GB file 3.08GB

Llama-3.2-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q2_K_S
0.94GB file 2.99GB
Q2_M
0.94GB file 2.99GB
Q2_S
0.94GB file 2.99GB
Q2_XS
0.94GB file 2.99GB
Q2_XXS
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q3_M
1.31GB file 3.36GB
Q3_S
1.31GB file 3.36GB
Q3_XS
1.31GB file 3.36GB
Q3_XXS
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_1
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q4_NL
1.69GB file 3.74GB
Q4_XS
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_1
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q1_M
3GB file 5.05GB
Q1_S
3GB file 5.05GB
Q8_0
3.19GB file 5.24GB

Meta-Llama-3-70B-Instruct-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_1
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_1
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB

Meta-Llama-3-8B-Instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_1
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_1
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB

Meta-Llama-3.1-405B-Instruct-16KV-heads-GGUF

405B 8K ctx
Q2_M
126.56GB file 135.2GB
Q2_S
126.56GB file 135.2GB
Q2_XS
126.56GB file 135.2GB
Q2_XXS
126.56GB file 135.2GB
Q3_XXS
177.19GB file 185.83GB
Q1_M
405GB file 413.64GB
Q1_S
405GB file 413.64GB

Meta-Llama-3.1-405B-Instruct-GGUF

405B 8K ctx
Q2_K
126.56GB file 135.2GB
Q2_K_S
126.56GB file 135.2GB
Q2_M
126.56GB file 135.2GB
Q2_S
126.56GB file 135.2GB
Q2_XS
126.56GB file 135.2GB
Q2_XXS
126.56GB file 135.2GB
Q3_K_L
177.19GB file 185.83GB
Q3_K_M
177.19GB file 185.83GB
Q3_K_S
177.19GB file 185.83GB
Q3_M
177.19GB file 185.83GB
Q3_S
177.19GB file 185.83GB
Q3_XS
177.19GB file 185.83GB
Q3_XXS
177.19GB file 185.83GB
Q4_K_M
227.81GB file 236.45GB
Q4_K_S
227.81GB file 236.45GB
Q4_NL
227.81GB file 236.45GB
Q4_XS
227.81GB file 236.45GB
Q1_M
405GB file 413.64GB
Q1_S
405GB file 413.64GB

Meta-Llama-3.1-70B-Instruct-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_1
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_1
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB

Meta-Llama-3.1-8B-Instruct-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_M
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_1
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_1
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB

Qwen2.5-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q2_K_S
0.16GB file 2.17GB
Q2_M
0.16GB file 2.17GB
Q2_S
0.16GB file 2.17GB
Q2_XS
0.16GB file 2.17GB
Q2_XXS
0.16GB file 2.17GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_M
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q3_M
0.22GB file 2.23GB
Q3_S
0.22GB file 2.23GB
Q3_XS
0.22GB file 2.23GB
Q3_XXS
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q4_1
0.28GB file 2.29GB
Q4_K_M
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q4_NL
0.28GB file 2.29GB
Q4_XS
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q5_1
0.34GB file 2.35GB
Q5_K_M
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q1_M
0.5GB file 2.51GB
Q1_S
0.5GB file 2.51GB
Q8_0
0.53GB file 2.54GB

Qwen2.5-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_1
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_1
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB

Qwen2.5-14B-Instruct-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q2_K_S
4.38GB file 6.61GB
Q2_M
4.38GB file 6.61GB
Q2_S
4.38GB file 6.61GB
Q2_XS
4.38GB file 6.61GB
Q2_XXS
4.38GB file 6.61GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_M
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q3_M
6.13GB file 8.36GB
Q3_S
6.13GB file 8.36GB
Q3_XS
6.13GB file 8.36GB
Q3_XXS
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q4_1
7.88GB file 10.11GB
Q4_K_M
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q4_NL
7.88GB file 10.11GB
Q4_XS
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_1
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q1_M
14GB file 16.23GB
Q1_S
14GB file 16.23GB
Q8_0
14.88GB file 17.11GB

Qwen2.5-32B-Instruct-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q2_K_S
10GB file 12.52GB
Q2_M
10GB file 12.52GB
Q2_S
10GB file 12.52GB
Q2_XS
10GB file 12.52GB
Q2_XXS
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q3_M
14GB file 16.52GB
Q3_S
14GB file 16.52GB
Q3_XS
14GB file 16.52GB
Q3_XXS
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_1
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q4_NL
18GB file 20.52GB
Q4_XS
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_1
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q1_M
32GB file 34.52GB
Q1_S
32GB file 34.52GB
Q8_0
34GB file 36.52GB

Qwen2.5-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q2_K_S
0.94GB file 2.99GB
Q2_M
0.94GB file 2.99GB
Q2_S
0.94GB file 2.99GB
Q2_XS
0.94GB file 2.99GB
Q2_XXS
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q3_M
1.31GB file 3.36GB
Q3_S
1.31GB file 3.36GB
Q3_XS
1.31GB file 3.36GB
Q3_XXS
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_1
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q4_NL
1.69GB file 3.74GB
Q4_XS
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_1
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q1_M
3GB file 5.05GB
Q1_S
3GB file 5.05GB
Q8_0
3.19GB file 5.24GB

Qwen2.5-72B-Instruct-GGUF

72B 8K ctx
Q2_K
22.5GB file 25.68GB
Q2_K_S
22.5GB file 25.68GB
Q2_M
22.5GB file 25.68GB
Q2_S
22.5GB file 25.68GB
Q2_XS
22.5GB file 25.68GB
Q2_XXS
22.5GB file 25.68GB
Q3_K_L
31.5GB file 34.68GB
Q3_K_M
31.5GB file 34.68GB
Q3_K_S
31.5GB file 34.68GB
Q3_M
31.5GB file 34.68GB
Q3_S
31.5GB file 34.68GB
Q3_XS
31.5GB file 34.68GB
Q3_XXS
31.5GB file 34.68GB
Q4_0
40.5GB file 43.68GB
Q4_1
40.5GB file 43.68GB
Q4_K_M
40.5GB file 43.68GB
Q4_K_S
40.5GB file 43.68GB
Q4_NL
40.5GB file 43.68GB
Q4_XS
40.5GB file 43.68GB
Q5_0
49.5GB file 52.68GB
Q5_1
49.5GB file 52.68GB
Q5_K_M
49.5GB file 52.68GB
Q5_K_S
49.5GB file 52.68GB
Q6_K
58.5GB file 61.68GB
Q1_M
72GB file 75.18GB
Q1_S
72GB file 75.18GB
Q8_0
76.5GB file 79.68GB

Qwen2.5-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_1
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_1
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB

Qwen2.5-Coder-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q2_K_S
0.16GB file 2.17GB
Q2_M
0.16GB file 2.17GB
Q2_S
0.16GB file 2.17GB
Q2_XS
0.16GB file 2.17GB
Q2_XXS
0.16GB file 2.17GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_M
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q3_M
0.22GB file 2.23GB
Q3_S
0.22GB file 2.23GB
Q3_XS
0.22GB file 2.23GB
Q3_XXS
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q4_1
0.28GB file 2.29GB
Q4_K_M
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q4_NL
0.28GB file 2.29GB
Q4_XS
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q5_1
0.34GB file 2.35GB
Q5_K_M
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q1_M
0.5GB file 2.51GB
Q1_S
0.5GB file 2.51GB
Q8_0
0.53GB file 2.54GB

Qwen2.5-Coder-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_M
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_1
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q5_1
1.03GB file 3.05GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB

Qwen2.5-Coder-32B-Instruct-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q2_K_S
10GB file 12.52GB
Q2_M
10GB file 12.52GB
Q2_S
10GB file 12.52GB
Q2_XS
10GB file 12.52GB
Q2_XXS
10GB file 12.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_M
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q3_M
14GB file 16.52GB
Q3_S
14GB file 16.52GB
Q3_XS
14GB file 16.52GB
Q3_XXS
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q4_1
18GB file 20.52GB
Q4_K_M
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q4_NL
18GB file 20.52GB
Q4_XS
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_1
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q1_M
32GB file 34.52GB
Q1_S
32GB file 34.52GB
Q8_0
34GB file 36.52GB

Qwen2.5-Coder-3B-Instruct-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q2_K_S
0.94GB file 2.99GB
Q2_M
0.94GB file 2.99GB
Q2_S
0.94GB file 2.99GB
Q2_XS
0.94GB file 2.99GB
Q2_XXS
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q3_M
1.31GB file 3.36GB
Q3_S
1.31GB file 3.36GB
Q3_XS
1.31GB file 3.36GB
Q3_XXS
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_1
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q4_NL
1.69GB file 3.74GB
Q4_XS
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_1
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q1_M
3GB file 5.05GB
Q1_S
3GB file 5.05GB
Q8_0
3.19GB file 5.24GB

Qwen2.5-Coder-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_1
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_1
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
T

Thomaskuo

1 models

gemma-3-12b-it-heretic

12B 8K ctx
Q3_K_M
5.25GB file 7.45GB
Q4_K_M
6.75GB file 8.95GB
Q4_K_S
6.75GB file 8.95GB
Q5_K_M
8.25GB file 10.45GB
Q5_K_S
8.25GB file 10.45GB
Q6_K
9.75GB file 11.95GB
Q8_0
12.75GB file 14.95GB
T

Thorge A I

1 models

llama-guard-3-1b-q8_0.gguf

1B 8K ctx
Q8_0
1.06GB file 3.08GB
T

Thorh

1 models

Phi-3-medium-4k-instruct-Q4_K_M-GGUF

3.8B 8K ctx
Q4_K_M
2.14GB file 4.2GB
T

Thos0412

2 models

llama-2-ko-7b-Q4_K_M-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB

Mistral-ko-7B-v0.1-Q4_K_M-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB
T

Thread Abort

1 models

h2o-danube3-500m-chat-GGUF

Unknown 8K ctx
Q4_K_M
0GB file 2GB
Q5_K_M
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB
T

Thrithep

1 models

openthaigpt1.5-7b-instruct-Q4_K_M-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB
T

Thuan220401

1 models

Vistral-7B-Chat-gguf

7B 8K ctx
Q2_0
2.19GB file 4.3GB
Q3_0
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q1_0
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
FP16
14GB file 16.11GB