All Supported Models

29681 AI models · 198875 quantization formats tracked

L

Legionlm

1 models

sonnet-llama-3.2-3b

3B 8K ctx
Q8_0
3.19GB file 5.24GB
L

Legol

2 models

gemma-3-1b-it-Q4_K_M-GGUF

1B 8K ctx
Q4_0
0.56GB file 2.58GB
FP16
2GB file 4.02GB

QwQ-32B-Q2_K-GGUF

32B 8K ctx
Q2_0
10GB file 12.52GB
FP16
64GB file 66.52GB
L

Legraphista

138 models

Athene-70B-IMat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K
30.63GB file 33.78GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB
FP16
140GB file 143.15GB

AutoCoder_S_6.7B-IMat-GGUF

6.7B 8K ctx
Q2_K
2.09GB file 4.2GB
Q2_K_S
2.09GB file 4.2GB
Q2_M
2.09GB file 4.2GB
Q2_S
2.09GB file 4.2GB
Q2_XS
2.09GB file 4.2GB
Q2_XXS
2.09GB file 4.2GB
Q3_K
2.93GB file 5.04GB
Q3_K_L
2.93GB file 5.04GB
Q3_K_S
2.93GB file 5.04GB
Q3_M
2.93GB file 5.04GB
Q3_S
2.93GB file 5.04GB
Q3_XS
2.93GB file 5.04GB
Q3_XXS
2.93GB file 5.04GB
Q4_K
3.77GB file 5.88GB
Q4_K_S
3.77GB file 5.88GB
Q4_NL
3.77GB file 5.88GB
Q4_XS
3.77GB file 5.88GB
Q5_K
4.61GB file 6.72GB
Q5_K_S
4.61GB file 6.72GB
Q6_K
5.44GB file 7.55GB
Q1_M
6.7GB file 8.81GB
Q1_S
6.7GB file 8.81GB
Q8_0
7.12GB file 9.23GB
BF16
13.4GB file 15.51GB
FP16
13.4GB file 15.51GB

AutoCoder-IMat-GGUF

Unknown 8K ctx
BF16
0GB file 2GB
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

aya-23-35B-GGUF

35B 8K ctx
Q2_0
10.94GB file 13.51GB
Q3_0
15.31GB file 17.88GB
Q4_0
19.69GB file 22.26GB
Q5_0
24.06GB file 26.63GB
Q6_0
28.44GB file 31.01GB
Q8_0
37.19GB file 39.76GB
BF16
70GB file 72.57GB
FP16
70GB file 72.57GB

aya-23-35B-IMat-GGUF

35B 8K ctx
Q2_K
10.94GB file 13.51GB
Q2_K_S
10.94GB file 13.51GB
Q2_M
10.94GB file 13.51GB
Q2_S
10.94GB file 13.51GB
Q2_XS
10.94GB file 13.51GB
Q2_XXS
10.94GB file 13.51GB
Q3_K
15.31GB file 17.88GB
Q3_K_L
15.31GB file 17.88GB
Q3_K_S
15.31GB file 17.88GB
Q3_M
15.31GB file 17.88GB
Q3_S
15.31GB file 17.88GB
Q3_XS
15.31GB file 17.88GB
Q3_XXS
15.31GB file 17.88GB
Q4_K
19.69GB file 22.26GB
Q4_K_S
19.69GB file 22.26GB
Q4_NL
19.69GB file 22.26GB
Q4_XS
19.69GB file 22.26GB
Q5_K
24.06GB file 26.63GB
Q5_K_S
24.06GB file 26.63GB
Q6_K
28.44GB file 31.01GB
Q1_M
35GB file 37.57GB
Q1_S
35GB file 37.57GB
Q8_0
37.19GB file 39.76GB
FP16
70GB file 72.57GB

aya-23-8B-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q6_0
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

aya-23-8B-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

c4ai-command-r-plus-08-2024-IMat-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

codegeex4-all-9b-IMat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

Codestral-22B-v0.1-IMat-GGUF

22B 8K ctx
Q2_K
6.88GB file 9.24GB
Q2_K_S
6.88GB file 9.24GB
Q2_M
6.88GB file 9.24GB
Q2_S
6.88GB file 9.24GB
Q2_XS
6.88GB file 9.24GB
Q2_XXS
6.88GB file 9.24GB
Q3_K
9.63GB file 11.99GB
Q3_K_L
9.63GB file 11.99GB
Q3_K_S
9.63GB file 11.99GB
Q3_M
9.63GB file 11.99GB
Q3_S
9.63GB file 11.99GB
Q3_XS
9.63GB file 11.99GB
Q3_XXS
9.63GB file 11.99GB
Q4_K
12.38GB file 14.74GB
Q4_K_S
12.38GB file 14.74GB
Q4_NL
12.38GB file 14.74GB
Q4_XS
12.38GB file 14.74GB
Q5_K
15.13GB file 17.49GB
Q5_K_S
15.13GB file 17.49GB
Q6_K
17.88GB file 20.24GB
Q1_M
22GB file 24.36GB
Q1_S
22GB file 24.36GB
Q8_0
23.38GB file 25.74GB
BF16
44GB file 46.36GB
FP16
44GB file 46.36GB

datagemma-rag-27b-it-IMat-GGUF

27B 8K ctx
Q2_K
8.44GB file 10.88GB
Q2_K_S
8.44GB file 10.88GB
Q2_M
8.44GB file 10.88GB
Q2_S
8.44GB file 10.88GB
Q2_XS
8.44GB file 10.88GB
Q2_XXS
8.44GB file 10.88GB
Q3_K
11.81GB file 14.25GB
Q3_K_L
11.81GB file 14.25GB
Q3_K_S
11.81GB file 14.25GB
Q3_M
11.81GB file 14.25GB
Q3_S
11.81GB file 14.25GB
Q3_XS
11.81GB file 14.25GB
Q3_XXS
11.81GB file 14.25GB
Q4_K
15.19GB file 17.63GB
Q4_K_S
15.19GB file 17.63GB
Q4_NL
15.19GB file 17.63GB
Q4_XS
15.19GB file 17.63GB
Q5_K
18.56GB file 21GB
Q5_K_S
18.56GB file 21GB
Q6_K
21.94GB file 24.38GB
Q1_M
27GB file 29.44GB
Q1_S
27GB file 29.44GB
Q8_0
28.69GB file 31.13GB
BF16
54GB file 56.44GB
FP16
54GB file 56.44GB

datagemma-rig-27b-it-IMat-GGUF

27B 8K ctx
Q2_K
8.44GB file 10.88GB
Q2_K_S
8.44GB file 10.88GB
Q2_M
8.44GB file 10.88GB
Q2_S
8.44GB file 10.88GB
Q2_XS
8.44GB file 10.88GB
Q2_XXS
8.44GB file 10.88GB
Q3_K
11.81GB file 14.25GB
Q3_K_L
11.81GB file 14.25GB
Q3_K_S
11.81GB file 14.25GB
Q3_M
11.81GB file 14.25GB
Q3_S
11.81GB file 14.25GB
Q3_XS
11.81GB file 14.25GB
Q3_XXS
11.81GB file 14.25GB
Q4_K
15.19GB file 17.63GB
Q4_K_S
15.19GB file 17.63GB
Q4_NL
15.19GB file 17.63GB
Q4_XS
15.19GB file 17.63GB
Q5_K
18.56GB file 21GB
Q5_K_S
18.56GB file 21GB
Q6_K
21.94GB file 24.38GB
Q1_M
27GB file 29.44GB
Q1_S
27GB file 29.44GB
Q8_0
28.69GB file 31.13GB
BF16
54GB file 56.44GB
FP16
54GB file 56.44GB

DeepSeek-Coder-V2-Instruct-0724-IMat-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q2_K_S
0.63GB file 2.66GB
Q2_M
0.63GB file 2.66GB
Q2_S
0.63GB file 2.66GB
Q2_XS
0.63GB file 2.66GB
Q2_XXS
0.63GB file 2.66GB
Q3_K
0.88GB file 2.91GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q3_M
0.88GB file 2.91GB
Q3_S
0.88GB file 2.91GB
Q3_XS
0.88GB file 2.91GB
Q3_XXS
0.88GB file 2.91GB
Q4_K
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q4_NL
1.13GB file 3.16GB
Q4_XS
1.13GB file 3.16GB
Q5_K
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q1_M
2GB file 4.03GB
Q1_S
2GB file 4.03GB
Q8_0
2.13GB file 4.16GB

DeepSeek-Coder-V2-Instruct-IMat-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q2_K_S
0.63GB file 2.66GB
Q2_M
0.63GB file 2.66GB
Q2_S
0.63GB file 2.66GB
Q2_XS
0.63GB file 2.66GB
Q2_XXS
0.63GB file 2.66GB
Q3_K
0.88GB file 2.91GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q3_M
0.88GB file 2.91GB
Q3_S
0.88GB file 2.91GB
Q3_XS
0.88GB file 2.91GB
Q3_XXS
0.88GB file 2.91GB
Q4_K
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q4_NL
1.13GB file 3.16GB
Q4_XS
1.13GB file 3.16GB
Q5_K
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q1_M
2GB file 4.03GB
Q1_S
2GB file 4.03GB
Q8_0
2.13GB file 4.16GB

DeepSeek-Coder-V2-Lite-Base-IMat-GGUF

Unknown 8K ctx
BF16
0GB file 2GB
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

DeepSeek-Coder-V2-Lite-Instruct-IMat-GGUF

Unknown 8K ctx
BF16
0GB file 2GB
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

DeepSeek-V2-Lite-Chat-IMat-GGUF

Unknown 8K ctx
BF16
0GB file 2GB
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

DeepSeek-V2-Lite-IMat-GGUF

Unknown 8K ctx
BF16
0GB file 2GB
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

DeepSeek-V2.5-IMat-GGUF

Unknown 8K ctx
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

dolphin-2.9.1-llama-3-70b-IMat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K
30.63GB file 33.78GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB
FP16
140GB file 143.15GB

dolphin-2.9.1-llama-3-8b-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

dolphin-2.9.1-mixtral-1x22b-IMat-GGUF

22B 8K ctx
Q2_K
6.88GB file 9.24GB
Q2_K_S
6.88GB file 9.24GB
Q2_M
6.88GB file 9.24GB
Q2_S
6.88GB file 9.24GB
Q2_XS
6.88GB file 9.24GB
Q2_XXS
6.88GB file 9.24GB
Q3_K
9.63GB file 11.99GB
Q3_K_L
9.63GB file 11.99GB
Q3_K_S
9.63GB file 11.99GB
Q3_M
9.63GB file 11.99GB
Q3_S
9.63GB file 11.99GB
Q3_XS
9.63GB file 11.99GB
Q3_XXS
9.63GB file 11.99GB
Q4_K
12.38GB file 14.74GB
Q4_K_S
12.38GB file 14.74GB
Q4_NL
12.38GB file 14.74GB
Q4_XS
12.38GB file 14.74GB
Q5_K
15.13GB file 17.49GB
Q5_K_S
15.13GB file 17.49GB
Q6_K
17.88GB file 20.24GB
Q1_M
22GB file 24.36GB
Q1_S
22GB file 24.36GB
Q8_0
23.38GB file 25.74GB
BF16
44GB file 46.36GB
FP16
44GB file 46.36GB

dolphin-2.9.2-Phi-3-Medium-abliterated-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

dolphin-2.9.2-Phi-3-Medium-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

gemma-2-27b-it-IMat-GGUF

27B 8K ctx
Q2_K
8.44GB file 10.88GB
Q2_K_S
8.44GB file 10.88GB
Q2_M
8.44GB file 10.88GB
Q2_S
8.44GB file 10.88GB
Q2_XS
8.44GB file 10.88GB
Q2_XXS
8.44GB file 10.88GB
Q3_K
11.81GB file 14.25GB
Q3_K_L
11.81GB file 14.25GB
Q3_K_S
11.81GB file 14.25GB
Q3_M
11.81GB file 14.25GB
Q3_S
11.81GB file 14.25GB
Q3_XS
11.81GB file 14.25GB
Q3_XXS
11.81GB file 14.25GB
Q4_K
15.19GB file 17.63GB
Q4_K_S
15.19GB file 17.63GB
Q4_NL
15.19GB file 17.63GB
Q4_XS
15.19GB file 17.63GB
Q5_K
18.56GB file 21GB
Q5_K_S
18.56GB file 21GB
Q6_K
21.94GB file 24.38GB
Q1_M
27GB file 29.44GB
Q1_S
27GB file 29.44GB
Q8_0
28.69GB file 31.13GB
BF16
54GB file 56.44GB
FP16
54GB file 56.44GB

gemma-2-2b-IMat-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q2_K_S
0.63GB file 2.66GB
Q2_M
0.63GB file 2.66GB
Q2_S
0.63GB file 2.66GB
Q2_XS
0.63GB file 2.66GB
Q2_XXS
0.63GB file 2.66GB
Q3_K
0.88GB file 2.91GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q3_M
0.88GB file 2.91GB
Q3_S
0.88GB file 2.91GB
Q3_XS
0.88GB file 2.91GB
Q3_XXS
0.88GB file 2.91GB
Q4_K
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q4_NL
1.13GB file 3.16GB
Q4_XS
1.13GB file 3.16GB
Q5_K
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q1_M
2GB file 4.03GB
Q1_S
2GB file 4.03GB
Q8_0
2.13GB file 4.16GB
BF16
4GB file 6.03GB
FP16
4GB file 6.03GB

gemma-2-2b-it-IMat-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q2_K_S
0.63GB file 2.66GB
Q2_M
0.63GB file 2.66GB
Q2_S
0.63GB file 2.66GB
Q2_XS
0.63GB file 2.66GB
Q2_XXS
0.63GB file 2.66GB
Q3_K
0.88GB file 2.91GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q3_M
0.88GB file 2.91GB
Q3_S
0.88GB file 2.91GB
Q3_XS
0.88GB file 2.91GB
Q3_XXS
0.88GB file 2.91GB
Q4_K
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q4_NL
1.13GB file 3.16GB
Q4_XS
1.13GB file 3.16GB
Q5_K
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q1_M
2GB file 4.03GB
Q1_S
2GB file 4.03GB
Q8_0
2.13GB file 4.16GB
BF16
4GB file 6.03GB
FP16
4GB file 6.03GB

gemma-2-9b-it-IMat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

Gemma-2-9B-It-SPPO-Iter3-IMat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

glm-4-9b-chat-1m-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

glm-4-9b-chat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

glm-4-9b-chat-IMat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

granite-20b-code-instruct-IMat-GGUF

20B 8K ctx
Q2_K
6.25GB file 8.58GB
Q2_K_S
6.25GB file 8.58GB
Q2_M
6.25GB file 8.58GB
Q2_S
6.25GB file 8.58GB
Q2_XS
6.25GB file 8.58GB
Q2_XXS
6.25GB file 8.58GB
Q3_K
8.75GB file 11.08GB
Q3_K_L
8.75GB file 11.08GB
Q3_K_S
8.75GB file 11.08GB
Q3_M
8.75GB file 11.08GB
Q3_S
8.75GB file 11.08GB
Q3_XS
8.75GB file 11.08GB
Q3_XXS
8.75GB file 11.08GB
Q4_K
11.25GB file 13.58GB
Q4_K_S
11.25GB file 13.58GB
Q4_NL
11.25GB file 13.58GB
Q4_XS
11.25GB file 13.58GB
Q5_K
13.75GB file 16.08GB
Q5_K_S
13.75GB file 16.08GB
Q6_K
16.25GB file 18.58GB
Q1_M
20GB file 22.33GB
Q1_S
20GB file 22.33GB
Q8_0
21.25GB file 23.58GB
BF16
40GB file 42.33GB
FP16
40GB file 42.33GB

Hermes-2-Theta-Llama-3-8B-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Hermes-3-Llama-3.1-70B-IMat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K
30.63GB file 33.78GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB
FP16
140GB file 143.15GB

Hermes-3-Llama-3.1-8B-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Higgs-Llama-3-70B-IMat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K
30.63GB file 33.78GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB
FP16
140GB file 143.15GB

internlm2_5-1_8b-chat-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

internlm2_5-20b-chat-IMat-GGUF

20B 8K ctx
Q2_K
6.25GB file 8.58GB
Q2_K_S
6.25GB file 8.58GB
Q2_M
6.25GB file 8.58GB
Q2_S
6.25GB file 8.58GB
Q2_XS
6.25GB file 8.58GB
Q2_XXS
6.25GB file 8.58GB
Q3_K
8.75GB file 11.08GB
Q3_K_L
8.75GB file 11.08GB
Q3_K_S
8.75GB file 11.08GB
Q3_M
8.75GB file 11.08GB
Q3_S
8.75GB file 11.08GB
Q3_XS
8.75GB file 11.08GB
Q3_XXS
8.75GB file 11.08GB
Q4_K
11.25GB file 13.58GB
Q4_K_S
11.25GB file 13.58GB
Q4_NL
11.25GB file 13.58GB
Q4_XS
11.25GB file 13.58GB
Q5_K
13.75GB file 16.08GB
Q5_K_S
13.75GB file 16.08GB
Q6_K
16.25GB file 18.58GB
Q1_M
20GB file 22.33GB
Q1_S
20GB file 22.33GB
Q8_0
21.25GB file 23.58GB
BF16
40GB file 42.33GB
FP16
40GB file 42.33GB

internlm2_5-7b-chat-1m-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

internlm2_5-7b-chat-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

internlm2-math-plus-1_8b-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

internlm2-math-plus-20b-IMat-GGUF

20B 8K ctx
Q2_K
6.25GB file 8.58GB
Q2_K_S
6.25GB file 8.58GB
Q2_M
6.25GB file 8.58GB
Q2_S
6.25GB file 8.58GB
Q2_XS
6.25GB file 8.58GB
Q2_XXS
6.25GB file 8.58GB
Q3_K
8.75GB file 11.08GB
Q3_K_L
8.75GB file 11.08GB
Q3_K_S
8.75GB file 11.08GB
Q3_M
8.75GB file 11.08GB
Q3_S
8.75GB file 11.08GB
Q3_XS
8.75GB file 11.08GB
Q3_XXS
8.75GB file 11.08GB
Q4_K
11.25GB file 13.58GB
Q4_K_S
11.25GB file 13.58GB
Q4_NL
11.25GB file 13.58GB
Q4_XS
11.25GB file 13.58GB
Q5_K
13.75GB file 16.08GB
Q5_K_S
13.75GB file 16.08GB
Q6_K
16.25GB file 18.58GB
Q1_M
20GB file 22.33GB
Q1_S
20GB file 22.33GB
Q8_0
21.25GB file 23.58GB
BF16
40GB file 42.33GB
FP16
40GB file 42.33GB

internlm2-math-plus-7b-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

internlm2-math-plus-mixtral8x22b-IMat-GGUF

22B 8K ctx
Q2_K
6.88GB file 9.24GB
Q2_K_S
6.88GB file 9.24GB
Q2_M
6.88GB file 9.24GB
Q2_S
6.88GB file 9.24GB
Q2_XS
6.88GB file 9.24GB
Q2_XXS
6.88GB file 9.24GB
Q3_K
9.63GB file 11.99GB
Q3_K_L
9.63GB file 11.99GB
Q3_K_S
9.63GB file 11.99GB
Q3_M
9.63GB file 11.99GB
Q3_S
9.63GB file 11.99GB
Q3_XS
9.63GB file 11.99GB
Q3_XXS
9.63GB file 11.99GB
Q4_K
12.38GB file 14.74GB
Q4_K_S
12.38GB file 14.74GB
Q4_NL
12.38GB file 14.74GB
Q4_XS
12.38GB file 14.74GB
Q5_K
15.13GB file 17.49GB
Q5_K_S
15.13GB file 17.49GB
Q6_K
17.88GB file 20.24GB
Q1_M
22GB file 24.36GB
Q1_S
22GB file 24.36GB
Q8_0
23.38GB file 25.74GB
BF16
44GB file 46.36GB
FP16
44GB file 46.36GB

K2-Chat-IMat-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

K2-ckpt_360-IMat-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

K2-IMat-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Llama-3-8B-Instruct-MopeyMule-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Llama-3-Instruct-8B-SimPO-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Llama-3-Instruct-8B-SPPO-Iter3-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Llama-3.1-Minitron-4B-Width-Base-GGUF

4B 8K ctx
Q2_K
1.25GB file 3.32GB
Q3_K
1.75GB file 3.82GB
Q3_K_L
1.75GB file 3.82GB
Q3_K_S
1.75GB file 3.82GB
Q3_M
1.75GB file 3.82GB
Q3_S
1.75GB file 3.82GB
Q3_XS
1.75GB file 3.82GB
Q4_K
2.25GB file 4.32GB
Q4_K_S
2.25GB file 4.32GB
Q4_NL
2.25GB file 4.32GB
Q4_XS
2.25GB file 4.32GB
Q5_K
2.75GB file 4.82GB
Q5_K_S
2.75GB file 4.82GB
Q6_K
3.25GB file 5.32GB
Q8_0
4.25GB file 6.32GB
BF16
8GB file 10.07GB
FP16
8GB file 10.07GB

Llama-3.1-Storm-8B-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Llama-3.1-SuperNova-Lite-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Llama-3.2-1B-Instruct-IMat-GGUF

1B 8K ctx
Q2_K
0.31GB file 2.33GB
Q2_K_S
0.31GB file 2.33GB
Q2_M
0.31GB file 2.33GB
Q2_S
0.31GB file 2.33GB
Q2_XS
0.31GB file 2.33GB
Q2_XXS
0.31GB file 2.33GB
Q3_K
0.44GB file 2.46GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q3_M
0.44GB file 2.46GB
Q3_S
0.44GB file 2.46GB
Q3_XS
0.44GB file 2.46GB
Q3_XXS
0.44GB file 2.46GB
Q4_K
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q4_NL
0.56GB file 2.58GB
Q4_XS
0.56GB file 2.58GB
Q5_K
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q1_M
1GB file 3.02GB
Q1_S
1GB file 3.02GB
Q8_0
1.06GB file 3.08GB
BF16
2GB file 4.02GB
FP16
2GB file 4.02GB

Llama-3.2-3B-Instruct-IMat-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q2_K_S
0.94GB file 2.99GB
Q2_M
0.94GB file 2.99GB
Q2_S
0.94GB file 2.99GB
Q2_XS
0.94GB file 2.99GB
Q2_XXS
0.94GB file 2.99GB
Q3_K
1.31GB file 3.36GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q3_M
1.31GB file 3.36GB
Q3_S
1.31GB file 3.36GB
Q3_XS
1.31GB file 3.36GB
Q3_XXS
1.31GB file 3.36GB
Q4_K
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q4_NL
1.69GB file 3.74GB
Q4_XS
1.69GB file 3.74GB
Q5_K
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q1_M
3GB file 5.05GB
Q1_S
3GB file 5.05GB
Q8_0
3.19GB file 5.24GB
BF16
6GB file 8.05GB
FP16
6GB file 8.05GB

Llama-Guard-3-8B-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Llama3-ChatQA-1.5-8B-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
FP16
16GB file 18.13GB

llm-compiler-13b-ftd-IMat-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q2_K_S
4.06GB file 6.27GB
Q2_M
4.06GB file 6.27GB
Q2_S
4.06GB file 6.27GB
Q2_XS
4.06GB file 6.27GB
Q2_XXS
4.06GB file 6.27GB
Q3_K
5.69GB file 7.9GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q3_M
5.69GB file 7.9GB
Q3_S
5.69GB file 7.9GB
Q3_XS
5.69GB file 7.9GB
Q3_XXS
5.69GB file 7.9GB
Q4_K
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q4_NL
7.31GB file 9.52GB
Q4_XS
7.31GB file 9.52GB
Q5_K
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q1_M
13GB file 15.21GB
Q1_S
13GB file 15.21GB
Q8_0
13.81GB file 16.02GB
BF16
26GB file 28.21GB
FP16
26GB file 28.21GB

llm-compiler-13b-IMat-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q2_K_S
4.06GB file 6.27GB
Q2_M
4.06GB file 6.27GB
Q2_S
4.06GB file 6.27GB
Q2_XS
4.06GB file 6.27GB
Q2_XXS
4.06GB file 6.27GB
Q3_K
5.69GB file 7.9GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q3_M
5.69GB file 7.9GB
Q3_S
5.69GB file 7.9GB
Q3_XS
5.69GB file 7.9GB
Q3_XXS
5.69GB file 7.9GB
Q4_K
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q4_NL
7.31GB file 9.52GB
Q4_XS
7.31GB file 9.52GB
Q5_K
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q1_M
13GB file 15.21GB
Q1_S
13GB file 15.21GB
Q8_0
13.81GB file 16.02GB
BF16
26GB file 28.21GB
FP16
26GB file 28.21GB

llm-compiler-7b-ftd-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

llm-compiler-7b-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

mathstral-7B-v0.1-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Meta-Llama-3-70B-Instruct-abliterated-v3.5-IMat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K
30.63GB file 33.78GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB
FP16
140GB file 143.15GB

Meta-Llama-3-8B-Instruct-abliterated-v3-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Meta-Llama-3.1-70B-Instruct-IMat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K
30.63GB file 33.78GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB
FP16
140GB file 143.15GB

Meta-Llama-3.1-8B-Instruct-abliterated-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Meta-Llama-3.1-8B-Instruct-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Minitron-4B-Base-GGUF

4B 8K ctx
Q2_K
1.25GB file 3.32GB
Q3_K
1.75GB file 3.82GB
Q3_K_L
1.75GB file 3.82GB
Q3_K_S
1.75GB file 3.82GB
Q3_M
1.75GB file 3.82GB
Q3_S
1.75GB file 3.82GB
Q3_XS
1.75GB file 3.82GB
Q4_K
2.25GB file 4.32GB
Q4_K_S
2.25GB file 4.32GB
Q4_NL
2.25GB file 4.32GB
Q4_XS
2.25GB file 4.32GB
Q5_K
2.75GB file 4.82GB
Q5_K_S
2.75GB file 4.82GB
Q6_K
3.25GB file 5.32GB
Q8_0
4.25GB file 6.32GB
BF16
8GB file 10.07GB
FP16
8GB file 10.07GB

Minitron-8B-Base-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Mistral-7B-Instruct-v0.3-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Mistral-7B-v0.3-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Mistral-Large-Instruct-2407-IMat-GGUF

Unknown 8K ctx
BF16
0GB file 2GB
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Mistral-Nemo-Instruct-2407-IMat-GGUF

12B 8K ctx
Q2_K
3.75GB file 5.95GB
Q2_K_S
3.75GB file 5.95GB
Q2_M
3.75GB file 5.95GB
Q2_S
3.75GB file 5.95GB
Q2_XS
3.75GB file 5.95GB
Q2_XXS
3.75GB file 5.95GB
Q3_K
5.25GB file 7.45GB
Q3_K_L
5.25GB file 7.45GB
Q3_K_S
5.25GB file 7.45GB
Q3_M
5.25GB file 7.45GB
Q3_S
5.25GB file 7.45GB
Q3_XS
5.25GB file 7.45GB
Q3_XXS
5.25GB file 7.45GB
Q4_K
6.75GB file 8.95GB
Q4_K_S
6.75GB file 8.95GB
Q4_NL
6.75GB file 8.95GB
Q4_XS
6.75GB file 8.95GB
Q5_K
8.25GB file 10.45GB
Q5_K_S
8.25GB file 10.45GB
Q6_K
9.75GB file 11.95GB
Q1_M
12GB file 14.2GB
Q1_S
12GB file 14.2GB
Q8_0
12.75GB file 14.95GB
BF16
24GB file 26.2GB
FP16
24GB file 26.2GB

Mistral-NeMo-Minitron-8B-Base-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Mistral-Small-Instruct-2409-IMat-GGUF

Unknown 8K ctx
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q4_XS
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB
BF16
0GB file 2GB
FP16
0GB file 2GB
Q1_M
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB

neo_7b_instruct_v0.1-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

neo_7b_sft_v0.1-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

neo_7b-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

NuminaMath-7B-TIR-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Nxcode-CQ-7B-orpo-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

openchat-3.6-8b-20240522-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

Palmyra-Fin-70B-32K-IMat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K
30.63GB file 33.78GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB
FP16
140GB file 143.15GB

Phi-3-medium-128k-instruct-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

Phi-3-medium-4k-instruct-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

Phi-3-mini-128k-instruct-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

Phi-3-mini-4k-instruct-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

Phi-3-mini-4k-instruct-update2024_07_03-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

Phi-3.5-mini-instruct-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

Qwen2-0.5B-IMat-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q2_K_S
0.16GB file 2.17GB
Q2_M
0.16GB file 2.17GB
Q2_S
0.16GB file 2.17GB
Q2_XS
0.16GB file 2.17GB
Q2_XXS
0.16GB file 2.17GB
Q3_K
0.22GB file 2.23GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q3_M
0.22GB file 2.23GB
Q3_S
0.22GB file 2.23GB
Q3_XS
0.22GB file 2.23GB
Q3_XXS
0.22GB file 2.23GB
Q4_K
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q4_NL
0.28GB file 2.29GB
Q4_XS
0.28GB file 2.29GB
Q5_K
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q1_M
0.5GB file 2.51GB
Q1_S
0.5GB file 2.51GB
Q8_0
0.53GB file 2.54GB
BF16
1GB file 3.01GB
FP16
1GB file 3.01GB

Qwen2-0.5B-Instruct-IMat-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q2_K_S
0.16GB file 2.17GB
Q2_M
0.16GB file 2.17GB
Q2_S
0.16GB file 2.17GB
Q2_XS
0.16GB file 2.17GB
Q2_XXS
0.16GB file 2.17GB
Q3_K
0.22GB file 2.23GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q3_M
0.22GB file 2.23GB
Q3_S
0.22GB file 2.23GB
Q3_XS
0.22GB file 2.23GB
Q3_XXS
0.22GB file 2.23GB
Q4_K
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q4_NL
0.28GB file 2.29GB
Q4_XS
0.28GB file 2.29GB
Q5_K
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q1_M
0.5GB file 2.51GB
Q1_S
0.5GB file 2.51GB
Q8_0
0.53GB file 2.54GB
BF16
1GB file 3.01GB
FP16
1GB file 3.01GB

Qwen2-1.5B-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB
FP16
3GB file 5.02GB

Qwen2-1.5B-Instruct-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB
FP16
3GB file 5.02GB

Qwen2-57B-A14B-Instruct-GGUF

57B 8K ctx
Q2_K
17.81GB file 20.74GB
Q3_K
24.94GB file 27.87GB
Q3_K_L
24.94GB file 27.87GB
Q3_K_S
24.94GB file 27.87GB
Q3_M
24.94GB file 27.87GB
Q3_S
24.94GB file 27.87GB
Q3_XS
24.94GB file 27.87GB
Q4_K
32.06GB file 34.99GB
Q4_K_S
32.06GB file 34.99GB
Q4_NL
32.06GB file 34.99GB
Q4_XS
32.06GB file 34.99GB
Q5_K
39.19GB file 42.12GB
Q5_K_S
39.19GB file 42.12GB
Q6_K
46.31GB file 49.24GB
Q8_0
60.56GB file 63.49GB
BF16
114GB file 116.93GB
FP16
114GB file 116.93GB

Qwen2-57B-A14B-Instruct-IMat-GGUF

57B 8K ctx
Q5_K
39.19GB file 42.12GB
Q5_K_S
39.19GB file 42.12GB
Q6_K
46.31GB file 49.24GB
Q8_0
60.56GB file 63.49GB
BF16
114GB file 116.93GB
FP16
114GB file 116.93GB

Qwen2-72B-Instruct-IMat-GGUF

72B 8K ctx
Q2_K
22.5GB file 25.68GB
Q2_K_S
22.5GB file 25.68GB
Q2_M
22.5GB file 25.68GB
Q2_S
22.5GB file 25.68GB
Q2_XS
22.5GB file 25.68GB
Q2_XXS
22.5GB file 25.68GB
Q3_K
31.5GB file 34.68GB
Q3_K_L
31.5GB file 34.68GB
Q3_K_S
31.5GB file 34.68GB
Q3_M
31.5GB file 34.68GB
Q3_S
31.5GB file 34.68GB
Q3_XS
31.5GB file 34.68GB
Q3_XXS
31.5GB file 34.68GB
Q4_K
40.5GB file 43.68GB
Q4_K_S
40.5GB file 43.68GB
Q4_NL
40.5GB file 43.68GB
Q4_XS
40.5GB file 43.68GB
Q5_K
49.5GB file 52.68GB
Q5_K_S
49.5GB file 52.68GB
Q6_K
58.5GB file 61.68GB
Q1_M
72GB file 75.18GB
Q1_S
72GB file 75.18GB
Q8_0
76.5GB file 79.68GB
BF16
144GB file 147.18GB
FP16
144GB file 147.18GB

Qwen2-7B-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Qwen2-Math-1.5B-Instruct-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB
FP16
3GB file 5.02GB

Qwen2-Math-72B-Instruct-IMat-GGUF

72B 8K ctx
Q2_K
22.5GB file 25.68GB
Q2_K_S
22.5GB file 25.68GB
Q2_M
22.5GB file 25.68GB
Q2_S
22.5GB file 25.68GB
Q2_XS
22.5GB file 25.68GB
Q2_XXS
22.5GB file 25.68GB
Q3_K
31.5GB file 34.68GB
Q3_K_L
31.5GB file 34.68GB
Q3_K_S
31.5GB file 34.68GB
Q3_M
31.5GB file 34.68GB
Q3_S
31.5GB file 34.68GB
Q3_XS
31.5GB file 34.68GB
Q3_XXS
31.5GB file 34.68GB
Q4_K
40.5GB file 43.68GB
Q4_K_S
40.5GB file 43.68GB
Q4_NL
40.5GB file 43.68GB
Q4_XS
40.5GB file 43.68GB
Q5_K
49.5GB file 52.68GB
Q5_K_S
49.5GB file 52.68GB
Q6_K
58.5GB file 61.68GB
Q1_M
72GB file 75.18GB
Q8_0
76.5GB file 79.68GB
BF16
144GB file 147.18GB
FP16
144GB file 147.18GB

Qwen2-Math-7B-Instruct-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Qwen2.5-0.5B-Instruct-IMat-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q2_K_S
0.16GB file 2.17GB
Q2_M
0.16GB file 2.17GB
Q2_S
0.16GB file 2.17GB
Q2_XS
0.16GB file 2.17GB
Q2_XXS
0.16GB file 2.17GB
Q3_K
0.22GB file 2.23GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q3_M
0.22GB file 2.23GB
Q3_S
0.22GB file 2.23GB
Q3_XS
0.22GB file 2.23GB
Q3_XXS
0.22GB file 2.23GB
Q4_K
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q4_NL
0.28GB file 2.29GB
Q4_XS
0.28GB file 2.29GB
Q5_K
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q1_M
0.5GB file 2.51GB
Q1_S
0.5GB file 2.51GB
Q8_0
0.53GB file 2.54GB
BF16
1GB file 3.01GB
FP16
1GB file 3.01GB

Qwen2.5-1.5B-Instruct-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB
FP16
3GB file 5.02GB

Qwen2.5-14B-Instruct-IMat-GGUF

14B 8K ctx
Q2_K
4.38GB file 6.61GB
Q2_K_S
4.38GB file 6.61GB
Q2_M
4.38GB file 6.61GB
Q2_S
4.38GB file 6.61GB
Q2_XS
4.38GB file 6.61GB
Q2_XXS
4.38GB file 6.61GB
Q3_K
6.13GB file 8.36GB
Q3_K_L
6.13GB file 8.36GB
Q3_K_S
6.13GB file 8.36GB
Q3_M
6.13GB file 8.36GB
Q3_S
6.13GB file 8.36GB
Q3_XS
6.13GB file 8.36GB
Q3_XXS
6.13GB file 8.36GB
Q4_K
7.88GB file 10.11GB
Q4_K_S
7.88GB file 10.11GB
Q4_NL
7.88GB file 10.11GB
Q4_XS
7.88GB file 10.11GB
Q5_K
9.63GB file 11.86GB
Q5_K_S
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q1_M
14GB file 16.23GB
Q1_S
14GB file 16.23GB
Q8_0
14.88GB file 17.11GB
BF16
28GB file 30.23GB
FP16
28GB file 30.23GB

Qwen2.5-32B-Instruct-IMat-GGUF

32B 8K ctx
Q2_K
10GB file 12.52GB
Q2_K_S
10GB file 12.52GB
Q2_M
10GB file 12.52GB
Q2_S
10GB file 12.52GB
Q2_XS
10GB file 12.52GB
Q2_XXS
10GB file 12.52GB
Q3_K
14GB file 16.52GB
Q3_K_L
14GB file 16.52GB
Q3_K_S
14GB file 16.52GB
Q3_M
14GB file 16.52GB
Q3_S
14GB file 16.52GB
Q3_XS
14GB file 16.52GB
Q3_XXS
14GB file 16.52GB
Q4_K
18GB file 20.52GB
Q4_K_S
18GB file 20.52GB
Q4_NL
18GB file 20.52GB
Q4_XS
18GB file 20.52GB
Q5_K
22GB file 24.52GB
Q5_K_S
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q1_M
32GB file 34.52GB
Q1_S
32GB file 34.52GB
Q8_0
34GB file 36.52GB
BF16
64GB file 66.52GB
FP16
64GB file 66.52GB

Qwen2.5-3B-Instruct-IMat-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q2_K_S
0.94GB file 2.99GB
Q2_M
0.94GB file 2.99GB
Q2_S
0.94GB file 2.99GB
Q2_XS
0.94GB file 2.99GB
Q2_XXS
0.94GB file 2.99GB
Q3_K
1.31GB file 3.36GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q3_M
1.31GB file 3.36GB
Q3_S
1.31GB file 3.36GB
Q3_XS
1.31GB file 3.36GB
Q3_XXS
1.31GB file 3.36GB
Q4_K
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q4_NL
1.69GB file 3.74GB
Q4_XS
1.69GB file 3.74GB
Q5_K
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q1_M
3GB file 5.05GB
Q1_S
3GB file 5.05GB
Q8_0
3.19GB file 5.24GB
BF16
6GB file 8.05GB
FP16
6GB file 8.05GB

Qwen2.5-72B-Instruct-IMat-GGUF

72B 8K ctx
Q2_K
22.5GB file 25.68GB
Q2_K_S
22.5GB file 25.68GB
Q2_M
22.5GB file 25.68GB
Q2_S
22.5GB file 25.68GB
Q2_XS
22.5GB file 25.68GB
Q2_XXS
22.5GB file 25.68GB
Q3_K
31.5GB file 34.68GB
Q3_K_L
31.5GB file 34.68GB
Q3_K_S
31.5GB file 34.68GB
Q3_M
31.5GB file 34.68GB
Q3_S
31.5GB file 34.68GB
Q3_XS
31.5GB file 34.68GB
Q3_XXS
31.5GB file 34.68GB
Q4_K
40.5GB file 43.68GB
Q4_K_S
40.5GB file 43.68GB
Q4_NL
40.5GB file 43.68GB
Q4_XS
40.5GB file 43.68GB
Q5_K
49.5GB file 52.68GB
Q5_K_S
49.5GB file 52.68GB
Q6_K
58.5GB file 61.68GB
Q1_M
72GB file 75.18GB
Q8_0
76.5GB file 79.68GB
BF16
144GB file 147.18GB
FP16
144GB file 147.18GB

Qwen2.5-7B-Instruct-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Qwen2.5-Coder-1.5B-Instruct-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB
FP16
3GB file 5.02GB

Qwen2.5-Coder-7B-Instruct-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Qwen2.5-Math-1.5B-Instruct-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB
FP16
3GB file 5.02GB

Qwen2.5-Math-7B-Instruct-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

reader-lm-0.5b-IMat-GGUF

0.5B 8K ctx
Q2_K
0.16GB file 2.17GB
Q2_K_S
0.16GB file 2.17GB
Q2_M
0.16GB file 2.17GB
Q2_S
0.16GB file 2.17GB
Q2_XS
0.16GB file 2.17GB
Q2_XXS
0.16GB file 2.17GB
Q3_K
0.22GB file 2.23GB
Q3_K_L
0.22GB file 2.23GB
Q3_K_S
0.22GB file 2.23GB
Q3_M
0.22GB file 2.23GB
Q3_S
0.22GB file 2.23GB
Q3_XS
0.22GB file 2.23GB
Q3_XXS
0.22GB file 2.23GB
Q4_K
0.28GB file 2.29GB
Q4_K_S
0.28GB file 2.29GB
Q4_NL
0.28GB file 2.29GB
Q4_XS
0.28GB file 2.29GB
Q5_K
0.34GB file 2.35GB
Q5_K_S
0.34GB file 2.35GB
Q6_K
0.41GB file 2.42GB
Q1_M
0.5GB file 2.51GB
Q1_S
0.5GB file 2.51GB
Q8_0
0.53GB file 2.54GB

reader-lm-1.5b-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB

Reflection-Llama-3.1-70B-IMat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q2_K_S
21.88GB file 25.03GB
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_K
30.63GB file 33.78GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_M
70GB file 73.15GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB
FP16
140GB file 143.15GB

RoGemma-7b-Instruct-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

RoLlama2-7b-Base-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB

RoLlama2-7b-Base-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

RoLlama2-7b-Chat-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB

RoLlama2-7b-Chat-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

RoLlama2-7b-Instruct-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB

RoLlama2-7b-Instruct-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

RoLlama3-8b-Instruct-IMat-GGUF

8B 8K ctx
Q2_K
2.5GB file 4.63GB
Q2_K_S
2.5GB file 4.63GB
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_K
3.5GB file 5.63GB
Q3_K_L
3.5GB file 5.63GB
Q3_K_S
3.5GB file 5.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_K
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB
FP16
16GB file 18.13GB

RoMistral-7b-Instruct-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB

RoMistral-7b-Instruct-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

shieldgemma-27b-IMat-GGUF

27B 8K ctx
Q2_K
8.44GB file 10.88GB
Q2_K_S
8.44GB file 10.88GB
Q2_M
8.44GB file 10.88GB
Q2_S
8.44GB file 10.88GB
Q2_XS
8.44GB file 10.88GB
Q2_XXS
8.44GB file 10.88GB
Q3_K
11.81GB file 14.25GB
Q3_K_L
11.81GB file 14.25GB
Q3_K_S
11.81GB file 14.25GB
Q3_M
11.81GB file 14.25GB
Q3_S
11.81GB file 14.25GB
Q3_XS
11.81GB file 14.25GB
Q3_XXS
11.81GB file 14.25GB
Q4_K
15.19GB file 17.63GB
Q4_K_S
15.19GB file 17.63GB
Q4_NL
15.19GB file 17.63GB
Q4_XS
15.19GB file 17.63GB
Q5_K
18.56GB file 21GB
Q5_K_S
18.56GB file 21GB
Q6_K
21.94GB file 24.38GB
Q1_M
27GB file 29.44GB
Q1_S
27GB file 29.44GB
Q8_0
28.69GB file 31.13GB
BF16
54GB file 56.44GB
FP16
54GB file 56.44GB

shieldgemma-2b-IMat-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q2_K_S
0.63GB file 2.66GB
Q2_M
0.63GB file 2.66GB
Q2_S
0.63GB file 2.66GB
Q2_XS
0.63GB file 2.66GB
Q2_XXS
0.63GB file 2.66GB
Q3_K
0.88GB file 2.91GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q3_M
0.88GB file 2.91GB
Q3_S
0.88GB file 2.91GB
Q3_XS
0.88GB file 2.91GB
Q3_XXS
0.88GB file 2.91GB
Q4_K
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q4_NL
1.13GB file 3.16GB
Q4_XS
1.13GB file 3.16GB
Q5_K
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q1_M
2GB file 4.03GB
Q1_S
2GB file 4.03GB
Q8_0
2.13GB file 4.16GB
BF16
4GB file 6.03GB
FP16
4GB file 6.03GB

shieldgemma-9b-IMat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

unsloth-Phi-3.5-mini-instruct-IMat-GGUF

3.8B 8K ctx
Q2_K
1.19GB file 3.25GB
Q2_K_S
1.19GB file 3.25GB
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_K
1.66GB file 3.72GB
Q3_K_L
1.66GB file 3.72GB
Q3_K_S
1.66GB file 3.72GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_K
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_M
3.8GB file 5.86GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB
FP16
7.6GB file 9.66GB

xLAM-7b-r-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

xLAM-8x22b-r-IMat-GGUF

22B 8K ctx
Q2_K
6.88GB file 9.24GB
Q2_K_S
6.88GB file 9.24GB
Q2_M
6.88GB file 9.24GB
Q2_S
6.88GB file 9.24GB
Q2_XS
6.88GB file 9.24GB
Q2_XXS
6.88GB file 9.24GB
Q3_K
9.63GB file 11.99GB
Q3_K_L
9.63GB file 11.99GB
Q3_K_S
9.63GB file 11.99GB
Q3_M
9.63GB file 11.99GB
Q3_S
9.63GB file 11.99GB
Q3_XS
9.63GB file 11.99GB
Q3_XXS
9.63GB file 11.99GB
Q4_K
12.38GB file 14.74GB
Q4_K_S
12.38GB file 14.74GB
Q4_NL
12.38GB file 14.74GB
Q4_XS
12.38GB file 14.74GB
Q5_K
15.13GB file 17.49GB
Q5_K_S
15.13GB file 17.49GB
Q6_K
17.88GB file 20.24GB
Q1_M
22GB file 24.36GB
Q1_S
22GB file 24.36GB
Q8_0
23.38GB file 25.74GB
BF16
44GB file 46.36GB
FP16
44GB file 46.36GB

xLAM-8x7b-r-IMat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q2_K_S
2.19GB file 4.3GB
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_K
3.06GB file 5.17GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_K
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB
FP16
14GB file 16.11GB

Yi-9B-Coder-IMat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

Yi-Coder-1.5B-Chat-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB
FP16
3GB file 5.02GB

Yi-Coder-1.5B-IMat-GGUF

1.5B 8K ctx
Q2_K
0.47GB file 2.49GB
Q2_K_S
0.47GB file 2.49GB
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_K
0.66GB file 2.68GB
Q3_K_L
0.66GB file 2.68GB
Q3_K_S
0.66GB file 2.68GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_K
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_M
1.5GB file 3.52GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB
FP16
3GB file 5.02GB

Yi-Coder-9B-Chat-IMat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

Yi-Coder-9B-IMat-GGUF

9B 8K ctx
Q2_K
2.81GB file 4.96GB
Q2_K_S
2.81GB file 4.96GB
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_K
3.94GB file 6.09GB
Q3_K_L
3.94GB file 6.09GB
Q3_K_S
3.94GB file 6.09GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_K
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB
FP16
18GB file 20.15GB

Yuan2-M32-hf-IMat-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q1_S
0GB file 2GB
Q2_K
0GB file 2GB
Q2_K_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_S
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_K
0GB file 2GB
Q4_K_S
0GB file 2GB
Q4_NL
0GB file 2GB
Q5_K
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB
L

Lehungquangminh

1 models

gemma-3-1b-it-Q4_K_M-GGUF

1B 8K ctx
Q4_0
0.56GB file 2.58GB
FP16
2GB file 4.02GB
L

Lekt8

1 models

tdp

4B 8K ctx
BF16
8GB file 10.07GB
L

Leliuga

3 models

gemma-2b-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

gemma-2b-it-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

mamba-2.8b-hf-GGUF

2.8B 8K ctx
Q2_K
0.88GB file 2.93GB
Q3_K_L
1.22GB file 3.27GB
Q3_K_M
1.22GB file 3.27GB
Q3_K_S
1.22GB file 3.27GB
Q4_0
1.58GB file 3.63GB
Q4_K_M
1.58GB file 3.63GB
Q4_K_S
1.58GB file 3.63GB
Q5_0
1.92GB file 3.97GB
Q5_K_M
1.92GB file 3.97GB
Q5_K_S
1.92GB file 3.97GB
Q6_K
2.28GB file 4.33GB
Q8_0
2.97GB file 5.02GB
L

Lelloman

1 models

smollm-torrent-metadata

Unknown 8K ctx
FP16
0GB file 2GB
Q4_0
0GB file 2GB
L

Lemongooooo

1 models

model

Unknown 8K ctx
Q4_K_M
0GB file 2GB
L

Lemonilia

3 models

AshhLimaRP-Mistral-7B

7B 8K ctx
Q4_K_M
3.94GB file 6.05GB
Q6_K
5.69GB file 7.8GB

Limamono-Mistral-7B-v0.50

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB

ShoriRP-v0.75d

7B 8K ctx
Q6_0
5.69GB file 7.8GB
FP16
14GB file 16.11GB
L

Lemonyins

1 models

Qwen3.6-27B-uncensored-abliterated-MTP-i1-IQ4_XS-GGUF-Smaller

35B 8K ctx
Q4_XS
19.69GB file 22.26GB