All Supported Models

29681 AI models · 198875 quantization formats tracked

Q

Quixote12

1 models

Qwen3-Coder-30B-A3B-Instruct-GGUF

30B 8K ctx
Q2_K
9.38GB file 11.87GB
Q2_K_L
9.38GB file 11.87GB
Q2_K_XL
9.38GB file 11.87GB
Q2_M
9.38GB file 11.87GB
Q2_XXS
9.38GB file 11.87GB
Q3_K_M
13.13GB file 15.62GB
Q3_K_S
13.13GB file 15.62GB
Q3_K_XL
13.13GB file 15.62GB
Q3_XXS
13.13GB file 15.62GB
Q4_0
16.88GB file 19.37GB
Q4_1
16.88GB file 19.37GB
Q4_K_M
16.88GB file 19.37GB
Q4_K_S
16.88GB file 19.37GB
Q4_K_XL
16.88GB file 19.37GB
Q4_NL
16.88GB file 19.37GB
Q4_XS
16.88GB file 19.37GB
Q5_K_M
20.63GB file 23.12GB
Q5_K_S
20.63GB file 23.12GB
Q5_K_XL
20.63GB file 23.12GB
Q6_K
24.38GB file 26.87GB
Q6_K_XL
24.38GB file 26.87GB
Q1_0
30GB file 32.49GB
Q1_M
30GB file 32.49GB
Q1_S
30GB file 32.49GB
Q8_0
31.88GB file 34.37GB
Q8_K_XL
31.88GB file 34.37GB
BF16
60GB file 62.49GB
Q

Quixotedav

1 models

gemma-2-9b-it-Q4_K_M-GGUF

9B 8K ctx
Q4_0
5.06GB file 7.21GB
FP16
18GB file 20.15GB
Q

Qurk41

1 models

ERNIE-4.5-21B-A3B-PT-Q4_K_M-GGUF

21B 8K ctx
Q4_0
11.81GB file 14.15GB
FP16
42GB file 44.34GB
Q

Qurtana

1 models

Qwen-2.5-0.5B-MathInstruct.rev.2

0.5B 8K ctx
Q4_K_M
0.28GB file 2.29GB
Q5_K_M
0.34GB file 2.35GB
Q8_0
0.53GB file 2.54GB
Q

Qvac

4 models

fabric-llm-finetune

Unknown 8K ctx
FP16
0GB file 2GB

fabric-llm-finetune-bitnet

Unknown 8K ctx
Q1_0
0GB file 2GB
Q2_0
0GB file 2GB

MedPsy-1.7B-GGUF

1.7B 8K ctx
Q3_0
0.74GB file 2.77GB
Q4_0
0.96GB file 2.99GB
Q5_0
1.17GB file 3.2GB
Q8_0
1.81GB file 3.84GB
BF16
3.4GB file 5.43GB

MedPsy-4B-GGUF

4B 8K ctx
Q3_0
1.75GB file 3.82GB
Q4_0
2.25GB file 4.32GB
Q5_0
2.75GB file 4.82GB
Q8_0
4.25GB file 6.32GB
BF16
8GB file 10.07GB
Q

Qwe1325

1 models

gemma2-2b-it-tw-Q4_K_M-GGUF

2B 8K ctx
Q4_0
1.13GB file 3.16GB
FP16
4GB file 6.03GB
Q

Qweaksdbkajsbdk

1 models

Qwen2.5-Coder-14B-n8n-Workflow-Generator

14B 8K ctx
Q4_0
7.88GB file 10.11GB
FP16
28GB file 30.23GB
Q

Qwen

39 models

CodeQwen1.5-7B-Chat-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
Q3_0
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Qwen1.5-0.5B-Chat-GGUF

0.5B 8K ctx
Q2_0
0.16GB file 2.17GB
Q3_0
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q6_0
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB

Qwen1.5-1.8B-Chat-GGUF

1.8B 8K ctx
Q2_0
0.56GB file 2.59GB
Q3_0
0.79GB file 2.82GB
Q4_0
1.01GB file 3.04GB
Q5_0
1.24GB file 3.27GB
Q6_0
1.46GB file 3.49GB
Q8_0
1.91GB file 3.94GB

Qwen1.5-110B-Chat-GGUF

110B 8K ctx
Q2_0
34.38GB file 38.18GB
FP16
220GB file 223.8GB

Qwen1.5-14B-Chat-GGUF

14B 8K ctx
Q2_0
4.38GB file 6.61GB
Q3_0
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q6_0
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Qwen1.5-32B-Chat-GGUF

32B 8K ctx
Q2_0
10GB file 12.52GB
Q3_0
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q6_0
26GB file 28.52GB
Q8_0
34GB file 36.52GB

Qwen1.5-4B-Chat-GGUF

4B 8K ctx
Q2_0
1.25GB file 3.32GB
Q3_0
1.75GB file 3.82GB
Q4_0
2.25GB file 4.32GB
Q5_0
2.75GB file 4.82GB
Q6_0
3.25GB file 5.32GB
Q8_0
4.25GB file 6.32GB

Qwen1.5-72B-Chat-GGUF

72B 8K ctx
Q2_0
22.5GB file 25.68GB
Q3_0
31.5GB file 34.68GB
FP16
144GB file 147.18GB

Qwen1.5-7B-Chat-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
Q3_0
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Qwen2-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_0
0.16GB file 2.17GB
Q3_0
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q6_0
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB
FP16
1GB file 3.01GB

Qwen2-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_0
0.47GB file 2.49GB
Q3_0
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q6_0
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB
FP16
3GB file 5.02GB

Qwen2-57B-A14B-Instruct-GGUF

57B 8K ctx
Q3_0
24.94GB file 27.87GB
Q4_0
32.06GB file 34.99GB
Q5_0
39.19GB file 42.12GB
Q6_0
46.31GB file 49.24GB
Q8_0
60.56GB file 63.49GB
FP16
114GB file 116.93GB

Qwen2-72B-Instruct-GGUF

72B 8K ctx
Q2_0
22.5GB file 25.68GB
Q3_0
31.5GB file 34.68GB
Q4_0
40.5GB file 43.68GB
Q5_0
49.5GB file 52.68GB
Q6_0
58.5GB file 61.68GB
Q1_0
72GB file 75.18GB
Q8_0
76.5GB file 79.68GB
FP16
144GB file 147.18GB

Qwen2-7B-Instruct-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
Q3_0
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
FP16
14GB file 16.11GB

Qwen2.5-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_0
0.16GB file 2.17GB
Q3_0
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q6_0
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB
FP16
1GB file 3.01GB

Qwen2.5-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_0
0.47GB file 2.49GB
Q3_0
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q6_0
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB
FP16
3GB file 5.02GB

Qwen2.5-14B-Instruct-GGUF

14B 8K ctx
Q2_0
4.38GB file 6.61GB
Q3_0
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q6_0
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB
FP16
28GB file 30.23GB

Qwen2.5-32B-Instruct-GGUF

32B 8K ctx
Q2_0
10GB file 12.52GB
Q3_0
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q6_0
26GB file 28.52GB
Q8_0
34GB file 36.52GB
FP16
64GB file 66.52GB

Qwen2.5-3B-Instruct-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Qwen2.5-72B-Instruct-GGUF

72B 8K ctx
Q2_0
22.5GB file 25.68GB
Q3_0
31.5GB file 34.68GB
Q4_0
40.5GB file 43.68GB
Q5_0
49.5GB file 52.68GB
Q6_0
58.5GB file 61.68GB
Q8_0
76.5GB file 79.68GB
FP16
144GB file 147.18GB

Qwen2.5-7B-Instruct-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
Q3_0
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
FP16
14GB file 16.11GB

Qwen2.5-Coder-0.5B-Instruct-GGUF

0.5B 8K ctx
Q2_0
0.16GB file 2.17GB
Q3_0
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q6_0
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB
FP16
1GB file 3.01GB

Qwen2.5-Coder-1.5B-Instruct-GGUF

1.5B 8K ctx
Q2_0
0.47GB file 2.49GB
Q3_0
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q6_0
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Qwen2.5-Coder-14B-Instruct-GGUF

14B 8K ctx
Q2_0
4.38GB file 6.61GB
Q3_0
6.13GB file 8.36GB
Q4_0
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q6_0
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB
FP16
28GB file 30.23GB

Qwen2.5-Coder-32B-Instruct-GGUF

32B 8K ctx
Q2_0
10GB file 12.52GB
Q3_0
14GB file 16.52GB
Q4_0
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q6_0
26GB file 28.52GB
Q8_0
34GB file 36.52GB
FP16
64GB file 66.52GB

Qwen2.5-Coder-3B-Instruct-GGUF

3B 8K ctx
Q2_0
0.94GB file 2.99GB
Q3_0
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q6_0
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB
FP16
6GB file 8.05GB

Qwen2.5-Coder-7B-Instruct-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
Q3_0
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
FP16
14GB file 16.11GB

Qwen3-0.6B-GGUF

0.6B 8K ctx
Q8_0
0.64GB file 2.65GB

Qwen3-1.7B-GGUF

1.7B 8K ctx
Q8_0
1.81GB file 3.84GB

Qwen3-14B-GGUF

14B 8K ctx
Q4_K_M
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_K_M
9.63GB file 11.86GB
Q6_K
11.38GB file 13.61GB
Q8_0
14.88GB file 17.11GB

Qwen3-235B-A22B-GGUF

235B 8K ctx
Q4_K_M
132.19GB file 138.04GB
Q5_0
161.56GB file 167.41GB
Q5_K_M
161.56GB file 167.41GB
Q6_K
190.94GB file 196.79GB
Q8_0
249.69GB file 255.54GB

Qwen3-30B-A3B-GGUF

30B 8K ctx
Q4_K_M
16.88GB file 19.37GB
Q5_0
20.63GB file 23.12GB
Q5_K_M
20.63GB file 23.12GB
Q6_K
24.38GB file 26.87GB
Q8_0
31.88GB file 34.37GB

Qwen3-32B-GGUF

32B 8K ctx
Q4_K_M
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q5_K_M
22GB file 24.52GB
Q6_K
26GB file 28.52GB
Q8_0
34GB file 36.52GB

Qwen3-4B-GGUF

4B 8K ctx
Q4_K_M
2.25GB file 4.32GB
Q5_0
2.75GB file 4.82GB
Q5_K_M
2.75GB file 4.82GB
Q6_K
3.25GB file 5.32GB
Q8_0
4.25GB file 6.32GB

Qwen3-8B-GGUF

8B 8K ctx
Q4_K_M
4.5GB file 6.63GB
Q5_0
5.5GB file 7.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q8_0
8.5GB file 10.63GB

Qwen3-Coder-Next-GGUF

57B 8K ctx
Q4_K_M
32.06GB file 34.99GB
Q5_0
39.19GB file 42.12GB
Q5_K_M
39.19GB file 42.12GB
Q6_K
46.31GB file 49.24GB
Q8_0
60.56GB file 63.49GB

Qwen3-Next-80B-A3B-Instruct-GGUF

80B 8K ctx
Q4_K_M
45GB file 48.31GB
Q5_0
55GB file 58.31GB
Q5_K_M
55GB file 58.31GB
Q6_K
65GB file 68.31GB
Q8_0
85GB file 88.31GB

Qwen3-Next-80B-A3B-Thinking-GGUF

80B 8K ctx
Q4_K_M
45GB file 48.31GB
Q5_0
55GB file 58.31GB
Q5_K_M
55GB file 58.31GB
Q6_K
65GB file 68.31GB
Q8_0
85GB file 88.31GB

QwQ-32B-GGUF

32B 8K ctx
Q4_0
18GB file 20.52GB
Q5_0
22GB file 24.52GB
Q6_0
26GB file 28.52GB
Q8_0
34GB file 36.52GB
FP16
64GB file 66.52GB
Q

Qweqweqe132

1 models

Teuken-7B-instruct-research-v0.4-Q4_K_M-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB
Q

Qwp4W3Hyb

28 models

Cerebrum-1.0-8x7b-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_K_M
3.94GB file 6.05GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB

Codestral-22B-v0.1-iMat-GGUF

22B 8K ctx
Q2_M
6.88GB file 9.24GB
Q2_S
6.88GB file 9.24GB
Q2_XS
6.88GB file 9.24GB
Q2_XXS
6.88GB file 9.24GB
Q3_M
9.63GB file 11.99GB
Q3_S
9.63GB file 11.99GB
Q3_XS
9.63GB file 11.99GB
Q3_XXS
9.63GB file 11.99GB
Q4_0
12.38GB file 14.74GB
Q4_K_M
12.38GB file 14.74GB
Q4_K_S
12.38GB file 14.74GB
Q4_NL
12.38GB file 14.74GB
Q4_XS
12.38GB file 14.74GB
Q5_K_M
15.13GB file 17.49GB
Q5_K_S
15.13GB file 17.49GB
Q6_K
17.88GB file 20.24GB
Q1_S
22GB file 24.36GB
Q8_0
23.38GB file 25.74GB
BF16
44GB file 46.36GB

gemma-2-27b-it-iMat-GGUF

27B 8K ctx
Q2_XXS
8.44GB file 10.88GB
Q3_XXS
11.81GB file 14.25GB
Q4_K_L
15.19GB file 17.63GB
Q4_XS
15.19GB file 17.63GB
Q5_K_L
18.56GB file 21GB
Q6_K_L
21.94GB file 24.38GB
Q1_S
27GB file 29.44GB
Q8_0
28.69GB file 31.13GB

gemma-2-9b-it-iMat-GGUF

9B 8K ctx
Q2_XXS
2.81GB file 4.96GB
Q3_XXS
3.94GB file 6.09GB
Q4_K_L
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K_L
6.19GB file 8.34GB
Q6_K_L
7.31GB file 9.46GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB

Gemma-2-9B-It-SPPO-Iter3-iMat-GGUF

9B 8K ctx
Q2_XXS
2.81GB file 4.96GB
Q3_XXS
3.94GB file 6.09GB
Q4_K_L
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K_L
6.19GB file 8.34GB
Q6_K_L
7.31GB file 9.46GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB

Hunyuan-A13B-Instruct-hf-WIP-GGUF

13B 8K ctx
Q3_K_L
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q4_NL
7.31GB file 9.52GB
Q4_XS
7.31GB file 9.52GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB
BF16
26GB file 28.21GB

Llama-3-11.5B-Instruct-V2-iMat-GGUF

11.5B 8K ctx
Q2_M
3.59GB file 5.78GB
Q2_S
3.59GB file 5.78GB
Q2_XS
3.59GB file 5.78GB
Q2_XXS
3.59GB file 5.78GB
Q3_M
5.03GB file 7.22GB
Q3_S
5.03GB file 7.22GB
Q3_XS
5.03GB file 7.22GB
Q3_XXS
5.03GB file 7.22GB
Q4_0
6.47GB file 8.66GB
Q4_K_M
6.47GB file 8.66GB
Q4_K_S
6.47GB file 8.66GB
Q4_NL
6.47GB file 8.66GB
Q4_XS
6.47GB file 8.66GB
Q5_K_M
7.91GB file 10.1GB
Q5_K_S
7.91GB file 10.1GB
Q6_K
9.34GB file 11.53GB
Q1_S
11.5GB file 13.69GB
Q8_0
12.22GB file 14.41GB
BF16
23GB file 25.19GB

Llama-3-8B-Instruct-Gradient-1048k-iMat-GGUF

8B 8K ctx
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB

Meta-Llama-3-70B-Instruct-iMat-GGUF

70B 8K ctx
Q2_M
21.88GB file 25.03GB
Q2_S
21.88GB file 25.03GB
Q2_XS
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_M
30.63GB file 33.78GB
Q3_S
30.63GB file 33.78GB
Q3_XS
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB

Meta-Llama-3-8B-Instruct-iMat-GGUF

8B 8K ctx
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB

Meta-Llama-3.1-70B-Instruct-iMat-GGUF

70B 8K ctx
Q2_M
21.88GB file 25.03GB
Q2_XXS
21.88GB file 25.03GB
Q3_M
30.63GB file 33.78GB
Q3_XXS
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_L
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_NL
39.38GB file 42.53GB
Q4_XS
39.38GB file 42.53GB
Q5_K_L
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q6_K
56.88GB file 60.03GB
Q6_K_L
56.88GB file 60.03GB
Q1_S
70GB file 73.15GB
Q8_0
74.38GB file 77.53GB
BF16
140GB file 143.15GB

Meta-Llama-3.1-8B-Instruct-iMat-GGUF

8B 8K ctx
Q2_M
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_M
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB

Mistral-Nemo-Instruct-2407-iMat-GGUF

12B 8K ctx
Q2_L
3.75GB file 5.95GB
Q2_M
3.75GB file 5.95GB
Q2_XXS
3.75GB file 5.95GB
Q3_L
5.25GB file 7.45GB
Q3_M
5.25GB file 7.45GB
Q3_XXS
5.25GB file 7.45GB
Q4_0
6.75GB file 8.95GB
Q4_K_L
6.75GB file 8.95GB
Q4_K_M
6.75GB file 8.95GB
Q4_XS
6.75GB file 8.95GB
Q5_K_L
8.25GB file 10.45GB
Q5_K_M
8.25GB file 10.45GB
Q6_K
9.75GB file 11.95GB
Q6_K_L
9.75GB file 11.95GB
Q1_S
12GB file 14.2GB
Q8_0
12.75GB file 14.95GB
BF16
24GB file 26.2GB

Mistral-Small-3.1-24B-Instruct-2503-HF-iMat-GGUF

24B 8K ctx
Q2_M
7.5GB file 9.89GB
Q2_S
7.5GB file 9.89GB
Q2_XS
7.5GB file 9.89GB
Q2_XXS
7.5GB file 9.89GB
Q3_M
10.5GB file 12.89GB
Q3_S
10.5GB file 12.89GB
Q3_XS
10.5GB file 12.89GB
Q3_XXS
10.5GB file 12.89GB
Q4_0
13.5GB file 15.89GB
Q4_K_M
13.5GB file 15.89GB
Q4_K_S
13.5GB file 15.89GB
Q4_NL
13.5GB file 15.89GB
Q4_XS
13.5GB file 15.89GB
Q5_K_M
16.5GB file 18.89GB
Q5_K_S
16.5GB file 18.89GB
Q6_K
19.5GB file 21.89GB
Q1_S
24GB file 26.39GB
Q8_0
25.5GB file 27.89GB
BF16
48GB file 50.39GB

openchat-3.6-8b-20240522-iMat-GGUF

8B 8K ctx
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB

Phi-3-medium-128k-instruct-iMat-GGUF

3.8B 8K ctx
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB

Phi-3-medium-4k-instruct-iMat-GGUF

3.8B 8K ctx
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB

Phi-3-mini-128k-instruct-iMat-GGUF

3.8B 8K ctx
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB

Phi-3-mini-4k-instruct-iMat-GGUF

3.8B 8K ctx
Q2_M
1.19GB file 3.25GB
Q2_S
1.19GB file 3.25GB
Q2_XS
1.19GB file 3.25GB
Q2_XXS
1.19GB file 3.25GB
Q3_M
1.66GB file 3.72GB
Q3_S
1.66GB file 3.72GB
Q3_XS
1.66GB file 3.72GB
Q3_XXS
1.66GB file 3.72GB
Q4_0
2.14GB file 4.2GB
Q4_K_M
2.14GB file 4.2GB
Q4_K_S
2.14GB file 4.2GB
Q4_NL
2.14GB file 4.2GB
Q4_XS
2.14GB file 4.2GB
Q5_K_M
2.61GB file 4.67GB
Q5_K_S
2.61GB file 4.67GB
Q6_K
3.09GB file 5.15GB
Q1_S
3.8GB file 5.86GB
Q8_0
4.04GB file 6.1GB
BF16
7.6GB file 9.66GB

Qwen2-1.5B-Instruct-iMat-GGUF

1.5B 8K ctx
Q2_M
0.47GB file 2.49GB
Q2_S
0.47GB file 2.49GB
Q2_XS
0.47GB file 2.49GB
Q2_XXS
0.47GB file 2.49GB
Q3_M
0.66GB file 2.68GB
Q3_S
0.66GB file 2.68GB
Q3_XS
0.66GB file 2.68GB
Q3_XXS
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q4_K_M
0.84GB file 2.86GB
Q4_K_S
0.84GB file 2.86GB
Q4_NL
0.84GB file 2.86GB
Q4_XS
0.84GB file 2.86GB
Q5_K_M
1.03GB file 3.05GB
Q5_K_S
1.03GB file 3.05GB
Q6_K
1.22GB file 3.24GB
Q1_S
1.5GB file 3.52GB
Q8_0
1.59GB file 3.61GB
BF16
3GB file 5.02GB

Qwen2-72B-Instruct-iMat-GGUF

72B 8K ctx
Q2_M
22.5GB file 25.68GB
Q2_S
22.5GB file 25.68GB
Q2_XS
22.5GB file 25.68GB
Q2_XXS
22.5GB file 25.68GB
Q3_M
31.5GB file 34.68GB
Q3_S
31.5GB file 34.68GB
Q3_XS
31.5GB file 34.68GB
Q3_XXS
31.5GB file 34.68GB
Q4_0
40.5GB file 43.68GB
Q4_K_M
40.5GB file 43.68GB
Q4_K_S
40.5GB file 43.68GB
Q4_NL
40.5GB file 43.68GB
Q4_XS
40.5GB file 43.68GB
Q5_K_M
49.5GB file 52.68GB
Q5_K_S
49.5GB file 52.68GB
Q6_K
58.5GB file 61.68GB
Q1_S
72GB file 75.18GB
Q8_0
76.5GB file 79.68GB
BF16
144GB file 147.18GB

Qwen2-7B-Instruct-iMat-GGUF

7B 8K ctx
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q4_NL
3.94GB file 6.05GB
Q4_XS
3.94GB file 6.05GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q1_S
7GB file 9.11GB
Q8_0
7.44GB file 9.55GB
BF16
14GB file 16.11GB

SFR-Iterative-DPO-LLaMA-3-8B-R-iMat-GGUF

8B 8K ctx
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_0
4.5GB file 6.63GB
Q4_K_M
4.5GB file 6.63GB
Q4_K_S
4.5GB file 6.63GB
Q4_NL
4.5GB file 6.63GB
Q4_XS
4.5GB file 6.63GB
Q5_K_M
5.5GB file 7.63GB
Q5_K_S
5.5GB file 7.63GB
Q6_K
6.5GB file 8.63GB
Q1_S
8GB file 10.13GB
Q8_0
8.5GB file 10.63GB
BF16
16GB file 18.13GB

Yi-1.5-34B-Chat-16K-iMat-GGUF

34B 8K ctx
Q2_M
10.63GB file 13.19GB
Q2_S
10.63GB file 13.19GB
Q2_XS
10.63GB file 13.19GB
Q2_XXS
10.63GB file 13.19GB
Q3_M
14.88GB file 17.44GB
Q3_S
14.88GB file 17.44GB
Q3_XS
14.88GB file 17.44GB
Q3_XXS
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q4_NL
19.13GB file 21.69GB
Q4_XS
19.13GB file 21.69GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q1_S
34GB file 36.56GB
Q8_0
36.13GB file 38.69GB
BF16
68GB file 70.56GB

Yi-1.5-34B-Chat-iMat-GGUF

34B 8K ctx
Q2_M
10.63GB file 13.19GB
Q2_S
10.63GB file 13.19GB
Q2_XS
10.63GB file 13.19GB
Q2_XXS
10.63GB file 13.19GB
Q3_M
14.88GB file 17.44GB
Q3_S
14.88GB file 17.44GB
Q3_XS
14.88GB file 17.44GB
Q3_XXS
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q4_NL
19.13GB file 21.69GB
Q4_XS
19.13GB file 21.69GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q1_S
34GB file 36.56GB
Q8_0
36.13GB file 38.69GB
BF16
68GB file 70.56GB

Yi-1.5-6B-Chat-iMat-GGUF

6B 8K ctx
Q2_M
1.88GB file 3.98GB
Q2_S
1.88GB file 3.98GB
Q2_XS
1.88GB file 3.98GB
Q2_XXS
1.88GB file 3.98GB
Q3_M
2.63GB file 4.73GB
Q3_S
2.63GB file 4.73GB
Q3_XS
2.63GB file 4.73GB
Q3_XXS
2.63GB file 4.73GB
Q4_0
3.38GB file 5.48GB
Q4_K_M
3.38GB file 5.48GB
Q4_K_S
3.38GB file 5.48GB
Q4_NL
3.38GB file 5.48GB
Q4_XS
3.38GB file 5.48GB
Q5_K_M
4.13GB file 6.23GB
Q5_K_S
4.13GB file 6.23GB
Q6_K
4.88GB file 6.98GB
Q1_S
6GB file 8.1GB
Q8_0
6.38GB file 8.48GB
BF16
12GB file 14.1GB

Yi-1.5-9B-Chat-16K-iMat-GGUF

9B 8K ctx
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB

Yi-1.5-9B-Chat-iMat-GGUF

9B 8K ctx
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_0
5.06GB file 7.21GB
Q4_K_M
5.06GB file 7.21GB
Q4_K_S
5.06GB file 7.21GB
Q4_NL
5.06GB file 7.21GB
Q4_XS
5.06GB file 7.21GB
Q5_K_M
6.19GB file 8.34GB
Q5_K_S
6.19GB file 8.34GB
Q6_K
7.31GB file 9.46GB
Q1_S
9GB file 11.15GB
Q8_0
9.56GB file 11.71GB
BF16
18GB file 20.15GB