All Supported Models

29681 AI models · 198875 quantization formats tracked

C

C H A R K A

1 models

llama3.2-hg-education

Unknown 8K ctx
FP16
0GB file 2GB
Q4_0
0GB file 2GB
C

C H E 72

50 models

Breeze-7B-Instruct-v1_0-Q2_K-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
FP16
14GB file 16.11GB

Breeze-7B-Instruct-v1_0-Q3_K_L-GGUF

7B 8K ctx
Q3_0
3.06GB file 5.17GB
FP16
14GB file 16.11GB

Breeze-7B-Instruct-v1_0-Q3_K_M-GGUF

7B 8K ctx
Q3_0
3.06GB file 5.17GB
FP16
14GB file 16.11GB

Breeze-7B-Instruct-v1_0-Q3_K_S-GGUF

7B 8K ctx
Q3_0
3.06GB file 5.17GB
FP16
14GB file 16.11GB

Breeze-7B-Instruct-v1_0-Q4_0-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB

Breeze-7B-Instruct-v1_0-Q4_K_M-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB

Breeze-7B-Instruct-v1_0-Q4_K_S-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB

Breeze-7B-Instruct-v1_0-Q5_0-GGUF

7B 8K ctx
Q5_0
4.81GB file 6.92GB

Breeze-7B-Instruct-v1_0-Q5_K_M-GGUF

7B 8K ctx
Q5_0
4.81GB file 6.92GB
FP16
14GB file 16.11GB

Breeze-7B-Instruct-v1_0-Q6_K-GGUF

7B 8K ctx
Q6_0
5.69GB file 7.8GB
FP16
14GB file 16.11GB

Breeze-7B-Instruct-v1_0-Q8_0-GGUF

7B 8K ctx
Q8_0
7.44GB file 9.55GB

Phi-3-medium-128k-instruct-Q3_K_L-GGUF

3.8B 8K ctx
Q3_0
1.66GB file 3.72GB
FP16
7.6GB file 9.66GB

Phi-3-medium-128k-instruct-Q4_0-GGUF

0B 8K ctx
Q4_0
0GB file 2GB

Phi-3-medium-128k-instruct-Q4_K_M-GGUF

3.8B 8K ctx
Q4_0
2.14GB file 4.2GB
FP16
7.6GB file 9.66GB

Phi-3-medium-128k-instruct-Q4_K_S-GGUF

3.8B 8K ctx
Q4_0
2.14GB file 4.2GB
FP16
7.6GB file 9.66GB

Phi-3-medium-128k-instruct-Q5_0-GGUF

0B 8K ctx
Q5_0
0GB file 2GB

Phi-3-medium-128k-instruct-Q5_K_M-GGUF

3.8B 8K ctx
Q5_0
2.61GB file 4.67GB
FP16
7.6GB file 9.66GB

Phi-3-medium-128k-instruct-Q5_K_S-GGUF

3.8B 8K ctx
Q5_0
2.61GB file 4.67GB
FP16
7.6GB file 9.66GB

Phi-3-medium-128k-instruct-Q6_K-GGUF

3.8B 8K ctx
Q6_0
3.09GB file 5.15GB
FP16
7.6GB file 9.66GB

Phi-3-medium-128k-instruct-Q8_0-GGUF

0B 8K ctx
Q8_0
0GB file 2GB

Phi-3-mini-128k-instruct-Q6_K-GGUF

3.8B 8K ctx
Q6_0
3.09GB file 5.15GB
FP16
7.6GB file 9.66GB

Phi-3-mini-128k-instruct-Q8_0-GGUF

0B 8K ctx
Q8_0
0GB file 2GB

Qwen1.5-4B-Chat-Q2_K-GGUF

4B 8K ctx
Q2_0
1.25GB file 3.32GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q3_K_L-GGUF

4B 8K ctx
Q3_0
1.75GB file 3.82GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q3_K_M-GGUF

4B 8K ctx
Q3_0
1.75GB file 3.82GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q3_K_S-GGUF

4B 8K ctx
Q3_0
1.75GB file 3.82GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q4_0-GGUF

4B 8K ctx
Q4_0
2.25GB file 4.32GB

Qwen1.5-4B-Chat-Q4_K_M-GGUF

4B 8K ctx
Q4_0
2.25GB file 4.32GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q4_K_S-GGUF

4B 8K ctx
Q4_0
2.25GB file 4.32GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q5_0-GGUF

4B 8K ctx
Q5_0
2.75GB file 4.82GB

Qwen1.5-4B-Chat-Q5_K_M-GGUF

4B 8K ctx
Q5_0
2.75GB file 4.82GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q5_K_S-GGUF

4B 8K ctx
Q5_0
2.75GB file 4.82GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q6_K-GGUF

4B 8K ctx
Q6_0
3.25GB file 5.32GB
FP16
8GB file 10.07GB

Qwen1.5-4B-Chat-Q8_0-GGUF

4B 8K ctx
Q8_0
4.25GB file 6.32GB

Qwen2-0.5B-Instruct-Q6_K-GGUF

0.5B 8K ctx
Q6_0
0.41GB file 2.42GB
FP16
1GB file 3.01GB

Qwen2-0.5B-Instruct-Q8_0-GGUF

0.5B 8K ctx
Q8_0
0.53GB file 2.54GB

Qwen2-1.5B-Instruct-Q6_K-GGUF

1.5B 8K ctx
Q6_0
1.22GB file 3.24GB
FP16
3GB file 5.02GB

Qwen2-1.5B-Instruct-Q8_0-GGUF

1.5B 8K ctx
Q8_0
1.59GB file 3.61GB

Qwen2-7B-Instruct-Q2_K-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q3_K_L-GGUF

7B 8K ctx
Q3_0
3.06GB file 5.17GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q3_K_M-GGUF

7B 8K ctx
Q3_0
3.06GB file 5.17GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q3_K_S-GGUF

7B 8K ctx
Q3_0
3.06GB file 5.17GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q4_0-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB

Qwen2-7B-Instruct-Q4_K_M-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q4_K_S-GGUF

7B 8K ctx
Q4_0
3.94GB file 6.05GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q5_0-GGUF

7B 8K ctx
Q5_0
4.81GB file 6.92GB

Qwen2-7B-Instruct-Q5_K_M-GGUF

7B 8K ctx
Q5_0
4.81GB file 6.92GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q5_K_S-GGUF

7B 8K ctx
Q5_0
4.81GB file 6.92GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q6_K-GGUF

7B 8K ctx
Q6_0
5.69GB file 7.8GB
FP16
14GB file 16.11GB

Qwen2-7B-Instruct-Q8_0-GGUF

7B 8K ctx
Q8_0
7.44GB file 9.55GB
C

C H E 72 Z Lab

4 models

Alibaba-Qwen2-0_5B-Instruct-GGUF

0.5B 8K ctx
Q2_0
0.16GB file 2.17GB
Q3_0
0.22GB file 2.23GB
Q4_0
0.28GB file 2.29GB
Q5_0
0.34GB file 2.35GB
Q6_0
0.41GB file 2.42GB
Q8_0
0.53GB file 2.54GB

Alibaba-Qwen2-1_5B-Instruct-GGUF

1.5B 8K ctx
Q2_0
0.47GB file 2.49GB
Q3_0
0.66GB file 2.68GB
Q4_0
0.84GB file 2.86GB
Q5_0
1.03GB file 3.05GB
Q6_0
1.22GB file 3.24GB
Q8_0
1.59GB file 3.61GB

Alibaba-Qwen2-7B-Instruct-GGUF

7B 8K ctx
Q2_0
2.19GB file 4.3GB
Q3_0
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q6_0
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Microsoft-Phi3-14B-Instruct128K-GGUF

14B 8K ctx
Q2_0
4.38GB file 6.61GB
Q3_0
6.13GB file 8.36GB
FP16
28GB file 30.23GB
C

C Hunter789

2 models

Qwen3.6-27B-i1-IQ4_KS-GGUF

27B 8K ctx
Q4_K
15.19GB file 17.63GB

Qwen3.6-27B-i1-IQ4_XS-GGUF

27B 8K ctx
Q4_XS
15.19GB file 17.63GB
C

C I S Cai

15 models

Cerebrum-1.0-8x7b-SOTA-GGUF

7B 8K ctx
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_XS
3.94GB file 6.05GB

CodeQwen1.5-7B-Chat-SOTA-GGUF

7B 8K ctx
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_NL
3.94GB file 6.05GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
FP16
14GB file 16.11GB

gemma-4-31B-it-NVFP4-turbo-GGUF

31B 8K ctx
FP4
31GB file 33.51GB

gorilla-openfunctions-v2-SOTA-GGUF

Unknown 8K ctx
FP16
0GB file 2GB
Q1_S
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q2_XS
0GB file 2GB
Q2_XXS
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_XS
0GB file 2GB

granite-3.2-8b-instruct-SOTA-GGUF

8B 8K ctx
Q2_M
2.5GB file 4.63GB
Q2_S
2.5GB file 4.63GB
Q2_XS
2.5GB file 4.63GB
Q2_XXS
2.5GB file 4.63GB
Q3_M
3.5GB file 5.63GB
Q3_S
3.5GB file 5.63GB
Q3_XS
3.5GB file 5.63GB
Q3_XXS
3.5GB file 5.63GB
Q4_XS
4.5GB file 6.63GB
Q1_M
8GB file 10.13GB
Q1_S
8GB file 10.13GB
BF16
16GB file 18.13GB

Mistral-7B-Instruct-v0.3-SOTA-GGUF

7B 8K ctx
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_XS
3.94GB file 6.05GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
FP16
14GB file 16.11GB

Mistral-Nemo-Instruct-2407-SOTA-GGUF

12B 8K ctx
Q2_M
3.75GB file 5.95GB
Q2_S
3.75GB file 5.95GB
Q2_XS
3.75GB file 5.95GB
Q2_XXS
3.75GB file 5.95GB
Q3_M
5.25GB file 7.45GB
Q3_S
5.25GB file 7.45GB
Q3_XS
5.25GB file 7.45GB
Q3_XXS
5.25GB file 7.45GB
Q4_XS
6.75GB file 8.95GB
Q1_M
12GB file 14.2GB
Q1_S
12GB file 14.2GB
BF16
24GB file 26.2GB

Nxcode-CQ-7B-orpo-SOTA-GGUF

7B 8K ctx
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_NL
3.94GB file 6.05GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
FP16
14GB file 16.11GB

OpenCodeInterpreter-DS-6.7B-SOTA-GGUF

6.7B 8K ctx
Q2_M
2.09GB file 4.2GB
Q2_S
2.09GB file 4.2GB
Q2_XS
2.09GB file 4.2GB
Q2_XXS
2.09GB file 4.2GB
Q3_M
2.93GB file 5.04GB
Q3_S
2.93GB file 5.04GB
Q3_XS
2.93GB file 5.04GB
Q3_XXS
2.93GB file 5.04GB
Q4_XS
3.77GB file 5.88GB
Q1_S
6.7GB file 8.81GB
FP16
13.4GB file 15.51GB

pydevmini1-SOTA-GGUF

Unknown 8K ctx
BF16
0GB file 2GB
Q2_M
0GB file 2GB
Q2_S
0GB file 2GB
Q3_M
0GB file 2GB
Q3_S
0GB file 2GB
Q3_XS
0GB file 2GB
Q3_XXS
0GB file 2GB
Q4_XS
0GB file 2GB

Qwen2.5-Coder-0.5B-Instruct-SOTA-GGUF

0.5B 8K ctx
Q4_NL
0.28GB file 2.29GB
BF16
1GB file 3.01GB

Qwen2.5-Coder-32B-Instruct-SOTA-GGUF

32B 8K ctx
Q2_M
10GB file 12.52GB
Q2_S
10GB file 12.52GB
Q2_XS
10GB file 12.52GB
Q2_XXS
10GB file 12.52GB
Q3_M
14GB file 16.52GB
Q3_S
14GB file 16.52GB
Q3_XS
14GB file 16.52GB
Q3_XXS
14GB file 16.52GB
Q4_XS
18GB file 20.52GB
Q1_M
32GB file 34.52GB
Q1_S
32GB file 34.52GB
BF16
64GB file 66.52GB

Qwen2.5-Coder-7B-Instruct-SOTA-GGUF

7B 8K ctx
Q2_M
2.19GB file 4.3GB
Q2_S
2.19GB file 4.3GB
Q2_XS
2.19GB file 4.3GB
Q2_XXS
2.19GB file 4.3GB
Q3_M
3.06GB file 5.17GB
Q3_S
3.06GB file 5.17GB
Q3_XS
3.06GB file 5.17GB
Q3_XXS
3.06GB file 5.17GB
Q4_XS
3.94GB file 6.05GB
Q1_M
7GB file 9.11GB
Q1_S
7GB file 9.11GB
BF16
14GB file 16.11GB

Qwen3-Coder-30B-A3B-Instruct-SOTA-GGUF

30B 8K ctx
Q2_M
9.38GB file 11.87GB
Q2_S
9.38GB file 11.87GB
Q2_XS
9.38GB file 11.87GB
Q2_XXS
9.38GB file 11.87GB
Q3_M
13.13GB file 15.62GB
Q3_S
13.13GB file 15.62GB
Q3_XS
13.13GB file 15.62GB
Q3_XXS
13.13GB file 15.62GB
Q4_XS
16.88GB file 19.37GB
Q1_M
30GB file 32.49GB
Q1_S
30GB file 32.49GB
BF16
60GB file 62.49GB

Yi-Coder-9B-Chat-SOTA-GGUF

9B 8K ctx
Q2_M
2.81GB file 4.96GB
Q2_S
2.81GB file 4.96GB
Q2_XS
2.81GB file 4.96GB
Q2_XXS
2.81GB file 4.96GB
Q3_M
3.94GB file 6.09GB
Q3_S
3.94GB file 6.09GB
Q3_XS
3.94GB file 6.09GB
Q3_XXS
3.94GB file 6.09GB
Q4_XS
5.06GB file 7.21GB
Q1_M
9GB file 11.15GB
Q1_S
9GB file 11.15GB
BF16
18GB file 20.15GB
C

C J B Adair

1 models

UserLM-8b-Q8_0-GGUF

8B 8K ctx
Q8_0
8.5GB file 10.63GB
C

C Srose X

1 models

Legal_llama3.1_part5

8B 8K ctx
Q8_0
8.5GB file 10.63GB
C

C T Angarato

1 models

Broken-Tutu-24B-Unslop-v2.0-Q4_K_M-GGUF

24B 8K ctx
Q4_0
13.5GB file 15.89GB
FP16
48GB file 50.39GB
C

C T Ati

3 models

llama3.1-typhoon2-8b-instruct-Q4_K_M-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
FP16
16GB file 18.13GB

llama3.1-typhoon2-8b-Q4_K_M-GGUF

8B 8K ctx
Q4_0
4.5GB file 6.63GB
FP16
16GB file 18.13GB

llama3.2-typhoon2-t1-3b-research-preview-Q4_K_M-GGUF

3B 8K ctx
Q4_0
1.69GB file 3.74GB
FP16
6GB file 8.05GB
C

C Xx

1 models

gemma-4-E4B-it-OBLITERATED

4B 8K ctx
Q4_K_M
2.25GB file 4.32GB
Q5_K_M
2.75GB file 4.82GB
Q8_0
4.25GB file 6.32GB