All Supported Models

29681 AI models · 198875 quantization formats tracked

T

Tharunrega

1 models

qwen2.5-1.5b-finance-dpo

1.5B 8K ctx
Q4_K_M
0.84GB file 2.86GB
T

That Cultivator

2 models

sarvam-30b-gguf

30B 8K ctx
Q4_K_M
16.88GB file 19.37GB

sarvam-30b-Uncensored-gguf

30B 8K ctx
Q3_K_M
13.13GB file 15.62GB
Q4_K_M
16.88GB file 19.37GB
Q6_K
24.38GB file 26.87GB
T

Thataigod

2 models

QwQ-32B-Q3_K_M-GGUF

32B 8K ctx
Q3_0
14GB file 16.52GB
FP16
64GB file 66.52GB

QwQ-32B-Q3_K_S-GGUF

32B 8K ctx
Q3_0
14GB file 16.52GB
FP16
64GB file 66.52GB
T

Thatoneguy69

1 models

Opus4.7-GODs.Ghost.Codex-4B.GGuF

4B 8K ctx
Q4_K_M
2.25GB file 4.32GB
Q5_K_M
2.75GB file 4.82GB
Q8_0
4.25GB file 6.32GB
T

The Block

2 models

LLmRA-3B-v0.1-GGUF-V2

3B 8K ctx
Q3_K_M
1.31GB file 3.36GB
Q4_K_M
1.69GB file 3.74GB
Q5_K_M
2.06GB file 4.11GB
Q8_0
3.19GB file 5.24GB

Mistral-7B-Instruct-v0.2-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
T

The Bloke

274 models

13B-Ouroboros-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Aetheria-L2-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Airochronos-L2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Airolima-Chronos-Grad-L2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Akins-3B-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Amber-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

AmberChat-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

ARIA-70B-V2-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Asclepius-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

CalliopeDS-L2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

CaPlatTessDolXaBoros-Yi-34B-200K-DARE-Ties-HighDensity-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

Capybara-Tess-Yi-34B-200K-DARE-Ties-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

Capybara-Tess-Yi-34B-200K-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

CapyTessBorosYi-34B-200K-DARE-Ties-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

CausalLM-14B-GGUF

14B 8K ctx
Q4_0
7.88GB file 10.11GB
Q4_1
7.88GB file 10.11GB
Q5_0
9.63GB file 11.86GB
Q5_1
9.63GB file 11.86GB
Q8_0
14.88GB file 17.11GB

CausalLM-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Chronoboros-Grad-L2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Chronohermes-Grad-L2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Chronolima-Airo-Grad-L2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Claire-7B-0.1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

CodeLlama-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

CodeLlama-13B-Instruct-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

CodeLlama-13B-Python-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

CodeLlama-34B-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

CodeLlama-34B-Instruct-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

CodeLlama-34B-Python-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

CodeLlama-70B-hf-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

CodeLlama-70B-Instruct-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

CodeLlama-70B-Python-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

CodeLlama-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

CodeLlama-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

CodeLlama-7B-Python-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

CodeNinja-1.0-OpenChat-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

DaringMaid-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

DaringMaid-20B-GGUF

20B 8K ctx
Q2_K
6.25GB file 8.58GB
Q3_K_L
8.75GB file 11.08GB
Q3_K_M
8.75GB file 11.08GB
Q3_K_S
8.75GB file 11.08GB
Q4_0
11.25GB file 13.58GB
Q4_K_M
11.25GB file 13.58GB
Q4_K_S
11.25GB file 13.58GB
Q5_0
13.75GB file 16.08GB
Q5_K_M
13.75GB file 16.08GB
Q5_K_S
13.75GB file 16.08GB
Q6_K
16.25GB file 18.58GB
Q8_0
21.25GB file 23.58GB

Deacon-34B-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

deepsex-34b-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

DiscoLM-120b-GGUF

120B 8K ctx
Q2_K
37.5GB file 41.47GB

DiscoLM-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

DPOpenHermes-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

DPOpenHermes-7B-v2-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Dr_Samantha-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

em_german_13b_v01-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

em_german_70b_v01-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

em_german_7b_v01-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

em_german_leo_mistral-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

em_german_mistral_v01-GGUF

01B 8K ctx
Q2_K
0.31GB file 2.33GB
Q3_K_L
0.44GB file 2.46GB
Q3_K_M
0.44GB file 2.46GB
Q3_K_S
0.44GB file 2.46GB
Q4_0
0.56GB file 2.58GB
Q4_K_M
0.56GB file 2.58GB
Q4_K_S
0.56GB file 2.58GB
Q5_0
0.69GB file 2.71GB
Q5_K_M
0.69GB file 2.71GB
Q5_K_S
0.69GB file 2.71GB
Q6_K
0.81GB file 2.83GB
Q8_0
1.06GB file 3.08GB

fiction.live-Kimiko-V2-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

finance-chat-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

finance-LLM-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

finance-LLM-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

FusionNet_34Bx2_MoE-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB

GEITje-7B-chat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

genz-13B-v2-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Genz-70b-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

go-bruins-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

go-bruins-v2-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

GodziLLa2-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

goliath-120b-GGUF

120B 8K ctx
Q2_K
37.5GB file 41.47GB

Guanaco-13B-Uncensored-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Guanaco-7B-Uncensored-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

hippogriff-30b-chat-GGUF

30B 8K ctx
Q2_K
9.38GB file 11.87GB
Q3_K_L
13.13GB file 15.62GB
Q3_K_M
13.13GB file 15.62GB
Q3_K_S
13.13GB file 15.62GB
Q4_0
16.88GB file 19.37GB
Q4_K_M
16.88GB file 19.37GB
Q4_K_S
16.88GB file 19.37GB
Q5_0
20.63GB file 23.12GB
Q5_K_M
20.63GB file 23.12GB
Q5_K_S
20.63GB file 23.12GB
Q6_K
24.38GB file 26.87GB
Q8_0
31.88GB file 34.37GB

Instruct_Mixtral-8x7B-v0.1_Dolly15K-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

jackalope-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

japanese-stablelm-base-beta-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

japanese-stablelm-instruct-beta-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

japanese-stablelm-instruct-beta-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

japanese-stablelm-instruct-gamma-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

KafkaLM-70B-German-V0.1-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

KAI-7B-beta-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Kimiko-v2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

law-chat-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

law-LLM-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Lemur-70B-Chat-v1-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

leo-hessianai-13B-chat-bilingual-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

leo-hessianai-13B-chat-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

leo-hessianai-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

leo-hessianai-70B-chat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

leo-hessianai-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

leo-hessianai-7B-chat-bilingual-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

leo-hessianai-7B-chat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

leo-hessianai-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Leo-Mistral-Hessianai-7B-Chat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

lince-zero-GGUF

Unknown 8K ctx
Q4_0
0GB file 2GB
Q4_1
0GB file 2GB
Q5_0
0GB file 2GB
Q5_1
0GB file 2GB
Q8_0
0GB file 2GB

Llama-2-13B-chat-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

llama-2-13B-chat-limarp-v2-merged-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Llama-2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Llama-2-70B-Chat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Llama-2-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Llama-2-70B-Orca-200k-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Llama-2-7B-Chat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Llama-2-7B-ft-instruct-es-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Llama-2-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Llama-2-7B-vietnamese-20k-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Llama-2-Coder-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Llama2-70B-OASST-SFT-v10-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

LlongOrca-13B-16K-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

LlongOrca-7B-16K-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Luban-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

LUNA-SOLARkrautLM-Instruct-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Magicoder-S-DS-6.7B-GGUF

6.7B 8K ctx
Q2_K
2.09GB file 4.2GB
Q3_K_L
2.93GB file 5.04GB
Q3_K_M
2.93GB file 5.04GB
Q3_K_S
2.93GB file 5.04GB
Q4_0
3.77GB file 5.88GB
Q4_K_M
3.77GB file 5.88GB
Q4_K_S
3.77GB file 5.88GB
Q5_0
4.61GB file 6.72GB
Q5_K_M
4.61GB file 6.72GB
Q5_K_S
4.61GB file 6.72GB
Q6_K
5.44GB file 7.55GB
Q8_0
7.12GB file 9.23GB

manticore-13b-chat-pyg-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Manticore-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Marcoroni-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Marcoroni-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Marcoroni-7b-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Marx-3B-v3-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

med42-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

medalpaca-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

medicine-chat-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

medicine-LLM-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

medicine-LLM-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

meditron-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Merged-AGI-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

MetaMath-Cybertron-Starling-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Mini_Synatra_7B_02-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Mistral-7B-Instruct-v0.1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Mistral-7B-Instruct-v0.2-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Mistral-7B-OpenOrca-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Mistral-7B-v0.1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Mistral-Pygmalion-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Mixtral-8x7B-Instruct-v0.1-LimaRP-ZLoss-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Mixtral-SlimOrca-8x7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

MonadGPT-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

MysticFusion-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Mythalion-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Nanbeige-16B-Base-32K-GGUF

16B 8K ctx
Q2_K
5GB file 7.26GB
Q3_K_L
7GB file 9.26GB
Q3_K_M
7GB file 9.26GB
Q3_K_S
7GB file 9.26GB
Q4_0
9GB file 11.26GB
Q4_K_M
9GB file 11.26GB
Q4_K_S
9GB file 11.26GB
Q5_0
11GB file 13.26GB
Q5_K_M
11GB file 13.26GB
Q5_K_S
11GB file 13.26GB
Q6_K
13GB file 15.26GB
Q8_0
17GB file 19.26GB

Nanbeige-16B-Base-GGUF

16B 8K ctx
Q2_K
5GB file 7.26GB
Q3_K_L
7GB file 9.26GB
Q3_K_M
7GB file 9.26GB
Q3_K_S
7GB file 9.26GB
Q4_0
9GB file 11.26GB
Q4_K_M
9GB file 11.26GB
Q4_K_S
9GB file 11.26GB
Q5_0
11GB file 13.26GB
Q5_K_M
11GB file 13.26GB
Q5_K_S
11GB file 13.26GB
Q6_K
13GB file 15.26GB
Q8_0
17GB file 19.26GB

Nanbeige-16B-Chat-32K-GGUF

16B 8K ctx
Q2_K
5GB file 7.26GB
Q3_K_L
7GB file 9.26GB
Q3_K_M
7GB file 9.26GB
Q3_K_S
7GB file 9.26GB
Q4_0
9GB file 11.26GB
Q4_K_M
9GB file 11.26GB
Q4_K_S
9GB file 11.26GB
Q5_0
11GB file 13.26GB
Q5_K_M
11GB file 13.26GB
Q5_K_S
11GB file 13.26GB
Q6_K
13GB file 15.26GB
Q8_0
17GB file 19.26GB
FP16
32GB file 34.26GB

Nanbeige-16B-Chat-GGUF

16B 8K ctx
Q2_K
5GB file 7.26GB
Q3_K_L
7GB file 9.26GB
Q3_K_M
7GB file 9.26GB
Q3_K_S
7GB file 9.26GB
Q4_0
9GB file 11.26GB
Q4_K_M
9GB file 11.26GB
Q4_K_S
9GB file 11.26GB
Q5_0
11GB file 13.26GB
Q5_K_M
11GB file 13.26GB
Q5_K_S
11GB file 13.26GB
Q6_K
13GB file 15.26GB
Q8_0
17GB file 19.26GB

Norobara-ZLoss-8x7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Norocetacean-20B-10k-GGUF

20B 8K ctx
Q2_K
6.25GB file 8.58GB
Q3_K_L
8.75GB file 11.08GB
Q3_K_M
8.75GB file 11.08GB
Q3_K_S
8.75GB file 11.08GB
Q4_0
11.25GB file 13.58GB
Q4_K_M
11.25GB file 13.58GB
Q4_K_S
11.25GB file 13.58GB
Q5_0
13.75GB file 16.08GB
Q5_K_M
13.75GB file 16.08GB
Q5_K_S
13.75GB file 16.08GB
Q6_K
16.25GB file 18.58GB
Q8_0
21.25GB file 23.58GB

notus-7B-v1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

notux-8x7b-v1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

OpenAssistant-Llama2-13B-Orca-8K-3319-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

openbuddy-coder-34b-v11-bf16-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

openbuddy-deepseek-67b-v15-base-GGUF

67B 8K ctx
Q2_K
20.94GB file 24.04GB
Q3_K_L
29.31GB file 32.41GB
Q3_K_M
29.31GB file 32.41GB
Q3_K_S
29.31GB file 32.41GB
Q4_0
37.69GB file 40.79GB
Q4_K_M
37.69GB file 40.79GB
Q4_K_S
37.69GB file 40.79GB
Q5_0
46.06GB file 49.16GB
Q5_K_M
46.06GB file 49.16GB
Q5_K_S
46.06GB file 49.16GB

OpenBuddy-Llama2-13B-v11.1-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

openbuddy-llama2-13b64k-v15-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

openbuddy-llama2-34b-v11.1-bf16-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

OpenBuddy-Llama2-70b-v10.1-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

openbuddy-llama2-70B-v13-base-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

openbuddy-mistral-7B-v13-base-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

openbuddy-mistral-7B-v13-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

openbuddy-mistral-7B-v13.1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

openbuddy-mixtral-7bx8-v16.3-32k-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

openbuddy-mixtral-8x7b-v15.1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

openbuddy-openllama-7B-v12-bf16-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

openchat-3.5-0106-GGUF

0106B 8K ctx
Q2_K
33.13GB file 36.87GB
Q3_K_L
46.38GB file 50.12GB
Q3_K_M
46.38GB file 50.12GB
Q3_K_S
46.38GB file 50.12GB
Q4_0
59.63GB file 63.37GB
Q4_K_M
59.63GB file 63.37GB
Q4_K_S
59.63GB file 63.37GB
Q5_0
72.88GB file 76.62GB
Q5_K_M
72.88GB file 76.62GB
Q5_K_S
72.88GB file 76.62GB
Q6_K
86.13GB file 89.87GB
Q8_0
112.63GB file 116.37GB

openchat-3.5-1210-GGUF

1210B 8K ctx
Q2_K
378.13GB file 399.95GB
Q3_K_L
529.38GB file 551.2GB
Q3_K_M
529.38GB file 551.2GB
Q3_K_S
529.38GB file 551.2GB
Q4_0
680.63GB file 702.45GB
Q4_K_M
680.63GB file 702.45GB
Q4_K_S
680.63GB file 702.45GB
Q5_0
831.88GB file 853.7GB
Q5_K_M
831.88GB file 853.7GB
Q5_K_S
831.88GB file 853.7GB
Q6_K
983.13GB file 1004.95GB
Q8_0
1285.63GB file 1307.45GB

openinstruct-mistral-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

OpenOrca-Platypus2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

openthaigpt-1.0.0-beta-13B-chat-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Optimus-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

opus-v0-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

opus-v0-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

opus-v0.5-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

orca_mini_v3_13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

orca_mini_v3_70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

orca_mini_v3_7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Orca-2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Orca-2-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

OrcaMaid-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

OrcaMaid-v2-FIX-13B-32k-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

OrcaMaid-v3-13B-32k-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

OrcaMaidXL-17B-32k-GGUF

17B 8K ctx
Q2_K
5.31GB file 7.59GB
Q3_K_L
7.44GB file 9.72GB
Q3_K_M
7.44GB file 9.72GB
Q3_K_S
7.44GB file 9.72GB
Q4_0
9.56GB file 11.84GB
Q4_K_M
9.56GB file 11.84GB
Q4_K_S
9.56GB file 11.84GB
Q5_0
11.69GB file 13.97GB
Q5_K_M
11.69GB file 13.97GB
Q5_K_S
11.69GB file 13.97GB
Q6_K
13.81GB file 16.09GB
Q8_0
18.06GB file 20.34GB

OrionStar-Yi-34B-Chat-Llama-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

phi-2-dpo-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

phi-2-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

Pirouette-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

PiVoT-0.1-early-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

PiVoT-0.1-Evil-a-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

PiVoT-0.1-Starling-LM-RP-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

PiVoT-10.7B-Mistral-v0.2-GGUF

10.7B 8K ctx
Q2_K
3.34GB file 5.52GB
Q3_K_L
4.68GB file 6.86GB
Q3_K_M
4.68GB file 6.86GB
Q3_K_S
4.68GB file 6.86GB
Q4_0
6.02GB file 8.2GB
Q4_K_M
6.02GB file 8.2GB
Q4_K_S
6.02GB file 8.2GB
Q5_0
7.36GB file 9.54GB
Q5_K_M
7.36GB file 9.54GB
Q5_K_S
7.36GB file 9.54GB
Q6_K
8.69GB file 10.87GB
Q8_0
11.37GB file 13.55GB

PiVoT-10.7B-Mistral-v0.2-RP-GGUF

10.7B 8K ctx
Q2_K
3.34GB file 5.52GB
Q3_K_L
4.68GB file 6.86GB
Q3_K_M
4.68GB file 6.86GB
Q3_K_S
4.68GB file 6.86GB
Q4_0
6.02GB file 8.2GB
Q4_K_M
6.02GB file 8.2GB
Q4_K_S
6.02GB file 8.2GB
Q5_0
7.36GB file 9.54GB
Q5_K_M
7.36GB file 9.54GB
Q5_K_S
7.36GB file 9.54GB
Q6_K
8.69GB file 10.87GB
Q8_0
11.37GB file 13.55GB

PiVoT-Merge-A-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

PlatYi-34B-Llama-Q-v3-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

prometheus-13B-v1.0-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

prometheus-7B-v1.0-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Pygmalion-2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Pygmalion-2-13B-SuperCOT-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Pygmalion-2-13B-SuperCOT-weighed-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Pygmalion-2-13B-SuperCOT2-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Pygmalion-2-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

qCammel-13-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

qCammel-70-x-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB

quantum-dpo-v0.1-GGUF

0.1B 8K ctx
Q2_K
0.03GB file 2.03GB
Q3_K_L
0.04GB file 2.04GB
Q3_K_M
0.04GB file 2.04GB
Q3_K_S
0.04GB file 2.04GB
Q4_0
0.06GB file 2.06GB
Q4_K_M
0.06GB file 2.06GB
Q4_K_S
0.06GB file 2.06GB
Q5_0
0.07GB file 2.07GB
Q5_K_M
0.07GB file 2.07GB
Q5_K_S
0.07GB file 2.07GB
Q6_K
0.08GB file 2.08GB
Q8_0
0.11GB file 2.11GB

quantum-v0.01-GGUF

0.01B 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0.01GB file 2.01GB
Q4_K_M
0.01GB file 2.01GB
Q4_K_S
0.01GB file 2.01GB
Q5_0
0.01GB file 2.01GB
Q5_K_M
0.01GB file 2.01GB
Q5_K_S
0.01GB file 2.01GB
Q6_K
0.01GB file 2.01GB
Q8_0
0.01GB file 2.01GB

Rose-20B-GGUF

20B 8K ctx
Q2_K
6.25GB file 8.58GB
Q3_K_L
8.75GB file 11.08GB
Q3_K_M
8.75GB file 11.08GB
Q3_K_S
8.75GB file 11.08GB
Q4_0
11.25GB file 13.58GB
Q4_K_M
11.25GB file 13.58GB
Q4_K_S
11.25GB file 13.58GB
Q5_0
13.75GB file 16.08GB
Q5_K_M
13.75GB file 16.08GB
Q5_K_S
13.75GB file 16.08GB
Q6_K
16.25GB file 18.58GB
Q8_0
21.25GB file 23.58GB

Sakura-SOLAR-Instruct-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

SauerkrautLM-13B-v1-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

SauerkrautLM-3B-v1-GGUF

3B 8K ctx
Q4_0
1.69GB file 3.74GB
Q4_1
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_1
2.06GB file 4.11GB
Q8_0
3.19GB file 5.24GB

SauerkrautLM-70B-v1-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

SauerkrautLM-7B-HerO-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SauerkrautLM-7B-v1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SauerkrautLM-7B-v1-mistral-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SauerkrautLM-Mixtral-8x7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SauerkrautLM-Mixtral-8x7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_M
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SauerkrautLM-SOLAR-Instruct-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

SG-Raccoon-Yi-200k-2.0-GGUF

55B 8K ctx
Q2_K
17.19GB file 20.09GB
Q3_K_L
24.06GB file 26.96GB
Q3_K_M
24.06GB file 26.96GB
Q3_K_S
24.06GB file 26.96GB
Q4_0
30.94GB file 33.84GB
Q4_K_M
30.94GB file 33.84GB
Q4_K_S
30.94GB file 33.84GB
Q5_0
37.81GB file 40.71GB
Q5_K_M
37.81GB file 40.71GB
Q5_K_S
37.81GB file 40.71GB
Q6_K
44.69GB file 47.59GB

SG-Raccoon-Yi-55B-200k-GGUF

55B 8K ctx
Q2_K
17.19GB file 20.09GB
Q3_K_L
24.06GB file 26.96GB
Q3_K_M
24.06GB file 26.96GB
Q3_K_S
24.06GB file 26.96GB
Q4_0
30.94GB file 33.84GB
Q4_K_M
30.94GB file 33.84GB
Q4_K_S
30.94GB file 33.84GB
Q5_0
37.81GB file 40.71GB
Q5_K_M
37.81GB file 40.71GB
Q5_K_S
37.81GB file 40.71GB
Q6_K
44.69GB file 47.59GB

SG-Raccoon-Yi-55B-GGUF

55B 8K ctx
Q2_K
17.19GB file 20.09GB
Q3_K_L
24.06GB file 26.96GB
Q3_K_M
24.06GB file 26.96GB
Q3_K_S
24.06GB file 26.96GB
Q4_0
30.94GB file 33.84GB
Q4_K_M
30.94GB file 33.84GB
Q4_K_S
30.94GB file 33.84GB
Q5_0
37.81GB file 40.71GB
Q5_K_M
37.81GB file 40.71GB
Q5_K_S
37.81GB file 40.71GB
Q6_K
44.69GB file 47.59GB

Sheep-Duck-Llama-2-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

ShiningValiant-1.2-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

ShiningValiant-1.3-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

ShiningValiantXS-1.1-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

ShiningValiantXS-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

smol-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SOLAR-Platypus-10.7B-v2-GGUF

10.7B 8K ctx
Q2_K
3.34GB file 5.52GB
Q3_K_L
4.68GB file 6.86GB
Q3_K_M
4.68GB file 6.86GB
Q3_K_S
4.68GB file 6.86GB
Q4_0
6.02GB file 8.2GB
Q4_K_M
6.02GB file 8.2GB
Q4_K_S
6.02GB file 8.2GB
Q5_0
7.36GB file 9.54GB
Q5_K_M
7.36GB file 9.54GB
Q5_K_S
7.36GB file 9.54GB
Q6_K
8.69GB file 10.87GB
Q8_0
11.37GB file 13.55GB

SOLARC-MOE-10.7Bx4-GGUF

10.7B 8K ctx
Q2_K
3.34GB file 5.52GB
Q3_K_M
4.68GB file 6.86GB
Q4_0
6.02GB file 8.2GB
Q4_K_M
6.02GB file 8.2GB
Q5_0
7.36GB file 9.54GB
Q5_K_M
7.36GB file 9.54GB
Q6_K
8.69GB file 10.87GB
Q8_0
11.37GB file 13.55GB

speechless-code-mistral-7B-v1.0-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

speechless-codellama-34b-v2.0-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

Speechless-Llama2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Speechless-Llama2-Hermes-Orca-Platypus-WizardLM-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

speechless-mistral-dolphin-orca-platypus-samantha-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

speechless-mistral-moloras-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

speechless-tora-code-7B-v1.0-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

sqlcoder-34b-alpha-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

sqlcoder-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

sqlcoder-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

sqlcoder2-GGUF

2B 8K ctx
Q2_K
0.63GB file 2.66GB
Q3_K_L
0.88GB file 2.91GB
Q3_K_M
0.88GB file 2.91GB
Q3_K_S
0.88GB file 2.91GB
Q4_0
1.13GB file 3.16GB
Q4_K_M
1.13GB file 3.16GB
Q4_K_S
1.13GB file 3.16GB
Q5_0
1.38GB file 3.41GB
Q5_K_M
1.38GB file 3.41GB
Q5_K_S
1.38GB file 3.41GB
Q6_K
1.63GB file 3.66GB
Q8_0
2.13GB file 4.16GB

StableBeluga-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

StableBeluga-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

StableBeluga2-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

StellarBright-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

stockmark-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

SUS-Chat-34B-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

Swallow-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Swallow-13B-Instruct-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Swallow-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Swallow-70B-instruct-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Swallow-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Swallow-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Synatra-7B-v0.3-base-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Synatra-7B-v0.3-RP-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Synatra-V0.1-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Synthia-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Synthia-13B-v1.2-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Synthia-70B-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Synthia-70B-v1.1-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Synthia-7B-v1.2-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SynthIA-7B-v1.5-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SynthIA-7B-v2.0-16k-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

SynthIA-7B-v2.0-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Terminis-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

tora-13B-v1.0-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

tora-70B-v1.0-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

tora-7B-v1.0-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

tora-code-13B-v1.0-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

tora-code-34b-v1.0-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

tora-code-7B-v1.0-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Trurl-2-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

Trurl-2-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

typhoon-7B-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Upstage-Llama-2-70B-instruct-v2-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

upstage-llama-30b-instruct-2048-GGUF

30B 8K ctx
Q2_K
9.38GB file 11.87GB
Q3_K_L
13.13GB file 15.62GB
Q3_K_M
13.13GB file 15.62GB
Q3_K_S
13.13GB file 15.62GB
Q4_0
16.88GB file 19.37GB
Q4_K_M
16.88GB file 19.37GB
Q4_K_S
16.88GB file 19.37GB
Q5_0
20.63GB file 23.12GB
Q5_K_M
20.63GB file 23.12GB
Q5_K_S
20.63GB file 23.12GB
Q6_K
24.38GB file 26.87GB
Q8_0
31.88GB file 34.37GB

Upstage-Llama1-65B-Instruct-GGUF

65B 8K ctx
Q2_K
20.31GB file 23.37GB
Q3_K_L
28.44GB file 31.5GB
Q3_K_M
28.44GB file 31.5GB
Q3_K_S
28.44GB file 31.5GB
Q4_0
36.56GB file 39.62GB
Q4_K_M
36.56GB file 39.62GB
Q4_K_S
36.56GB file 39.62GB
Q5_0
44.69GB file 47.75GB
Q5_K_M
44.69GB file 47.75GB
Q5_K_S
44.69GB file 47.75GB

Velara-11B-V2-GGUF

11B 8K ctx
Q2_K
3.44GB file 5.62GB
Q3_K_L
4.81GB file 6.99GB
Q3_K_M
4.81GB file 6.99GB
Q3_K_S
4.81GB file 6.99GB
Q4_0
6.19GB file 8.37GB
Q4_K_M
6.19GB file 8.37GB
Q4_K_S
6.19GB file 8.37GB
Q5_0
7.56GB file 9.74GB
Q5_K_M
7.56GB file 9.74GB
Q5_K_S
7.56GB file 9.74GB
Q6_K
8.94GB file 11.12GB
Q8_0
11.69GB file 13.87GB

Velara-GGUF

Unknown 8K ctx
Q2_K
0GB file 2GB
Q3_K_L
0GB file 2GB
Q3_K_M
0GB file 2GB
Q3_K_S
0GB file 2GB
Q4_0
0GB file 2GB
Q4_K_M
0GB file 2GB
Q4_K_S
0GB file 2GB
Q5_0
0GB file 2GB
Q5_K_M
0GB file 2GB
Q5_K_S
0GB file 2GB
Q6_K
0GB file 2GB
Q8_0
0GB file 2GB

Vigogne-2-13B-Instruct-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

vigogne-2-70B-chat-GGUF

70B 8K ctx
Q2_K
21.88GB file 25.03GB
Q3_K_L
30.63GB file 33.78GB
Q3_K_M
30.63GB file 33.78GB
Q3_K_S
30.63GB file 33.78GB
Q4_0
39.38GB file 42.53GB
Q4_K_M
39.38GB file 42.53GB
Q4_K_S
39.38GB file 42.53GB
Q5_0
48.13GB file 51.28GB
Q5_K_M
48.13GB file 51.28GB
Q5_K_S
48.13GB file 51.28GB

Vigogne-2-7B-Chat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Vigogne-2-7B-Instruct-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Vigostral-7B-Chat-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

wizard-mega-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

WizardMath-7B-V1.1-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

Yi-34B-200K-DARE-merge-v5-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

Yi-34B-Chat-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB

Yi-6B-GGUF

6B 8K ctx
Q2_K
1.88GB file 3.98GB
Q3_K_L
2.63GB file 4.73GB
Q3_K_M
2.63GB file 4.73GB
Q3_K_S
2.63GB file 4.73GB
Q4_0
3.38GB file 5.48GB
Q4_K_M
3.38GB file 5.48GB
Q4_K_S
3.38GB file 5.48GB
Q5_0
4.13GB file 6.23GB
Q5_K_M
4.13GB file 6.23GB
Q5_K_S
4.13GB file 6.23GB
Q6_K
4.88GB file 6.98GB
Q8_0
6.38GB file 8.48GB

You_can_cry_Snowman-13B-GGUF

13B 8K ctx
Q2_K
4.06GB file 6.27GB
Q3_K_L
5.69GB file 7.9GB
Q3_K_M
5.69GB file 7.9GB
Q3_K_S
5.69GB file 7.9GB
Q4_0
7.31GB file 9.52GB
Q4_K_M
7.31GB file 9.52GB
Q4_K_S
7.31GB file 9.52GB
Q5_0
8.94GB file 11.15GB
Q5_K_M
8.94GB file 11.15GB
Q5_K_S
8.94GB file 11.15GB
Q6_K
10.56GB file 12.77GB
Q8_0
13.81GB file 16.02GB

ZephRP-m7b-GGUF

7B 8K ctx
Q2_K
2.19GB file 4.3GB
Q3_K_L
3.06GB file 5.17GB
Q3_K_M
3.06GB file 5.17GB
Q3_K_S
3.06GB file 5.17GB
Q4_0
3.94GB file 6.05GB
Q4_K_M
3.94GB file 6.05GB
Q4_K_S
3.94GB file 6.05GB
Q5_0
4.81GB file 6.92GB
Q5_K_M
4.81GB file 6.92GB
Q5_K_S
4.81GB file 6.92GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

zephyr-quiklang-3b-4K-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

zephyr-quiklang-3b-GGUF

3B 8K ctx
Q2_K
0.94GB file 2.99GB
Q3_K_L
1.31GB file 3.36GB
Q3_K_M
1.31GB file 3.36GB
Q3_K_S
1.31GB file 3.36GB
Q4_0
1.69GB file 3.74GB
Q4_K_M
1.69GB file 3.74GB
Q4_K_S
1.69GB file 3.74GB
Q5_0
2.06GB file 4.11GB
Q5_K_M
2.06GB file 4.11GB
Q5_K_S
2.06GB file 4.11GB
Q6_K
2.44GB file 4.49GB
Q8_0
3.19GB file 5.24GB

Ziya-Coding-34B-v1.0-GGUF

34B 8K ctx
Q2_K
10.63GB file 13.19GB
Q3_K_L
14.88GB file 17.44GB
Q3_K_M
14.88GB file 17.44GB
Q3_K_S
14.88GB file 17.44GB
Q4_0
19.13GB file 21.69GB
Q4_K_M
19.13GB file 21.69GB
Q4_K_S
19.13GB file 21.69GB
Q5_0
23.38GB file 25.94GB
Q5_K_M
23.38GB file 25.94GB
Q5_K_S
23.38GB file 25.94GB
Q6_K
27.63GB file 30.19GB
Q8_0
36.13GB file 38.69GB
T

The Clanker Lover

1 models

steelman-14b-ada-GGUF

14B 8K ctx
Q4_K_M
7.88GB file 10.11GB
Q8_0
14.88GB file 17.11GB
T

The Ep Tic

2 models

IQuest-Coder-V1-7B-Instruct-GGUF

7B 8K ctx
Q4_K_M
3.94GB file 6.05GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB

IQuest-Coder-V1-7B-Thinking-GGUF

7B 8K ctx
Q4_K_M
3.94GB file 6.05GB
Q6_K
5.69GB file 7.8GB
Q8_0
7.44GB file 9.55GB
T

The Fall Of Man

1 models

nightbloom-v1de-gguf

Unknown 8K ctx
Q4_K_M
0GB file 2GB
T

The Hierophant

1 models

GrayLine-Qwen3-8B-Q8_0-GGUF

8B 8K ctx
Q8_0
8.5GB file 10.63GB