-
-
Notifications
You must be signed in to change notification settings - Fork 2.3k
/
Copy pathindex.yaml
13996 lines (13480 loc) · 790 KB
/
index.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
---
- &phi4
url: "github:mudler/LocalAI/gallery/phi-4-chat.yaml@master"
name: "phi-4"
icon: https://avatars.githubusercontent.com/u/6154722
license: mit
tags:
- llm
- gguf
- phi
- cpu
- gpu
- text-generation
urls:
- https://huggingface.co/microsoft/phi-4
- https://huggingface.co/bartowski/phi-4-GGUF
description: |
phi-4 is a state-of-the-art open model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning.
phi-4 underwent a rigorous enhancement and alignment process, incorporating both supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures. Phi-4 is a 14B parameters, dense decoder-only Transformer model.
overrides:
parameters:
model: phi-4-Q4_K_M.gguf
files:
- filename: phi-4-Q4_K_M.gguf
uri: huggingface://bartowski/phi-4-GGUF/phi-4-Q4_K_M.gguf
sha256: 009aba717c09d4a35890c7d35eb59d54e1dba884c7c526e7197d9c13ab5911d9
- !!merge <<: *phi4
url: "github:mudler/LocalAI/gallery/phi-4-chat-fcall.yaml@master"
name: "LocalAI-functioncall-phi-4-v0.3"
icon: https://cdn-uploads.huggingface.co/production/uploads/647374aa7ff32a81ac6d35d4/Dzbdzn27KEc3K6zNNi070.png
urls:
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.3
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.3-Q4_K_M-GGUF
description: |
A model tailored to be conversational and execute function calls with LocalAI. This model is based on phi-4.
overrides:
parameters:
model: localai-functioncall-phi-4-v0.3-q4_k_m.gguf
files:
- filename: localai-functioncall-phi-4-v0.3-q4_k_m.gguf
sha256: 23fee048ded2a6e2e1a7b6bbefa6cbf83068f194caa9552aecbaa00fec8a16d5
uri: huggingface://mudler/LocalAI-functioncall-phi-4-v0.3-Q4_K_M-GGUF/localai-functioncall-phi-4-v0.3-q4_k_m.gguf
- !!merge <<: *phi4
url: "github:mudler/LocalAI/gallery/phi-4-chat-fcall.yaml@master"
name: "LocalAI-functioncall-phi-4-v0.2"
icon: https://cdn-uploads.huggingface.co/production/uploads/647374aa7ff32a81ac6d35d4/Dzbdzn27KEc3K6zNNi070.png
description: |
A model tailored to be conversational and execute function calls with LocalAI. This model is based on phi-4.
This is the second iteration of https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.1 with added CoT (o1) capabilities from the marco-o1 dataset.
urls:
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.2
- https://huggingface.co/mudler/localai-functioncall-phi-4-v0.2-Q4_K_M-GGUF
overrides:
parameters:
model: localai-functioncall-phi-4-v0.2-q4_k_m.gguf
files:
- filename: localai-functioncall-phi-4-v0.2-q4_k_m.gguf
uri: huggingface://mudler/localai-functioncall-phi-4-v0.2-Q4_K_M-GGUF/localai-functioncall-phi-4-v0.2-q4_k_m.gguf
sha256: 681b5fb5070f23323a9cc8cbd1306b1c348c2f292041d3ba2335b26b071757b7
- !!merge <<: *phi4
url: "github:mudler/LocalAI/gallery/phi-4-chat-fcall.yaml@master"
name: "LocalAI-functioncall-phi-4-v0.1"
icon: https://cdn-uploads.huggingface.co/production/uploads/647374aa7ff32a81ac6d35d4/Dzbdzn27KEc3K6zNNi070.png
description: |
A model tailored to be conversational and execute function calls with LocalAI. This model is based on phi-4.
urls:
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.1
- https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.1-Q4_K_M-GGUF
overrides:
parameters:
model: localai-functioncall-phi-4-v0.1-q4_k_m.gguf
files:
- filename: localai-functioncall-phi-4-v0.1-q4_k_m.gguf
uri: huggingface://mudler/LocalAI-functioncall-phi-4-v0.1-Q4_K_M-GGUF/localai-functioncall-phi-4-v0.1-q4_k_m.gguf
sha256: 0ae4e5e4ba89c16c1e810285c5c8b84416fa67f8ed7c175aa0b6fc0a103017aa
- !!merge <<: *phi4
name: "sicariussicariistuff_phi-lthy4"
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
icon: https://huggingface.co/SicariusSicariiStuff/Phi-lthy4/resolve/main/Images/Phi-Lthy4.png
urls:
- https://huggingface.co/SicariusSicariiStuff/Phi-lthy4
- https://huggingface.co/bartowski/SicariusSicariiStuff_Phi-lthy4-GGUF
description: |
- The BEST Phi-4 Roleplay finetune in the world (Not that much of an achievement here, Phi roleplay finetunes can probably be counted on a single hand).
- Compact size & fully healed from the brain surgery Only 11.9B parameters. Phi-4 wasn't that hard to run even at 14B, now with even fewer brain cells, your new phone could probably run it easily. (SD8Gen3 and above recommended).
- Strong Roleplay & Creative writing abilities. This really surprised me. Actually good.
Writes and roleplays quite uniquely, probably because of lack of RP\writing slop in the pretrain. Who would have thought?
- Smart assistant with low refusals - It kept some of the smarts, and our little Phi-Lthy here will be quite eager to answer your naughty questions.
- Quite good at following the character card. Finally, it puts its math brain to some productive tasks. Gooner technology is becoming more popular by the day.
overrides:
parameters:
model: SicariusSicariiStuff_Phi-lthy4-Q4_K_M.gguf
files:
- filename: SicariusSicariiStuff_Phi-lthy4-Q4_K_M.gguf
sha256: a5004b2d0f3eb869f07285f53ec283aa383063085987113d2a41cb54708fb5ad
uri: huggingface://bartowski/SicariusSicariiStuff_Phi-lthy4-GGUF/SicariusSicariiStuff_Phi-lthy4-Q4_K_M.gguf
- !!merge <<: *phi4
name: "sicariussicariistuff_phi-line_14b"
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
icon: https://huggingface.co/SicariusSicariiStuff/Phi-Line_14B/resolve/main/Images/Phi-Line_14B.png
urls:
- https://huggingface.co/SicariusSicariiStuff/Phi-Line_14B
- https://huggingface.co/bartowski/SicariusSicariiStuff_Phi-Line_14B-GGUF
description: |
Excellent Roleplay with more brains. (Who would have thought Phi-4 models would be good at this? so weird... )
Medium length response (1-4 paragraphs, usually 2-3).
Excellent assistant that follows instructions well enough, and keeps good formating.
Strong Creative writing abilities. Will obey requests regarding formatting (markdown headlines for paragraphs, etc).
Writes and roleplays quite uniquely, probably because of lack of RP\writing slop in the pretrain. This is just my guesstimate.
LOW refusals - Total freedom in RP, can do things other RP models won't, and I'll leave it at that. Low refusals in assistant tasks as well.
VERY good at following the character card. Math brain is used for gooner tech, as it should be.
overrides:
parameters:
model: SicariusSicariiStuff_Phi-Line_14B-Q4_K_M.gguf
files:
- filename: SicariusSicariiStuff_Phi-Line_14B-Q4_K_M.gguf
sha256: 552c5a613bc5f24494646858795837ac42d3c216c5caedd7f4d6b954e5df58f2
uri: huggingface://bartowski/SicariusSicariiStuff_Phi-Line_14B-GGUF/SicariusSicariiStuff_Phi-Line_14B-Q4_K_M.gguf
- !!merge <<: *phi4
name: "microsoft_phi-4-mini-instruct"
urls:
- https://huggingface.co/microsoft/Phi-4-mini-instruct
- https://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF
description: |
Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model belongs to the Phi-4 model family and supports 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning and direct preference optimization to support precise instruction adherence and robust safety measures.
overrides:
parameters:
model: microsoft_Phi-4-mini-instruct-Q4_K_M.gguf
files:
- filename: microsoft_Phi-4-mini-instruct-Q4_K_M.gguf
sha256: 01999f17c39cc3074afae5e9c539bc82d45f2dd7faa3917c66cbef76fce8c0c2
uri: huggingface://bartowski/microsoft_Phi-4-mini-instruct-GGUF/microsoft_Phi-4-mini-instruct-Q4_K_M.gguf
- &falcon3
name: "falcon3-1b-instruct"
url: "github:mudler/LocalAI/gallery/falcon3.yaml@master"
icon: https://huggingface.co/datasets/tiiuae/documentation-images/resolve/main/general/falco3-logo.png
urls:
- https://huggingface.co/tiiuae/Falcon3-1B-Instruct
- https://huggingface.co/bartowski/Falcon3-1B-Instruct-GGUF
description: |
Falcon3 family of Open Foundation Models is a set of pretrained and instruct LLMs ranging from 1B to 10B parameters.
This repository contains the Falcon3-1B-Instruct. It achieves strong results on reasoning, language understanding, instruction following, code and mathematics tasks. Falcon3-1B-Instruct supports 4 languages (English, French, Spanish, Portuguese) and a context length of up to 8K.
overrides:
parameters:
model: Falcon3-1B-Instruct-Q4_K_M.gguf
files:
- filename: Falcon3-1B-Instruct-Q4_K_M.gguf
uri: huggingface://bartowski/Falcon3-1B-Instruct-GGUF/Falcon3-1B-Instruct-Q4_K_M.gguf
sha256: 1c92013dac1ab6e703e787f3e0829ca03cc95311e4c113a77950d15ff6dea7b3
tags:
- llm
- gguf
- gpu
- cpu
- falcon
license: falcon-llm
- !!merge <<: *falcon3
name: "falcon3-3b-instruct"
urls:
- https://huggingface.co/tiiuae/Falcon3-3B-Instruct
- https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF
overrides:
parameters:
model: Falcon3-3B-Instruct-Q4_K_M.gguf
files:
- filename: Falcon3-3B-Instruct-Q4_K_M.gguf
uri: huggingface://bartowski/Falcon3-3B-Instruct-GGUF/Falcon3-3B-Instruct-Q4_K_M.gguf
sha256: 6ea6cecba144fe5b711ca07ae4263ccdf6ee6419807a46220419189da8446557
- !!merge <<: *falcon3
name: "falcon3-10b-instruct"
urls:
- https://huggingface.co/tiiuae/Falcon3-10B-Instruct
- https://huggingface.co/bartowski/Falcon3-10B-Instruct-GGUF
overrides:
parameters:
model: Falcon3-10B-Instruct-Q4_K_M.gguf
files:
- filename: Falcon3-10B-Instruct-Q4_K_M.gguf
uri: huggingface://bartowski/Falcon3-10B-Instruct-GGUF/Falcon3-10B-Instruct-Q4_K_M.gguf
sha256: 0a33327bd71e1788a8e9f17889824a17a65efd3f96a4b2a5e2bc6ff2f39b8241
- !!merge <<: *falcon3
name: "falcon3-1b-instruct-abliterated"
urls:
- https://huggingface.co/huihui-ai/Falcon3-1B-Instruct-abliterated
- https://huggingface.co/bartowski/Falcon3-1B-Instruct-abliterated-GGUF
description: |
This is an uncensored version of tiiuae/Falcon3-1B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
overrides:
parameters:
model: Falcon3-1B-Instruct-abliterated-Q4_K_M.gguf
files:
- filename: Falcon3-1B-Instruct-abliterated-Q4_K_M.gguf
sha256: 416d15ce58334b7956818befb088d46c1e3e7153ebf2da2fb9769a5b1ff934a1
uri: huggingface://bartowski/Falcon3-1B-Instruct-abliterated-GGUF/Falcon3-1B-Instruct-abliterated-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "falcon3-3b-instruct-abliterated"
urls:
- https://huggingface.co/huihui-ai/Falcon3-3B-Instruct-abliterated
- https://huggingface.co/bartowski/Falcon3-3B-Instruct-abliterated-GGUF
description: |
This is an uncensored version of tiiuae/Falcon3-3B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
overrides:
parameters:
model: Falcon3-3B-Instruct-abliterated-Q4_K_M.gguf
files:
- filename: Falcon3-3B-Instruct-abliterated-Q4_K_M.gguf
sha256: 83773b77b0e34ef115f8a6508192e9f1d3426a61456744493f65cfe1e7f90aa9
uri: huggingface://bartowski/Falcon3-3B-Instruct-abliterated-GGUF/Falcon3-3B-Instruct-abliterated-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "falcon3-10b-instruct-abliterated"
urls:
- https://huggingface.co/huihui-ai/Falcon3-10B-Instruct-abliterated
- https://huggingface.co/bartowski/Falcon3-10B-Instruct-abliterated-GGUF
description: |
This is an uncensored version of tiiuae/Falcon3-10B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
overrides:
parameters:
model: Falcon3-10B-Instruct-abliterated-Q4_K_M.gguf
files:
- filename: Falcon3-10B-Instruct-abliterated-Q4_K_M.gguf
sha256: 5940df2ff88e5be93dbe0766b2a9683d7e73c204a69a1348a37f835cf2b5f767
uri: huggingface://bartowski/Falcon3-10B-Instruct-abliterated-GGUF/Falcon3-10B-Instruct-abliterated-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "falcon3-7b-instruct-abliterated"
urls:
- https://huggingface.co/huihui-ai/Falcon3-7B-Instruct-abliterated
- https://huggingface.co/bartowski/Falcon3-7B-Instruct-abliterated-GGUF
description: |
This is an uncensored version of tiiuae/Falcon3-7B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
overrides:
parameters:
model: Falcon3-7B-Instruct-abliterated-Q4_K_M.gguf
files:
- filename: Falcon3-7B-Instruct-abliterated-Q4_K_M.gguf
sha256: 68e10e638668acaa49fb7919224c7d8bcf1798126c7a499c4d9ec3b81313f8c8
uri: huggingface://bartowski/Falcon3-7B-Instruct-abliterated-GGUF/Falcon3-7B-Instruct-abliterated-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "nightwing3-10b-v0.1"
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
icon: https://cdn-uploads.huggingface.co/production/uploads/642265bc01c62c1e4102dc36/C6gY9vxCl3_SFzQLpLG0S.png
urls:
- https://huggingface.co/Nitral-AI/NightWing3-10B-v0.1
- https://huggingface.co/bartowski/NightWing3-10B-v0.1-GGUF
description: |
Base model: (Falcon3-10B)
overrides:
parameters:
model: NightWing3-10B-v0.1-Q4_K_M.gguf
files:
- filename: NightWing3-10B-v0.1-Q4_K_M.gguf
sha256: 2e87671542d22fe1ef9a68e43f2fdab7c2759479ad531946d9f0bdeffa6f5747
uri: huggingface://bartowski/NightWing3-10B-v0.1-GGUF/NightWing3-10B-v0.1-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "virtuoso-lite"
urls:
- https://huggingface.co/arcee-ai/Virtuoso-Lite
- https://huggingface.co/bartowski/Virtuoso-Lite-GGUF
description: |
Virtuoso-Lite (10B) is our next-generation, 10-billion-parameter language model based on the Llama-3 architecture. It is distilled from Deepseek-v3 using ~1.1B tokens/logits, allowing it to achieve robust performance at a significantly reduced parameter count compared to larger models. Despite its compact size, Virtuoso-Lite excels in a variety of tasks, demonstrating advanced reasoning, code generation, and mathematical problem-solving capabilities.
overrides:
parameters:
model: Virtuoso-Lite-Q4_K_M.gguf
files:
- filename: Virtuoso-Lite-Q4_K_M.gguf
sha256: 1d21bef8467a11a1e473d397128b05fb87b7e824606cdaea061e550cb219fee2
uri: huggingface://bartowski/Virtuoso-Lite-GGUF/Virtuoso-Lite-Q4_K_M.gguf
- !!merge <<: *falcon3
name: "suayptalha_maestro-10b"
icon: https://huggingface.co/suayptalha/Maestro-10B/resolve/main/Maestro-Logo.png
urls:
- https://huggingface.co/suayptalha/Maestro-10B
- https://huggingface.co/bartowski/suayptalha_Maestro-10B-GGUF
description: |
Maestro-10B is a 10 billion parameter model fine-tuned from Virtuoso-Lite, a next-generation language model developed by arcee-ai. Virtuoso-Lite itself is based on the Llama-3 architecture, distilled from Deepseek-v3 using approximately 1.1 billion tokens/logits. This distillation process allows Virtuoso-Lite to achieve robust performance with a smaller parameter count, excelling in reasoning, code generation, and mathematical problem-solving. Maestro-10B inherits these strengths from its base model, Virtuoso-Lite, and further enhances them through fine-tuning on the OpenOrca dataset. This combination of a distilled base model and targeted fine-tuning makes Maestro-10B a powerful and efficient language model.
overrides:
parameters:
model: suayptalha_Maestro-10B-Q4_K_M.gguf
files:
- filename: suayptalha_Maestro-10B-Q4_K_M.gguf
sha256: c570381da5624782ce6df4186ace6f747429fcbaf1a22c2a348288d3552eb19c
uri: huggingface://bartowski/suayptalha_Maestro-10B-GGUF/suayptalha_Maestro-10B-Q4_K_M.gguf
- &intellect1
name: "intellect-1-instruct"
url: "github:mudler/LocalAI/gallery/llama3.1-instruct.yaml@master"
icon: https://huggingface.co/PrimeIntellect/INTELLECT-1-Instruct/resolve/main/intellect-1-map.png
urls:
- https://huggingface.co/PrimeIntellect/INTELLECT-1-Instruct
- https://huggingface.co/bartowski/INTELLECT-1-Instruct-GGUF
tags:
- llm
- gguf
- gpu
- cpu
- intellect
license: apache-2.0
description: |
INTELLECT-1 is the first collaboratively trained 10 billion parameter language model trained from scratch on 1 trillion tokens of English text and code.
This is an instruct model. The base model associated with it is INTELLECT-1.
INTELLECT-1 was trained on up to 14 concurrent nodes distributed across 3 continents, with contributions from 30 independent community contributors providing compute. The training code utilizes the prime framework, a scalable distributed training framework designed for fault-tolerant, dynamically scaling, high-perfomance training on unreliable, globally distributed workers. The key abstraction that allows dynamic scaling is the ElasticDeviceMesh which manages dynamic global process groups for fault-tolerant communication across the internet and local process groups for communication within a node. The model was trained using the DiLoCo algorithms with 100 inner steps. The global all-reduce was done with custom int8 all-reduce kernels to reduce the communication payload required, greatly reducing the communication overhead by a factor 400x.
overrides:
parameters:
model: INTELLECT-1-Instruct-Q4_K_M.gguf
files:
- filename: INTELLECT-1-Instruct-Q4_K_M.gguf
sha256: 5df236fe570e5998d07fb3207788eac811ef3b77dd2a0ad04a2ef5c6361f3030
uri: huggingface://bartowski/INTELLECT-1-Instruct-GGUF/INTELLECT-1-Instruct-Q4_K_M.gguf
- &llama33
url: "github:mudler/LocalAI/gallery/llama3.1-instruct.yaml@master"
icon: https://avatars.githubusercontent.com/u/153379578
license: llama3.3
description: |
The Meta Llama 3.3 multilingual large language model (LLM) is a pretrained and instruction tuned generative model in 70B (text in/text out). The Llama 3.3 instruction tuned text only model is optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.
tags:
- llm
- gguf
- gpu
- cpu
- llama3.3
name: "llama-3.3-70b-instruct"
urls:
- https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct
- https://huggingface.co/MaziyarPanahi/Llama-3.3-70B-Instruct-GGUF
overrides:
parameters:
model: Llama-3.3-70B-Instruct.Q4_K_M.gguf
files:
- filename: Llama-3.3-70B-Instruct.Q4_K_M.gguf
sha256: 4f3b04ecae278bdb0fd545b47c210bc5edf823e5ebf7d41e0b526c81d54b1ff3
uri: huggingface://MaziyarPanahi/Llama-3.3-70B-Instruct-GGUF/Llama-3.3-70B-Instruct.Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-70b-euryale-v2.3"
icon: https://huggingface.co/Sao10K/L3.3-70B-Euryale-v2.3/resolve/main/Eury.png
urls:
- https://huggingface.co/Sao10K/L3.3-70B-Euryale-v2.3
- https://huggingface.co/bartowski/L3.3-70B-Euryale-v2.3-GGUF
description: |
A direct replacement / successor to Euryale v2.2, not Hanami-x1, though it is slightly better than them in my opinion.
overrides:
parameters:
model: L3.3-70B-Euryale-v2.3-Q4_K_M.gguf
files:
- filename: L3.3-70B-Euryale-v2.3-Q4_K_M.gguf
sha256: 4e78bb0e65886bfcff89b829f6d38aa6f6846988bb8291857e387e3f60b3217b
uri: huggingface://bartowski/L3.3-70B-Euryale-v2.3-GGUF/L3.3-70B-Euryale-v2.3-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-ms-evayale-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/HFCaVzRpiE05Y46p41qRy.webp
urls:
- https://huggingface.co/Steelskull/L3.3-MS-Evayale-70B
- https://huggingface.co/bartowski/L3.3-MS-Evayale-70B-GGUF
description: |
This model was created as I liked the storytelling of EVA but the prose and details of scenes from EURYALE, my goal is to merge the robust storytelling of both models while attempting to maintain the positives of both models.
overrides:
parameters:
model: L3.3-MS-Evayale-70B-Q4_K_M.gguf
files:
- filename: L3.3-MS-Evayale-70B-Q4_K_M.gguf
sha256: f941d88870fec8343946517a1802d159d23f3971eeea50b6cf12295330bd29cc
uri: huggingface://bartowski/L3.3-MS-Evayale-70B-GGUF/L3.3-MS-Evayale-70B-Q4_K_M.gguf
- !!merge <<: *llama33
name: "anubis-70b-v1"
icon: https://cdn-uploads.huggingface.co/production/uploads/65f2fd1c25b848bd061b5c2e/qQbZvnrWYvH8dMZORLBJn.webp
urls:
- https://huggingface.co/TheDrummer/Anubis-70B-v1
- https://huggingface.co/bartowski/Anubis-70B-v1-GGUF
description: |
It's a very balanced model between the L3.3 tunes. It's very creative, able to come up with new and interesting scenarios on your own that will thoroughly surprise you in ways that remind me of a 123B model. It has some of the most natural sounding dialogue and prose can come out of any model I've tried with the right swipe, in a way that truly brings your characters and RP to life that makes you feel like you're talking to a human writer instead of an AI - a quality that reminds me of Character AI in its prime. This model loves a great prompt and thrives off instructions.
overrides:
parameters:
model: Anubis-70B-v1-Q4_K_M.gguf
files:
- filename: Anubis-70B-v1-Q4_K_M.gguf
sha256: 9135f7090c675726469bd3a108cfbdddaa18638bad8e513928410de4b8bfd4d4
uri: huggingface://bartowski/Anubis-70B-v1-GGUF/Anubis-70B-v1-Q4_K_M.gguf
- !!merge <<: *llama33
name: "llama-3.3-70b-instruct-ablated"
icon: https://cdn-uploads.huggingface.co/production/uploads/6587d8dd1b44d0e694104fbf/0dkt6EhZYwXVBxvSWXdaM.png
urls:
- https://huggingface.co/NaniDAO/Llama-3.3-70B-Instruct-ablated
- https://huggingface.co/bartowski/Llama-3.3-70B-Instruct-ablated-GGUF
description: |
Llama 3.3 instruct 70B 128k context with ablation technique applied for a more helpful (and based) assistant.
This means it will refuse less of your valid requests for an uncensored UX. Use responsibly and use common sense.
We do not take any responsibility for how you apply this intelligence, just as we do not for how you apply your own.
overrides:
parameters:
model: Llama-3.3-70B-Instruct-ablated-Q4_K_M.gguf
files:
- filename: Llama-3.3-70B-Instruct-ablated-Q4_K_M.gguf
sha256: 090b2288810c5f6f680ff5cb4bc97665393d115c011fcd54dca6aec02e74a983
uri: huggingface://bartowski/Llama-3.3-70B-Instruct-ablated-GGUF/Llama-3.3-70B-Instruct-ablated-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-ms-evalebis-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/e49ykknqXee3Ihr-3BIl_.png
urls:
- https://huggingface.co/Steelskull/L3.3-MS-Evalebis-70b
- https://huggingface.co/bartowski/L3.3-MS-Evalebis-70b-GGUF
description: |
This model was created as I liked the storytelling of EVA, the prose and details of scenes from EURYALE and Anubis, my goal is to merge the robust storytelling of all three models while attempting to maintain the positives of the models.
overrides:
parameters:
model: L3.3-MS-Evalebis-70b-Q4_K_M.gguf
files:
- filename: L3.3-MS-Evalebis-70b-Q4_K_M.gguf
sha256: 5515110ab6a583f6eb360533e3c5b3dda6d402af407c0b0f2b34a2a57b5224d5
uri: huggingface://bartowski/L3.3-MS-Evalebis-70b-GGUF/L3.3-MS-Evalebis-70b-Q4_K_M.gguf
- !!merge <<: *llama33
name: "rombos-llm-70b-llama-3.3"
icon: "https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/QErypCEKD5OZLxUcSmYaR.jpeg"
urls:
- https://huggingface.co/rombodawg/Rombos-LLM-70b-Llama-3.3
- https://huggingface.co/bartowski/Rombos-LLM-70b-Llama-3.3-GGUF
- https://docs.google.com/document/d/1OjbjU5AOz4Ftn9xHQrX3oFQGhQ6RDUuXQipnQ9gn6tU/edit?usp=sharing
description: |
You know the drill by now.
Here is the paper. Have fun.
https://docs.google.com/document/d/1OjbjU5AOz4Ftn9xHQrX3oFQGhQ6RDUuXQipnQ9gn6tU/edit?usp=sharing
overrides:
parameters:
model: Rombos-LLM-70b-Llama-3.3-Q4_K_M.gguf
files:
- filename: Rombos-LLM-70b-Llama-3.3-Q4_K_M.gguf
uri: huggingface://bartowski/Rombos-LLM-70b-Llama-3.3-GGUF/Rombos-LLM-70b-Llama-3.3-Q4_K_M.gguf
sha256: 613008b960f6fff346b5dec71a87cd7ecdaff205bfea6332bd8fe2bb46177352
- !!merge <<: *llama33
name: "70b-l3.3-cirrus-x1"
icon: https://huggingface.co/Sao10K/70B-L3.3-Cirrus-x1/resolve/main/venti.png
urls:
- https://huggingface.co/Sao10K/70B-L3.3-Cirrus-x1
- https://huggingface.co/bartowski/70B-L3.3-Cirrus-x1-GGUF
description: |
- Same data composition as Freya, applied differently, trained longer too.
- Merging with its checkpoints was also involved.
- Has a nice style, with occasional issues that can be easily fixed.
- A more stable version compared to previous runs.
overrides:
parameters:
model: 70B-L3.3-Cirrus-x1-Q4_K_M.gguf
files:
- filename: 70B-L3.3-Cirrus-x1-Q4_K_M.gguf
sha256: 07dd464dddba959df8eb2f937787c2210b4c51c2375bd7c7ab2abbe198142a19
uri: huggingface://bartowski/70B-L3.3-Cirrus-x1-GGUF/70B-L3.3-Cirrus-x1-Q4_K_M.gguf
- !!merge <<: *llama33
name: "negative_llama_70b"
icon: https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B/resolve/main/Images/Negative_LLAMA_70B.png
urls:
- https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B
- https://huggingface.co/bartowski/Negative_LLAMA_70B-GGUF
description: |
- Strong Roleplay & Creative writing abilities.
- Less positivity bias.
- Very smart assistant with low refusals.
- Exceptionally good at following the character card.
- Characters feel more 'alive', and will occasionally initiate stuff on their own (without being prompted to, but fitting to their character).
- Strong ability to comprehend and roleplay uncommon physical and mental characteristics.
overrides:
parameters:
model: Negative_LLAMA_70B-Q4_K_M.gguf
files:
- filename: Negative_LLAMA_70B-Q4_K_M.gguf
sha256: 023c6bd38f6a66178529e6bb77b6e76379ae3ee031adc6885531986aa12750d9
uri: huggingface://bartowski/Negative_LLAMA_70B-GGUF/Negative_LLAMA_70B-Q4_K_M.gguf
- !!merge <<: *llama33
name: "negative-anubis-70b-v1"
icon: https://huggingface.co/knifeayumu/Negative-Anubis-70B-v1/resolve/main/Negative-Anubis.png
urls:
- https://huggingface.co/knifeayumu/Negative-Anubis-70B-v1
- https://huggingface.co/bartowski/Negative-Anubis-70B-v1-GGUF
description: |
Enjoyed SicariusSicariiStuff/Negative_LLAMA_70B but the prose was too dry for my tastes. So I merged it with TheDrummer/Anubis-70B-v1 for verbosity. Anubis has positivity bias so Negative could balance things out.
This is a merge of pre-trained language models created using mergekit.
The following models were included in the merge:
SicariusSicariiStuff/Negative_LLAMA_70B
TheDrummer/Anubis-70B-v1
overrides:
parameters:
model: Negative-Anubis-70B-v1-Q4_K_M.gguf
files:
- filename: Negative-Anubis-70B-v1-Q4_K_M.gguf
sha256: ac088da9ca70fffaa70c876fbada9fc5a02e7d6049ef68f16b11a9c3256f2510
uri: huggingface://bartowski/Negative-Anubis-70B-v1-GGUF/Negative-Anubis-70B-v1-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-ms-nevoria-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/dtlCF4LbekmDD2y3LNpdH.jpeg
urls:
- https://huggingface.co/Steelskull/L3.3-MS-Nevoria-70b
- https://huggingface.co/bartowski/L3.3-MS-Nevoria-70b-GGUF
description: |
This model was created as I liked the storytelling of EVA, the prose and details of scenes from EURYALE and Anubis, enhanced with Negative_LLAMA to kill off the positive bias with a touch of nemotron sprinkeled in.
The choice to use the lorablated model as a base was intentional - while it might seem counterintuitive, this approach creates unique interactions between the weights, similar to what was achieved in the original Astoria model and Astoria V2 model . Rather than simply removing refusals, this "weight twisting" effect that occurs when subtracting the lorablated base model from the other models during the merge process creates an interesting balance in the final model's behavior. While this approach differs from traditional sequential application of components, it was chosen for its unique characteristics in the model's responses.
overrides:
parameters:
model: L3.3-MS-Nevoria-70b-Q4_K_M.gguf
files:
- filename: L3.3-MS-Nevoria-70b-Q4_K_M.gguf
sha256: e8b0763f263089a19d4b112b7ed5085cc5f1ed9ca49c5085baa8d51f4ded1f94
uri: huggingface://bartowski/L3.3-MS-Nevoria-70b-GGUF/L3.3-MS-Nevoria-70b-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-70b-magnum-v4-se"
urls:
- https://huggingface.co/Doctor-Shotgun/L3.3-70B-Magnum-v4-SE
- https://huggingface.co/bartowski/L3.3-70B-Magnum-v4-SE-GGUF
description: |
The Magnum v4 series is complete, but here's something a little extra I wanted to tack on as I wasn't entirely satisfied with the results of v4 72B. "SE" for Special Edition - this model is finetuned from meta-llama/Llama-3.3-70B-Instruct as an rsLoRA adapter. The dataset is a slightly revised variant of the v4 data with some elements of the v2 data re-introduced.
The objective, as with the other Magnum models, is to emulate the prose style and quality of the Claude 3 Sonnet/Opus series of models on a local scale, so don't be surprised to see "Claude-isms" in its output.
overrides:
parameters:
model: L3.3-70B-Magnum-v4-SE-Q4_K_M.gguf
files:
- filename: L3.3-70B-Magnum-v4-SE-Q4_K_M.gguf
sha256: 9724a6364a42caa3d5a1687258eb329c9af6cbb2ce01c8dd556c1a222a2e0352
uri: huggingface://bartowski/L3.3-70B-Magnum-v4-SE-GGUF/L3.3-70B-Magnum-v4-SE-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-prikol-70b-v0.2"
icon: https://files.catbox.moe/x9t3zo.png
urls:
- https://huggingface.co/Nohobby/L3.3-Prikol-70B-v0.2
- https://huggingface.co/bartowski/L3.3-Prikol-70B-v0.2-GGUF
description: |
A merge of some Llama 3.3 models because um uh yeah
Went extra schizo on the recipe, hoping for an extra fun result, and... Well, I guess it's an overall improvement over the previous revision. It's a tiny bit smarter, has even more distinct swipes and nice dialogues, but for some reason it's damn sloppy.
I've published the second step of this merge as a separate model, and I'd say the results are more interesting, but not as usable as this one. https://huggingface.co/Nohobby/AbominationSnowPig
Prompt format: Llama3 OR Llama3 Context and ChatML Instruct. It actually works a bit better this way
overrides:
parameters:
model: L3.3-Prikol-70B-v0.2-Q4_K_M.gguf
files:
- filename: L3.3-Prikol-70B-v0.2-Q4_K_M.gguf
sha256: fc0ff514efbc0b67981c2bf1423d5a2e1b8801e4266ba0c653ea148414fe5ffc
uri: huggingface://bartowski/L3.3-Prikol-70B-v0.2-GGUF/L3.3-Prikol-70B-v0.2-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-nevoria-r1-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/_oWpsvCZ-graNKzJBBjGo.jpeg
urls:
- https://huggingface.co/Steelskull/L3.3-Nevoria-R1-70b
- https://huggingface.co/bartowski/L3.3-Nevoria-R1-70b-GGUF
description: |
This model builds upon the original Nevoria foundation, incorporating the Deepseek-R1 reasoning architecture to enhance dialogue interaction and scene comprehension. While maintaining Nevoria's core strengths in storytelling and scene description (derived from EVA, EURYALE, and Anubis), this iteration aims to improve prompt adherence and creative reasoning capabilities. The model also retains the balanced perspective introduced by Negative_LLAMA and Nemotron elements. Also, the model plays the card to almost a fault, It'll pick up on minor issues and attempt to run with them. Users had it call them out for misspelling a word while playing in character.
Note: While Nevoria-R1 represents a significant architectural change, rather than a direct successor to Nevoria, it operates as a distinct model with its own characteristics.
The lorablated model base choice was intentional, creating unique weight interactions similar to the original Astoria model and Astoria V2 model. This "weight twisting" effect, achieved by subtracting the lorablated base model during merging, creates an interesting balance in the model's behavior. While unconventional compared to sequential component application, this approach was chosen for its unique response characteristics.
overrides:
parameters:
model: L3.3-Nevoria-R1-70b-Q4_K_M.gguf
files:
- filename: L3.3-Nevoria-R1-70b-Q4_K_M.gguf
sha256: 9f32f202fb5b1465c942693bb11eea9e8a1c5686b00602715b495c068eaf1c58
uri: huggingface://bartowski/L3.3-Nevoria-R1-70b-GGUF/L3.3-Nevoria-R1-70b-Q4_K_M.gguf
- !!merge <<: *llama33
name: "nohobby_l3.3-prikol-70b-v0.4"
icon: https://files.catbox.moe/x9t3zo.png
urls:
- https://huggingface.co/Nohobby/L3.3-Prikol-70B-v0.4
- https://huggingface.co/bartowski/Nohobby_L3.3-Prikol-70B-v0.4-GGUF
description: |
I have yet to try it UPD: it sucks, bleh
Sometimes mistakes {{user}} for {{char}} and can't think. Other than that, the behavior is similar to the predecessors.
It sometimes gives some funny replies tho, yay!
overrides:
parameters:
model: Nohobby_L3.3-Prikol-70B-v0.4-Q4_K_M.gguf
files:
- filename: Nohobby_L3.3-Prikol-70B-v0.4-Q4_K_M.gguf
sha256: e1d67a40bdf0526bdfcaa16c6e4dfeecad41651e201b4009b65f4f444b773604
uri: huggingface://bartowski/Nohobby_L3.3-Prikol-70B-v0.4-GGUF/Nohobby_L3.3-Prikol-70B-v0.4-Q4_K_M.gguf
- !!merge <<: *llama33
name: "arliai_llama-3.3-70b-arliai-rpmax-v1.4"
urls:
- https://huggingface.co/ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4
- https://huggingface.co/bartowski/ArliAI_Llama-3.3-70B-ArliAI-RPMax-v1.4-GGUF
description: |
RPMax is a series of models that are trained on a diverse set of curated creative writing and RP datasets with a focus on variety and deduplication. This model is designed to be highly creative and non-repetitive by making sure no two entries in the dataset have repeated characters or situations, which makes sure the model does not latch on to a certain personality and be capable of understanding and acting appropriately to any characters or situations.
overrides:
parameters:
model: ArliAI_Llama-3.3-70B-ArliAI-RPMax-v1.4-Q4_K_M.gguf
files:
- filename: ArliAI_Llama-3.3-70B-ArliAI-RPMax-v1.4-Q4_K_M.gguf
sha256: 7c79e76e5c057cfe32529d930360fbebd29697948e5bac4e4b2eb6d2ee596e31
uri: huggingface://bartowski/ArliAI_Llama-3.3-70B-ArliAI-RPMax-v1.4-GGUF/ArliAI_Llama-3.3-70B-ArliAI-RPMax-v1.4-Q4_K_M.gguf
- !!merge <<: *llama33
name: "black-ink-guild_pernicious_prophecy_70b"
icon: https://huggingface.co/Black-Ink-Guild/Pernicious_Prophecy_70B/resolve/main/header.gif
urls:
- https://huggingface.co/Black-Ink-Guild/Pernicious_Prophecy_70B
- https://huggingface.co/bartowski/Black-Ink-Guild_Pernicious_Prophecy_70B-GGUF
description: |
Pernicious Prophecy 70B is a Llama-3.3 70B-based, two-step model designed by Black Ink Guild (SicariusSicariiStuff and invisietch) for uncensored roleplay, assistant tasks, and general usage.
NOTE: Pernicious Prophecy 70B is an uncensored model and can produce deranged, offensive, and dangerous outputs. You are solely responsible for anything that you choose to do with this model.
overrides:
parameters:
model: Black-Ink-Guild_Pernicious_Prophecy_70B-Q4_K_M.gguf
files:
- filename: Black-Ink-Guild_Pernicious_Prophecy_70B-Q4_K_M.gguf
sha256: d8d4874b837993546b750db3faf1c6e5d867883a6750f04f1f4986973d7c107b
uri: huggingface://bartowski/Black-Ink-Guild_Pernicious_Prophecy_70B-GGUF/Black-Ink-Guild_Pernicious_Prophecy_70B-Q4_K_M.gguf
- !!merge <<: *llama33
name: "nohobby_l3.3-prikol-70b-v0.5"
icon: https://files.catbox.moe/x9t3zo.png
urls:
- https://huggingface.co/Nohobby/L3.3-Prikol-70B-v0.5
- https://huggingface.co/bartowski/Nohobby_L3.3-Prikol-70B-v0.5-GGUF
description: |
99% of mergekit addicts quit before they hit it big.
Gosh, I need to create an org for my test runs - my profile looks like a dumpster.
What was it again? Ah, the new model.
Exactly what I wanted. All I had to do was yank out the cursed official DeepSeek distill and here we are.
From the brief tests it gave me some unusual takes on the character cards I'm used to. Just this makes it worth it imo. Also the writing is kinda nice.
overrides:
parameters:
model: Nohobby_L3.3-Prikol-70B-v0.5-Q4_K_M.gguf
files:
- filename: Nohobby_L3.3-Prikol-70B-v0.5-Q4_K_M.gguf
sha256: 36f29015f1f420f51569603445a3ea5fe72e3651c2022ef064086f5617578fe6
uri: huggingface://bartowski/Nohobby_L3.3-Prikol-70B-v0.5-GGUF/Nohobby_L3.3-Prikol-70B-v0.5-Q4_K_M.gguf
- !!merge <<: *llama33
name: "theskullery_l3.3-exp-unnamed-model-70b-v0.5"
urls:
- https://huggingface.co/TheSkullery/L3.3-exp-unnamed-model-70b-v0.5
- https://huggingface.co/bartowski/TheSkullery_L3.3-exp-unnamed-model-70b-v0.5-GGUF
description: |
No description available for this model
overrides:
parameters:
model: TheSkullery_L3.3-exp-unnamed-model-70b-v0.5-Q4_K_M.gguf
files:
- filename: TheSkullery_L3.3-exp-unnamed-model-70b-v0.5-Q4_K_M.gguf
sha256: b8f7a0bcbccf79507ee28c8f6ca4e88625d9aa17f92deb12635775fb2eb42a2a
uri: huggingface://bartowski/TheSkullery_L3.3-exp-unnamed-model-70b-v0.5-GGUF/TheSkullery_L3.3-exp-unnamed-model-70b-v0.5-Q4_K_M.gguf
- !!merge <<: *llama33
name: "sentientagi_dobby-unhinged-llama-3.3-70b"
icon: https://huggingface.co/SentientAGI/Dobby-Unhinged-Llama-3.3-70B/resolve/main/assets/Dobby-70B.png
urls:
- https://huggingface.co/SentientAGI/Dobby-Unhinged-Llama-3.3-70B
- https://huggingface.co/bartowski/SentientAGI_Dobby-Unhinged-Llama-3.3-70B-GGUF
description: |
Dobby-Unhinged-Llama-3.3-70B is a language model fine-tuned from Llama-3.3-70B-Instruct. Dobby models have a strong conviction towards personal freedom, decentralization, and all things crypto — even when coerced to speak otherwise. Dobby-Unhinged-Llama-3.3-70B, Dobby-Mini-Leashed-Llama-3.1-8B and Dobby-Mini-Unhinged-Llama-3.1-8B have their own unique personalities, and this 70B model is being released in response to the community feedback that was collected from our previous 8B releases.
overrides:
parameters:
model: SentientAGI_Dobby-Unhinged-Llama-3.3-70B-Q4_K_M.gguf
files:
- filename: SentientAGI_Dobby-Unhinged-Llama-3.3-70B-Q4_K_M.gguf
sha256: b768e3828f8a72b7374bcf71600af8621563f1b002459b4dcd002ab144f68aa6
uri: huggingface://bartowski/SentientAGI_Dobby-Unhinged-Llama-3.3-70B-GGUF/SentientAGI_Dobby-Unhinged-Llama-3.3-70B-Q4_K_M.gguf
- !!merge <<: *llama33
name: "steelskull_l3.3-mokume-gane-r1-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/F_aK-DO_bMK7fWpDaHoNd.jpeg
urls:
- https://huggingface.co/Steelskull/L3.3-Mokume-Gane-R1-70b
- https://huggingface.co/bartowski/Steelskull_L3.3-Mokume-Gane-R1-70b-GGUF
description: |
Named after the Japanese metalworking technique 'Mokume-gane' (木目金), meaning 'wood grain metal', this model embodies the artistry of creating distinctive layered patterns through the careful mixing of different components. Just as Mokume-gane craftsmen blend various metals to create unique visual patterns, this model combines specialized AI components to generate creative and unexpected outputs.
overrides:
parameters:
model: Steelskull_L3.3-Mokume-Gane-R1-70b-Q4_K_M.gguf
files:
- filename: Steelskull_L3.3-Mokume-Gane-R1-70b-Q4_K_M.gguf
sha256: 301534a01cec1434c9d0a1b6f13be4e1b5896015d28cee393c3f323ee94efa50
uri: huggingface://bartowski/Steelskull_L3.3-Mokume-Gane-R1-70b-GGUF/Steelskull_L3.3-Mokume-Gane-R1-70b-Q4_K_M.gguf
- !!merge <<: *llama33
name: "steelskull_l3.3-cu-mai-r1-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/i3DSObqtHDERbQeh18Uf0.png
urls:
- https://huggingface.co/Steelskull/L3.3-Cu-Mai-R1-70b
- https://huggingface.co/bartowski/Steelskull_L3.3-Cu-Mai-R1-70b-GGUF
description: |
Cu-Mai, a play on San-Mai for Copper-Steel Damascus, represents a significant evolution in the three-part model series alongside San-Mai (OG) and Mokume-Gane. While maintaining the grounded and reliable nature of San-Mai, Cu-Mai introduces its own distinct "flavor" in terms of prose and overall vibe. The model demonstrates strong adherence to prompts while offering a unique creative expression.
L3.3-Cu-Mai-R1-70b integrates specialized components through the SCE merge method:
EVA and EURYALE foundations for creative expression and scene comprehension
Cirrus and Hanami elements for enhanced reasoning capabilities
Anubis components for detailed scene description
Negative_LLAMA integration for balanced perspective and response
Users consistently praise Cu-Mai for its:
Exceptional prose quality and natural dialogue flow
Strong adherence to prompts and creative expression
Improved coherency and reduced repetition
Performance on par with the original model
While some users note slightly reduced intelligence compared to the original, this trade-off is generally viewed as minimal and doesn't significantly impact the overall experience. The model's reasoning capabilities can be effectively activated through proper prompting techniques.
overrides:
parameters:
model: Steelskull_L3.3-Cu-Mai-R1-70b-Q4_K_M.gguf
files:
- filename: Steelskull_L3.3-Cu-Mai-R1-70b-Q4_K_M.gguf
sha256: 7e61cf7b3126414a7d7a54264e2ba42f663aefb7f82af6bb06da9d35e6a8843a
uri: huggingface://bartowski/Steelskull_L3.3-Cu-Mai-R1-70b-GGUF/Steelskull_L3.3-Cu-Mai-R1-70b-Q4_K_M.gguf
- !!merge <<: *llama33
name: "nohobby_l3.3-prikol-70b-extra"
icon: https://files.catbox.moe/x9t3zo.png
urls:
- https://huggingface.co/Nohobby/L3.3-Prikol-70B-EXTRA
- https://huggingface.co/bartowski/Nohobby_L3.3-Prikol-70B-EXTRA-GGUF
description: |
After banging my head against the wall some more - I actually managed to merge DeepSeek distill into my mess! Along with even more models (my hand just slipped, I swear)
The prose is better than in v0.5, but has a different feel to it, so I guess it's more of a step to the side than forward (hence the title EXTRA instead of 0.6).
The context recall may have improved, or I'm just gaslighting myself to think so.
And of course, since it now has DeepSeek in it - <think> tags!
They kinda work out of the box if you add <think> to the 'Start Reply With' field in ST - that way the model will write a really short character thought in it. However, if we want some OOC reasoning, things get trickier.
My initial thought was that this model could be instructed to use <think> either only for {{char}}'s inner monologue or for detached analysis, but actually it would end up writing character thoughts most of the time anyway, and the times when it did reason stuff it threw the narrative out of the window by making it too formal and even adding some notes at the end.
overrides:
parameters:
model: Nohobby_L3.3-Prikol-70B-EXTRA-Q4_K_M.gguf
files:
- filename: Nohobby_L3.3-Prikol-70B-EXTRA-Q4_K_M.gguf
sha256: 0efb34490e9714d6c8cc5dd4bf59ea894bf766af8a038982f5eba7bab9d0f962
uri: huggingface://bartowski/Nohobby_L3.3-Prikol-70B-EXTRA-GGUF/Nohobby_L3.3-Prikol-70B-EXTRA-Q4_K_M.gguf
- !!merge <<: *llama33
name: "latitudegames_wayfarer-large-70b-llama-3.3"
icon: https://huggingface.co/LatitudeGames/Wayfarer-Large-70B-Llama-3.3/resolve/main/wayfarer-large.jpg
urls:
- https://huggingface.co/LatitudeGames/Wayfarer-Large-70B-Llama-3.3
- https://huggingface.co/bartowski/LatitudeGames_Wayfarer-Large-70B-Llama-3.3-GGUF
description: |
We’ve heard over and over from AI Dungeon players that modern AI models are too nice, never letting them fail or die. While it may be good for a chatbot to be nice and helpful, great stories and games aren’t all rainbows and unicorns. They have conflict, tension, and even death. These create real stakes and consequences for characters and the journeys they go on.
Similarly, great games need opposition. You must be able to fail, die, and may even have to start over. This makes games more fun!
However, the vast majority of AI models, through alignment RLHF, have been trained away from darkness, violence, or conflict, preventing them from fulfilling this role. To give our players better options, we decided to train our own model to fix these issues.
The Wayfarer model series are a set of adventure role-play models specifically trained to give players a challenging and dangerous experience.
We wanted to contribute back to the open source community that we’ve benefitted so much from so we open sourced a 12b parameter version version back in Jan. We thought people would love it but people were even more excited than we expected.
Due to popular request we decided to train a larger 70b version based on Llama 3.3.
overrides:
parameters:
model: LatitudeGames_Wayfarer-Large-70B-Llama-3.3-Q4_K_M.gguf
files:
- filename: LatitudeGames_Wayfarer-Large-70B-Llama-3.3-Q4_K_M.gguf
sha256: 5b9f6923e247e5c6db3fc0f6fe558939b51b5fe1003d83cf5c10e74b586a1bf8
uri: huggingface://bartowski/LatitudeGames_Wayfarer-Large-70B-Llama-3.3-GGUF/LatitudeGames_Wayfarer-Large-70B-Llama-3.3-Q4_K_M.gguf
- !!merge <<: *llama33
name: "steelskull_l3.3-mokume-gane-r1-70b-v1.1"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/F_aK-DO_bMK7fWpDaHoNd.jpeg
urls:
- https://huggingface.co/Steelskull/L3.3-Mokume-Gane-R1-70b-v1.1
- https://huggingface.co/bartowski/Steelskull_L3.3-Mokume-Gane-R1-70b-v1.1-GGUF
description: |
Named after the Japanese metalworking technique 'Mokume-gane' (木目金), meaning 'wood grain metal', this model embodies the artistry of creating distinctive layered patterns through the careful mixing of different components. Just as Mokume-gane craftsmen blend various metals to create unique visual patterns, this model combines specialized AI components to generate creative and unexpected outputs.
overrides:
parameters:
model: Steelskull_L3.3-Mokume-Gane-R1-70b-v1.1-Q4_K_M.gguf
files:
- filename: Steelskull_L3.3-Mokume-Gane-R1-70b-v1.1-Q4_K_M.gguf
sha256: f91b7f7f35b0d23971595773cdc8151f6d6a33427f170dc2216e005b5fd09776
uri: huggingface://bartowski/Steelskull_L3.3-Mokume-Gane-R1-70b-v1.1-GGUF/Steelskull_L3.3-Mokume-Gane-R1-70b-v1.1-Q4_K_M.gguf
- !!merge <<: *llama33
name: "l3.3-geneticlemonade-unleashed-70b-i1"
icon: https://cdn-uploads.huggingface.co/production/uploads/65b19c6c638328850e12d38c/P8HgQAzAjEWE67u9sSKJz.png
urls:
- https://huggingface.co/zerofata/L3.3-GeneticLemonade-Unleashed-70B
- https://huggingface.co/mradermacher/L3.3-GeneticLemonade-Unleashed-70B-i1-GGUF
description: |
Inspired to learn how to merge by the Nevoria series from SteelSkull.
This model is the result of a few dozen different attempts of learning how to merge.
Designed for RP, this model is mostly uncensored and focused around striking a balance between writing style, creativity and intelligence.
overrides:
parameters:
model: L3.3-GeneticLemonade-Unleashed-70B.i1-Q4_K_M.gguf
files:
- filename: L3.3-GeneticLemonade-Unleashed-70B.i1-Q4_K_M.gguf
sha256: c1f5527ee6a5dec99d19d795430570c3af7efc969c30aca2c22b601af6ac4fe4
uri: huggingface://mradermacher/L3.3-GeneticLemonade-Unleashed-70B-i1-GGUF/L3.3-GeneticLemonade-Unleashed-70B.i1-Q4_K_M.gguf
- !!merge <<: *llama33
name: "llama-3.3-magicalgirl-2"
icon: https://cdn-uploads.huggingface.co/production/uploads/633e85093a17ab61de8d9073/FGK0qBGmELj6DEUxbbrdR.png
urls:
- https://huggingface.co/KaraKaraWitch/Llama-3.3-MagicalGirl-2
- https://huggingface.co/mradermacher/Llama-3.3-MagicalGirl-2-GGUF
description: |
New merge. This an experiment to increase the "Madness" in a model. Merge is based on top UGI-Bench models (So yeah, I would think this would be benchmaxxing.)
This is the second time I'm using SCE. The previous MagicalGirl model seems to be quite happy with it.
Added KaraKaraWitch/Llama-MiraiFanfare-3.3-70B based on feedback I got from others (People generally seem to remember this rather than other models). So I'm not sure how this would play into the merge.
The following models were included in the merge:
TheDrummer/Anubis-70B-v1
SicariusSicariiStuff/Negative_LLAMA_70B
LatitudeGames/Wayfarer-Large-70B-Llama-3.3
KaraKaraWitch/Llama-MiraiFanfare-3.3-70B
Black-Ink-Guild/Pernicious_Prophecy_70B
overrides:
parameters:
model: Llama-3.3-MagicalGirl-2.Q4_K_M.gguf
files:
- filename: Llama-3.3-MagicalGirl-2.Q4_K_M.gguf
sha256: 01bd7e23c764d18279da4dbd20de19e60009d6e66e8aad1c93732a33f214e6a2
uri: huggingface://mradermacher/Llama-3.3-MagicalGirl-2-GGUF/Llama-3.3-MagicalGirl-2.Q4_K_M.gguf
- !!merge <<: *llama33
name: "steelskull_l3.3-electra-r1-70b"
icon: https://cdn-uploads.huggingface.co/production/uploads/64545af5ec40bbbd01242ca6/GXLpDNkbGEvESfLmWkKpD.jpeg
urls:
- https://huggingface.co/Steelskull/L3.3-Electra-R1-70b
- https://huggingface.co/bartowski/Steelskull_L3.3-Electra-R1-70b-GGUF
description: |
L3.3-Electra-R1-70b is the newest release of the Unnamed series, this is the 6th iteration based of user feedback.
Built on a custom DeepSeek R1 Distill base (TheSkullery/L3.1x3.3-Hydroblated-R1-70B-v4.4), Electra-R1 integrates specialized components through the SCE merge method. The model uses float32 dtype during processing with a bfloat16 output dtype for optimized performance.
Electra-R1 serves newest gold standard and baseline. User feedback consistently highlights its superior intelligence, coherence, and unique ability to provide deep character insights. Through proper prompting, the model demonstrates advanced reasoning capabilities and unprompted exploration of character inner thoughts and motivations.
The model utilizes the custom Hydroblated-R1 base, created for stability and enhanced reasoning. The SCE merge method's settings are precisely tuned based on extensive community feedback (of over 10 diffrent models from Nevoria to Cu-Mai), ensuring optimal component integration while maintaining model coherence and reliability. This foundation establishes Electra-R1 as the benchmark upon which its variant models build and expand.
overrides:
parameters:
model: Steelskull_L3.3-Electra-R1-70b-Q4_K_M.gguf
files:
- filename: Steelskull_L3.3-Electra-R1-70b-Q4_K_M.gguf
sha256: 1f39e1d398ef659ad7074c827dc6993c2007813a303ee72c189e88c4c76f70db
uri: huggingface://bartowski/Steelskull_L3.3-Electra-R1-70b-GGUF/Steelskull_L3.3-Electra-R1-70b-Q4_K_M.gguf
- &rwkv
url: "github:mudler/LocalAI/gallery/rwkv.yaml@master"
name: "rwkv-6-world-7b"
icon: https://avatars.githubusercontent.com/u/132652788
license: apache-2.0
urls:
- https://huggingface.co/RWKV/rwkv-6-world-7b
- https://huggingface.co/bartowski/rwkv-6-world-7b-GGUF
tags:
- llm
- rwkv
- cpu
- gpu
- rnn
description: |
RWKV (pronounced RwaKuv) is an RNN with GPT-level LLM performance, and can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7.
So it's combining the best of RNN and transformer - great performance, fast inference, fast training, saves VRAM, "infinite" ctxlen, and free text embedding. Moreover it's 100% attention-free, and a Linux Foundation AI project.
overrides:
parameters:
model: rwkv-6-world-7b-Q4_K_M.gguf
files:
- filename: rwkv-6-world-7b-Q4_K_M.gguf
sha256: f74574186fa4584f405e92198605680db6ad00fd77974ffa14bf02073bb90273
uri: huggingface://bartowski/rwkv-6-world-7b-GGUF/rwkv-6-world-7b-Q4_K_M.gguf
- &qwen25coder
name: "qwen2.5-coder-14b"
icon: https://avatars.githubusercontent.com/u/141221163
url: "github:mudler/LocalAI/gallery/chatml.yaml@master"
license: apache-2.0
tags:
- llm
- gguf
- gpu
- qwen
- qwen2.5
- cpu
urls:
- https://huggingface.co/Qwen/Qwen2.5-Coder-14B
- https://huggingface.co/mradermacher/Qwen2.5-Coder-14B-GGUF
description: |
Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5:
Significantly improvements in code generation, code reasoning and code fixing. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o.
A more comprehensive foundation for real-world applications such as Code Agents. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies.
Long-context Support up to 128K tokens.
overrides:
parameters:
model: Qwen2.5-Coder-14B.Q4_K_M.gguf
files:
- filename: Qwen2.5-Coder-14B.Q4_K_M.gguf
sha256: 94f277a9ac7caf117140b2fff4e1ccf4bc9f35395b0112f0d0d7c82c6f8d860e
uri: huggingface://mradermacher/Qwen2.5-Coder-14B-GGUF/Qwen2.5-Coder-14B.Q4_K_M.gguf
- !!merge <<: *qwen25coder
name: "qwen2.5-coder-3b-instruct"
urls:
- https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct
- https://huggingface.co/bartowski/Qwen2.5-Coder-3B-Instruct-GGUF
overrides:
parameters:
model: Qwen2.5-Coder-3B-Instruct-Q4_K_M.gguf
files:
- filename: Qwen2.5-Coder-3B-Instruct-Q4_K_M.gguf
sha256: 3da3afe6cf5c674ac195803ea0dd6fee7e1c228c2105c1ce8c66890d1d4ab460
uri: huggingface://bartowski/Qwen2.5-Coder-3B-Instruct-GGUF/Qwen2.5-Coder-3B-Instruct-Q4_K_M.gguf
- !!merge <<: *qwen25coder
name: "qwen2.5-coder-32b-instruct"
urls:
- https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct
- https://huggingface.co/bartowski/Qwen2.5-Coder-32B-Instruct-GGUF
overrides:
parameters:
model: Qwen2.5-Coder-32B-Instruct-Q4_K_M.gguf
files:
- filename: Qwen2.5-Coder-32B-Instruct-Q4_K_M.gguf
sha256: 8e2fd78ff55e7cdf577fda257bac2776feb7d73d922613caf35468073807e815
uri: huggingface://bartowski/Qwen2.5-Coder-32B-Instruct-GGUF/Qwen2.5-Coder-32B-Instruct-Q4_K_M.gguf
- !!merge <<: *qwen25coder
name: "qwen2.5-coder-14b-instruct"
urls:
- https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct
- https://huggingface.co/bartowski/Qwen2.5-Coder-14B-Instruct-GGUF
overrides:
parameters:
model: Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf
files:
- filename: Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf
sha256: 2946d28c9e1bb2bcae6d42e8678863a31775df6f740315c7d7e6d6b6411f5937
uri: huggingface://bartowski/Qwen2.5-Coder-14B-Instruct-GGUF/Qwen2.5-Coder-14B-Instruct-Q4_K_M.gguf
- !!merge <<: *qwen25coder
name: "qwen2.5-coder-1.5b-instruct"
urls:
- https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct
- https://huggingface.co/bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF
overrides:
parameters:
model: Qwen2.5-Coder-1.5B-Instruct-Q4_K_M.gguf
files:
- filename: Qwen2.5-Coder-1.5B-Instruct-Q4_K_M.gguf
sha256: f530705d447660a4336c329981af164b471b60b974b1d808d57e8ec9fe23b239
uri: huggingface://bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF/Qwen2.5-Coder-1.5B-Instruct-Q4_K_M.gguf
- !!merge <<: *qwen25coder
name: "qwen2.5-coder-7b-instruct"
urls:
- https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct
- https://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF
overrides:
parameters:
model: Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf
files:
- filename: Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf
sha256: 1664fccab734674a50763490a8c6931b70e3f2f8ec10031b54806d30e5f956b6
uri: huggingface://bartowski/Qwen2.5-Coder-7B-Instruct-GGUF/Qwen2.5-Coder-7B-Instruct-Q4_K_M.gguf
- !!merge <<: *qwen25coder
name: "qwen2.5-coder-7b-3x-instruct-ties-v1.2-i1"
urls:
- https://huggingface.co/BenevolenceMessiah/Qwen2.5-Coder-7B-3x-Instruct-TIES-v1.2
- https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-3x-Instruct-TIES-v1.2-i1-GGUF
description: |
The following models were included in the merge:
BenevolenceMessiah/Qwen2.5-Coder-7B-Chat-Instruct-TIES-v1.2
MadeAgents/Hammer2.0-7b
huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated
overrides:
parameters:
model: Qwen2.5-Coder-7B-3x-Instruct-TIES-v1.2.i1-Q4_K_M.gguf
files:
- filename: Qwen2.5-Coder-7B-3x-Instruct-TIES-v1.2.i1-Q4_K_M.gguf
sha256: c28a4da700f634f1277f02391d81fa3c0ba783fa4b02886bd4bfe5f13b6605ef
uri: huggingface://mradermacher/Qwen2.5-Coder-7B-3x-Instruct-TIES-v1.2-i1-GGUF/Qwen2.5-Coder-7B-3x-Instruct-TIES-v1.2.i1-Q4_K_M.gguf
- !!merge <<: *qwen25coder
name: "qwen2.5-coder-7b-instruct-abliterated-i1"
urls:
- https://huggingface.co/huihui-ai/Qwen2.5-Coder-7B-Instruct-abliterated
- https://huggingface.co/mradermacher/Qwen2.5-Coder-7B-Instruct-abliterated-i1-GGUF
description: |
This is an uncensored version of Qwen2.5-Coder-7B-Instruct created with abliteration (see this article to know more about it).
Special thanks to @FailSpy for the original code and technique. Please follow him if you're interested in abliterated models.
overrides:
parameters:
model: Qwen2.5-Coder-7B-Instruct-abliterated.i1-Q4_K_M.gguf
files:
- filename: Qwen2.5-Coder-7B-Instruct-abliterated.i1-Q4_K_M.gguf
sha256: 9100ccd9e8167cefda98bd1c97d5d765a21e70e124e4d6b89945fd66ebb481b4
uri: huggingface://mradermacher/Qwen2.5-Coder-7B-Instruct-abliterated-i1-GGUF/Qwen2.5-Coder-7B-Instruct-abliterated.i1-Q4_K_M.gguf
- !!merge <<: *qwen25coder
name: "rombos-coder-v2.5-qwen-7b"
icon: https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/QErypCEKD5OZLxUcSmYaR.jpeg
urls:
- https://huggingface.co/rombodawg/Rombos-Coder-V2.5-Qwen-7b
- https://huggingface.co/bartowski/Rombos-Coder-V2.5-Qwen-7b-GGUF
- https://docs.google.com/document/d/1OjbjU5AOz4Ftn9xHQrX3oFQGhQ6RDUuXQipnQ9gn6tU/edit?usp=sharing
description: |
Rombos-Coder-V2.5-Qwen-7b is a continues finetuned version of Qwen2.5-Coder-7B-Instruct. I took it upon myself to merge the instruct model with the base model myself using the * Ties* merge method as demonstrated in my own "Continuous Finetuning" method (link available).
This version of the model shows higher performance than the original instruct and base models.
overrides:
parameters:
model: Rombos-Coder-V2.5-Qwen-7b-Q4_K_M.gguf
files:
- filename: Rombos-Coder-V2.5-Qwen-7b-Q4_K_M.gguf
sha256: ca16a550f1be00b7e92f94c0c18ea6af1e5c158d5d1cb3994f9f0a0d13922272
uri: huggingface://bartowski/Rombos-Coder-V2.5-Qwen-7b-GGUF/Rombos-Coder-V2.5-Qwen-7b-Q4_K_M.gguf
- !!merge <<: *qwen25coder