Nick M

veldierin

54 2

AI & ML interests

None yet

Recent Activity

new activity 1 day ago

nightmedia/Qwen3.8-27B-Brainwaves-WFH-mxfp4-mlx:Keep us updated on this recipe, it looks interesting

new activity 2 days ago

peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP:Agentic Coding using OpenCode facing the issue

new activity 3 days ago

DavidAU/Qwen3.8-27B-Cold-Fable-Fusion-GAIN-V1.1-732-Heretic-Uncensored-stage1:Was this finetune done to retain the level of thinking of the base so we can have proper xhigh reasoning?

View all activity

Organizations

None yet

New activity in nightmedia/Qwen3.8-27B-Brainwaves-WFH-mxfp4-mlx 1 day ago

Keep us updated on this recipe, it looks interesting

❤️ 1

#2 opened 2 days ago by

veldierin

New activity in peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP 2 days ago

Agentic Coding using OpenCode facing the issue

#9 opened 2 days ago by

SolutionsDealer

New activity in DavidAU/Qwen3.8-27B-Cold-Fable-Fusion-GAIN-V1.1-732-Heretic-Uncensored-stage1 3 days ago

Was this finetune done to retain the level of thinking of the base so we can have proper xhigh reasoning?

#6 opened 3 days ago by

veldierin

New activity in nightmedia/Qwen3.8-27B-Brainwaves-Heretic-mxfp4-mlx 4 days ago

Gemini trace analysis

#1 opened 4 days ago by

nightmedia

New activity in peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF-MTP 4 days ago

MTP enabled: unbounded VRAM growth during inference until CUDA abort (V100)

#4 opened 4 days ago by

fozosan

New activity in mradermacher/Qwen3.8-27B-OBLITERATED-GGUF 4 days ago

Model stops answering prematurely

➕ 1

#1 opened 6 days ago by

theblackcat

New activity in DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF 5 days ago

Great model so far

❤️🔥 3

#17 opened 8 days ago by

mythrime

New activity in peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF 5 days ago

request for re-quantization w/fixed MTP heads

👍 1

#3 opened 5 days ago by

veldierin

How it compares to Ornith 1.5 35B?

#2 opened 5 days ago by

MartinPatterson

New activity in DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 6 days ago

Mtp quality? 🤔

#7 opened 13 days ago by

LinkuStarto

New activity in peculiar-ragdoll/Occult-Nail-1.0-35B-A3B-GGUF 8 days ago

APEX quant

#1 opened 10 days ago by

benoe

New activity in SC117/Ornith-1.5-35B-A3B-MTP-APEX-GGUF 8 days ago

Waiting for Ornith-1.5-35B-Heretic-MTP-APEX-GGUF

#2 opened 8 days ago by

atfa

New activity in DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 8 days ago

VERSION NVFP4 : Would a nvfp4 version possible ?

➕ 1

#14 opened 11 days ago by

crazyhenres

New activity in DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF 8 days ago

Testing it on greenboost-cli with good results

❤️👍 2

#15 opened 9 days ago by

hyphaed

replied to nightmedia's post 9 days ago

I appreciate your merges, @nightmedia - even if we have to fix the MTP 😃

New activity in DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NM-DAU-NEO-MAX-MTP-GGUF 10 days ago

request (424318 tokens) exceeds the available context size (250112 tokens), try increasing it: {"error":{"code":400,"message":"request (424318 tokens)

#14 opened 10 days ago by

akierum

reacted to nightmedia's post with 👍 10 days ago

Post

4177

Qwen3.8-27B metrics

It's hard to track all model cards where I post these, so I figured people would get more value out of seeing these in the open.

quant     arc   arc/e boolq hswag obkqa piqa  wino
mxfp8     0.591,0.782,0.896,0.746,0.448,0.801,0.711
q8-hi     0.602,0.779,0.896,0.747,0.446,0.793,0.703
q6-hi     0.602,0.775,0.895,0.748,0.448,0.795,0.710
q4-hi     0.604,0.780,0.898,0.744,0.454,0.795,0.708
mxfp4     0.581,0.771,0.889,0.738,0.442,0.798,0.713
1M
mxfp8     0.590,0.787,0.897,0.744,0.446,0.801,0.709

Quant     Perplexity      Peak Memory   Tokens/sec
mxfp8     6.090 ± 0.054   34.74 GB      138
mxfp4     5.952 ± 0.051   21.30 GB      148

The performance is as measured on a M4 MBP 128GB, speed may vary depending on your platform.

These are all instruct metrics, generated by including this line in the jinja template:

{%- set enable_thinking = false %}

Then run the test suite to generate the metrics:

mlx_lm.evaluate --model MODEL --tasks winogrande boolq arc_challenge arc_easy hellaswag openbookqa piqa

This will generate the file:

eval_MODEL_0.4.9_winogrande_boolq_arc_challenge_arc_easy_hellaswag_openbookqa_piqa

This is a JSON containing all gathered metrics; for example the q4-hi:

"arc_challenge": {
        "alias": "arc_challenge",
        "acc,none": 0.5819112627986348,
        "acc_stderr,none": 0.014413988396996116,
        "acc_norm,none": 0.6040955631399317,
        "acc_norm_stderr,none": 0.01429122839353657
    },

I use the value of acc_norm for metrics, rounded to 3 decimals.

As I get more quants tested, I will add them here.

A complete test run for a single quant takes 7-9 hours depending on quant size, 10-12 hours for BF16 depending on perplexity: this is why you see on my model cards that I usually post the first three, that only take 2-3 hours :)

-G