🔄 In a Training Loop

Stas Veretennikov PRO

ironbcc

·

AI & ML interests

None yet

Recent Activity

liked a dataset about 22 hours ago

allenai/wildguardmix

published a model 1 day ago

ironbcc/Seb-9B-v2-NVFP4

updated a model 1 day ago

ironbcc/Seb-9B-v2-NVFP4

View all activity

Organizations

None yet

liked a dataset about 22 hours ago

allenai/wildguardmix

Viewer • Updated Jun 29, 2024 • 88.5k • 8.03k • 110

published a model 1 day ago

ironbcc/Seb-9B-v2-NVFP4

Text Classification • 9B • Updated 1 day ago • 23 • 1

updated a model 1 day ago

ironbcc/Seb-9B-v2-NVFP4

Text Classification • 9B • Updated 1 day ago • 23 • 1

liked a model 2 days ago

ironbcc/Seb-9B-v2

Text Classification • 10B • Updated 2 days ago • 97 • 1

New activity in Cloudflare/clef 2 days ago

Seb-9B-v2: another small decision model that also speaks /v1/systemone

#13 opened 2 days ago by

commented on Seb-9B vs Jev 1.13, JEV-27B-VL and Laya on identical rows 2 days ago

Update: Seb-9B-v2 is out. This post covers Seb-9B (v1); its numbers were not re-run on v2. Same disclosure applies: everything below is self-run.

Decision Index 0.3 public index: 59.16 (board numbers: Jev 1.13 57.96, Jev 1.14 60.70)
Laya suites: 0.801 mean accuracy on the same 17 public suites (v1 0.792, Jev 1.13 0.786, Laya 0.694)
Speed: about 22 ms per decision on one RTX PRO 6000
New: reads images, inputs up to 32k tokens, speaks both /v1/systemone and a plain chat API

New write-up, including where it loses: https://huggingface.co/blog/ironbcc/seb-9b-v2
Model: https://huggingface.co/ironbcc/Seb-9B-v2

New activity in LocalLLaMA/typed-decisions 2 days ago

Seb-9B (9B, fitted on train, one question per request): 0.754 acc / KL 0.349 / Brier 0.131 / ECE 0.031 on the official test split

#13 opened 18 days ago by

New activity in autotrust/JEV-27B-VL 2 days ago

Seb-9B vs JEV-27B-VL on identical rows: 17 Laya public suites, a 6,920-row decision suite and two image suites (self-run)

#4 opened 18 days ago by

New activity in convaiinnovations/laya 2 days ago

Seb-9B vs Jev 1.13 vs Laya on the 17 public suites from Laya's benchmark report (identical questions, self-run)

#30 opened 18 days ago by

New activity in multimodalart/jev-decision-index 2 days ago

Self-scored Decision Index 0.3 public run for Seb-9B-v2 (9B, Apache-2.0): 59.16

#96 opened 2 days ago by

New activity in nace-ai/drex-v1.5 2 days ago

Seb-9B-v2: another 9B decision model, public index comparison

#3 opened 2 days ago by

New activity in openjev/openjev 2 days ago

Seb-9B-v2: Apache-2.0 decision model with images, 32k context and both APIs

#7 opened 2 days ago by

New activity in Mapika/decider-4b 2 days ago

Seb-9B-v2: a 9B decision model, cross-check welcome

#5 opened 2 days ago by

New activity in bespokelabs/Bespoke-Nimble-9B-v3 2 days ago

Seb-9B-v2 (9B): closest peers on the public index, per-area comparison

#2 opened 2 days ago by

New activity in jaredpalmer/kev-9b 2 days ago

Seb-9B-v2: same size class, happy to compare on identical rows

#5 opened 2 days ago by

New activity in perplexity-ai/pplx-decider-v1.1-27b 2 days ago

Seb-9B-v2 (9B): the same 'decision model first, LLM fallback' idea, with numbers

#6 opened 2 days ago by

New activity in autotrust/JEV-27B-VL 2 days ago

Update to our earlier comparison (#4): Seb-9B-v2 reads images and goes to 32k context

#10 opened 2 days ago by

New activity in vllm-sr/d3 2 days ago

Seb-9B-v2: a 9B decision model for comparison (and a router result)

#8 opened 2 days ago by

updated a dataset 2 days ago

ironbcc/seb-9b-v2-decision-index

Updated 2 days ago • 27

published a dataset 2 days ago

ironbcc/seb-9b-v2-decision-index

Updated 2 days ago • 27